It's a personal preference, no doubt. If you think it's the right thing to do, you probably wouldn't be questioning it. And it really comes down to what your ultimate goals are.
As for being 'stuck' at an A-, that's really a chicken/egg concept. Are you stuck at an A- prestige because you haven't been able to land good recruits? Have you had trouble getting the recruits because your conference as a whole wasn't earning enough money? This is something that only you are really going to know the answer to.
If you're a good coach now, you should be a good coach there. And you know whether or not you're a good coach.
Ask yourself a simple question, if both schools could only achieve an A- and be a contender, who would I prefer to coach? Factor everything in, including the school, the conference the coaches in the conference.