You are looking at the wrong things. I qualified to jump from
UAB to
NC State in Wooden. Never made the NT in 5 seasons. Fortunately, NC State has been unspeakably bad for many seasons. So, it is possible to jump from D3 to D1 and then jump to a Big 6 positions 5 seasons later.
The thing holding you back will not be prestige (quit thinking about it), but, after only 3 seasons at Yale, loyalty. I would hope that, if Yale gets to the NT again or wins >20 games next season, that you would be eligible to jump to a truly devastated program similar to this one.
Looking at what I walked into...maybe I should have waited another season or two to get something better, but
here I am.