I don't think its that big of an issue. You're pretty much going to run into ath/spd being important in any defense. In a conference with all sims, you should be able to dominate if you aim for 70+ def on most of your players, which should be doable. You can give up a little bit of ath on your guards, and a little bit of spd on your bigs. For example a guard with 35 (green) ath and 50 (green) def will end up being a pretty solid player at that level, assuming 80+ spd and decent ball skills (bh, pass).
You might have to make some sacrifices in non-core attributes, but you can pull it off. Win your conference tournaments, make some national tournaments, increase your prestige a bit, and then aim a little higher on your recruits.