You can if you have some advantages to overcome the Prestige disadvantage. If you have an extra scholarship, or you've tracked his recruiting and he's gotten beat up in a battle or two, or you have a significant distance advantage, then it's green light. However, you probably should prepare a backup plan if you've misjudged.
kmason must mean D2 & D3. As long as a recruit is already accepting visits, then prestige makes very little difference at lower levels. If you have to spend a bunch of money pulling the recruit down first, then that is easy to attribute to a prestige advantage, but it's more likely just that you're forced to spend your funds a little less efficiently early on.