Which is why you should go for the players that will help you the most, regardless of potential. Potential is important, but only one factor in what may help your team.
If you REALLY owned the New England Patriots and you had a player you could sign that your scouts told you had low potential. But he was much better than any other that could help your team Right Now. Wouldn't you sign him. So maybe he wouldn't get much better. But if he can help your team more than another that might grow more you'd certainly have to consider him.