Here are the factors to consider, IMO:
1. Are you looking for a CB or a Safety?
2. As mentioned, all the ratings and their potential must be known also.
3. What is your defensive philosophy? Example: I have built a secondary based on speed as my number 1 requirement. Some coaches build a secondary based on TECH as their number 1 requirement. Depending on the style you play and the expected opponents you will face, you may need to have more safety type players...or if you play a lot of Dime, you will want more CB type players.