There aren't any drop downs or new players in the football game, which is why there is more emphasis on filling the majority of your class on signing day. You can always find guys with high potential late, but strong cores and high potential will be much more rare, especially when it comes to linemen, QB's, RB's, and WR's. As recruiting draws to a close, the ability to recruit long distance becomes easier and less expensive, so in theory you can recruit the entire country towards the end.
As for average vs high potential, the difference can be relatively small or very large, depending on the player's WE. Avg with very high WE can look a lot like high potential with middling WE, especially in D3. The problem is that the recruiting pool isn't evenly distributed when it comes to certain positions vs others. For example, most QB recruits are pretty terrible, making their potential and work ethic critically important. They need to be grown in order to be good. Meanwhile, you will occasionally find linemen whose cores are high enough that even average potential will be more than enough to make them worthwhile.