your question is a strange one. i think you have a misunderstanding about what potential means, so i will give you the quick rundown.
potential as displayed to us is merely a derivative. it is NOT a core attribute of a player.
really, a player has 2 values for each rating - a current value, and a maximum value. for example, a player might have 50 speed, with a max of 60 speed, and thus, we are told he has medium potential.
so, in your question, the guy having low potential just means the different of his max and current value is 5 or less. if you end up losing a bunch of points, his potential will be higher - the potential is just the difference of the max and current. so for example (although outrageous, just to illustrate), if you have a guy with 80 speed low potential, and you don't practice and he drops to 60 speed, then he would have high potential.