It seems like a player is more likely to lose points in a category that they have high potential in, as opposed to a category with average or low potential. Does anyone know why that is? 

This is true on both ends of the spectrum.  When a player maxes out in a category and you get the email from your assistant coach, you can zero it out and not lose any points (probably lose a very small fraction, but nothing significant).

You can just look at potential as just a likelihood for the rating to move, be it positive or negative.