That thought occurred to me but you should be able to tell how quickly a player can react and how quickly he moves once he does react. Beyond that, it will measure how hard the ball was hit, how fast the runner ran and how hard the fielder threw the ball. Using this data, you should be able to generate averages. After doing that, you should be able to know which players are above/below average.
But, yeah, it throws speculation out the door. Rather than saying "75% of the SS make that play", you'll have to say "His poor jump, which was well below his average, and poor positioning is why he failed to make that play."
As I said, defensive stats will have some credibility now.