I am more inclined to pay attention to defense over offense, but can understand how many go the other way. Offense is easy to see the results in, it's one guys stats....AVG, OBP, home runs, etc.
Defense has many more moving parts...ratings, pitchers, ballpark, etc. and is much harder to quantify.