One of the "criticisms" of the GUESS reports is that it undervalues QB's. And I can't disagree with that - certain positions consistently score higher in the ratings than others.
I took a look at approximately 80 seasons of recruiting classes, across all 11 worlds, and pulled some raw stats and re-ran GUESS ratings, along with the "base" ratings from the GD 3.0 engine currently in Beta. The results below are for D1A; all other divisions showed similar results, albeit with QB coming in slightly lower. I tossed out some players with abnormally low scores - they were obviously AC signings and were skewing the result because the AC doesn't sign players in the same kinds of distribution as human coaches.
Most of the columns should be self explanatory - the " % " column shows how the average rating compares to the highest average rating across positions. For example, the QB average rating is 84% of the Running Back average rating. For D1AA, D2, and D3, that number runs closer to 80%.
However, as you can see, it's not so much a result in deficiencies of the formulas as it is deficiencies of the STATS. The CORES column, which is the result of just adding up the three main "core" attributes at each position, shows core stat scores for QB's are substantially lower than they are for other positions. Totals are included just for comparison sake.
One of the things we've talked about in the past is "normalizing" the ratings between positions, so that the average rating across all positions are near (and hopefully the MAX ratings follow suit). With a sample set this large, I think I can comfortably adjust the formulas to do so, not just for QB's but for all positions. The two big questions are -- 1) Should I do so, or should I let the raw stats dictate ratings regardless of position, and 2) Implement now, or wait until the big revision I'm working on in conjunction with the beta?
For those who are interested, current formulas can be found here. Note that for DL / LB / DB, and the purposes of this exercise, I averaged the weights between the two sub-positions (i.e. average of S/CB weights to get DB formula.)
Other thoughts and constructive criticism welcome.