Posted by harriswb3 on 1/4/2018 7:11:00 AM (view original):
Question: Have you done any "look back" to see how your predictions have worked out? This season and last seem to be pretty "spot on" from what little bit I've looked at.
Not in any official capacity. I've often quickly browsed the Top 25 (or so) visually and I've come to the same conclusion as you - pretty spot-on. I mean, I'm always missing badly (both in the overestimate and underestimate sense) on a team or two, but mostly it's pretty accurate. This typically happens in one of two main cases:
1. A coach changes teams within a given level (IA, IAA...) after having a great season prior. That first season, they carry their old "Coach Effect" from the prior team with them, but getting used to a new team is difficult, so they end up doing worse than I projected as they "learn" the new team. or
2. A coach wildly overperforms or underperforms, inflating/deflating their Coach Effect, then they return to the "norm" the next season and I'm way off.
It's not always the case that one of those two things happens. Sometimes my algorithm is just wrong. But I'm comfortable with where it's at these days and pretty proud of the accuracy.
I do have all the data from past seasons (actual game results). And for the past few seasons, I now have the game-by-game (that huge results matrix) of my regular season projections. If I get ambitious, I might go back through that data and do a compare.