Posted by dahsdebater on 2/19/2012 4:36:00 PM (view original):
I think that the confounding you are suggesting is much more of a problem analyzing offense than defense.  That's a big part of why I didn't bother.  Also because it's easier to learn about offensive efficiency based on the eye test, just watching what happens.  The game compiles offensive stats for us.  No such advantages for defensive stats.  Also, other than double teams, which are uncommon enough not to be a major confounding problem, the only defensive setting that coaches can adjust is +/-.  I figure over a large enough sample those should largely balance each other.  Also, a large portion of the sample was sim coached, so they always come out in a 0.  I really think the biggest flaw in the analysis is the assumption of starter-on-starter matchups.

the confounding you talk about here is all those little side things, IMO - the implicit variables, like what a coach does and who else is playing - not the ones in question like ath and def and spd. not to suggest they aren't important, the starting thing is probably pretty important.

what i am saying is, in general, and for defense too - i think the biggest problem is the lack of having independent variables in the key variables you are looking at - the players' ratings. you say you focus on defense - so on defense, you have ath, def, sb, and reb that are all of huge importance - and ath and def are very strongly correlated, as well as sb and reb. any judgement you make about any of those 4 stats, based on a regression, is going to be very shaky because you never know how much of what you are seeing is a result of the variable in question, or the variable that depends on it.

for example, a player with great ath almost always has great def. so, you look at the impact of ath on fg%. you see a player with 70 ath might result in his opponent having 10% higher fg% than 90 ath (obviously an over statement, but to keep #s simple...). well, if you could say, give me a 70 ath 70 def player, and compare him with a 90 ath 70 def player, you may be able to say look - that 10% higher FG% is due to the 20 ath. however - because the 90 ath guy probably has 90 def - now all you are saying is, going from 70/70 to 90/90 is 10%. well, is the 20 ath 9% and the 20 def 1%? or 20 ath 1% and 20 def 9%? or 5% each? how do you know? same goes for sb and reb!

on another note - what i was talking about so far was just a generic problem that would apply to any regression with dependent variables. thats really just a statistical thing that happens to apply to HD - my conclusions really have nothing to do with HD itself.

however, i would like to make a comment that does have an HD context. if you think about doing something like this regression in HD - even if you had independent variables from recruit generation - it is a very tough thing to do. for example, take the original statement. in short - shot blocking may be more important than it generally gets credit for - shot blocking correlates more to opponent fg% than any other stat, even for guards.

well, forget the guard part for a second. assume the correlation was true. does that make sb most important? not necessarily - and its EXTREMELY hard to say. for example, good def often results in the player not being able to take a shot in the first place, or even in a steal. neither of those qualities are reflected in fg%. i don't think sb has any impact on steals - ath, spd, and def do - so if sb has a bigger impact on fg%, you don't know if the steal part outweighs that difference or not. also, sb could have some impact on if a player takes a shot in the first place - not wanting it to get blocked - but i strongly feel ath and def play a big part in not letting you take the shot in the first place. so then what happens? most likely, another player on the team shoots instead. so you have a potentially huge benefit, of shutting down a key offensive player on that possession, so a lesser player has to take the shot. that has a huge impact on TEAM fg%, but not necessarily on the fg% of the individual you are guarding.

so, it is a very, very difficult and potentially dangerous thing to do - to try to take correlations from a regression like this and draw conclusions in the context of the game. im not suggestion you shouldn't try, nor that dahs is giving bad info or anything like that. i personally love to see this kind of stuff, i think its very interesting. i just think its also a very easy thing to draw incorrect conclusions from, if you aren't careful or aren't aware of the shortcomings inherent in the model - not dahs's model, but any model attempting to accomplish similar things.


finally - i find that guard comment very curious. despite everything ive said - these kind of regressions CAN give you some valuable insights. it is pretty surprising to hear you say the correlation on opponent fg% is highest for sb, for guards - even with whatever potential issues there might be - that is a pretty powerful statement. i always thought big man sb, in any defense, would have a significant impact on opposing guard fg%, because you have bigs trying to block shots at the time shots are taken, regardless of what defense you play. but i wouldn't think a guard having more sb would have much impact on another guard. guards usually have so little sb, that it might just be that its a smaller factor than things like ath and def - but it might be a larger factor per point. that would be pretty significant, i think, because i believe most people (myself included) totally ignore sb for guards.

one final clarification - dahs, when i say you are talking about these littler things, that is the wrong way to say it. heres what i really mean. you might suggest that 10 points of sb has a 5% impact on opponent fg%, the highest of any rating. well, because you aren't taking into account starters playing against other players than the opposing starter, maybe that number is wrong - maybe its a 3% impact. and because of all these other details, maybe its actually really 2.5%.

those things can all be important, so little is a terrible way to describe it. but those are all things that affect the number you come up with.

my primary concern is not with the number - although i agree there are definitely reasons the number can't be totally accurate - my primary concern is, once you have that number, how do you interpret it? what does it mean? if a player with 10 points of sb more, has - on average - 10 more points of reb - how much of that 5% or 2.5% or whatever the number is, is because the defender's SB caused him to block shots, and how much is because his rebounding caused him to prevent putbacks and offensive rebounds, both of which result in high percentage shots? because the ratings here are dependent, even if you took into account all the factors in the world to come up with the right % for the correlation, you could never know how to interpret it, how much of the effect was due to one rating or another. if the "real" number is 3.682%, is that 2% because of the 10 sb, and 1.682% from the 10 reb? or .1% from the 10 sb, and 3.582% from the 10 reb?

Posted by stinenavy on 2/19/2012 7:01:00 PM (view original):
I don't think I've seen so much humblebrag on these forums before. Congrats billyg.

This is why we can't have nice things.

I applaud the efforts to apply statistical analysis to the game in and effort to decipher the programming logic.  I do not have the time or energy to devote to that.  I just enjoy the game.  That is probably why I only have three titles.  I wish I did have that kind of time.  But I learned one thing early on playing this game after sending a ticket about a series of seemingly illogical losses.  The response I received changed my view of the game: "The system is programed for 25% of the game outcomes to be upsets, the same as the NCAA."  I don't know if that has been tweaked in the last few years, but randomness is programmed in.  So each night you have a 25% chance of being upset.  Whether you suffered an upset the previous two nights or not, you still have a 25% chance of being upset the next night.

Obviously, the better you are compared to your opponent, the less likely that upset will take place, but it leaves luck play a part in this game.  So even though statistics can be very helpful, sometimes it just doesn't matter due to the randomness factor.  It helps me understand why I may have a team I am not confident in do very well, and a team I think should be great, fall flat on their face.  Same with player production.  I'm not saying stats can't help you build a better team.  I believe they can.  But just realize that 25% of the time the stats might not mean anything.

Posted by commish118 on 2/20/2012 12:03:00 AM (view original):
I applaud the efforts to apply statistical analysis to the game in and effort to decipher the programming logic.  I do not have the time or energy to devote to that.  I just enjoy the game.  That is probably why I only have three titles.  I wish I did have that kind of time.  But I learned one thing early on playing this game after sending a ticket about a series of seemingly illogical losses.  The response I received changed my view of the game: "The system is programed for 25% of the game outcomes to be upsets, the same as the NCAA."  I don't know if that has been tweaked in the last few years, but randomness is programmed in.  So each night you have a 25% chance of being upset.  Whether you suffered an upset the previous two nights or not, you still have a 25% chance of being upset the next night.

Obviously, the better you are compared to your opponent, the less likely that upset will take place, but it leaves luck play a part in this game.  So even though statistics can be very helpful, sometimes it just doesn't matter due to the randomness factor.  It helps me understand why I may have a team I am not confident in do very well, and a team I think should be great, fall flat on their face.  Same with player production.  I'm not saying stats can't help you build a better team.  I believe they can.  But just realize that 25% of the time the stats might not mean anything.

commish, I love ya (esp. b/c you are a fellow X fan)... but I find this impossible to be true.  There are great teams every year that never seem to get upset.  Yes, they may lose some games - but most of these games are reasonable losses IMO (not upsets).  Perhaps my serious math deficiency is missing something here though.

BTW, I find all the comments by the stat guys fascinating and appreciate the insights/food for thought.  I think I have overvalued pass/bh among bigs as an aside, based on others' opinions.

billy_g - do you care to share why you are quitting?  Perhaps just bored at this point?

Brian, that was the response I received.  The majority of the upsets are going to be minor upsets and as the distance in quality gets wider, the less likely the potential for upset.  Proof can more clearly be seen in the NT.  But randomness is definitely built into the program.

That seems less true now than it did a couple years ago. 

i think it's also important to understand how a computer programmer would define an upset... that is something like: "any time a team that would win the game >50% of the time loses the game." so a total of 25% of all 51-49, 52-48, 53-47... 99-1 games are going to be upsets. meaning that most of those will come in the 50-50 to 60-40 range, and fewer as they go out from there. so if you have the clear best team in all the land, you're still going to win an awful lot of games.

though there is still the main caveat from billyg-- don't ever fully trust CS.

Commish, I don't think they are saying there is a high-level code that determines upfront whether a game is going to be an upset or not, or artificially forcing the % of upsets to 25% (and you probably realize that yourself). All they're saying is that there is sufficient variability (or high enough standard deviation) programmed into the individual play-by-play calculations that it will ultimately result in the same 25% of upsets you get in real life. 

And as you suggest above, the vast majority of those upsets are going to involve teams that are 1-5 point underdogs, simply because the point spread of those games is within the programmed variability/standard deviation. It is not saying that if you are a 44-point favorite, you have a 25% chance of losing. You are still going to win that game 98%+ of the time, even if they are trying to match the real-world 25% upset rate.

I'd read the 25% as a target toward which they design the game.  One could target 25% upsets when you design the extent to which there is random variability in each action that takes place in the game

Posted by nc2457829305 on 2/20/2012 10:20:00 AM (view original):
i think it's also important to understand how a computer programmer would define an upset... that is something like: "any time a team that would win the game >50% of the time loses the game." so a total of 25% of all 51-49, 52-48, 53-47... 99-1 games are going to be upsets. meaning that most of those will come in the 50-50 to 60-40 range, and fewer as they go out from there. so if you have the clear best team in all the land, you're still going to win an awful lot of games.

"best team in all the land"
let's not bring Manchester United into this.......

This response was a couple of years ago, maybe even three.  So it could have changed or been tweaked since then.  But the manner of the response seemed to indicate that there was a target here to match results in the NCAA.  It was "25% of the game results each night are upsets, just like the NCAA ". (I'm paraphrasing). Knowing programers, this sounds like a logical target they are trying to achieve and they are happy with the results.