I think the culture will make it tougher than people like to believe. How it works out will depend a lot on the demeanor of Sam. If it's Jonathan Martin-like, it's going to be a huge problem. If he basically ignores the slurs(which I'm sure he's already done) that he will hear, it will just become a matter of the players accepting him as a teammate. And that won't be a problem if he plays well. As far as the guy who worries that he might be checking out his weiner, that dude is already worried about players checking out his weiner. Can't fix him.
Think back to Riley Cooper. "OMG. He dropped a n-bomb and 70% of the guys he'll be playing with won't forget that. He's going to have tough year!!!" He played well and it became a non-story. Maybe RC isn't going out for drinks with everyone but there was no problem after the one flare-up during training camp. Sam may not have a bunch of dinner parties for teammates but no one will be going out of their way to give him grief if he can help his team win.