In 1976, cable TV was still in it's infancy. There was no satellite TV (I don't think), there was no internet, there was no MLB Network, there was no SportsCenter, etc.
To see the great players of the day, you got two nationally televised baseball games every week ("Game of the Week" on NBC on Saturday afternoon, and "Monday Night Baseball" on ABC), and often they were the popular teams of the day (I remember Reds/Dodgers being a common matchup whenever they played). Some players in some markets were difficult to ever have an opportunity to watch. It was always a treat to see Nolan Ryan pitching for the Angels on Monday night, maybe once every other season.
The All-Star game was typically the one opportunity every season one had to see all the MLB stars of the day. So that helped with high ratings.
Today the landscape is much different. No matter where you live, you have opportunities (should you be willing to pay) to watch any game, any player, at any time. The All-Star game is no longer special, it's no longer a "must-see". It's not because of the game itself, it's just because of the availability of season-long exposure to all of MLB.