In general I don’t think difficulty of competition will be an issue except for you and a few others who almost exclusively play the WISC.
The way I plan to get around this is to weight the impact of these predictions against number of teams owned (I.e. more teams = full impact, fewer teams = less impact)