Early on, there will still be a few worthy guys who get snubbed because of this. But I guess, as each world gets older and the original guys are phased out, it won't make much of a difference.
By season 20 of each world, players with prior years of service will cease to exist, and everyone in each world will have had their entire career occur within that world. And even by then, the guys who were very young when the world was created will have logged enough service within that world to make their prior years a moot point.
So I guess we can live with 3 or 4 guys not getting their day in court, in exchange for everything running more smoothly and the ballot not getting too bloated.