I just did this.
The trick is to create a Theme League, which lets you select the number of teams that will be in it.
Do you have a set number of owners locked in? You need a minimum of 12 (even number).
Here are the different formats:
https://help.whatifsports.com/hc/en-us/articles/360014305331-What-are-the-different-league-formats-
The trick is for the commissioner to create their team first; then create the theme league second; the invite their friends 3rd (so friends should NOT create teams/join simleague in anyway until the commish has created the league, which requires the commish to first create their team). Its screwy the way this works, but if you try to do it any in any other order, it won't work.
If you have a set number of teams that matches one of those formats, then you can build a "truly private" league.
We did this with 12 of us so we didn't get converted to an open league since we didn't have 24 owners.
Let me know if you any other questions, more than happy to help.
Roger