People who say that are generally talking about the signup where you have to pick an instance. And then there’s the worry over which other servers yours federates with. If you isolate your attention to a single instance, then all those worries go away.

The same already happens on the fediverse in regards to mastodon itself. A lot of people discuss the fediverse almost wholly in terms of mastodon.