It’s because the number of players on the server isn’t actively updated for you until you hit refresh. It’s annoying as well because even when a server says it’s full for you it could actually have less people but you still won’t be able to join it from the server browser. Usually requires me restarting steam a few times to get a decent ping server as when I try to refresh 2-3 times the server list disappears completely.
Hopefully this gets polished in the future.