Certainly in many rounds (as is life), many matches are determined by the actions of a few people. (base sneaks, snipers etc)
I think if people choose to become invested in the team they are on (their teammates that round, the faction etc) in that moment and even fight to the bitter end then the desire or ability to switch sides becomes moot. Much like nationality.
Is that overdramatizing it all? Yes. But I think it's hellva lot more fun to become emotionally invested in matches =D
The overarching point I'm making. Give players less reason to switch and the problem solves itself