In my opinion, it boils down to separate areas:
AFK at the start of the game
Putting everyone that is AFK in a third invisible team would already be a big improvement.
Wrong team allocation
Sometimes people are moved to the wrong team that has a lower amount of players. Maybe this happens due to score difference, but IMHO this should only happen after a significant score difference. Maybe the game could track the score per minute of each player internally and use this as a ranking instead of the total score. Especially in long games there can be a huge difference.
Use the average of the VP of both Teams as starting VP
In long games, players that join the worse team, get even more penalized by getting less starting VP. (as they get the average VP of their team) In the long run, this also makes it harder for a team that is far behind in VP to catch up and make a comeback.
In my opinion, the average VP of both teams should be used as the base for the calculation for the VP of a new joining player.