I guess that when the tournament first began in the 50's, there were hardly any participating players playing in European leagues, so teams weren't affected. Similarly, I think only 1 African team qualified for World Cup finals, so this tournament didn't affect much else.
Now that this has changed, it would make more sense to move the tournament and to only hold it every 4 years, instead of the current 2.