The six foot two John Charles is the reason I said six foot three !
All of the truly great players I've seen in my life time were all of a certain height except for Ronaldo (I saw John Charles play, but he was a long way past his best when I did so, therefore I wouldn't place him in that very high level - my old man was adamant that Charles was the best player he saw mind).
I had the same experience as you when I was in primary school in that I was generally considered to be a better footballer than the bigger lads who were picked in front of me by the rugby loving teacher in charge of the school team. As I grew up and got to know more about the two sports, I consoled myself with the thought that rugby was one of those size obsessed sports where brute force and ignorance can count for more than talent in a way that just doesn't happen in football - I think that still holds true, but I'm not quite as sure about it as I used to be.