I always get a size or two above what I would normally get as I rarely wear it on its own.
If I go to a game in winter, like most are, then I'd have several layers underneath.