I don't think the majority of people with issues with migration are actually doing it for racial issues but instead economic and keeping their world the same.
They'd be the same if there was a threat of working class white people were moving near.
But you're right it is mean minded.