Things can easily get misconstrued.

I remember going to Gillingham years ago. They had an ex-Jack (name escapes me) playing for them, who also happened to be black.
He had possession of the ball close to the corner flag and the end where the City fans were housed.
The inevitable... "You Jack bastard" was directed at him. The Gillingham fans to our left were livid. No prizes for guessing what they 'thought' we were chanting.