I doubt it was banter but tbh it is highly unlikely anyone would vote for something just because their boss told them to.
I am guessing one of the things he would not 'rethink if Labour got in' would be the obscene 'dividend' him and his wife take out of the company.