You're wrong. We live in a representative democracy. In that system we elect MPs and, if they don't do what we want, we elect a different MP at the next election.
There is one exception to that - when the government (in this case supported overwhelmingly by Parliament) decides to use direct democracy in the form of a plebiscite.
When Parliament passed the issue over to us, and the government promised to act on our decision, they ceded their sovereignty to us. Their job was to act on what we decided. They haven't.