Even though EU citizens had to show their passports they were allowed in under the free movement rules.
Yes, you are right that non-EU migration was always controlled by the UK, after all Priti Patel and her predecessors were for ever telling us they were going to reduce immigration, implying it was in their control. They just never did and then started peddling lies about the benefits Brexit