You're confusing the office of Prime Ninister with the leader of a party. The two are not the same.
We've had situations in the past where the leader of the party isn't PM. I think we are about to see this again: Truss remains leader of the party but Hunt has the support of the majority of MPs