I do worry about our players confidence

Much more of this or a loss against burnley and we could go into free fall.
These players then won't help us next year it will need to be a complete rebuild.

I think this is the danger. It was always unlikely we would pick up points against Man City, Chelsea, etc and we don't really need to. It's the rest of the division where we could and need to get some wins. However, the freak nature of our early season fixtures might mean that by the time we start getting more games where we can compete, our confidence will be shot. I guess the Burnley game might give us some idea.