It's a little bit odd that ain't it?
My mother was quite upset when I told her I was a non-believer, and in the end I had to tell a white lie when she asked me if I believe we will "all be together when we die".
Which, in my opinion, points to why religion exists. A lot of people struggle to come to terms with their own mortality, that it doesn't last for ever. I struggled with it for a long time as I tried to shake off the chains of indoctrination.
My mother was also fearful that, as a non believer, I would somehow lose any moral compass I had. I merely pointed her to the stories of catholic priests who abused young children and she said "yes, but they are the priests that don't believe in God!".
I didn't come from a particularly religious background, but I remember being petrified that I was going to burn in hell for the multitude of sins that I had unwittingly committed as a teenage boy. It's in such an environment that these "men of God" are able to commit such heinous crimes.