Your case is interesting to me in that you have prior family history of spiritual healing. Years ago spiritual healing was very important. Some people are able to absorb the spiritual energy from the spiritual dimension at the center of our Earth. This energy is contained within our souls. When the touch somebody there is a flow of spiritual energy from their soul to the other persons soul. This is explained in my latest book "Cosmic Reincarnation and the Dual Light-speed Universe".
Many people have new souls. They are not reincarnates. You are a reincarnate and have lived many times before. This gives you a degree of healing ability. It helps you to heal but to a lesser extent it permits you to touch another person and give them some of your spiritual energy.
Jesus was a spiritual healer. He could touch people and his excessive spiritual energy would flow into them. Nowadays fewer and fewer people are spiritual healers since medicine has advanced so much.
So you meet people who have new souls. Their spiritual energy is very low and they drain you. I cannot advise you what to do. I just want you to understand that you are spiritually superior and every man who you meet most likely will be spiritually inferior. Opposites often make good love mates.

just got me thinking you know I used to be so harsh; younger me was SO defensive (i'd been hurt--hadnt we all). Understanding and adapting to the fact of their young souls was impossible for me it seemed, I was too immature for the job after all, I was no better. There is a passage Christians know, Without Love, you are nothing. I am not preaching it because I was very far from living up to it, practicing it, but I am definitely agreeing with it and wish I'd lived by it more consistently.