Sorry to hear of your pain.
There could be various reasons why he is pushing you away. One could be he is experiencing a heavy period of the Dark Night Of The Soul in which layers of his old self (ego) is being shed so he is not thinking in a rational way. This has happened to me before and I backed off from contact with a person I was close to as I honestly felt insane.
Or, it could be he is coming from a place of fear, which again is the ego. Fear in that his ego is deeming him unworthy of happiness and love. This is quite common and a conditioned belief he may have been holding on to throughout all his life.
It has also happened to me before in the past too so I say it from first hand experience. At the time I was fearing rejection and abandonment from someone, and he may be fearing similar with you. He is subconsciously almost waiting for things to go wrong as this is his conditioned belief. He may or may not have meant his sweet words, but try not to keep going over that as it will serve no benefit, only add to your lack of self worth.
But, him suddenly flipping sounds as if he's fearing rejection or loss from you, so he's getting in there first. It is a very common sign, although I might be wrong.
In terms of you; well you sound as though you need some healing. I read a post yesterday on here by someone - can't remember who - may have been FairyCrystal or Inika, who said you will find it hard to heal AND be in a relationship where you can be completely you.
Also, remove the thought of him possibly meeting someone 'better' than you. There is nobody 'better' than anyone. Again, this is lack of self love - you're punishing yourself and you don't have any truth or fact in this at all.
Positive affirmations may help you here. Go within and start to heal and sit down and talk things through with your partner if he is wanting to do that, but ultimately it might be a sign that you need to focus on yourself right now.
We don't attract what we want, we attract what we are - I am a firm believer of that.