Do i move on or stay for the children?

Simple answer - move on, and leave him for the sake of yourself, and your family.

I really think you answered this dilemma yourself, your higher self, your heart speaks true, but your guilt is keeping you back. There should be no guilt, your kids will be okay, i think you feel too much and need to take care of yourself, and know that in the end it is the best answer for your kids to not be with him.
I very much think you should move on to be happy, free, your children will respect you for that decision when they can understand, and just let them know, it is not their fault, that both of you still love them dearly, that will never change. To have the family in a situation like it is now is hurtful to everyone. It will only become worse. You should leave him, and you can work out times where they can go see him. Do not let your worries tie your hands together, they will be okay.
You need to live your life, you do not need him, and your children will be better off without him there in the long run. families operate just fine with one parent out of the picture, and your children can still see him if they want to. you don't want to have them growing up near parent's that don't love each other under the same roof, that atmosphere is stressful and painful, it is better to be truthful about the situation and split. you need to be able to be happy enough to spend time and give your all to your children and yourself and your life, instead of the stress of him ruining it all.
So i think the best idea here is for you to move on instead of staying and suffering, and the problem just getting worse. Your family will be okay if you split. You will be a lot more happy in life being single, or maybe even meet your real soul mate, the positive possibilities are endless.
There are so many pros that way. And only cons the other way.