He still sounds like a dick if he hasn't tried to talk to you about it. That said, I'm giving you the benefit of the doubt that you have tried to talk to him about his sandy vagina.
Im not saying its anyone's actual fault, but sounds like this is going nowhere fast. You sound like you only want him for temporary security, which he isn't giving you anyway. If his paternal instincts aren't strong enough to want to do the right thing by his child (irrespective of what happened/s between you two), I say you're better off without him.
All this is based on you being honest in your post, but there's 2 sides to every story. I have a feeling there is a lot you haven't told us (and we don't need to know), and if so, maybe you just have to ride it out if you want him in your kids life. If you only want him as someone to support you, fuck that shit, it's going to end messy for all, especially the kids.