@happyhippopot Hey,
From the sounds of it, you're already thinking in the right direction, but let's be honest, that doesn't make it easy. I went through a similar situation where I became a single parent of 3, and felt the same kinds of things. I won't lie, there were times it really sucked, but the older I get, the better it gets, and the times along the way were rewarding. You're right, I wouldn't say I've lived my life to the fullest. But I've lived it the best I can for them, and as they get older, every day, every week, every vacation, I get to spend a little more of my energy on me instead of them.
I'll say, the small stuff isn't worth sweating, as difficult as it may seem at the time.
And please, most importantly, forgive for whatever guilt you're holding against yourself. There's absolutely nothing positive that can come from that. It will only continue to hurt you, and through that, impact your children. Hanging onto that will consume part of you that you should be using on you and your children, so by not forgiving, you're doing you and your kids a disservice (in my internet-stranger opinion).
One small plus-side of being a single parent I discovered: discipline and raising the kids became easier, in that there have never been conflicting opinions on parenting. There's no one else for them to run to, no one to leave them confused as to what the actual rules are. It's your way of the highway, and I honestly believe that's a good thing.