@apostoliclife I can't tell you for sure what to do as it's a really tough situation, but I can tell you my very similar circumstances and how it is affecting my daughter right now.
I (M40) got custody of my daughter when she was 3 years old because her mum was off the rails. Always on drugs...