@kiddoleknocker get a a lawyer and file. that'll male things much easier for you and the process shouldn't be to complicated as you have status quo on your side. plus make sure to always emphasise how important stability and consistency are for a kid your son's age.
@lee25
do you have a court order? if not I would tell the father that you do not feel comfortable leaving the child in his care under these circumstances. refusing to explain symptoms and ignoring illness are HUGE red flags