@darthklim I do think your anger and lack of patience is part of the problem. It seems like you resent your wife and son tbh. If you are losing your patience with him then of course he is going to want the nurturing mom over angry Dad.
Your son needs to feel safe and loved when you are tucking...