Just because the child is living with him does not mean he has legal custody. That is set in court by a judge and you agree to either sole or joint custody and child support payments.
Without proof of custody and child support payments, N-400 will be denied based on lack of moral character.