Not every officer likes kids. Small kids especially can cry or otherwise distract the parents and or adjudicator. In this case, officer asks one parent to step outside with kid while asking the other questions. Then parents switch. Then answers compared. Essentially, it becomes a stokes interview when otherwise it wouldn't. I'm quoting few lawyers who go to these interviews, it's from their experience. None of the times I've been to USCIS office I've seen kids around.
The wait for interview can be long. The interview itself can last from 15 minutes to an hour or more depending on officer. If you get same day oath, you may need to wait for it in the building. This happened to me, about 1.5 hrs after interview until I got called for ceremony. A small kid may find this place extremely boring.
I'm sure some VJ users brought kids and had positive experience, I'm just sharing some possible difficulties with this approach.