There is a big difference between inconvenient and extreme hardship. My mother immigrated from Germany and would travel with me and my brother back to Germany to see her family. First trip I was 2 and my brother 4. She managed to make the trip there and back by plane with us and no one to help her. Yes it was not as easy as flying by herself but she managed just fine. Military spouses do it all the time, I went from Fort Benning in Georgia to Frankfort Germany with a two year old, and three cats. You have many options and that is how USCIS will look at it. You can travel with the children, you can find someone to care for the children and travel alone, he can come to the US to visit, you can meet in a different country with or without the children. But saying you cannot travel because you have young children is not an extreme hardship.