My SO was at work and I took a couple hours off and went to my interview alone. We went together to my AOS interview, ROC was approved without an interview. I don't even think the officer asked where my spouse was during the N400 interview.
We sent in a bunch of documents shared accounts, insurance, loan etc.
I think it is totally up to you to make the decision and in case they were looking for him, then it is on you. We can't make this decision for you, you have to decide and take responsibility. Worst case you could explain and bring proof that the two of you are in vacation and you flew back to do the interview (boarding passes, itinerary etc)
I don't remember the interview letter stating that i need to bring my SO