They do give you a folder with bunch of stuff before ceremony. You can place certificate in there.
I have not tried getting authenticated copy. My goal was to leave USCIS building as soon as I could
Yes, my certificate was sent away with passport application. I took a digital scanner copy and few photos of it just in case. Yes, it was stapled and folded.
Yes, it actually arrived after passport, and unlike passport, it wasn't a tracked mail.
If you have history of mailman losing your mail, if you live in apartment building with known issues etc, then yes, maybe applying for urgent passport can eliminate the possibility of USPS losing the certificate. But in my case, I live in my own place and it's pretty reliable. There were no issues. Once I got certificate back I stored it in folder.