So at my oath ceremony in Minneapolis, SSA officials were actually at the ceremony, and they had us update our records during the ceremony online and then turn the paper forms in. So I actually updated mine on the day of my oath ceremony (along with a significant number of the other ~1000ish people there), and got my updated, restriction free SS card in the mail about two weeks later.
YMMV obviously, but I'd probably just go.