He would need to be in the US to join your bank account. My husband did so at our bank before having a SSN. I added my husband as a user in my credit card -again it depends on the credit card. We made Wills, Power of Attorney, got life insurance, made each other beneficiaries on each others 401k/pensions. There are lots of things to do—pretty much normal husband/wife stuff you would do regardless.