You would also need the expired card. The letter is worthless alone. This was a mistake by the interview officer. Hopefully, you will have your oath soon. You probably won't need the card, but you currently have no physical evidence of status.
Just tell the representative, at the oath, that the officer took your card. There will be no issue.