I'm not a native English speaker, but to me "by" = "before". And "before" is not "on" the day. I guess wording could have been better.
The best shot OP has is to mail everything today with post mark showing today's date. But again, wording in USCIS letter says they should receive it in the office by June 4th. So it's already late. It's unlikely petition is going to be revived. Motion to reconsider is also pointless. New N-400 petition is likely needed.