It was probably not a great idea to adjust on a non-immigrant visa, honestly. Why didn’t he just petition you properly for an IR-1, he literally had years to do it?
Also- now you are out of status and no longer in a period of authorized stay, ie subject to removal. Your EAD is also no longer valid.
What does your lawyer recommend?