As others have mentioned, USCIS usually conducts home visits when they suspect fraud. The fact that you got approved for your conditional GC suggests the evidence you guys submitted was strong enough to convince them of the legitimacy of your marriage. When it comes to your removal of conditions, the chances of getting an interview seem higher if you weren't interviewed for your AOS so I would expect to be called in for an interview. Note, this is not from first-hand experience, just from what I gathered in this forum.
Everyone's living situation is different; if you and your spouse are in a legitimate marriage (it seems you have solid evidence to back that up, too), and you choose to sleep in separate bedrooms for reasons that are nobody's business but your own (well, and USCIS', should they ask you during the interview) then you have nothing to worry about imo!