On the sly

This diversion was enjoyed on the sly, and unknown to the ladies of the house.

He was working for us on the sly at the Herald-Traveler-Record-American (before it shortened its topheavy name), then as now a reactionary, homophobic rag. Evidently he placed himself at some risk to help us, I learned later, because he was threatened with dismissal for his unauthorized use of the typesetter.