It is not immoral to have sex with one’s own wife or husband. However, it is not wholesome kamma either. Renunciation of sexual pleasures is wholesome kamma, and chastity is essential for those intent on gaining realisation of the Dhamma. “For as long as the slightest brushwood (of the passions) of man towards women is not cut down, so long is his mind in bondage, like the milch calf to its mother-cow.” (Dhp v.284) A lay person can enjoy sex from time to time, but it will inevitably lead to attachment, grief, and despair in the long term. It is therefore wise to treat it with respect, as one treats a fire in one’s own home.
A devout lay person should observe chastity on the Uposatha days of the full-moon and new-moon. That is the traditional practice. In Wat Pah Nanachat, when I was there, some of the villagers would spend the entire night in the monastery, meditating with the monks. That is the orthodox and traditional way to properly observe the Uposatha — not as is now often the case, taking five precepts in the early evening, and going home to eat an evening meal, and sleep with one's wife/husband as usual. That is called observing the cowherd's uposatha.
Please don't worry about not being able to enjoy sexual relations if you meditate too much. You're very unlikely to become a Non-returner (anāgāmī) after just one-night in the monastery.
In the Noble Eightfold Path, Right Thought includes renunciation of sensual pleasures (nekkhamma sankappa), so renunciation of sensual pleasures is an essential factor of the eightfold noble path.
• • • • (Upasampadā: 24th June, 1979)