I think the biggest difficulty in lay life (as compared to monastic) is not all the daily challenges, but rather the absence of a supportive Sangha 24/7. This is what I really envy monastics for. Yes, practice never stops, if you really know how to practice, to pay attention. But for a layperson whose practice has not reached that level of insight and stability, it's more like a whirlwind of happenings with habitual responses and some self-control and occasional clarity and insight into the situation.
I used to have a friend who lived right next to the Uni when I was studying and I dropped in regularly for a sit (40-50mins) and discuss the dhamma/dharma (she is Theravadin, I am Zen) afterwards. That was great (it did do my knee in eventually, a small price to pay
) These kalyana mitras are so invaluable for a layperson like me.
So in answer to the original question:
Challenges to Buddhism:
1. Materialistic Culture translating to either disinterest and cynical approach to spirituality or acquisition mentality in relation to things spiritual and greed.
2. Addiction to information, external stimuli and thinking.
3. Big egos stroked from early age - "you are so special" mentality.
1. Virtual absence of a supportive Sangha.
2. Laziness and force of habit.
3. Comfort-seeking rather than truth-seeking.