I doubt if the resistance to ritual is because it attacks the ego. (If you wanted that, you should stick to your original religion, not go shopping for Buddhist practices.) I suspect it is because ritual tends to be boring and meaningless.
But we want things to be not-boring
, and "meaningful" because that fits within the range of our happy experience, and that is what we cling to.
Take sitting meditation for example.
It is boring, and essentially meaningless
because as soon as you impute meaning to it,
rather than just watching your breath, or allowing the mind to rest in its natural state,
as soon as it starts to be something
it becomes something to cling to.
That is why, I think, for example,
one is generally advised not to grasp on to signs of 'progress' along the path,
but to let it go and just keep practicing.
Buddhist practice continuously pushes one out of one's comfort zone
because the more you move out of your comfort zone,
the bigger it becomes
until at last there is no dualism, no distinction,
because there is no "me" operating.
I think that the comforts of ritual, that you mention,
do bring people together if that is what they are into,
but if it isn't then it is a turn-off.
But then, 'ritual' is a pretty big word. it can mean just about anything.