Firstly to clarify atheism its a lack of a belief in a god, you can believe in ghosts fairies, and in invisible pink unicorns, and still be an atheist, you can believe in rebirth and still be atheist. I'm not sure where brahmas and devas fall on the deity thing.
Anyway personally I am a sceptic its not that I don't want to believe these things, its that I can't. I tried and can't, the only reality I could believe in is naturalistic pantheism, in the words of Ricky gervais I can't believe in what I don't.
I left Buddhism a while ago but was browsing ATI and the good ol kalama sutta for some discussion, anyway I found this section as part of it. I later found out that it was called the four solaces.
17. "The disciple of the Noble Ones, Kalamas, who has such a hate-free mind, such a malice-free mind, such an undefiled mind, and such a purified mind, is one by whom four solaces are found here and now.
"'Suppose there is a hereafter and there is a fruit, result, of deeds done well or ill. Then it is possible that at the dissolution of the body after death, I shall arise in the heavenly world, which is possessed of the state of bliss.' This is the first solace found by him.
"'Suppose there is no hereafter and there is no fruit, no result, of deeds done well or ill. Yet in this world, here and now, free from hatred, free from malice, safe and sound, and happy, I keep myself.' This is the second solace found by him.
"'Suppose evil (results) befall an evil-doer. I, however, think of doing evil to no one. Then, how can ill (results) affect me who do no evil deed?' This is the third solace found by him.
"'Suppose evil (results) do not befall an evil-doer. Then I see myself purified in any case.' This is the fourth solace found by him.
"The disciple of the Noble Ones, Kalamas, who has such a hate-free mind, such a malice-free mind, such an undefiled mind, and such a purified mind, is one by whom, here and now, these four solaces are found."
So the second solace felt like the Buddha was talking to me, cos practicing Buddhism does improve me as a person. I wrote a whole essay as a student nurse about how samadhi has scientific studies backing it up too and increases the size of the amygdala.
So yes I feel as far as the Buddha was concerned you can follow the dhamma and still be a sceptic. However I know he taught literal continuation/rebirth, I don't agree with bachelor on that. I'm even in the process of finding a group in my area.
As sceptical mind as you will find.