Science can't answer that question just yet. In fact, science can't say much about consciousness, its origin and fate, without helping itself with metaphysical speculation, even if sometimes that sort of assumptions pass to the general public as scientific facts. There are some interesting studies about it, but then you find different interpretations of the results. While some say that there is strong evidence in favor of a theory supporting reincarnation (usually people outside Buddhism don't make the difference between rebirth and reincarnation), others will maintain that such is not the case. The reasons for such scenario are vast and trying to understand both positions implies spending some dozens of hours researching and reading about philosophy, epistemology of science, history of science, sociology etc. Then it is possible to have some grasp about what's really going on. In the end, what you will have are opinions covering a large spectrum regarding this matter. You may prefer some over others, but in the end you will end with your own
opinion or not being able to form one.
If I were to tell you that I knew for a fact that rebirth was a fact, talking about memories of past lives, you couldn't be sure if I was talking about cold hard facts or my beliefs mixed with interpretations. So, again you would be stuck in the testimony of a 3rd party. It won't do much for you in terms of certainty.
Regarding your observations about nature, there are many cases of people who seem to have unexplained memories and feelings concerning what some assume to be past lives. Thousands of cases were already reported. Some are explainable, some hardly. You never seen an electron, a molecule or a supernova. Yet you trust experts word on this. You can try to figure out who are those you can consider experts about this subject of rebirth. Scientists? Not for me. Some Buddhist teachers? It's for you to decide.
In the end, it boils down to your own experience, since it seems that this phenomena is quite hidden and not easy to verify. Deepen your practice to a point in which you know for yourself if rebirth is a fact. When you discover, you may say to others that you know for a fact that rebirth theory can be confirmed, but then others will have nothing but your word to rely upon. If they wish to know for themselves, they need to follow the procedure and train their mind as you did.
By the way, I am thinking about an apple right now. There's no science on Earth that can prove this to be true. This doesn't mean I can't think about an apple right now, though. I know I am. Is it possible for you to think about an apple right now? Try it for yourself and tell me. However, to be sure that you indeed can think about an apple at this very moment, I would have to read your mind. Not even a polygraph could guarantee that you were speaking the truth, as we have nothing able to measure or detect mental phenomena directly (it's useful to avoid mistaking them for their neural correlates).
So, there you have it. IMO, the only way out in the current state of affairs is for you to try it for yourself. It will be a hell of a quest as it seems training your mind to such a point takes years of dedicated practice, thousands or tens of thousands hours of meditation and perhaps some more hundreds of hours studying. In the end you will know, but it's a knowledge obtained in a way that can't be proven to a 3rd person. Gladly, knowing for a fact that rebirth is true or not seems to bring many other positive side effects to our mind.

So it may worth your while.