Note that there are other possible answers to the Fermi paradox. In Peter Hamilton's Commonwealth books, sufficiently advanced aliens typically develop to a point where the physical universe isn't of very great concern to them. The central plot of Pandora's Star/Judas Unchained is humans having to clean up a mess that a previous race thought had been permanently solved, but had not. That previous race still exists, but one faction has deliberately regressed to a pretechnological stage, while the other is so advanced that humans can't even communicate with them. The plot of those books actually maps fairly well onto ME1, come to think of it.
See also 2001, although that's a different model since those hyper-advanced aliens are still around and very much in charge.
Actually, I made another post about exactly this in which I thought that they should take a different approach with the Remnant - namely, an advanced species that attained transcendence, and benevolently guided other advanced species to do the same. This would be different, indeed opposite to the Reapers, and would neatly account for the Fermi paradox. So in that sense I agree with you.
Another poster raised extinction level events as an answer as well - the thing is, this is a common objection but it isn't a very good one. Could the vast majority of intelligent species nuke themselves to extinction before spreading out of their solar system?
Sure. Absolutely. I mean, lets be honest - we will probably do exactly that to ourselves. If not, we will succumb to overpopulation, pandemics, environmental disasters and resource wars without a nuclear holocaust just as easily.
But, what is most impressive about the Fermi Paradox is that it only takes ONE civilization to not do that. Given the size and age of a galaxy, do you know how long it would take a single civilization, lets say the human species, to colonize every single feasible star system in the galaxy if they do it at a constant rate and sublight velocities?
About 50-100 million years. That's it. Earth is 4 billion years old and it is a young planet compared to many others, especially those around red dwarfs. If other intelligent life is common, the galaxy should be covered multiple times over by now.
That's the impressiveness of the paradox by anyone that understands the math and scales involved. It makes assumptions, yes, but pretty reasonable ones in that it is extremely anti-success for intelligent life and advanced civilizations. But it just takes one success story.
Mass Effect addresses this beautifully, I might add. The cycles explain the paradox, and every species in a given cycle flourishes whereas many probably would have perished of their own devices (like the Krogan or Drell) due to the easy access of the relay network and FTL travel.
So, if Andromeda has no relay network, more species would certainly go extinct prior to colonizing any other star systems. The majority probably would. But given Andromeda's age, it should be covered by at least one advanced civilization by now. Hopefully Bioware puts some intelligent thought into that, and it at least seems like they are attempting to with the Remnant.