Yes there is evidence. The reapers only ever operated in the Milky way. They only ever left the galaxy to "hide" between cycles. We learn of their history and the cycles in the first series. The reapers are a solution to a problem for the milky way, there is zero mention of any reaper being sent out to other galaxies why? because their mandate isn't to solve the organic/AI problem everywhere in the universe. The reapers are machines they do what they are programmed to do. There isn't a shred of evidence of them having any desire to spread their version of a solution to any other galaxy.
So don't need to provide any hand waving for the reapers not to be in Andromeda because there was no evidence they ever went to or wanted to go to Andromeda in any of the three previous games.
That's only evidence of their activities, but where is the evidence of their reasoning? They only operate in this galaxy, therefore, they are only programmed to operate in this galaxy" is not evidence.
And there is evidence from the Leviathans themselves that they are free to operate in other galaxies: The intelligence has one purpose: preservation of life. (direct quote from Leviathan)
Nowhere is there a hint that being in this galaxy is anything other than a physical limitation.
Faulty reasoning.
I could be eaten by a tiger in Western Canada. It is possible but not rational to believe it is a threat. And very irrational to devote any time or resources to protecting myself from being eaten by a tiger. So yes there could be a threat to organic life from Andromeda but it is HIGHLY unlikely so it is actually IRRATIONAL for the reapers to devote and resources on this possibility with no evidence of a threat. A bunch or milky way refugees in Andromeda does not a threat make so no need to deal with it.
Unless you believe that sooner or later, tigers will evolve and devour people in Western Canada. The Reapers operate along those lines of thinking. Heck, they practically invented faulty reasoning.
"Without us to stop it, synthetics would destroy all organics"
Synthetics could be created by organics in Andromeda. Therefore, synthetics could rise up, exterminate organics there, then spread to other galaxies.
Therefore, if getting to Andromeda is at all feasible, they need to go there ad harvest any organics they find before they can create synthetics that would...you get the idea.
Thus, if the Reapers didn't go to Andromeda, it could only be because the gulf of dark space is insurmountable. At least, insurmountable to Reaper tech.