The problem isn't being deserted per se but rather the sheer size of the Galaxy and the limits of conventional FTL travel.
There are an estimated 500 billion to 1 trillion stars on Andromeda (depends on the source). Let that number sink in for awhile.
Hey, you're the one who said "deserted." I figured you actually meant deserted.
So now it's size? What's wrong with the size of Andromeda that isn't wrong with staying in the Milky Way? Lots of stars here, too. The uncharted areas of the Milky Way aren't any better than Andromeda in this regard. The only difference is that there will be Reapers here. The charted areas would make it easier, but that just means the Reapers catch you right away.
You talk about colonizing garden worlds.
My point is, the expedition could comb those stars a thousand years and not find a single garden world or come across any intelligent lifeforms.
What's your evidence that the ratio of garden worlds to stars is anything like that low? The ME galaxy maps don't support such a conclusion. And again, moving to Andromeda won't make this problem worse, and the size of Andromeda can't hurt either.
There must be a reason for their destination and it better not be "let's just travel 2,5M lightyears and start combing the star systems from that corner there because Reapers and Shepard."
Why is "no Reapers" not a valid reason?
TL;DR: what problem does staying in the Milky Way solve, and how would you deal with the Reapers if you did stay?