not one to nitpick, but if we look at injustices done to both parties throughout history, as well as ways to escape their fate (I.E. tevinter, or dalish), then i think it has to go to the elves.
The mages are taken when they are first discovered to have magic, forced to live in a tower with abusive templars (usually) for the rest of their lives, but if not found they can blend and hide as just the race they were born, or they can escape to tevinter where magic rules. Like the elves, back in the days of the old tevinter empire, magic ruled the known world.
The elves in the cities are constantly discriminated against, basically treated like slaves, and are slaves in tevinter. Those of the dalish live close to the land and must do without the perks of living in a city, but they lack the amount of discrimination the elves in the city get (though they are still looked at like the other elves from outsiders). The elves, over their entire history, have been on a bad luck streak, they once owned a great empire that spanned the entire known world, before the humans appeared and took it over as their own starting the tevinter empire. Then after they helped free the people from the chains of the tevinter empire, they were given their own "country" The Dales. This did not last long as an attack presumably started by the elves prompted an exalted march leading to the destruction of the country, thus scattering them again. Elves have a few choices, live as a servant close to that of a slave in the alienage, live as a slave in tevinter, or join the dalish. The elves who join the circle of magi as mages, might see the life as an upgrade compared to their other options.