More like the world hates them, because they keep giving the world reasons to hate them.
Sitting out a blight? it didn't make them popular, The Dalish harassing, killing and interrupting trade? Doesn't make them popular.
And this is just the stuff the world knows about, how do you think people would feel if they found out elves cursing humans to be werewolves, or summoning an elder demon into the world?
It's a vicious cycle. They're mistreated, so they lash back, and then are seen in a worse light.
Also, I'm purely talking about city elves. As is briala. The Dalish are annoying and aren't doing anything for the elves. They could help their fellows in the cities, but they choose not to.
Briala is starting a city elf rebellion. She will accept Dalish aid but she doesn't expect it. The city elves have it the worst.