I suspect that all four races of Inquisitor are going to frighten the Chantry by being called the Herald of Andraste.
Short answer: True, but some races might frighten them more than others.
Long answer: Dalish elves and Andrastian humans particularly have been diametrically opposed for centuries, ever since the Dales resisted being converted to the worship of Andraste, and the Dalish chose the wilderness over conversion.
I think there are some socio-economic reasons too. Some have speculated that the humans felt threatened by elves having their own sovereign nation and religion apart from them, and so tried to exert their will over them while they had the Dales (trying to convert and "trade"), then demonized, conquered and subjugated them when the elves refused to bow to their will.
Now most elves live in human cities, practicing the humans' religion, dependent on humans socially, economically, and religiously. Socially since they're a small minority, economically because they rely on humans for whatever scut job they can get (thus assuring that humans have a never-ending source of cheap labor and social inferiors), and religiously since most city elves worship at the Chantry while humans control the Chantry, and Chantry teachings.
The Chantry has only dug itself deeper by spreading anti-elf propoganda over the centuries. They've scratched out accounts that relay positive elven involvement in human history, perpetuated accounts that portray the elves negatively (like the elves' supposed human kidnappings, blood sacrifices and war crimes before the Fall of the Dales), declared the Canticles of Shartan to be heretic, and have basically condemned all elves that don't bow to human will as heathens and savages.
Now suddenly one of the Dalish they've been demonizing for centuries (to keep the city elves marginalized) has become the most important and influential living person of their Andrastian religion. What's more, the Dalish elves (and elves in general) have every reason to be angry with humans over how they've been treated over the centuries, and a Dalish elf who is seen as the closest to their Lady can use the position to undermine Chantry authority and/or sway the people against them.