The Dalish speak some elvish, as we hear the Dalish use a number of elvish terms along with the Common Tongue, and Merrill even refers to a romanced Hawke with an elvish term of endearment.
As for Finn, you're conflating his familiarity with a single word with the entirety of the language, and that's only possible because the Circle took a number of elven artifacts and tomes after the fall of the Dales.
I can swear in Spanish, German, Klingon, Dwarvish, Elvish, Gnomish, and Elvish. That doesn't make me polylingual. The elves use the few words they know, but they couldn't hold a prolonged discussion entirely in Elvish. The mages know more because they have access to older books, which can even predate the formation, let alone the fall, of the Dales.