But... being afraid of Templars has little to do with embracing magic 
I think there might have been a misunderstanding.
I thought you were saying the Dalish kick extra mages out of their clan because they (the Dalish) are afraid of mages.
That goes against all established lore.
We do know that the Dalish are not single people. We know Minaeve and Dalish from Bull's Chargers were chased away. We know that Lavellan and Sabrae exchange their mages in Arlathven and that basically each clan is different to a point that it's becoming hard to call the Dalish as one people. It's been established in the lore that they all have different practices.
Heck, I laughed out loud when I've read some time ago on wiki that "the intense quarreling that occurs in these conferences [Arlathven] leads many to suspect the Dalish prefer their tribal isolation due to irreconcilable differences." I mean, I wouldn't be surprised that this is exactly what happens

I wouldn't particularly mind this "3 Mage Max" rule if they presented it as something that only some Dalish Clans do.
Instead, DAI presents it like all Dalish, as a collective culture, do this. All Dalish Clans have a strict three mage maximum, and while some are nicer about it than others (some leave their kids out to die, while others hold onto them as long as they can until they can find another Clan with less than three mages to pawn them off to), it's still universal. When Minaeve and Vivienne bring it up to Lavellan, there's no choice for you to say, "There's a three mage rule? I've never heard of this," or "Some Clans do that, not ours."
NO, you always have to agree that that is something the Dalish do; that this is the rule for your Clan too. You can only try to soften the blow by saying, "My Clan would never do that [leave an extra mage child to die], we'd try to find another Clan that needs an extra mage," but you can't deny that the extra mage child couldn't stay with your clan if you already have three. I think Minaeve, but I know Vivienne even poses the question, "And what would happen if you couldn't find an extra clan to take in the extra mage child?" and Lavellan can't give a good answer.
If it was presented as something that only some clans did, I could live with it. But its' not, it's presented as a "Yes, All Dalish" thing.
And this goes against lore from previous games. DAO reveals that while only mages can be a Keeper and Keeper's First and Second, it doesn't mention any upper limit of mages. Lanaya mentioned how she had to compete for several mages to get her title, and they took in the apostate Ainarin even though they already had plenty of mages. In DA2, Merrill mentioned how all clans need at least two mages, preferably three, to become Keeper and Keeper's First, and Clan Sabrae happened to have a shortage of mages while her own Clan had a surplus, so they pulled her out due to the shortage, not the surplus. She even laments that fewer Dalish children are being born mages, and she considers it sad because she feels it's a sign of their dying heritage. In DAI, suddenly the Dalish are popping out mage babies right and left and all of them have always had this "three mage maximum" upper limit, and they all try to dump spare mage children, and I'm thinking... something that would get me a warning point for sharing.
I just don't buy it.
Am I the only one who notice that the structure of Dalish society is very much reminiscent of the oppressive Ancient Elvhenan society ?
In Ancient Elvhenan, your place in society is determined by your magical abilities and magical prowess. In many ways, it is very similar to Tevinter Imperium.
We see this in Dalish Elves where only the mages are allowed to rule the Dalish clans and non-mages are relegated to other jobs. The fact that they shuffle mages amongst the Dalish to give mages to clans that have very little or no mages show that the Dalish, like their ancient counterparts, place a premium on mages over non-mages.
Does Solas view mages and non-mages differently ? I mean I don't think so, I have not seen any evidence of this, but he doesn't right. Even Flemythal doesn't seem to care about placing a premium on mages.
To be honest, I think that's a vast oversimplification.
That's like saying smaller war clans like the Avvar and Chasind are just as oppressive as "civilized" medieval nations just because the strongest warlords are the leaders, just like how the royals, nobles, and knights are war leaders with their own private armies to oppress the "little people" in sedentary nations like Ferelden and Orlais.
I'm serious. Medieval society was very loosely divided into three sections: those who fight (nobles, knights, and soldiers), those who pray (the clergy), and those those who work (peasants, serfs, laborers). The "ideal" kings and nobles in medieval society were accomplished warlords, or descended from accomplished warlords. They got their wealth from their land, made "peasants" work the land and give them the lion's share of resources to hoard for themselves, and they won that land ownership other oppressive warlords they defeated in battle, and/or inherited from their warlord parents/ancestors and protected by fighting off other warlords and their armies trying to kill them to take their land too.
And they ruled with iron, killing and pushing down those who didn't want to work under them. Their subjects had to do all the scut work of making society run (growing food, making clothes, etc) while the war lords got to feast in their castles, filled with fine silks and goods that others designed and/or brought them, with enough stores of grain and ale and meat to last long winters while the peasants and crafsmen who grew the food and made the goods they used barely got enough to provide their basic needs.
Everyone gets on mages for being supposed tyrants because of the "might makes right" dynamic of magic > non-mages, but the non-mage systems are just as strongly rooted in "might makes right." Big powerful warriors rule the smaller war tribes just as big powerful warrior kings and princes and nobles with private armies of knights and soldiers and guards ruled kingdoms. Big warriors from small war clans can cut off your head for back-talking just as surely as kings and nobles can either personally or send their paid knights and soldiers and guards (who were ideally loyal to them) to cut off your head for back-talking too.
As for Solas, in ancient Elvhenan all elves had magic, not just the Evanruis. It was just that, like current non-magic kingdoms, "might makes right." The strongest mages ruled the weaker mages just like how the strongest warriors rule the weaker peasants.