But it is still a wrong system . I mean a security that only a few enjoy while many do not...can you still call it security ?
I mean you could compare the Templar to the Warden system , in the sense they sacrifice themself to protect the mass and therefor offer security from the Blight .
And the Templar oppress and take mages freedoms and rights , to protect the commun folks from abominations and the fade or magic .
But the difference , is the Blight is a real threath that exist . while the mages threath is a mass fear born from a bloody past that none wanna find a better solution to then oppressing and tagging every mage as 'futur Tevinter' crap .
At least the warden are honest and willing to give their life for the threath that is the blight . The Templar , may seem to be able to do the same...but in DAO the fact that they wanted the Annulement instead of going in to investigate , tell me.....they are too extreme . One probleme and BAM! Annulement Annulement ! so much for being trained to fight maléficar... 
The group of non-mages is much larger than mages themselves. So it's not a few.
The fear is not unreasonable. People like Connor have murdered tons of people just by being taken over by one demon. One demon in one kid becomes an unstoppable mass murderer. That's terrifying.
They wanted the annulment because the tower seemed lost and let's be honest, the tower would have totally been lost had it not been for the Warden. Uldred was way too powerful. The whole tower was full of demons. That's not a tiny problem at all.
As for Wardens being honest? That's why Duncan told you about the potentially fatal blood ritual beforehand. Oh wait.
The situation isn't really that black and white. Considering most problems have born out of ignorance, education is of top importance for mages and non-mages. I'd advocate for that.