That safety is, punishing people for fear of what some of them might do. By that logic doesn't that give women/people the right to lock men/people with certain disorders, up for fear of what they might do? There is nothing just in that. Mages and non mages are born free, the chantry and templars rob them of that freedom. If someone kidnaps you, and you fight back, does it give the kidnappers the right to use force against you?
I don't think either has the right to impose anything, freedom is a right, until it's you've done something that deems you unworthy of it.
The Circle system works on the basis that restricting the freedoms of mages is preferable to having non-mages fall victims to magic.
The ideals of magical freedom work on the principles that, while some non-mages will inevitable fall victim to magic, this is preferable to imposing restrictions on the freedom of mages in the manner than the Circle system does.
They both choose to place the burden of suffering on the opposite side in the name of their own protection. What is the difference beyond that you support one side due to your belief that "freedom is a right"?
At least the Templars have a plan to diminish the number of victims beyond "Let's all be free and hope for the best".