On my end:
1. The Wardens have demons.
2. The Wardens take only skilled fighters they proceed to train mercilessly.
3. The Wardens are combat pragmatists who fight to kill, not show off.
4. The Taint may give them an edge. If we're assuming Grey Wardens are just sentient Darkspawn, it's implied they're stronger and more durable than any normal human beings.
5. Wardens are brutal, unforgiving, and fearless fighters used to horror.
6. The Wardens have a castle, which historically means you need about 10 to 1 to defeat in open combat as a result.
vs.
1. The Templars or Circle of Mage's entire army is there.
2. Varric says that the Inquisition's military power by Skyhold is, "equal to any City-State" and with military agents plus perks, may equal that of a kingdom according to the epilogue slide depending on your influence.
3. The Inquisition has Siege Equipment.
4. Cullen is a military expert.
5. Leliana is a subterfuge expert.
6. All of your companions are throwing in their lot off and on screen as well.
7. Historically, one of the big problems with the military has been most people actually don't want to kill their opponents. One of the big advantages of moral and especially zeal is the followers tend to actually want to hurt the enemy. This changed with military conditioning but is a big thing.
One thing that WOULD be a factor in RL military engagements and why the Wardens would lose many engagements is the fact they're not a mixed military.
The Wardens have bowmen, mages, and swordsmen but are small engagement special forces.
Whereas the Inquisition is a full-on army-army with all that implies.
Larry Hama ended his original G.I. Joe comic by having Cobra smashed not by G.I. Joe but the regular United States military as the former were good but the sheer overwhelming power of that was more than anything the former could touch or bring to bear.