You can feel that way but the game quite blatantly makes going to the mages a priority. Let's start off with why:
When you first get to Val Royeaux, the templars essentially tell you that you can go to hell and that they are their own power. Heck, Lucius straight up calls the templars who joined the Inquisition TRAITORS, so it's made blatantly clear the Templars want nothing to do with you. On the other hand, Fiona politely comes to you at great risk to herself and invites you to discuss a potential alliance. If you aren't a rampant mage-hater, there is literally no reason not to at least go to Redcliffe and hear the mages out.
Afterwards, you find out about the time rifts. After that, you find out that a bunch of Tevinters for some reason came all the way to Redcliffe to get the mages. Fiona not knowing about you should raise some concern. Not going to the church afterwards is the only way to justifiably consider going to the templars. If you do go, you learn about the Venatori, a group who is obsessed with you and is probably the closest thing you've got so far to discovering just who exactly caused the Breach, not to mention Dorian's warning how the time rifts are going to continue to expand and go farther. That is pretty much an immediate reason to deal with the mages.
Now, you've basically got a hostile enemy occupying territory practically next door to you with time magic that is starting to expand and are the primary suspects in finding the culprits who started the Breach. Before you suggest "lets get the Templars", your advisors straight up tell you that won't work as the Venatori are mobilizing for war and by the time you get the Templars, the mages will be long gone, plus your advisors straight up telling you actually assaulting Redcliffe Castle even with Templars would be seen as an act of war with Ferelden so that's out of the picture. So now you've got a hostile foreign power mobilizing for war who potentially caused the Breach that is right next door and will take the mages away from you if you try to get the Templars. That alone should be reason enough for why you should deal with the mages ASAP.
Furthermore, literally nothing indicates you'll lose the Templars if you help out the mages first, plus the plan to get the Templars involves essentially threatening them to help you. Something tells me the Templars wouldn't be too happy to help you when you threaten them into helping you.
Plus, finding out Cory's plans makes a lot more sense if you sided with the mages because they already happened but meanwhile the only reason you know about Cory's plans with the templars is by finding papers the Envy demon left lying about and Envy gloating about the demon army. Sorry, but the mage method makes a lot more sense.
Furthermore, having to recruit Cole TWICE is just plain ridiculous. It makes sense having to recruit him if you sided with the mages but zero if you already recruited him after siding with the templars. It makes sense everyone is suspicious if he first came during the attack on Haven but little if he's already been around for a while after siding with the Templars. Plus, the interactions with the dying Roderick make more sense if Cole is the one with him rather than Dorian.
Now lets talk about the Red Templars:
If you sided with the templars, HOW IS THERE AN ARMY OF RED TEMPLARS STILL AROUND? It makes sense that the Venatori are still around if you sided with the mages because a majority of them are from Tevinter but it makes zero sense for the Red Templars to have a large presence if you helped the Red Templars. For starters, you prevented them from bolstering their army with the entire Templar order which should drastically reduce their numbers. Furthermore, you killed a lot of red templars while recruiting the mages so there should be even less. That brings up the question of how are the red templars still a fighting force if you sided with the templars? They are apparently large enough to stand up to the Orlesian army during WPHW despite that making no sense if you sided with the templars yet makes complete sense if you sided with the mages.
Furthermore, the Templars stop being relevant if Barris dies or you conscript them. There is no redemption arc in that case. Plus, the Templars get absolutely no mention in Trespasser apart from their rebellion against Divine Vivienne.
All things considered, the game blatantly wants you to go towards the mages.