Bioware may be better at shaping characters and cultures rather than main storylines, but here, I think there is a method to Cory's madness. Here's the sequence of events as I imagine it:
0.) A millennia ago, Cory is 1 of 7 powerful magisters trying to take over the Golden City, the seat of the Maker, in the Fade. Even with all 7 of them, they failed, maybe partly because they were also scheming against each other. However, Cory always wanted to return, but by himself, to take all the power for his own.
1.) A millennia later, an opportunity! Cory somehow is given or steals Solas's Orb, an ancient Elven construct.
2.) He plans on using the Orb to create some way to get access to the Fade: the Anchor. Dagna the crafter hypothesized the Anchor is a key; it can open doors, or it can close them, which was how we mainly used it. If that's true, then Cory was trying to create a key for him to open every door before him in the Fade.
3.) He needs Blood Magic to forge the Achor, the key. Out of sheer arrogance, he wanted to symbolically use the Divine's blood as his tool to get to the Maker. Cory believes in symbolism, as described by Morrigan for why he had bound a dragon, just as a gesture to show he's like an old god.
4.) Disaster strikes! This nobody comes and takes the Orb at a critical moment, steals the Anchor, runs away.
5.) Cory tries to recover the Anchor, finally tracks it down at Haven. He goes to get it, finds it is bound to the Herald. Fumes, but now turns to Plan B.
6.) Cory figures there must be other things that could do the same thing as the Anchor, likely to be Elven devices. He sends his minions everywhere to look through Elven ruins, that's the mid-game. Of all the leads he had, he finds a real good one: the magic mirror and the key at the Well of Sorrows at the Temple of Mythal.
7.) He is again foiled at the Temple of Mythal by the Herald. This is the 3rd time he met the Herald, who keeps on foiling him. This is ridiculous, he gets really pissed. He knows the smarter strategy is to retreat and rebuild, as Leliana said, but right now, he doesn't care about succeeding anymore. If he can't win, he'll make the whole world lose instead.
8.) He goes back to where this feud with the Herald started. He might not have the forces to assault Skyhold, but he goes nearby: the Temple of Sacred Ashes. He knows it holds symbolic significance to the Herald, that the Herald must meet his challenge. IMHO, he was no longer trying to open a Rift into the Fade to cross it, but to rip one open to flood the world with wild magic or whatever is in the Fade. He's already given up trying to become a god, except to bring Hell on Earth.
---
Thus, his plan looks really weird, but it's due to 2 things:
- at the start, his plan was already flawed, as he's trying to succeed alone where 7 like him failed
- in the end, he had already tossed his plan out the window, he just wanted revenge
To put it in game terms, Cory was playing a PvP game with us, but when he started to lose, he was just trying to ragequit. That's why he seemed more and more erratic, like an upset player.