As for how he became king, well, that's a tale in and of itself. Though I can only speak of what he did with in-game events, as I have to envision what headcanon things there might be (the in-game things are enough to justify it however). And when I say in-game events, I mean the quests, as it's not possible for a DN to take the crown back in the game despite the historical and cultural precedent. A lot of what I say happens in-game, but it's not given any sort of connection to the idea of taking the crown back.
He first returned to Orzammar and found it in turmoil. A dozen votes held by the Assembly in the span of the three weeks since Endrin's death, with civil war imminent on the horizon. The Assembly is deadlocked, stone for stone, with Bhelen and Harrowmont having 40 votes each.
Many in the Assembly were suspicious of Bhelen's rise to power (at this point Trian is still believed to be dead). Many of the common folk were similarly suspicious, often venting the things Harrowmont wouldn't openly declare, though it was not enough for any meaningful difference. That it existed though was a great deal.
After seeking out Steward Bandelor and seeing how Bhelen's allies were trying to cut off the surface trade temporarily so that they could take over all the contracts, he talked to Vartag and "convinced" him that he was willing to put aside his own revenge for the means to put Bhelen on the throne.
Vartag bought it hook line and sinker and handed over the forged promissory notes to Xanthos. Xanthos knew Harrowmont well enough to understand that he would never do something so irresponsible and stupid and dishonest, and so he took them to Shaper Czibor to verify this. He studied the originals and compared them and Czibor said that the Ministry of Lands would investigate the Gavorn clan for this.
And since the Gavorn clan, and Vartag especially, are close-knit with Bhelen, this merits an investigation into Bhelen as well.
Now, Bhelen is the kind of person who will sacrifice anyone for his ambitions. He got rid of his brothers and even his lesser cousin Revelka doesn't mean anything to him. If he's willing to cast aside family loyalty, I have no doubt that he'd throw the Gavorns under the bus to save his own face when faced with scrutiny.
Xanthos capitalizes on this and offers Vartag a way out. Testify against Bhelen and he'll be allowed to join the Legion, or be arrested and tried by the Shaperate (the guards were behind Xanthos).
Next, he talked with Orta and, upon hearing that she may have descended from House Ortan, told her that he would go and find the records if he should head out into the Deep Roads. In exchange, as she would be put onto the Assembly immediately, she would give him her vote and make a donation to the campaign against the Blight.
Next, he heads to the Glory Proving and enters independently and upon winning, proclaims in the loudest voice possible:
"Look on me fair Orzammar, and know that I am the one the Ancestors have chosen as their victor! This Proving, held to honor the memory of my father, was brought about to decide who should rule, whom the Ancestors favor! I have entered under no name but my own, so who here can deny that I am to be your king, meant to champion Orzammar into a new age?! Who here would say that I do not have valos atredum?!"
This was meant merely to put doubt and confusion in the hearts of the people and rally some of the nobility to him, as Nerav Helmi assures us that many of the nobles no doubt regret their decision to exile him. More to the point, he wanted to get the attention of Lord Harrowmont.
After meeting with Harrowmont, he tells him that he should support his bid for the crown when the time is right. He decides that he needs to take down Jarvia and her Carta in order to properly show to Orzammar that he is the strong king it needs, the one who can defend it from any sort of threat it faces -- foreign and domestic.
Upon coming down here with Darius and some other companions -- and meeting Leske, where Xanthos gave him a speech that broke his resolve at betraying his old friend -- he took down the Carta and also ingratiated himself with Zerlinda's family. Having a Mining Caste clan on his side was a good step. He also got the incriminating papers that put Bhelen and the Carta as responsible for Trian's death.
(he also took Rogek's deal, but he had plans to use smuggling for Orzammar's benefit)
And finally, upon getting Branka to forge for him a crown and write down that she indeed gave her support to whomever Xanthos wished, he told the Assembly that she wished for him to become king. Many thought this merely a power grab by an exile, but when someone from the Shaperate presented the evidence linking Bhelen to Trian's death -- which was itself brought to the Shapers by someone from House Harrowmont, as Xanthos couldn't bring it himself -- and he presented the writ from Branka detailing her words, the Assembly had no choice but to accept.
More so when Trian came in, to the surprise of all, and confirmed the story.
Bhelen wouldn't capitulate, saying that the entire thing was the work of Xanthos' mind and that he was forced to do it, and so Xanthos called for an Honor Proving like Darvianik Vollney had done when he was accused of fratricide. Bhelen accepted and was defeated, but Xanthos didn't kill him.
He offered him one last chance to atone and repent for his crimes by joining the Legion and heading to Bownammar (but he knew that Bhelen wouldn't accept because Bhelen knew he'd never survive due to Xanthos and Kardol being friends, it was merely for the public relations that he did it) and so Bhelen was arrested and sentenced to death.
Rica was understandably pissed but Darius assured her that Bhelen didn't really care for her (something I kinda believe as I watch how Bhelen treats her. He showers her with gifts and jewelry but I don't think he treats her as a person he can trust, since he doesn't tell her the truth about himself -- which politically speaking he doesn't have to do and shouldn't do). I think he viewed her more as a tool, rather then a person, like the rest of Orzammar society.
But as people no doubt know by now, I hate Bhelen with a passion so maybe that colors my perspective.