During Here Lies the Abyss, when Nasir is running with Loghain and Hawke and they encounter the bloated spider coming back to them, he wastes no time in shutting down both of their offers to distract the creature so they can escape. He instead says it should be him that distracts the creature, as out of the three of them he's the one who can traverse through the Fade and back again physically at will (by this point, he and Solas and Dagna have begun to study it enough that he's mastering it).
He's tired of seeing people he considers friends or allies dying before him, with nothing he can do to stop it. It happened back when he was a mercenary leader during the Blight. People he cared about died fighting against the Darkspawn. It happened again when he was with the Carta running his own mercenary group there as well. Friends he considered family would die in the cutthroat game of the Merchant's Guild.
Then the Conclave happened, and though he knew none of those people all that well the shock was still too much for him.
"All those people... dead?"
And then Haven? That was just the final nail in the coffin for him. How many people died because he failed as a leader, because he failed to properly make Haven defensible or realize that they couldn't stay there forever?
"For every one I saved, two more were cut down. I failed them..."
So he doesn't accept either of their attempts. He orders them to march through the rift and go and fight until he makes it back there. Before Hawke vanished through it though he sent a searing fireball at the back of the creature, which bought Nasir just enough time to open up his own rift and make it out, whereupon he then sealed it immediately.
Unfortunately, this rift sent him outside of Adamant, so he had to fight his way through again, killing any demons that hadn't been dealt with before and saving anyone he could. Eventually, he managed to make it towards where the main rift was and seal it, though he surmised that it perhaps was not worth the cost. He begins to think more Inquisition soldiers died as a result of his heroism in the Fade, but he had no way of knowing he'd be thrown so far away from the main rift upon leaving the Fade.
Ultimately Inquisition is proving to really **** up Nasir, as he had PTSD to start with and it just got worse with the Breach and Haven lol.
EDIT: Ooh I forgot to mention. Cassandra was both angry and distraught at the prospect of him having not come through the rift with Hawke and Loghain. She was panicking, worried.