Well, I think it's in question when exactly he left, is all-- I know that's more or less undefined, and I know it's not clear which one you're dealing with in the Val Royeaux encounter. That corpse in the locked room in Therinfal's been there for a while; that all could have happened easily before the swap, but it's hard to say. He certainly willfully placed Envy in command, and Envy did have the capacity to make decisions for itself, though. So I wouldn't necessarily write him/them off so easily. But I see what you're saying. Certainly the text could support either reading.
As far as justification-- no, I don't think Samson is justified in his actions. I'm not sure anyone's making that claim. I do think he tries to justify his actions to himself, and I do think that sometimes he successfully convinces himself, a little bit. Usually I think he fails to do that.
Like, the fact that I do think his actions came out of a place of fairly rational thought doesn't make them justified-- in a lot of ways it makes it worse, I find, because it means he can sit down, think it through, and still willfully choose to do what he does. It's kind of a fine point of distinction-- it is cruel, but he didn't do it to be cruel. A sin of ill action, rather than of ill intent.
But I mean I think we've got a pretty good understanding of each other at least, now, so yay! 
Maybe justified in the wrong word, I think its the fact people can see why he is angry at the Chantry, thats where I disagree.
Barris says that only seasoned or high ranking templars can detect a rare demon like Envy, so to me thats why the Knight Vigilant was killed, because he would have detected Envy. Even if the explosion at the Conclave was a accident I dont believe Corypheus would have left anyone alive. Knight Vigliant, Knight Divines and several moderates Knight Commanders were there, only the corrupted ones werent send, to me they were sent there to die as a part of the plan to put Envy in power.
To Samson, giving them the red is giving them a fitting end, he was willing to go through the pain too, to die for and with them (and possibly he did die for them if you choose the Templar path) no one is excusing it as anything but monumental **** up. It was the one thing that the chantry couldn't or didn't give them (it's alluded that they can die without lyrium - and that's the regular blue stuff). Who abandoned those Templars at Kirkwall? Was it Samson? No. Who gave those in pain of withdrawal relief? Was it the Chantry? No. Look it's more complex than what I can put here. Whatever you say - he cared, whatever you say he is not a socio-path, canon and the writing, the actual words written says he cares (however warped you think it might be).
In regard to the Chantry, its destruction and reformation and in general 'spiritual' themes play a huge part of inquisition's narrative, who we choose as divine influences that outcome. It's complex and interwoven and not always done particularly well but its there. I can't, given what is given to me in the story lay the blame solely at Samson's feet any more than I lay the destruction of the elven pantheon on one god.
Monster, villain, 'the destruction of your beloved Templars' - if that's all you see, okay, but I think you're missing a much more interesting (and broader) story.
Fine I shall bite.
The only one who abandones the templars at Kirkwall as Cullen, the templars decided to leave the Chantry they werent left.
The templars had Lyrium supply, there was plenty of it at Therinfal, those at Kirkwall who drunk knew it was bad, they deserve what they got, and they started using it before The departure of Cullen.
He is a sociopath, look what he did at Therinfal and Emprise du Lion, no one same would have orchestraded those things and see it as something good. If he cared about his he wouldnt have turned them, there is no life being a red templar, you are turned into a beast with the purpose of serve a magister Darkspawn, the very thing they had sworn to destroy. He says he cares about them to make him feel good, to not face the truth that he is worse than Meredith and the Chantry.
I dont even care about the Chantry, I care about the Templar Order, and Samson is responsible to turning or trying to turn them into monsters.