Like others have said, he misjudged. I'm sure he knew that it would be bad...not HOW bad. Cutting off a major portion of their magic? Yeah. Making them mortal? Yeah, I think THAT he didn't know. This was a species that lived, and breathed magic. It was what they knew. Their whole society thrived on it. They couldn't imagine a world without it. So, when he cut them off from it in a major way, they lost near everything.
Aaaannnndddd...here it is. The Elves became the panda of Thedas. Sans magic, they went from dominant, to dominated. Humans DIDN'T destroy them...oh, they picked up and enslaved the pieces, sure, but they didn't DESTROY the elves. THAT came due to the fact that it is a LOT easier to blame someone else for your own mistakes. Like Solas. Sure, DON'T blame the nigh immortal despots who rule over you in fear...no, blame the guy who SAVED you. Fair enough, right? Solas ATTEMPTED to make things better. To him, removing the Evanuris was worth the cost. Well, until he got the receipt, and saw the bill. Now, he feels he owes a debt. He HAS to pay that debt. Yet...the elves time has past. They had their shot, and they collapsed. Their time is OVER. Is it sad? Yes, but time marches on. A new society has risen. Humanity is now dominant (for now), and the Elves are stuck munching on bamboo and waving at tourists. A sad end, for certain...but Pride toppled them (literally).
Solas doesn't WANT to kill humans (and dwarves...not so certain about the Qunari though), but to him, he must. I sympathize with him. Oh, he's wrong in more ways than a cannibalistic childrens cooking show host, but I get where he is coming from. Let's be honest...if we woke up one day, and found our ENTIRE civilization GONE, we'd freak. If we found a way to restore it, we'd take it...would EVERYONE do so knowing that you'd be condemning an untold amount to certain doom? I HOPE not, but how do we know until we are actually IN that position?
Solas KNEW that it would be bad. I don't think he knew HOW bad, but he knew what he was doing. He admits it. What he is attempting to do to FIX the problem is just the same wrong but from a different direction. He knows that to. This guy isn't looking for glory, he THINKS he HAS to set right a terrible wrong. Which, I'd say should make Thedas a bit MORE worried. Guilt and a death seeker attitude will PROBABLY make him reckless...and when your plan is going to cause the world to burn, RECKLESS is not what you want to hear.