He literally says "The return of my people will mean the end of your people". With his people specifically being ancient elves and no one else, since he doesn't consider modern city elves to be his people as evidenced by his talk after Halamshiral and actively despises the dalish culture. Sounds like planning to kill millions all right.
Again making all kinds of assumptions here, "End of your people" to whom, it's a generic line used on all inquisitors or matter their origin, which is all the credence I need to know it's not a literal thing, it's a metaphoric line meant to do exactly what it has in the masses like yourself inspire fear if not hate. The return of his people, is more than likely a shout out to the fact that his people will not be so easily cowed of or controlled by the existing masses. Also no he makes credence to the fact that he is just unhappy with what has happened to the Elves, he is pride therefore of course he distinguishes what he calls Elves different from what they have been reduced to in modern Thedas, but several times he's come to the defense of modern Thedas elves and seeks to show them the right way.
1.) He mentions in argument with a Dalish inquisitor that he went to the Dalish first but they refuted him and called him flat-ear and thought they knew better.
2.) He tries to awaken Sera several times to the possibility that her being a Elf is more than she realizes, and the potential there in.
3.) With a romanced Lavellan he may have misjudged the Dalish, and even compares Lavellan herself as being akin to a "Wisdom he has seen in a very long time." A obvious comparison to the best of the Ancient Elves maybe even on a level similar to Mythal as again Solas from what we know of his interactions with the ancient Elves only held Mythal in high esteem, calling her the best of them and vaunting her intelligence, compassion and wisdom.
4.) Abelas like Solas also made the distinction that the Elves of present Thedas, "Aren't his people." pointing out the Dalish pretend at something they know nothing about. Why would he need to point especially to the Dalish if there wasn't any relation to them being Elves at all? He could have easily "My people are all dead, we died in X Y Z." He did that because Elves of modern Thedas have Forgotten what they were, and only pretend to know what they lost, this was made clear several times if I recall, but the best tell was when Abelas came to Solas and sensed he was more akin to what he knew as the ancient Elves but just by looking at Abelas we know he is one sworn to Mythal by Solas own admission he is a slave, something akin to at best being beneath Solas and those who walked on even footing with Evanuris. The only real distinction however between what the Ancient Elves and their thralls in comparison to modern Thedasian Elves is the magic and knowledge of magic/lore the two have, I say this because every time Solas or another ancient Elf talk about the comparison it always seems to fall back to knowledge or power. They never make a distinction biologically, (because obviously they are one and the same except they have become lesser.).
A lot of people like to say Solas is telling half truths, and that might even be a valid counter argument in some cases, but I don't think it applies when it comes to his association with who is and who isn't a elf. Pride is his given name for a reason, for better or worse, there are numerous real world examples of the very same thing being done/said all based on the idea of pride/vanity. Few people who see themselves as better ever get the chance to meet someone who they thought was a lesser and thus are then humbled by such an individual, but I think a romanced Solas is an exception.
5.) Solas uses "our people" several times with a elf inquisitor when directly interacting with her/him, first notable time is on the way to Skyhold, when he reveals the orb to be a artifact of "Our People." He makes this distinction again when a Dalish Mage becomes an Knight Enchanter and fills you in on the history of where the Knight Enchanter likely got its start, point out it was "our ancestors."
I could point out the last scene with Flemeth and Solas, but I get the feeling it's a dead horse and no matter how true it may seem to the cause ot show he has errored in his judgement of Thedasian Elves, people who want him to be the villain would find some way to refute it. So I'll skip it and save myself the time arguing it.
Lastly its just obvious, it's no different if you woke up tomorrow and whatever privilege you consciously or unconsciously have was stripped from you and you were told your kind are weak, barely worth soil beneath your feet, told you will never elevate to any position higher than a paid serf, or laborer. Of course you like Solas would be outraged but more so you'd be probably heart broken when you looked around and saw all of your kin and alike actually accepting this as simple fact. So is it any wonder that he'd distant himself from such association, seems pretty self evident to me. Also without context you paint a Solas that isn't the same one everyone else got, as again some people played the game different and thus had different interactions with him. Again with a romanced Dalish inquisitor and I would bet a male befriended Dalish inquisitor aren't nearly as inflammatory, and again I've already said he knows that people are going to do die, but it's not directly because of the veil being torn down but because they want be able to adjust to the world being reset.