Well, there's one more thing. Not only Tevinter's had slaves. Dwarves apparently did too:
By Corypheus
I couldn't see a mention of this in any other source... Seems that the great dwarven Empire was not so different from the Tevinter. Yet no one speaks about it...
The Dwarves do already have a horrible caste system, so they having slavery doesn't make them much worse.
Also, it could be Coripheus' deduction. Let's remember that he comes from a time when there were only two countries in Thedas: Tevinter and the Dwarven Empire. Nothing more. So if they were humans and weren't part of the Imperium, they could only be part of the Dwarven Empire. But since he comes from a time when empires were race-based (if you aren't from the country's race, you can only be a slave), he also may have thought that humans should be slaves under the Dwarves.