I think it's interesting that Dorian appears more physically fit than mages we've seen so far in the series.
Well... you do realize that he, the Inquisitor (regardless of class), and Cullen all have the same body, right? It's always been this way in Dragon Age.
In all of the games all of the people (with a few specific exceptions -- Varric, Isabela, Bull, male mage Hawke in Champion armor) have the same body. Unclothed in DAO, a male human Warden (mage or non-mage) looks like Alistair, Loghain, Jowan, Anders, Irving, and so on. A male Hawke (mage or non-mage) has the same body as Keran.
Because of the way that the Dragon Age developers have specifically chosen to use bodies in their games I really don't think that anyone's body, with a few exceptions, should be taken at "face" value in terms of how the person actually looks, or should be something to go on for specific details like you are suggesting.
Do I think Dorian is physically fit? Yes I do. It's by design; he's got the genes for it.
Do I think he would have the same physique as my warrior? No, I don't.
Do I think he's completely hairless? No, I don't. I don't doubt his grooming habits, but maintaining that would take an inordinate amount of time that he doesn't appear to actually have. And I don't think he is so vain as to have that take priority over something else.
It's kind of difficult to determine the level of care he puts into himself though.
There is this banter with Solas...
Solas: Your magical skill is impressive, Dorian.
Dorian: You're not the first to say so.
Solas: Would you not conserve magical energy with a less... flashy... style, however?
Dorian: Yes, and I'd live longer if I only ate rice and boiled vegetables, but that's just as unlikely.
There is also his alcohol consumption. While I don't refer to him as an alcoholic, he does like his drink, and that has a deleterious affect on the body. But he also gets a lot of exercise following the Inquisitor around (as do they all), and I would imagine that one has to be relatively fit to use combat magic effectively, which would be important to him.