That's what I find so hard to understand about Dorian. When I played through with my female Lavellan he was her devoted BFF and based off that I was regretting not going with my gay male Lavellan on my first run. So now I am playing my male Lavellan to be honest he's being a real S**t in comparison. Okay I did get the lovely speech about how he thought he'd lost me but apart from that he's generally cynical and mocking at my attempts to be nice to him and what is really frustrating is that my lovely guy has to put up with it if he wants to keep the relationship going; I'd like to be able to stick up for myself a bit more.
There are two things at work here. That is part of Dorian's general personality, to a degree. BUT he also uses sarcasm to deflect from talking seriously about his own feelings, or from delving too deeply into the Inquisitor's feelings. Both the post-Adamant dialog and the "about the future" dialog perfectly illustrate this.
After Adamant he starts off being pissy and then you have to ask him what his problem is. THAT is Dorian in a nutshell. If you don't ask, he won't tell you. He keeps things inside, dwells on them, and that affects his behavior.
The "about the future dialog" also has this. When you bring up the subject he immediately goes for the self-depreciating line, which on the surface seems to question the strength of the Inquisitor's attachment. Dorian is not confident in himself (despite outward appearances), and is afraid that he is going to care too much for the Inquisitor and get hurt because of it.
This is the reason I get frustrated at people who characterize Dorian as one-dimensional or who say he has no depth as the "SJW gay agenda insert" character -- I'm NOT saying this of you, Gervaise -- because it's obvious that those people have not done the romance and seen Dorian in this other light.
I really enjoy the friendship my female mage characters have with him. But I don't really feel that you can actually know Dorian as a character unless you do the romance. But that's OK. People do have issues that are related to romantic relationships that their close friends might not be aware of. It seems very real and natural to me that there exists the same dichotomy with Dorian's character.