I agree that your inquisitor couldn't be replaced easily. If the hero inquisitor would leave the inquisition then there probably would be a lot of negative consequences but somebody else would probably chosen to be the next inquisitor. Anyway I meant that Leliana or Cassandra could have been inquisitor if you wouldn't have been at the conclave.
My elven inquisitors would gladly join Solas because to them the People who Solas always speaks of are more important to them than the humans in Thedas. My elven inquisitors have already saved the world but if the humans want to destroy it then they can do it. It's no longer their problem.
Considering that neither Leliana nor Cassandra stepped up (and after destruction of Haven they've decided to hand the leadership to Herald) I don't think they'd want or make effective Inquisitors. Aside form everything else, it's Giselle who hints one of the main reasons for it - both of them are too tied to the Chantry.
This is also why they both searched for Hawke (and HoF) earlier.
As for whether my elven Inquisitor would join Solas.... well it would depend what she'd learn.
But I know that she wouldn't join him solely on the basis of picking ancient elves over humans. I also don't think Solas would approve of that, seeing that he's a person sensitive to the plight of other races or groups that are treated unfairly - and he always approves of Inquisitors who never focus just on one group of people, but is fair to all.
Also, I don't think Solas just wants to restore ancient elves for the sake of restoring them. But even if he does - what if the way to restore ancient elves leads through changing the minds of people of Thedas? It would explain why Solas basically gave Skyhold to Inquisition. By his own admission, he did that in order to steer any suspicions away from elves and make people think of them as valuable allies.
So, what if Inquisitor and Solas would have to stay separated in order to work on bringing elves and other races together, with merely a promise that they'd reconnect one day, when things go well? If such is the case, I'm tearing them apart faster than I can say "The needs of the many outweigh the needs of the few" 
Besides, leaving 'just like that, for the sake of the People' would speak badly of my Inquisitor - she said it herself at the end of the game that Inquisition is just getting started, while Morrigan warned that Corypheus was just first of ancient terrors that will haunt the land. Plus, she's made friends and learned stuff from people she met along the way... and she'd just abandon them, for the allure of 'ancient elfiness'? Of the People she knows little to nothing about, who would likely flat-out reject her (like Abelas did in ToM), since for them she's just a common shemlen? I don't think they'd look at her favorably, even if the Anchor is the key of their salvation or she's recognized as beloved of their (assumed) leader.
In fact - what if shemlen can't even enter where the last ancient elves are, because they'd taint them with mortality? Solas might be able to withstand it, probably because of who he is... or does he? (I don't think he was born with all these wrinkles, lol).
So yeah, there are more than a few reasons why she wouldn't want or be able to go with him. And that's probably just a tip of an iceberg.
I believe that the main reason why Solas and inquisitor can't be together right now is that he has his mission and he thinks that he should complete it alone. I believe that he tried to be distant because he was afraid that he couldn't break up with you and he was afraid of revealing himself to you.
I suspect that he knows that he won't accomplish this mission unscathed - either he'd have to sacrifice himself or part of himself (or pay the price for whatever he's doing), which is why he sometimes sounds like he's about to die. What exactly this means is anyone's guess at this point