All he did was end the relationship. Poorly, but still.
If one feels used or violated from that, I think that's their problem really. After all, Lavellan advanced on him. His words before everything was "No, I shouldn't."
Hurt and resentful shouldn't be confused with violated or used.
She might have made the first move, but discounting his obviously wanting it and pursuing the relationship in spite of his caution is a bad thing.
If you choose, after Fade Haven, to say you'd be willing to take that risk if he is, his response is Yes, I may be (yes, I want it, maybe...the return of his caution)..." and asking for time due to "considerations".
Yet HE is the one who brings up the kiss again, after kind of feeling Lavellan out a bit to maybe try and make sense of his feelings. "I have not forgotten the kiss" isn't just her "advancing on him". He also caps off the second round of kisses by telling her he loves her. Even though, yes, he does the adorable head shake again and tries to walk away. Yes, she stops him, but yes, also part of him was looking for her to stop him from leaving, while another part of him wanted to shut it all down and, should Lavellan break it off with him, he seems quite relieved, though it's also sad in its own way.
That said, I agree that this shouldn't really lead to feeling used. However, couple this with later finding out he disappeared suddenly AND Leliana's report about his named village being an ancient ruin, uncovering some deception....well, it's justified if someone wants to feel like their Lavellan feels rather used. Partly because there are multiple things at work she doesn't know.