Sometimes I'm not sure if I dislike him or I just dislike what he represents in the game. It's like others have said, he's quite deliberately calculated to be your bro, he wasn't even inoffensively pleasant like Bethany, but was pushed too far into trying to be oh-so-very awesome in every way, and it falls completely flat for me. His jokes in particular take on an annoyingly condescending note because of how much less flawed he is than everyone else, so even though I'm already rolling my eyes at the drama and stupidity of the other characters, I still resent him doing it, too. Aveline, for example, was able to be a breath of fresh sanity without coming across as insufferably superior - she could be petty, stubborn, short-sighted and a bit of bully. She felt very human. Varric, in contrast, felt very...forced. Like the group comedian in a sitcom with canned laughter switched on after every oh-so-clever quip. I left him on the bench.
I suppose the only reason Varric comes off so well in DA2 is because he's telling the story, so he gets to control the nuances - or lack of the same - of his own portrayal. He might be an actual person in DA:I, but I'm already irritated with him taking up a companion slot that could have gone to another dwarf who hadn't been king of a whole game already (like Sigrun), so I'll be about six or seven playthroughs deep before I pay him any mind, if I ever do.