The other issue is Anders.. I never played as a male, so I couldn't say much on this subject matter. As a female, he never hit on me. But apparently if you're a man he hits on you pretty hard (regardless of whether you hit the heart icon or not?
I just want to clarify the Anders bit. This applies to both male AND female here.
After you recruit him, during the dialog for Talk to Anders he apologizes for the previous scene where he explained the whole Justice debacle, "Sorry for dumping all that on you." During the second dialog prompt if you select "[Blue] You did the right thing" -- which comes out as something similar to (the very cheesy) "We can't always know the result of our actions, we can only make them with a true heart," -- he takes that very positively and DOES hit on you, regardless of gender, and without you initiating anything via a heart icon.
You can completely avoid this by selecting Purple or Red during that second dialog, but you will still get 5 or 10 Rivalry respectively. Anders goes off on a tirade against the Chantry, you get an additional dialog prompt, and the conversation ends. But at least in this case you aren't being hit on, and neither do you have to refuse his advances.
If he does hit on you, the resultant dialog does vary for gender, with the females getting "You don't want to be with me" doom and gloom, and the males getting a chance to ask about Karl and Anders's history with men.
It always frustrated me that people spoke about this dialog as if Anders hit on you regardless, when it only happens under specific circumstances, as I described above.
I could go on about my issues with the differences in the dialog between the male and female version but this thread is not the place for that.