In that case, why not question sexuality too ? "Why ? What exactly makes you "not attracted to" that gender? Is it particular features?" and that is a slippery slope.
I don't see why attraction to gender should be questionned, therefore I don't see why attraction to species should be questionned, for I don't see what in their core is different.
The difference being that when we say gender what we mean is biological sex, as in: I only want to have sex with people whose genitalia is the same as mine. Yeah, I went there. Deal with it. Also, I assume you're referring to sexual preferences of gay, bi, straight, and asexual, btw. Gender preferences, on the other hand, are different and could encompass a wide spectrum of possibilities. A gay man could prefer men who are more feminine in behavior or dress. That doesn't mean said gay man prefers women. He prefers--although it'd be more accurate to say, only goes for; preference is not the ideal term to use for sexual orientation--men, but finds feminine qualities attractive and prefers those.
With race, no one is born with an ingrained sense of, "I'll only bonk men of ___ skin color." I agree with Ailith Tycane on that point. Tycane also touched upon internalized racism and self-loathing because of institutionalized racism. Just as institutionalized homophobia can lead someone to hating themselves for preferring someone of the same sex, racism can lead someone to hating people of the same "race" as themselves.
The way you're phrasing it, humanity would have an ingrained, biological reason for preferring people of their own skin color and only people of the same skin color. But that's not the case. And that's different from preferring someone who looks like you, which can still be influenced in external ways because of the racial biases in media and culture that Tycane mentioned.
It's different at its core because someone who thinks black people are ugly because they think of black people as thugs and see dark skin as disgusting due to associating it unattractiveness and nasty human behavior is, at its core, different from someone who does not date or have sex with men because they have no attraction to male bodies.
Does that make the distinction clearer?