Class based? Seems outlandish honestly. The only time I can see that happening is if a party member had a huge aversion to mages and magic, and even that a 'rivalry' path (or significant changes in dialogue) would work much better.
Gender restricted makes sense only if there are an equal amount of m/f and m/m or f/f options available. My solution has always been to have six LIs (may or may not be companions).
Male 1- females only
Male 2- males and females
Male 3- males only
Female 1- males only
Female 2- males and females
Female 3- females only
As far as race restrictions go, again I would have to say it applies based on character. I'm in favor of it if it flavors the character's reasoning and mentality. For instance a heavily pro-Dalish elf most likely wouldn't want a shem coming onto him/her. Dwarves heavily embedded in their culture generally frown on interspecies relations and pretty much 'stick to their own'. Varric in DA2 says he's not into humans, so if he were an LI for example, having him suddenly be into a human Femquisitor would be awkward handwaving to please the masses. But as with anything that would be much harder to implement since the backlash would be tremendous.
At the very least, if an LI must be playersexual, I do like the idea of different romantic dialogue based on race/gender if/when applicable.