I hate to tell you this - actually no i don't, it means you haven't missed anything: banter is severely bugged for many.
The good news is, you'll get to hear it at some point.
Party composition does not matter. All characters have things to say to each other, and they are all supposed to fire off their banter at the same approximate rate. Bioware claims should be around 2 per hour. Many, including myself, report nothing at all for several hours at a time.
There's a counter that secretly chooses a random number, maybe 15 minutes, maybe 40, and decides that's when the next conversation will start.
All testing and evidence points to one most likely cause: the timer is interrupted by everything. Combat, mounting, dismounting, being near NPCs in combat, sneezing too loud...
The problem is that it does not PAUSE the timer, it RESETS it. So if it was going to be 22 minutes til your next convo, and you go 20 minutes, and then you attack something, it re-rolls the timer from the beginning.
The original idea was sound: you don't want banter firing when you're in combat. Not only is it silly, you'll miss it.
And you can't hear your party members when they're not on screen, like when mounted. (I guess they're in my pocket..?)
There might even be other things breaking it. Poking around for three hours with little or no combat, and I sometimes don't hear more than 1 or 2 conversations. And sometimes none.
TL;DR they all speak the same amount with each other, in any combination, but you're probably bugged.
Side note: eventually, of course, if you keep the same 2 people in party, they will run out of recorded lines. But it sounds like that's not what's happening to you.