It's complicated because of procs and various stackable buffs from gear. Too many of them have probabilities and dependencies on situational context, like flanking.
It would be easier to calculate a theoretical max if you remove a lot of the variability, like:
- Ignore gear (except weapon) and accessories
- Standardize the weapon to be an unmodified 100 base damage (low for a warrior, high for a mage)
- Assume all attacks are frontal (ignore flanking) and the target has no status effects (ignore combos and skill interactions)
- Target has no resistances and weaknesses
- Target is human size (because some skills don't work on targets larger than AoE)
- Target has no guard and no barrier and no armor
- The target will stay put during the attack, won't fly/jump/teleport off, so stationary DoT does full amount
Finally, you need to put some kind of time limit. Are you looking for a single peak damage number, or total damage over a period of time? Based on the OP, where you mention MoD, it has to be over a period of time. And since damage stacks for MoD or Static Cage, the amount of time needs to be a constant, so let's arbitrarily say 5 seconds, counting from the first hit that would aggro the target. Any amount of prep time to take MO potion or whatnot is fine, clock doesn't start until first hit.
Now that the problem is sufficiently constrained, it's pretty much a puzzle game or min/max exercise. For a full party of your choice of classes, what combination of passive and active skills and potions/tonics/grenades yields the highest accumulated damage? Show your work.
That is obviously not the true maximum. From there you can predict some high-water marks above that nominal point, like if all characters were flanking or if the target is weak to fire attacks, that sort of stuff.