Well... there's always the attempt to sneak and a cutscene yanks you out of the mode.
But then, there's also the whole: (H)old party, sneak past an encounter, get far enough from your party, and then teleport them to you by releasing the (H)old command, so everyone avoids the encounter.
It started with a fireball in our faces which of course all the enemies were immune to the knockdown effect (bloody Nightmare mode). Good thing the tank caught the agro while we looked up at the clouds for 5 secs...