@Verrenus After you uninstall GPU drivers using DDU in safe mode, do you download and/or install the new drivers in safe mode as well? Or do you restart windows normally first before downloading and/or installing new GPU drivers? And if you don't mind me asking, are you using Windows 10?
Well, I know this wasn't addressed to me, but it shouldn't really matter when you download the new drivers if it's just an .exe sitting in a download folder somewhere. For the sake of convenience, you could download the new drivers before you start the whole process just so it's there and ready to go. I do it this way myself.
Before I do anything with my current GPU driver installation, I always have the newest drivers downloaded in advance, just like you, nightscrawl! 
That way, I can just run the latest version of DDU, have it reboot my machine into Windows Safe Mode automatically, then select to remove the NVIDIA drivers (since I'm running one of their cards) and click the "Clean and restart (Highly Recommended)" option.
DDU will then remove any traces of my current GPU drivers and when that's done, it will also automatically reboot my machine into normal Windows mode. Then I just have to run the "Custom" installation for the newest GPU drivers that I had downloaded in advance, with both the 3D Vision Driver and 3D Vision Controller Driver options unticked (since I never use my monitor to view 3D content) and the "Perform a clean installation" option ticked!
Do note that after installing the newest GPU drivers, you need to go back into DDU again (just in normal Windows Mode this time) and click the "Set Windows Automatic Driver installation to default" button to restore Windows' automatic plug-and-play driver installation to normal. DDU normally disables this functionality before uninstalling your current GPU drivers so that, upon rebooting into normal Windows mode, the OS won't pull a fast one on you and automatically install whatever GPU drivers it deems best before you had any chance of clean installing the drivers that you yourself had downloaded! 
And, yes, I am running Windows 10 and I've never had any problems whatsoever from running DDU every single time I want to upgrade to a newer GPU driver version! 