Yep, none of the upgrades mean much, no usefulness added at all. Not even codex entries in there. Your forces don't kill more enemies on scouting or at Arbor wilds because of a sparing ring and, you don't get any sort of healing bonus for the infirmary. The mage tower doesn't have harrowing, or even young mages to train, there isn't an enchanter there you can get a quest from. The Templar hold does nothing either. Basically you spend hours hunting down the supplies to upgrade and get nothing in return. You don't even get the rest of Skyhold repaired for your efforts.
I did the upgrades intentionally once, now I just grab quarries or logging stands if I happen across them, not worth any effort. Everything you need is found out in the world for crafting, and any upgrades don't do anything drastic cosmetically even.