It doesn't have to be jam packed of course, but a few NPCs would be nice--some servants in the kitchens, or off-duty soldiers or guardsmen wandering about. If there's stables a few stable hands would feel appropriate that tend to the mounts while the Inquisitor is away. It would be really cool if there were one or two NPCs that had their own tiny plot lines detailed through conversations like that one guy in the crew quarters in ME3 who was trying to get his family off Earth. I always got the sense from novels set in medieval or even Victorian times that each castle/manor was almost a tiny little village contained within the grounds. It might be asking a bit much but it would be awesome for Skyhold to have that same feel.
Castle Cousland felt large and appropriately castle-y. There was a big hall where you first meet Howe and Duncan, there were big and little rooms for nobles and servants and soldiers, there was the kitchen and Nan and the elven servants... add to that function aspects a la the Normandy, like a research library, a bathroom and vanity so you can tidy yourself up like the Black Emporium, vendors, healers, maybe a messenger service so you can write to your MP buddies... oh and trophy rooms like in Skyrim!
I get the vibe from Bioware's publicity materials that Skyhold will be rather like this - I certainly hope so.