The elves build pocket dimensions. The "Fade" wasn't a place - Solas makes that clear in Haven, when you first speak with him. The implication is that Solas stuffed what the Fade "was" into a pocket dimension, and otherwise wrecked everything that depended on the "Fade" being the way it originally "was". What Solas likely did is take the thing that is the "Fade" - that things are connected to - and stuffed it all in one place. That's the "splitting in two" that we learn about in the library.
its clear from codex entries that there was a distinction between the Fade and the physical world before the Fade, even if the borders were open, but I don't think that means we're actually touching real physical stone in the Fade at Adamant.
Hear me out here. I think everything we see there is the result of spirit fabrication. They have to be able to make physical objects, otherwise those Avvar statues from the First Blight couldn't have gotten there. There's even a codex talking about why there are floating rocks in the Fade, its because the dreamers don't focus on the ground and the spirits don't get connections between ideas.. The codex speculates that if dwarves could dream, then the stones wouldn't float, because dwarves are so attuned with the ground. So everything in this place is runs on the rules of spirit perceptions. So why couldn't the very ground we're standing on also be some spirit's craft? So is the reason the Fade seems different at Adamant because its physical or because we're not looking through the eyes of a dreamer?
I'm suggesting that we didn't enter a physical place in the Fade, we entered a non-physical space with physical bodies.