I think AmstradHero's list of some of the major obstacles is a good one. In particular, the increasing focus of major developers in the years after NWN came out on cinematic elements like the latest graphics, voice-acting, etc., in preference to core gameplay has made it a lot harder to make modder-friendly games.
Several other issues that I think play a role:
1) The "sweet spot" for a toolset between versatility/potential and ease of use.
NWN managed to achieve an almost perfect balance between these two. It is relatively easy to learn how to use the NWN toolset, and once learned, one can produce content with it relatively quickly. At the same time, it is versatile enough to enormously modify the type of gaming experience offered for those who wish to do so. Most game toolsets fall short in one or other of these areas: either the toolset is too simple and does not allow one to make anything all that different from the official campaign it is using as a base, or else it requires more technical skills (or simply more time to work with) than the typical amateur would-be modder is likely to have.
For me personally this aspect of NWN is enormously important. If I need to, I can throw together a basic area in the NWN toolset (excluding "meatier" elements like scripts, conversations and encounters) in less than 15 minutes (in fact, I will probably spend a lot more time than that fussing over details, but it is still a relatively quick process). Given that I, one guy working off and on in his spare time, am embarked on the insanely ambitious project of making an RPG campaign that rivals professionally produced ones in size and scope, that kind of ease of use is absolutely vital. Since I also aim to surpass such campaigns in quality (at least as far as my particular tastes as a player are concerned), I also need a toolset sufficiently versatile to alter the way many things work by default, by, for example, creating a more robust ethical system in which all kinds of actions a PC performs can affect alignment. I find it hard to imagine even attempting to do what I am doing with anything other than the NWN toolset.
The trouble, of course, with regards to any new game imitating this advantage, is that balancing two contradictory elements in a satisfactory way is by its nature always going to be tricky and difficult to pull off.
2) D&D
One reason there has been no NWN3 (by which term I include a game that would be a de facto NWN3 even if it did not use that specific name), is that Atari owned the rights to make D&D related CRPGs, and since after releasing NWN2 Atari was in the process of both going bankrupt and getting sued over those rights it was not in a position to do much with them. At the same time, of course, its possession prevented anyone else from doing anything D&D related either.
One could argue that this should not have mattered because there is no reason why a toolset-oriented, modder-friendly game must be based on D&D. While I would certainly agree it was not necessary, I do think being based on D&D was enormously helpful in making NWN modding what it was. It provided a set of rules and content that was not overly specific to a particular storyline or setting, and a base of players who were highly accustomed to thinking of their game as modular and customizable (in D&D a campaign could often be essentially the creation of the DM, even if using published materials as a base).
3) Lack of awareness that this particular gaming niche even exists and might be worth catering to.
One has to remember that there are a lot of people, even people involved in gaming, who are barely aware, or not aware at all, of the existence of the NWN modding scene. One still encounters people who, if they know about NWN at all, still somehow think that the official campaigns are the main thing it offers. The brief description of the game on GOG (for one example of this blind spot) only talks about the official campaigns and makes no mention of the toolset, the masses of community created modules and content, PWs, etc. (albeit some of the user reviews do refer to some of these elements). The game developers who make official campaigns, of course, are understandably especially likely to focus on them, and I suspect very few of them have any clue as to just how much the NWN community has been doing with this game. If developers do not even know a potential market exists, then naturally it will not occur to them to make a product for it.