50 réponses à ce sujet

[kamal_]

https://forum.biowar...ms-closing-down

My apologies for brevity as I'm on my phone....

[henesua]

Never fear. There is always light at the end of the tunnel, and it is not always a train.

 

In the mean time you can hang out in the Vault with us just to make sure that that light is not a train.

 

In IRC you can even drink some tea while you wait:

/server chat.freenode.net

/join #nwvault

 

 

And there's also the Beamdog forums to check out

https://forums.beamdog.com/

[T0r0]

Damn I feel old. I thought everything on the internet lasted forever ..

[Tchos]

There's also the dedicated NWN1 and NWN2 forums on the Nexus, complete with specific subforums similar to what we had here (General discussion, tech support, mod talk, mod tech support, and mod requests), running IP Board with all its usual amenities, like proper message quoting and colour-coded scripting formats.

[brgjoe]

Very sad to hear this.  I have been around NWN forums since they were hosted by the original Bioware website.  Then I moved to the "Social Bioware forums".  Then to this place.  Seemed like we lost a few with each successive transition. 

 

But it still has been a fantastic run.  People are still modding and playing a game that's over a decade and a half old.   Amazing.  And even though these forums will end, there will still be places to discuss this old game and it's mod's.

 

Hate to see this forum end.  Even though I had heard rumors that it's demise was near.  It still sucks to get the official word that is the case. 

 

Yet I am sure this game will continue to live on.

[henesua]

it certainly will. i think for a long time.

[The Amethyst Dragon]

Of course. Looks like I'll be spending the next couple of months archiving stuff and maybe putting some of it on the community site.

 

It's amazing how much information and creativity is on this forum.

[Baaleos]

Is there any automated archiving tool we can use to facilitate this, or would we have to create our own?

[kamal_]

Is there any automated archiving tool we can use to facilitate this, or would we have to create our own?

The scraper written for the old Vault did get the first page of the comments for a submission. Of course the new Vault team had permission from old Vault to do scraping.

[Baaleos]

Since they are closing the forums, I would like to know what threat they would use to stop someone making a scraper.

Banning us from the forums they intend to shut down?

Im working on one right now.

Worked out the logic already

 

Every forum has a collection of subforums, which has a collection of topics, which has a collection of posts.

Once I get the logic working for a single forum-> subforum, then a subforum-> topic, then a topic->posts

Grabbing the info is easy.

 

So far so good - able to programmatically get the Sub Forums for the Main NWN Forum.

Now I just need to code for the sub forums to get topics, then topics to get posts.

[Wall3T]

Just so I know for clerity the content being archived now will be placed into the new vault, or somewhere else? I've been a part of the content community for some time and hate to see all those wonderful posts vanish

[Baaleos]

If anyone is interested in getting a dump from my scraping efforts, let me know.

I've already got it to the point where it can iterate through all the pages on a given sub-forum and get the title and href to the topics.

 

The next and arguably final phase is to then iterate through all pages of a given topic, and grab post information.

Im also not entirely sure how much information we would want to archive.

 

I am thinking.

 

Topic Title

Which then contains a collection of posts

 Posts then has poster, post date, post content. (in text)

 Only problem with grabbing the date from posts, is that it displays the date relative to today.

Eg: It displays 'Posted today, or Yesterday' - 3 weeks from now, that wouldn't make much sense.

[Faerinaal]

Hi Baaleos, I tend to think the date is irrelevant as a data point, myself.  It could be useful for attempts to contact a poster, but it's not as if we have their email addresses anyway.

 

Please include us in your list for a data-dump.

 

We can be contacted at projectmanager@functiond20.com, or contact@functiond20.com.

[AndrueD]

The next and arguably final phase is to then iterate through all pages of a given topic, and grab post information.

Im also not entirely sure how much information we would want to archive.

Am thinking best stuff for save is all fixing and setting for running new WinOS, Linux, Mac nd with many problum that video mayb has too.  That matter much for new player who try running NWN with Win8 or 10 and fancy vcard/drivers.

 

Also think all design stuff where they figger how to do tricky things shud archive.  So many script ppl post here in many years for help it sad to think all that gud work just poof away end month.

[Baaleos]

I've had some progress on capturing the date information.

The date information is contained in the title for each topics link on the subforum.

I just replace Yesterday and  Today with the DateTime equivalents to allow the program to parse.

 

Grabbing the Forum - Sub Forums and all the topics minus the post data is relatively fast. It completed within about 5 minutes.

Actually grabbing the post data will probably take hours. (So far I have left that bit out)

 

I think for sanity sake, I am going to pump this into a database and then provide a means for the community to access it.

That way I can leave it running overnight and not worry about having to start from scratch.

[Baaleos]

Am thinking best stuff for save is all fixing and setting for running new WinOS, Linux, Mac nd with many problum that video mayb has too.  That matter much for new player who try running NWN with Win8 or 10 and fancy vcard/drivers.

 

Also think all design stuff where they figger how to do tricky things shud archive.  So many script ppl post here in many years for help it sad to think all that gud work just poof away end month.

 

I wouldnt worry about specific types of posts that we want to save.

When this is done, everything will be archived - regardless of importance.

[Baaleos]

According to the tool I am making, there will be 6964 topics to archive in Neverwinter Nights 1 forums.

I think it got the wrong number some how.... as manual adding of the topics from the summary on the main forum page suggests there is much more.

It managed to get the Topic title and start date/time without any exceptions or errors being thrown, so I think im ready to move onto the all important job of grabbing post information.

Then I can just leave it running.

 

Here is a pic to show my progress

 

 

 

Strange.

After it parses all the Sub-Forums, it gets this figure for each of them.

Finished Parsing SubForum:Hordes of the Underdark Official Campaign : Topics present:117

Finished Parsing SubForum:Shadows of Undrentide Official Campaign : Topics present:44

Finished Parsing SubForum:Dungeon Masters Realm : Topics present:79

Finished Parsing SubForum:Guilds and Registry - Ye Olde Tavern : Topics present:101

Finished Parsing SubForum:Official Campaign : Topics present:185

Finished Parsing SubForum:Tools and Plugin Developers : Topics present:165

Finished Parsing SubForum:Modules : Topics present:501

Finished Parsing SubForum:Persistent Worlds & Multiplayer : Topics present:577

Finished Parsing SubForum:Builders - Toolset : Topics present:701

Finished Parsing SubForum:Technical Support : Topics present:809

Finished Parsing SubForum:General Discussion : Topics present:1274

Finished Parsing SubForum:Custom Content and Community Expansion Pack : Topics present:2411

Finished Parsing SubForum:Builders - Scripting : Topics present:2984

 

Which matches the main displayed counts.

Not sure where it got the 6000 number before.

 

Turned out it was due to the way I was threading.

It was spitting out the count, before it had finished  counting the last Sub Forum.

[omen_shepperd]

We also have members constantly discussing a wide range of topics  over on the NWN community group we have on facebook. We will continue to have great discussions as long as we can. 

https://www.facebook.com/groups/nwncp/
 

[meaglyn]

Thanks Baaleos! That will be excellent to have. I'll be sad to see this go though.

[Baaleos]

Hey guys,

Just an update - the Scraper is go.

Its able to scrape around 186 Topics per 5 minutes. (Size of Topics will make that rate variable)

Eg: It will iterate over all pages within a topic.

We have over 9000 topics, with over 100,000 posts among them.

So this might take a day or two to capture all the information.

 

Some screenies can be found here:

 

 

 

For the post content - I basically capture the innerhtml from the post_body per post element on a page.

This is the example content I capture.

 

<div itemprop="commentText" class='post entry-content '>   <!--cached-Sat, 30 Jul 2016 13:25:47 +0000--><blockquote class="ipsBlockquote" data-author="WiredWraith" data-cid="20270158" data-time="1464020440"> <div> <p>I checked and double-checked all the stats of my henchman and no negative status effects show up other than that annoying Movement Speed Decreased one.&nbsp; Made sure conversation options weren't enabling Stealth or Search mode too.&nbsp; She's just... bugged.&nbsp; How can I kill her to try and reset her?&nbsp; She appears to have some sort of "I'm dying but I can't die" flag set.</p> </div> </blockquote> <p>Try kill her in shadow plane cuz script make her ded for gud on mythlar level wher snakehead lady stay. Hench respawn in SoU suppose to happen next to wher last master was.&nbsp; Am thinking hench respawn in Ch1 at dwarf house mayb specal for module but shud work like I say in all other mods/area cept the wind04 (mythlar).&nbsp; Mayb u rite that her bic went corrupt.&nbsp; Kill/rez dun mostly to fix faction but mayb work to fix bad effect.</p> <p>&nbsp;</p> <p>I know too late now to help much but save LOTS LOTS in NWN SP.&nbsp; CC not debug real gud sometime so it alway a gud idea to save so not have to redo lots.&nbsp; Guess mayb SoU have some bug left too. <img src='https://lvlt.forum.b...)</span></span>' class='bbc_emoticon' alt=':('>&nbsp; PW is better debug cuz alway want many ppl there so will fix bug very fast.</p>     <br>   </div>         <div class='ipsLikeBar right clearfix' id='rep_post_20270217'> <ul class='ipsList_inline'> <li id="like_post_20270217" class='ipsLikeBar_info' style="display:none">   </li>   </ul> </div> <script type='text/javascript'> ipb.global.registerReputation( 'rep_post_20270217', { domLikeStripId: 'like_post_20270217', app: 'forums', type: 'pid', typeid: '20270217' }, parseInt('0') ); </script>             <ul id='postControlsNormal_20270217' class='post_controls clear clearfix'> <li class='top hide'><a href='#ipboard_body' class='top' title='Back to top'><img src="https://lvlt.forum.b...)</span></span>" alt=""> Back to top</a></li>                                </ul>  

 

 

Its my hope that by saving the html source - we can keep image links intact.

Unfortunately it pulls through some bioware javascript code. That can be removed after we scrape it all though.

 

 

The progress bar is increasing, albeit slowly.

[kamal_]

Can you also scrape the nwn2 forums please?

[azaz1234]

this is great work that's being done to preserve the content already on the bio boards !

 

however, as a few posters have pointed out, there's still quite a bit of activity on these boards. there's good info coming out here every single day, new ideas, and new creative offerings. making sure we have an archive of existing content is an excellent idea, but i think doing only that and nothing else would be tantamount to consigning nwn to davy jones.

 

is it just me, or do other people also see a need for a continued, active forum ?

 

one poster suggested the nwn forums at nexusmods. i took a look and there are something like... 30 posts on that forum [which isn't to say it couldn't grow... ]  does anyone know if they could handle a forum similarly-configured to this one ?  i.e., categories for general discussion, the games themselves, scripting, custom-content developers, etc.

 

another obvious location would be http://www.neverwinternights.info ; it's already a central repository of good info on nwn, and there's a forum link -- albeit a link that happens to point here at the moment. so similar question to the above : does anyone know if that site could host a phpbb ?

 

imho we can't give up this valuable resource !

 

thoughts, ideas ?

[Cassandra Saturn]

try uBSN: http://biowaresocial...ork.boards.net/

[Neverwinter_Knight77]

I think the local vault and server vault players were pretty much two entirely different communities, so y'all don't know me (local vault roleplayer), but... Through all the ups and downs over the years, it's been good.

[Proleric]

I suggest the new vault as the best place for future discussion.

 

It's controlled by the NWN community. Most of us are there already.

 

The Nexus isn't a friendly place, in my experience. I don't think a relatively open community like ours would feel welcome there.