251 réponses à ce sujet

[leo_x]

Zarathustra217 wrote...

Whatever we do, let's see if we can make it something community wide without turning at each other in pedantries. It might be just me, but I think we could achieve more with doing a launcher than modifying the binaries. Still, it should be something simple yet adaptable and able to serve individual needs. Visually, the most neutral solution is just using textures from NWNs own GUI.

I'll see if I can persuade the guys from our community that were working on something similar to come here and join the discussion. Then I propose we set up something on sourceforge or GitHub.

From the top of my head, I can think of these features that would be generally useful to all:

1. Auto-downloader/installer for:
- the critical rebuild patch.
- HakPaks/tlks for specific servers.
- Single player modules?
- Music packages?
- Portraits?
- Overrides?

2. Server lists, possibly with player count.

We could then make a simple web-hosted form that submits XMLs (or whatever we use).

Well I would love to see something like this happen.  I was messing about with the idea...  Lua is the language I hack stuff in so the format I worked up was like this:

Server {   
   ip = "1.1.1.1", -- or hostname whatever.   
   port = 5121,   
   mod = { "optional uri to download module for SP", "md5" }   
   name = "Game/Server Name",   
   tlk = { "optional uri to tlk", "md5" },   
   homepage = "optional uri to homepage",   
   manifest = "optional uri to manifest as used by the old CEP2.3 updater..."

   Hak { "optionaluri to hak 1", "md5" },   
   Hak { "optionaluri to hak 2", "md5" },   
   Hak { "optionaluri to hak 3", "md5" }
   -- .. etc
}

I think a good analogy would be to make it like Magnet links for NWN, that downloaded and installed all the haks/tlks you need to play.

Website operators (there could be more than one) could accept these files and maybe a Markdown file with your servers description.  If people wanted a list of every server in the universe the website could offer them all zipped for download.  

Modifié par pope_leo, 08 décembre 2012 - 11:36 .

[Zarathustra217]

Sure, magnet links would be nice, but I think that given the relatively limited amount of servers still active, having them all included with the installer is realistic - and the same with other content really. You could have several thousand entries without it adding significantly to the download size. It'll also help the smaller servers and start out projects by not having to rely on people finding them by searching the web, so that people could explore all the community content of NWN in one easily accessible place.

[leo_x]

Zarathustra217 wrote...

Sure, magnet links would be nice, but I think that given the relatively limited amount of servers still active, having them all included with the installer is realistic - and the same with other content really. You could have several thousand entries without it adding significantly to the download size. It'll also help the smaller servers and start out projects by not having to rely on people finding them by searching the web, so that people could explore all the community content of NWN in one easily accessible place.

Yea, I see what you mean.

[Zarathustra217]

And thought about md5 check as well, but I also think we really should focus on making it forward for the content makers too. Since I think it would be too much to depend on centralized hosting of files, we would need people to manually fetch the md5s. Encouraging people to include a version number in filenames is probably simpler.

It's somewhat the same reason I propose XML - it's so forward to use and so well-established a standard.

Modifié par Zarathustra217, 08 décembre 2012 - 11:44 .

[leo_x]

No reason the md5s couldn't be optional... I agree that XML would probably be best.

[Zarathustra217]

Ah, yeah, md5 as optional is surely the way to go.

[Skipper24]

So gamespy pulling the plug and the gamespy chat/server lists for NWN 2, going down has also effected Nwn1 aswell?

This is terrible.

I dunno anything as far as putting something together to fix the problem/replace it but I know this much, people still buy Nwn1 and 2. What ever solution you people, you the ones who will hopefully take up this challange, with the knowledge skill and expertise to do so, will take into account, a new player will  click multiplayer, go through the login thing and then realise there is no connection too the server listings.

IF you want both games to continue to survive you need something to come up so these new people know, were to go to find servers or a specific server and so forth. Otherwiset they'll turn away from the game and that doesn't help.

What ever the solution is.. the dev's of either game should hopefully step up and do something so then the player community can take over.

It sucks but this is exactly what happend to one of the best space sims, Freelancer. Microsoft hosted the global server listings when that went down.. new players didn't know what to do.

Modifié par Skipper24, 08 décembre 2012 - 12:30 .

[Zarathustra217]

Yeah, it would be preferred if we could make something basic relatively fast so the current players don't abandon the game.

As for the actual coding, any proposals for a development framework? Have anyone had any experience developing with Mono? http://mono-project.com/

[Melkior_King]

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address. We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated. Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

I expect it would take me only an hour or two to cobble together a "hosts" file updater to do the same job and to make it foolproof (so the same entry doesn't get added twice or more).

It might take me a day or two to put together a version which would download the information from a known location and update it automatically, if that's an option.

[Visavant]

Melkior_King wrote...

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address. We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated. Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

I expect it would take me only an hour or two to cobble together a "hosts" file updater to do the same job and to make it foolproof (so the same entry doesn't get added twice or more).

It might take me a day or two to put together a version which would download the information from a known location and update it automatically, if that's an option.

This is a working server that is receiving gamespy updates from one of the NWN1 servers talking to it, the only edit was the hosts file change. The output you see on screen is to put the data also into an XML format to output to a webpage so that we have listings immediately seen on a website while I work on the game client's internal listing.

I've written an application that patches nwmain and nwserver for both NWN1 and NWN2, but also gives people the option to simply click a button and it modifies only the hosts file.

Modifié par Visavant, 08 décembre 2012 - 03:14 .

[Baaleos]

I was bored, so came up with a C# App that uses the cached/legacy info from the master server.
Using this app, players will have visible access to about 200-300 servers.

The info coming from here.
https://dl.dropbox.c...mespy_cache.htm

If we had a community website to point it at - we could have more updated info.


http://nwvault.ign.c....Detail&id=1540

The application and Source has been uploaded to the vault - awaiting approval.
Note - the app was coded very fast, so its bound to have bugs.
But the general usage is

Turn it on, wait a few seconds, and it should populate with server rows etc

Red Rows = Offline Servers, or Servers that just couldnt be communicated with.
Green = Online Servers

The Button on the right will perform a few actions.

1. It will look for the nwn install directory in the windows registry - some windows permissions may affect this - run as admin if u get errors.
2. It will start NWN up and give the server address to the application.
3. It will use a Memory class I wrote for my RhunDownloader application, this class modifies the nwmain.exe in memory, to disable the MasterServer authentication - so you should be able to jump right into the game with no wait time.


Some of those functions require elevated privilages - works without issue for me.



Note - Im not sure why, but some servers were not responding to the UDP query at all.
Which is odd - cause other servers would return a message to say offline or some sort of exception.
In the case of the weird ones - they just froze the udpClient receive method.
As result - I had to implement a 10 second max timeout for each server query.

If you see the server updating seemingly freeze, its just cause I have the max threadcount at 5, once the 10 seconds per thread timesout, it should free up threads and continue updating the table info.


Until the vault entry is approved u can get the binary from here.
http://31.132.2.110/...erverJoiner.exe

Note - I cannot promise to be making frequent updates to this, this is just meant as a proof of concept really.
Plus I was bored - so I made it to occupy myself.

Modifié par Baaleos, 08 décembre 2012 - 05:11 .

[Baaleos]

I just read the previous posts about a community updater/loader application?
Sounds interesting.


My own 2 cents on the subject (since its something I've been using for 2 years or so)

I upload my files to a webserver, (a folder on the webserver)
I call a URL when finished - with arguments pointint to that folder
It processes all the files in the folder.

It takes info such as

FileName,
Extension,
Size in bytes,
md5

It compares the md5 against any already existing entries in the database.
Updates as necessary.

Then, once per hour or when triggered manually,
another php file turns the mysql data into an xml

<?xml version="1.0" ?>
- <Files>
- <File>
<FileName>219.bmu</FileName>
<Size>2561160</Size>
<Date>1277584559</Date>
<MD5>ee61da9193b63333ca03df4ac6259bf3</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>ffxiii_dust2d.bmu</FileName>
<Size>3662064</Size>
<Date>1277089435</Date>
<MD5>0d1c1af6665dce0d41e2f2e4f01fa30e</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>229.bmu</FileName>
<Size>3305545</Size>
<Date>1277584693</Date>
<MD5>7003eda8ea94bf0af41a501e2bb8b85c</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>202.bmu</FileName>
<Size>2720820</Size>
<Date>1277584375</Date>
<MD5>1ae1cd90576c39d132a4a62daa842c96</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>mus_def_sluagh.bmu</FileName>
<Size>3831671</Size>
<Date>1277089722</Date>
<MD5>afa680c09d688f1ecee261efe074cc09</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>212.bmu</FileName>
<Size>5256997</Size>
<Date>1277584523</Date>
<MD5>03cc78ce65a79a9d134ca09077fd5d56</MD5>
<DELETE>FALSE</DELETE>
</File>

This XML is downloaded onto players machines, then parsed by my updater application.
Files that are marked as Delete = true
are ones that dont get downloaded, and get removed if they exist.

MD5s that dont match, are redownloaded fresh
And the file extension determines the location towhich they are transferred.

Although, in my case, I have some strong customizations.

eg - Instead of downloading wav files, I download mp3 files, then use ffmpeg - bundled in the executatable, to convert to wav.
Result is that it shortens download times - a wav file would be abour 3-4 mb for 30 seconds of audio or less, but ends up being 300-500kb as mp3.

I also recently introduced rar capabilities - that allows me to bundle override content, into a rar file - it gets downloaded, unrarred, and content transferred to override.

Also same for tlk files - I made a new extension called
rtlk (rarred tlk files)

The Cep2.3 tlk file being 7mb or so - ends up being about 400-500kb when rarred.


Now - in my case- I did alot of funny steps there, such as using the override directory etc alot, because my application is aimed towards allowing people to enter without needing the haks.
Hak content = compulsory
Override = Voluntary -(they might not see or hear the content, but they can still enter the server)


Hope this gives you guys some ideas?

[Zarathustra217]

The issue with using just regular C# is that it won't immediately support other platforms. NWN is one of the few major games that'll both run on Mac and linux and we shouldn't leave them out.

And yeah, the utility should definitely have at least zip capability (though rar is of course preferred) because most NWN content compile incredibly well.

I fundamentally believe though that our highest priority should be to make it as simple as possible for the end users (both players and content providers). NWN is an aging game and if there's too much hassle, it'll easily scare off people. If we can avoid manually editing the host file, I feel we should aspire to. By same reasoning, making file size, date and md5 optional makes it simpler.

In terms of how to manage content, wouldn't it be far the simplest solution if any package should have the same folder structure as NWN, so it simply extract to the NWN directory? (e.g. so that hakpaks are in the hak subfolder, talk tables in the tlk, override files in override etc.)

Also, I think it might be a good idea to allow for specifying some content as optional - so a server may have an optional override, music or portrait pack.

[leo_x]

Sounds kind of like you rolled your own rsync? I wonder ...if you could use nwnpatch.ini to load a hak client side while having the stuff in the override on the server or if client would ignore them if they weren't in the servers hak list. Not sure if that would even make anything more manageable tho.

Don't know if I will bother working on anything since it looks like it will be covered. I just wanted to be apart of the voices that say this is opportunity to make something better, something that makes every server and all the great custom content out there one-button accessible to players. My old nwnx_haks plugin was on kind of the same lines.

[Zarathustra217]

Oh, and a strong favouring point of not simply modifying the binary or using a custom host file is that a community made launcher would allow for auto-downloading and general content distribution.

[Rolo Kipp]

<*snoring softly*...>

Baaleos wrote...
...
If we had a community website to point it at - we could have more updated info.

http://nwvault.ign.c....Detail&id=1540

The application and Source has been uploaded to the vault - awaiting approval.
...

<boss! boss! wake up!>
*snort*...*grumble* <get *up* old man!>
*mumble* *grumble* mummy... <sheesh... *wing-slap*>
What! Where! I didn't do it! <that's the point *raven scowl*>
What are you blathering about now, bird. Where's my slippers? <i'm a raven, not a puppy. get up>
Why? Where's the fire? <not under you, evidently. big b has a project to approve on the vault>
Well, why didn't you *say* so?! You *mumble* that's what the problem is <*beak gapes in astonishment*>
What? <nothing, wizard>

<...and dreaming of good things to come>

Modifié par Rolo Kipp, 08 décembre 2012 - 06:04 .

[Visavant]

Doing something similar for another community as well, though I found that md5 checksumming on client machines with multiple files exceeding 200 MB to be problematic with older hardware. Serverside, I would still calculate the md5sum, though I entered in the file size and last modified entires which the auto downloader would launch on, only using MD5 when there was known corruption in a file.

Baaleos wrote...

I just read the previous posts about a community updater/loader application?
Sounds interesting.


My own 2 cents on the subject (since its something I've been using for 2 years or so)

I upload my files to a webserver, (a folder on the webserver)
I call a URL when finished - with arguments pointint to that folder
It processes all the files in the folder.

It takes info such as

FileName,
Extension,
Size in bytes,
md5

It compares the md5 against any already existing entries in the database.
Updates as necessary.

Then, once per hour or when triggered manually,
another php file turns the mysql data into an xml

<?xml version="1.0" ?>
- <Files>
- <File>
<FileName>219.bmu</FileName>
<Size>2561160</Size>
<Date>1277584559</Date>
<MD5>ee61da9193b63333ca03df4ac6259bf3</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>ffxiii_dust2d.bmu</FileName>
<Size>3662064</Size>
<Date>1277089435</Date>
<MD5>0d1c1af6665dce0d41e2f2e4f01fa30e</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>229.bmu</FileName>
<Size>3305545</Size>
<Date>1277584693</Date>
<MD5>7003eda8ea94bf0af41a501e2bb8b85c</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>202.bmu</FileName>
<Size>2720820</Size>
<Date>1277584375</Date>
<MD5>1ae1cd90576c39d132a4a62daa842c96</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>mus_def_sluagh.bmu</FileName>
<Size>3831671</Size>
<Date>1277089722</Date>
<MD5>afa680c09d688f1ecee261efe074cc09</MD5>
<DELETE>FALSE</DELETE>
</File>
- <File>
<FileName>212.bmu</FileName>
<Size>5256997</Size>
<Date>1277584523</Date>
<MD5>03cc78ce65a79a9d134ca09077fd5d56</MD5>
<DELETE>FALSE</DELETE>
</File>

This XML is downloaded onto players machines, then parsed by my updater application.
Files that are marked as Delete = true
are ones that dont get downloaded, and get removed if they exist.

MD5s that dont match, are redownloaded fresh
And the file extension determines the location towhich they are transferred.

Although, in my case, I have some strong customizations.

eg - Instead of downloading wav files, I download mp3 files, then use ffmpeg - bundled in the executatable, to convert to wav.
Result is that it shortens download times - a wav file would be abour 3-4 mb for 30 seconds of audio or less, but ends up being 300-500kb as mp3.

I also recently introduced rar capabilities - that allows me to bundle override content, into a rar file - it gets downloaded, unrarred, and content transferred to override.

Also same for tlk files - I made a new extension called
rtlk (rarred tlk files)

The Cep2.3 tlk file being 7mb or so - ends up being about 400-500kb when rarred.


Now - in my case- I did alot of funny steps there, such as using the override directory etc alot, because my application is aimed towards allowing people to enter without needing the haks.
Hak content = compulsory
Override = Voluntary -(they might not see or hear the content, but they can still enter the server)


Hope this gives you guys some ideas?

[Baaleos]

It is true - calculating md5's can be tricky - in order to calculate it, you need to read the entire files contents.
Reading 200mb + files x (amount of files)
could cause problems.

Unfortunately, I cant think of any other way of detecting changes in the files besides the md5, or some sort of relational version tracking similar to how svn or cvs works.

Keeping track of file size 'might' be sufficient - if we are talking about bytes, then chances of us having 2 versions of the file with exactly same file size are rare.

[Visavant]

We compared on the combination of filesize and last modified before resorting to md5.

[tzaeru]

Oh hai, seems I'm promptly killing my 8 year lurking spree, but here goes:

Melkior_King wrote...

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address. We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated. Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

Is the GameSpy IP in a data file? If it's directly in the executable, then the closest to a legal way to achieve that is to intercept the connection attempt to GameSpy's server and modify the IP on the fly.. But even that might fall under reverse engineering and be promptly illict, except if you happen to live in a country where reverse engineering for the purpose of maintaining compatibility and functionality is legal.

Baaleos wrote...

It is true - calculating md5's can be tricky - in order to calculate it, you need to read the entire files contents.
Reading 200mb + files x (amount of files)
could cause problems.

Unfortunately,
I cant think of any other way of detecting changes in the files besides
the md5, or some sort of relational version tracking similar to how svn
or cvs works.

I think the best way is to maintain version history in a data file and just do the baseline installation from scratch when the launcher/updater software is used for the first time.

So, for example, a simple resource for the files could look like this:

-1.1:
hak/some_server.hak
-1.0:
hak/some_server.hak
hak/cep_stuff.hak
tlk/some_server.tlk

Once the baseline installation (version 1.0) is done, the launcher/updater just marks down the installed version and when a new version is in, installs the files from that.

For convenience, there can be a simple options checkbox for telling the launcher/updater to assume that base installation is currently in so old players don't need to redownload everything.

Modifié par tzaeru, 08 décembre 2012 - 11:04 .

[MrZork]

I just want to say how impressed I am at how quickly you guys have come together with ideas and even implementations. And to say thanks.

Baaleos, is your C# project one that I could compile with the .NET Framework SDK? I just want to recompile with ServerBrowser.cs altered to point to a server list to which I have added a row for the server I've been playing on.

BTW, one potential fly in the changing-the-hosts-file ointment is that it may cause trouble with other (newer) games that still use gamespy... Perhaps the wrapper approach would allow use of the in-game server list without that issue?

[Visavant]

Unless the new game is also called nwn, it won't conflict.

nwn.master.gamespy.com

[Baaleos]

MrZork wrote...

I just want to say how impressed I am at how quickly you guys have come together with ideas and even implementations. And to say thanks.

Baaleos, is your C# project one that I could compile with the .NET Framework SDK? I just want to recompile with ServerBrowser.cs altered to point to a server list to which I have added a row for the server I've been playing on.

BTW, one potential fly in the changing-the-hosts-file ointment is that it may cause trouble with other (newer) games that still use gamespy... Perhaps the wrapper approach would allow use of the in-game server list without that issue?

Im not all the familiar with the .Net Framework SDK

The project on vault was compiled in VS 2010 - 
but should also compile in VS2010 Express


The Contents of the datagrid are come from an external website (a cached version of the master server list)


In theory, you could modify the DelayedStart method inside the class.

private void DelayedStart()
{
Thread.Sleep(2000); //Delayed Start to give windows time to get the form to display
string sText = GetMasterServerList(); 
string[] sSpliter = { "<tr>" };
string[] sRows = sText.Split(sSpliter, StringSplitOptions.RemoveEmptyEntries);
//MessageBox.Show(sRows.Length.ToString());
int i = 8;
string[] sSplit2 = { "<td class=\\"td1\\">" };
for (i = 8; i < sRows.Length - 1; i++)
{
string sRow = sRows[i];
string[] sColumns = sRow.Split(sSplit2, StringSplitOptions.RemoveEmptyEntries);
string sIP = sColumns[0].Replace("</td>", "");
string sPort = sColumns[1].Replace("</td>", "");
string sName = sColumns[5].Replace("</td>", "");
string[] sRowToAdd = { sName, sIP, sPort };
AddRow(dgServers,sRowToAdd);
}
//MessageBox.Show("Done");
ProcessList();
}

If you dont care about the master server list, and just want your own server in there.
Just change
string[] sRowToAdd = {"Your Server Name", "Your IP", "Your Port"};

Also - u can take it out of the loop, if you dont care about the rest.

private void DelayedStart()
{
Thread.Sleep(2000); //Delayed Start to give windows time to get the form to display
string sRow = {"ServerNameHere","IP Here","PORT HERE"};
AddRow(dgServers,sRow);
}

[MrZork]

Visavant: Cool, I didn't know GameSpy had things broken down into per-game subdomains. Good to know.

Baaleos: Thanks. I will give VS Express a shot.

[Melkior_King]

tzaeru wrote...

Melkior_King wrote...

The simplest solution (from the player/server point of view) is for someone to make a server which does the same thing as GameSpy (except only for nwn) and give us the IP address. We can then enter that IP address along with the name of the GameSpy nwn server into our local "hosts" file to redirect the nwn GameSpy calls to the new site.

The adding of the IP address and server name to the local "hosts" file could easily be automated. Distributors such as gog.com could add the code to the installer for nwn Diamond Edition.

Is the GameSpy IP in a data file? If it's directly in the executable, then the closest to a legal way to achieve that is to intercept the connection attempt to GameSpy's server and modify the IP on the fly.. But even that might fall under reverse engineering and be promptly illict, except if you happen to live in a country where reverse engineering for the purpose of maintaining compatibility and functionality is legal.

The GameSpy server base address is saved as a hard-coded string (master.gamespy.com) inside the exe files (nwmain.exe and nwserver.exe).  Changing that string will change the site which the game tries to access.

The game adds the nwn prefix to the base address before doing the dns enquiry so it ends up as nwn.master.gamespy.com

You can either edit the string inside each exe file so that the DNS servers resove to a different site, or you can enter the IP address along with the above site address into your local hosts file in order to redirect the game to a different site.

In short, this makes it fairly easy to redirect the GameSpy requests to a different site.