Im getting error 500 often these days, also the site is incredibly slow for me (im from middle-Europe).
Also, when I tried to write a new forum post there it didnt get saved. After I pressed the save button I get a new form to write a comment without any text and no new post appeared at the topic.
I find it pretty slow too and sometimes not too good... but aside from learning Drupal and giving Rolo a helping hand or donating what is there to be done? I'm not sure if the site could be 'open sourced', but if so I know some people that want to learn/work on Drupal.
<counting the damn few...>
The core of the problem is really quite simple (if no less difficult to correct); I'm broke and the site is cheap.
Basically, I am hosting a quite complex, MySQL-intensive site on the cheapest shared hosting I could find. I believe there are (a possibly flexible number) 4 other sites on the particular machine I am hosted on. When the combined traffic gets high, the database queries time out (those are the 500 internal server errors).
There are two relatively easy fixes to this problem. Move to a better server with the same company or find a dedicated server elsewhere.
In the past 18 months the Vault has received $150 in donations, leaving another $50 for me to make up out of pocket. I can do that, but I cannot do the $600 a year (approx) that a VPS would cost. Nor have I had any sterling gentlemen offer to host the Vault on their spare, idle t1 line :-P
Re opensourcing drupal: Leo, that is what drupal is all about! :-) I gladly accept help where and when I find it, and am quite willing to tutor newbies who want to learn what I know, whether it is drupal or 3DS Max or Nwscript :-P
<...chickens that hatched>
Yea, I suspected it. Unfortunately I cannot help in this regard in any way.
So the question is whether this community has a resources to maintain a site like vault on our own. The ultimate goal was to move everything there, but now we got only 1/10000 of content there and its already too much. If we dont have these resources, it would be better to abadon this idea than losing all the content in a future. There is still one other choice and thats nexus - ultimately the only reason why Im not using nexus for my work is that many peoples from this community doesnt support it and because a new vault was created. However we should reconsider nexus again - or is there a way how to keep old vault (better choice as no need to transfer anything)? I get the impression its going to be down in near future. Maybe if the donations we are giving to the new site would be sent to the old vault owners it would helped.
<cleaning the wax out of his ears...>
Now, Shad, I just gotta clarify a couple things...
1/10000 of the 40,000 projects on the old vault would be... *carefully counts out fingers and holds up four* four. I think there are about 400 on the new Vault, or about 1%.
Was only guess, I dont have an exact numbers as they are afaik not generally available. Still its not just about the number of project, also a number of comments, users and files. What we have now on new vault is definitely way less than 1%.
Re abandoning what we have so we don't lose it?! You lost me there. Completely.
I suggest that if the solution we tried to prepare is above our limits that we should abadon it. Better now than after four years of working on that site. How much time you, TAD and users put into running this site already? How much money? This is also probably reason why users dont move their projects from old vault into new one. As of now, the old vault seems like a more stable platform to be honest. By moving projects from old vault into new one, people risks that the time and effort they'll put into this will be lost again.
And perhaps you can explain better how taking away what few (greatly appreciated!) Donations I get will *help*!
Same argument as above. Whats the point of donating to something we have no real expectations to get running the way we need. Lost money imo.
Of course, this is based on a suspection that we don't have the needed resources to keep new vault running in the way we need - forums, users, project, terabites of datas, good connection. Maybe I am wrong and there are peoples in this community who can fund all of this ad infinitum. Please prove me wrong.
Ad. nexus: Agree the support for NWN is bad. However maybe the owners can help us if we tell them what do we need? Did anyone tried that already?
Hey Rollo, just got a "The card you entered cannot be used for this payment. Please enter a different credit or debit card number." error when I went to donate to the Vault with Paypal.
There's nothing wrong with my card, and I used it a few minutes ago on another site. Annoying. You can accept non-US paypal donations, right?
Do you have a Bitcoin address for donations?
Also, if bandwidth is an issue, could we use Bittorrent to do the heavy lifting for frequently-requested files? I'm sure a lot of people have torrent clients already, and have copies of the popular stuff on disk.
Yeah, torrents would definitely be a help. Could also use google for site search.
I would suggest that the more accurate reason why people aren't migrating their files from the old vault to the new one is because they are no longer part of the community. They have moved on, hence the initial run of volunteers mining old projects and migrating them instead.
I don't know if bittorrent would be a viable solution; it has this pesky necessity of someone having the data readily available at any one time and an even more pesky likelihood to eventually die out with decreasing demand for content. Demand for NWN content really doesn't have the demand necessary to start off with.
I'm not sure an abandon all hope ye who enter here outlook is productive but, as another old saying goes, nothing comes for free. I would think having a central option and discussions toward a more stable solution is paramount, and I would also think that a little vault access issue is probably a lot better than having to hunt down a myriad of dropbox or other "cloud" services- some of which are even less permanent unless every person were to invest in premium accounts and are already on shaky ground when you consider certain similar services subjected to government take downs, causing people to loose access to all their files, the irony being some government employees having lost theirs as well.
If one were to champion a move to drop it and kill all investment and support, the question remains: what then?
IGN is questionable at best and on borrowed time at worst. Nexus doesn't seem to be a viable or popular option. Gamefront is horrendous and Filefront isn't far behind IGN and extremely buggy the past year. So what then? Hit each other up by PM when we need a file?
Rolo took something on himself as an attempted solution to a dilemma and, were I in a more comfortable financial place, I'd be glad to contribute. As it is, I'm not, but I'm not resorting to "give up, dude" either.
If one were to champion a move to drop it and kill all investment and support, the question remains: what then?
As it is, I'm not, but I'm not resorting to "give up, dude" either.
Im not doing that, I am happily user of new vault and I support it by posting new stuff there and writing suggestions/opinions on forum. But the tech issues it has now made new vault unusable, the site is not working well for most part, every second request ends up with error 500, its impossible to upload something there because it might fail when saving as happened to me. Was only sending a reply to the forum for the ten times till it get sucesfully posted.
The VSP will be needed. So the real question is if this community is able to get 600$ together each year. If not, I fear this is lost cause and the efforts we've put int one vault are lost.
what then?
I actually have an idea. We dont really need to move the stuff from old vault. We can use the Omnibus solution. Archiving all the projects into downloadable program. Only files have to be somewhere and the links must be changed to the new location. Still just hosting the files is cheaper than hosting a live site with all the datas.
How much of the cost is storage? Maybe it would be better to purchase storage in a cloud service (e.g. Google Drive) - can get 1 TB for 9.99$/month.
Can we reconsider ads? They could be implemented in the way that there is a button "Turn ON the ads" and those of us that would like to could finance the site by turning the ads on?
Rolo, have you checked to see if you're getting throttled at all by your host?
There is a price war going on between all the cloud providers, so any prices you researched for file storage even a couple weeks ago would be way off today's prices (like the figure Zarathustra quoted used to be 50$ for 1 TB). I'm guessing there could be some nice space savings by working out a way to deduplicate old vault content too... even converting all zips and rars to 7z would help.
I think ShadoOow has a good idea there, put the old vault up as a blob (or torrent), focus the new vault on new stuff...
<pouring on a little...>
Frankly, Z's price for storage rather floored me, although google drive simply won't work. The account has to be able to accept uploads from multiple sources, not a single machine. Still, that very encouraging and I'll look at options. If I can get 1tb for $120 a year, I'd be willing to cover any shortfall out of pocket, as I do with the site hosting.
Let me get back to a few posts I didn't squint long enough to address earlier :-P
@ruadhri: I've never heard of Bitcoin. I'll research it. We have several european and australian sponsors who have never mentioned difficulties donating, but maybe they were just being stoic...
On the torrent: a true torrent would flagrantly violate our terms of use on this host, and very few site hosts are willing to host torrent servers (as opposed to clients) without specific arrangements (and money). And without a good population of clients, the server would be required.
Pain's pseudo-torrent idea (genius!) can take advantage of volunteer repositories of flexible size and inconsistent availability, but does not depend on them. I still want to go that route.
@Werelynx: I personally abhor ads for a non-profit community site. They just rub me wrong. I mean the ads for diapers, muscenex and deodorant. I actually am loving the idea (I think it was Shads) of replacing the What's New slideshow with *NwN* specific ads for servers and mods.
But, having stated how much I hate ads, I also said a loooong time ago that I'm building the Vault for the community and not for me. If enough of the vested members of this community want to monetize the Vault, we will monetize the Vault.
And now I want to mention something that may not be obvious to casual or new observers. There are a lot of people helping on the Vault already, and a lot of them are doing it so quietly and humbly that I am both astounded and grateful whenever I am somehow reminded of these background heroes. Specifically, I want to mention a few (only a few) that I feel have been under-appreciated by the community.
First, I want to recognize Shadooow. He is very active on the Vault, despite the tone of this thread. And his work, not only selfless, but against opposition, on the CPP is nothing short of incredible. Let no one think I have anything less than immense respect for the guy.
Second (probably the only place he's ever come in second!) is The Amethyst Dragon. The guy amazes me! Any time I start to feel over-burdened or buried alive, I look at what he's done, what he's doing and what he promises to do. If he ever drops buy HQ, I wouldn't buy him a coffee, I'd stand for a Venti java chip frap with double espresso blended in! =) What most people don't know is that he really stepped up - on the old Vault handling the ads system (until it finally broke), and recently, when I was buried with work, on the new Vault, just quietly handling business. Oh, yeah, and he took over the CCC when it really counted.
There are lots of others I could mention, but I'm only going to highlight three more.
Tarot Redhand singlehandedly mirrored at least a hundred high-value projects from the old vault to the new. He's still involved in preserving that content. Still working for *us*, without pay, with little recognition. I owe you a coffee, if you ever get over this side of the pond, bro.
Werelynx, the other crazy cat. This guy pestered me so much with corrections and pointers and links that I finally just made him an administrator. He's the guy working in the background finding broken links and spam and funny business and setting them right. But when you think of the Vault admin, you think of Rolo.... *shakes head in wonder* It takes a village to raise a Rolo =)
Finally I have a ninja old-timer for you. Fester Pot. The guy who's so quiet it takes you 20 minutes of intense googling just to find his blasted blog! Who's incredible mods are only considered "amateur" because he offers them free. *blink blink*. What you may not know about FP is the sheer *hours* he's put into moderating the Vault, fixing things and putting out fires.
Now, a thread about the difficulties of the Vault may be a funny place to recognize those contributions (oh, and I mean it, any one of you that drops by HQ gets a coffee on me. Except TAD. He gets a frap), but I did this to point out two things. First I don't think anyone is giving up on the Vault any time soon (though Shad has valid points highlighted by the fact we need to air our dirty laundry here instead of there :-P ). Second, a lot of this community is vested in the site through means other than money. That is incredibly valuable and why I keep finding ways to make up the financial shortfall.
Still, if we could get a php guru onboard, that could be very good. :-)
Heh, better get back to work. Have a little contract going live this afternoon :-P
Edit: There were 1436 different files downloaded from the new Vault in March. I don't have a count per file, though.
There are 607 active projects.
Site is using 116gb of disk space
Database is 1.2gb
<...catch-up>
I haven't looked overly much into it, but there are several plugins for drupal to make it integrate with Google Drive. Might be worth checking out 
<feeling...>
Update: Just upgraded to VPS. It will take a couple hours to migrate everything, but the first week's free and if this doesn't solve the issues, I'll downgrade and re-think :-P
Edit: *thump* <the sound of an old wizard falling over in a dead faint when *someone* digs deep for the cause... for the "foreseeable future">
If *that* person comes by HQ... I just might have to buy... *tries to think of something more special than a loaded frap* ...something *good*! :-)
<...broke>
Im not doing that, I am happily user of new vault and I support it by posting new stuff there and writing suggestions/opinions on forum...
Whats the point of donating to something we have no real expectations to get running the way we need. Lost money imo.
I always have to remind myself that English is not your primary language, and that's why I feel that your statements seem conflicting at times. I certainly don't question your contributions and do consider you a valuable contributor even if I don't necessarily agree with your approach all the time.
I would think that, rather than abandon anything, it would be beneficial to mitigate current issues while discussing possible solutions that the desired resources can be translated to. As it is, there are things on the new vault that never saw the old vault, and that would be enough incentive to maintain, for the time being, a means of mitigating new vault issues until such a time that those resources can also be transferred.
(Shadooow has) some valid points but I'm afraid, or saddened, that it really doesn't make much difference.
The primary cost is the file storage.
Honestly, the premium solution would be to finish what Pain started in developing the pseudo-torrent vault file system. It is based in php and both fast and robust. It allows distributed, but centralized file service and we already have offers of file repositories around the world.
This goes toward what I comment on initially. Nothing comes for free. The downfall of a torrent system I mentioned is also a plus depending on how one works it, but there is still a cost association with insuring a readily available seed. For now, the most viable solution for torrent would be investment in a seedbox. With a seedbox, the files are always there, can also be accessed via ed2k links (were one to desire avoiding ratio issues), and generally have high throughput. Seedboxes still have a cost association, however, and the cheaper ones are saddled with limitations like an active torrent limit and limited storage.
For this community, an active torrent limit wouldn't be too debilitating because the number of file requests would be fairly limited to begin with but the storage of the sheer amount of resources would easily bump that monthly cost up.
As Rolo points out, the largest issue is storage (as opposed to bandwidth) regardless what solution is settled upon- especially if one strives to keep this independent from larger sites like the Nexus.
I've never heard of Bitcoin. I'll research it. We have several european and australian sponsors who have never mentioned difficulties donating, but maybe they were just being stoic...
Pain's pseudo-torrent idea (genius!) can take advantage of volunteer repositories of flexible size and inconsistent availability, but does not depend on them. I still want to go that route.
Bitcoin, while new and pretty and sworn by the internet community, seems a bit sketchy. That may just be me and be because it's new but, if recent bitcoin issues are any indicator, a virtual currency not tied to any real-world money is problematic.
You'll have to refer me to this pseudo-torrent idea, though, because I'm not quite clear on what it entails and how it's different than torrent. I would still think something needs to be hosted somewhere in it's entirety which brings us back to square one.
Bitcoin, while new and pretty and sworn by the internet community, seems a bit sketchy. That may just be me and be because it's new but, if recent bitcoin issues are any indicator, a virtual currency not tied to any real-world money is problematic.
You'll have to refer me to this pseudo-torrent idea, though, because I'm not quite clear on what it entails and how it's different than torrent. I would still think something needs to be hosted somewhere in it's entirety which brings us back to square one.
Holding Bitcoin is sketchy, it bounces up and down like copper futures. But I wasn't suggesting Rolo hold any, just that I swing him $50 worth to a free address and he cashes it out at an online exchange the same day into his bank account. What's the worst that can happen over a few hours? He gets $45, or $55 - whatever... Or I could buy a USD prepaid credit card online, I guess, but that's not a very cost-effective way to send money.
The psuedo-torrent idea is genius, I'd love it if someone could finish it - uses far beyond our community! :-) My PHP skills are minimal :-/ but I'd still like to have a look
I'm curious what the pseudo-torrent idea is too. From what I gathered it's a system of (full or partial) mirrors set up so the vault could serve files from any of them to help defer bandwidth costs, but files wouldn't be broken among sources and reconstructed by a client... Is that right? Something like MirrorBrain? There is also http://metalinker.org It might be worthwhile investigating these instead, they're open source, used by some large groups, and are likely to be vastly more robust than anything anyone in the NWN community could make. They also have the benefit of being able to use torrents too and while there might not be many things in torrents, I imagine that at the very least server hosts would be willing to seed the custom content their own world uses.
<riding hard...>
Well, there've been some significant advances/retreats and flanking maneuvers the last 24hours.
We are now in the process of moving to a dedicated quad-core server with 32gb ram and 2x2TB HD. Turns out this solution is approximately $3 more than the VPS and orders of magnitude faster.
Niv, who some of you know as the guy who wrote the cool NwN Ruby library, is wading it at least knee deep (since I'm in a tad over my head).
In the meantime, the VPS has greatly reduced the 500 errors, but the site is insanely slow, using insane amounts of CPU and you can reply or comment on anything *except* forum posts.
My head hurts.
The psuedo-torrent thing is a lot like Pope-leo described with the addition that very large files *are* broken up into 100mb or 200mb chunks and torrented (if that's a verb). Also, the inactive files are stored on Repository zero, rather than One (the main vault site). So any file that is not accessed for more than a year moves to off-site storage on the Beast (Zero) and is only called up if needed.
Also, the whole thing is done in php for very fast, low-memory transfers with resume capability. Pain even released the php classes so anyone could write apps to access the project library.
Depending on whether I end up picking up the reins, I may indeed look into those links, PL. They look interesting. For now, I just want the site stable and safe.
<...with one foot on each tiger>
Retour en haut