28 réponses à ce sujet

[Shades of Night]

Hello! So I'm sure most people know about the forums closing now. I've seen a lot of posts about archiving bits, saving this, or moving here and there. Problem is everything is all higgledy-piggledy so chances are there will be duplicates of some things saved while others may be missed entirely. And whatever is archived could be stored anywhere...
 
One site I do know of that is specifically made for archiving sites is the Wayback Machine. Problem is as far as I can tell it does one page at a time... (if there is a way to make it save more, please tell me)
 
But this is where you come in.
 
I'm asking if anyone has some time, just a little, go to the Wayback Machine ( http://archive.org/web/ ) and start saving pages. However this will work best if you first make a post here specifying what section, sub-section, specific topics or whatever you're planning to save (because there are a loooot of pages) just so we don't get the same pages saved multiple times.
 
This would be a hugely daunting thing to take on alone, but if people work together and can break it up into bits I think it can be done.
 
And if it is done, we'll have an archive of the whole forums that anyone can access online.
 
Below is a list of all of the Forums, Subforums etc, I'll cross off any that are archived as people post about them.
 

Mass Effect: Andromeda - General Discussion
 
Mass Effect 3 -
        General Discussion
        Combat, Strategy, and Gameplay
        Story, Campaign, and Characters
        Multiplayer - Stickies
        Technical Support
        Archived

Mass Effect 2 - 
        General Discussion
        Combat, Strategy, and Gameplay
        Story, Campaign, and Characters - Suicide Mission Guide
        Technical Support
        Archived

Mass Effect -
        General Discussion
        Combat, Strategy, and Gameplay
        Story, Campaign, and Characters
        Technical Support
        Archived

Mass Effect Franchise -
        News & Announcements
        Lore Discussion
        Fan Creations
        Feedback & Suggestions

 

Dragon Age: Inquisition -
        General Discussion (Spoilers)
        General Discussion (NO Spoilers)
        Combat, Strategy, and Gameplay
        Story, Campaign, and Characters
        Multiplayer - Stickies
        Technical Support

 Dragon Age II  -
        General Discussion
        Combat, Strategy, and Gameplay
        Story, Campaign & Characters
        Technical Support
        Archived

 Dragon Age: Origins -
        General Discussion
        Combat, Strategy, & Gameplay
        Story, Campaign & Characters
        Dragon Age Toolset
        Technical Support
        Archived

 Dragon Age Franchise -
        News & Announcements
        Feedback & Suggestions
        Lore Discussion
        Fan Creations

 Dragon Age Keep -
        News & Announcements
        Feedback & Suggestions
        Tutorials & How To

 

BioWare Forum/BSN Help

BioWare Merchandise

 Legacy Games -
    Sonic Chronicles: The Dark Brotherhood
    Neverwinter Nights II
    Jade Empire
    Star Wars: Knights of the Old Republic
    Neverwinter Nights
    MDK2
    Baldur's Gate I & II
    Shattered Steel

 Archived -
    The Lobby (Off Topic)
    BioWare Store Support
    BioWare Blog
    BioWare Points Help

Social Groups

 

If you plan to help out a lot you might want to consider one of the quicker ways to archive pages mentioned here. That is, installing the Bookmarklet or Chrome Extension.

[KallenX]

i'll do at least the MP sticky sections

[Shades of Night]

i'll do at least the MP sticky sections

Thank you So Multiplayer Sticky sections for both DAI and ME3?

 

I'm going to pop a list of all the sections up in the first post too and I'll try to keep it updated with what people have/have not archived yet too to keep track of it more easily.

[Cassandra Saturn]

it's already coming up at uBSN: http://biowaresocial...ork.boards.net/

[Shades of Night]

it's already coming up at uBSN: http://biowaresocial...ork.boards.net/

Will that site be archiving every single forum post for people to access, not just the most popular/used ones? If not, then that's why I'm asking for people to help with archiving via the Wayback Machine

 

Forums have a lot of content, most of the things people backup will be mods and guides, but bug identification and possible workarounds among other things are also hugely useful for people who decide to play some of the older games (and the newer ones as bugs aren't always fixed).

[Cassandra Saturn]

yeah that's why i am hoping this will happen at uBSN

[Shades of Night]

yeah that's why i am hoping this will happen at uBSN

Hoping? So are you going to copy-paste every single post manually into the uBSN forum or... ?

[Cassandra Saturn]

manually. it's lot of work for me but worth it.

[Kaidan Fan]

I did this thread: https://forum.biowar...-mission/page-2

 

This is the web address for it:  http://web.archive.o...-final-mission/

 

Edit:  Hope I did it right.  It seems to all be there.

[CreeperLava]

I am backuping the whole forum through HTTrack, but having a web archive would be pretty nice too.

[Fexelea]

What are you storing it on @CreeperLava?

 

Perhaps we could collaborate to convert the data to phpbb in order to create a more forum-like backup. I'm working on a scraper but I don't want to duplicate work.

 

https://forum.biowar...-collaboration/

[CreeperLava]

What are you storing it on @CreeperLava?

 

Perhaps we could collaborate to convert the data to phpbb in order to create a more forum-like backup. I'm working on a scraper but I don't want to duplicate work.

 

https://forum.biowar...-collaboration/

Just my hard drive for now. It's still running, but once done I will archive it and post a link to it on forums. You could probably use it as is to make a complete mirror of the BSN. Not sure if you have the right to do so though.

[Shades of Night]

Cassandra Saturn: I'm sorry, but I'm pretty sure it's virtually impossible to do manually within the time frame unless you have something else to automate and speed up the process.

 

Kaidan Fan: Thank you! I'll add that to the list above.

 

CreeperLava: Really? You'll be able to create a fully functioning forum from your archive? That would be awesome, I don't know about rights but doesn't the Wayback Machine essentially do exactly that? I'm not sure how rights work with the Wayback Machine but if they can do it I don't see why someone else can't upload it all for archival purposes. Granted I'm guessing it would take up a looot of space.

 

Fexelea: Speaking of collaboration and not duplicating, I saw a post here (number 161) by malloc which seems to mention something similar to what you were talking about, perhaps they might be able to help?

[Fexelea]

I had seen that and contacted Malloc about that post and see if we can work together - thank you for the heads up regardless!

[FloraTheElf]

I would help but I'm a complete layperson when it comes to much more than turning on a computer! I just wanted to say how much i appreciate all the hard work you guys are voluntarily putting upon yourselves for everyone here. My gratitude just cannot be said enough! Just reading how much everyone is coming together makes me emotional

[CreeperLava]

CreeperLava: Really? You'll be able to create a fully functioning forum from your archive? That would be awesome, I don't know about rights but doesn't the Wayback Machine essentially do exactly that? I'm not sure how rights work with the Wayback Machine but if they can do it I don't see why someone else can't upload it all for archival purposes. Granted I'm guessing it would take up a looot of space.

Yes, that's the whole point of HTTrack : download websites for offline use. I should even be able to download all third party pictures that are scattered throughout the forums. It's running since about 15h now, up to 10 GB of data. I just upped the download speed, it should go faster. Still going to take days^^.

[Fexelea]

I have asked Bioware for permission/assistance, waiting on their internal decision. I imagine they will not mind archives at the very least.

 

If @CreeperLava and @ShadesofNight can complete their offline compilations, we can probably share that data between ourselves so I can figure out how importing into working forums with dotted usernames would work, that way people who claim their accounts could restore their posting history.

 

It's a huge forum, so if we can divide the tasks it could be faster. For example someone does one subforum only, then the next, etc. Not sure HTTrack can be used like that, but in any case I'll continue work on scraper to have an alternative

[CreeperLava]

I have asked Bioware for permission/assistance, waiting on their internal decision. I imagine they will not mind archives at the very least.

 

If @CreeperLava and @ShadesofNight can complete their offline compilations, we can probably share that data between ourselves so I can figure out how importing into working forums with dotted usernames would work, that way people who claim their accounts could restore their posting history.

 

It's a huge forum, so if we can divide the tasks it could be faster. For example someone does one subforum only, then the next, etc. Not sure HTTrack can be used like that, but in any case I'll continue work on scraper to have an alternative

Having alternatives is good. HTTrack will simply offer a mirror of the HTML/CSS, it doesn't save the backend of the website (afaik, I'm not an expert). So, not possible to just put everything on a server and expect that it will run fine.

 

I don't think it is a good idea to provide a way for people to log back into the saved website. It poses many problems of security for passwords, for example if someone decides to use the archives for phishing. For me, the archive must only be used as a mirror of the website at a given time, it can't be used to give a new home to the community. This must be done on another, new, forum.

[Fexelea]

Well there's no reason not to have a locked forum with a searchable archive on the same location. The idea to recover user posts would be for users who claim their accounts before the accounts here change. For example if I import my posts into a locked archive they can still be tied to me from the database because my account is the same. (for clarity I'm talking about importing here:http://fextralife.com/forums/f95/)

 

A similar result could happen for many others if they so wish

 

Not sure how this could be used for phising... I'd certainly not want that.

[Kantr]

can you get it to skip the images and just save the links to them instead?

[CreeperLava]

can you get it to skip the images and just save the links to them instead?

I can, yes. Not sure if I should though, the whole point of an offline backup is to be standalone. It would make the backup go much faster though, true.

[Kantr]

I can, yes. Not sure if I should though, the whole point of an offline backup is to be standalone. It would make the backup go much faster though, true.

It's up to you of course but in the long run it would probably save space.

[CreeperLava]

It's up to you of course but in the long run it would probably save space.

The problem is that I have no idea what the total volume of data of this forum is. I suppose it is a good idea, though, I'll add a restriction on jpg/png/gif.

[Kantr]

The problem is that I have no idea what the total volume of data of this forum is. I suppose it is a good idea, though, I'll add a restriction on jpg/png/gif.

I wouldn't be suprised if at least 500gb

[CreeperLava]

I wouldn't be suprised if at least 500gb

Should be much less once compressed, hopefully.