Aller au contenu

Photo

New forum for community: Importing threads and such


  • Veuillez vous connecter pour répondre
39 réponses à ce sujet

#26
Fexelea

Fexelea
  • Members
  • 1 625 messages

Progress update:

 

 

1. Thank you to everyone who already registered - please be patient for a little longer while we find the right balance. Please feel free to post

 

2. I will be making "Legacy" subforums as well - I simply thought to focus on the scraper first :)

 

3. Scraper is now exporting a first run of threads from a small subforum. Next it will export usernames/avatars. When these two are done, we will import this into our phpbb test database to see if it will work.

 

4. For those of you who wish to retain your post history: I am coming to the conclusion that you need to claim your account before the "real" import happens. Otherwise I'll have to make 500,000 accounts and manually activate them on request.

** If there are account conflicts (ie. your username exists or can't work do to symbols or spaces, please let me know)


  • vertigomez et mousestalker1 aiment ceci

#27
GoldenGail3

GoldenGail3
  • Members
  • 3 778 messages

I might give it a go, but just because I'm curious and bored, but eh, what I'll do after BSN is gone is uncertain as of late though -_-



#28
Probe Away

Probe Away
  • Members
  • 407 messages

Progress update:


1. Thank you to everyone who already registered - please be patient for a little longer while we find the right balance. Please feel free to post

2. I will be making "Legacy" subforums as well - I simply thought to focus on the scraper first :)

3. Scraper is now exporting a first run of threads from a small subforum. Next it will export usernames/avatars. When these two are done, we will import this into our phpbb test database to see if it will work.

4. For those of you who wish to retain your post history: I am coming to the conclusion that you need to claim your account before the "real" import happens. Otherwise I'll have to make 500,000 accounts and manually activate them on request.
** If there are account conflicts (ie. your username exists or can't work do to symbols or spaces, please let me know)


Firstly, thanks heaps for doing this. There is sooooooo much great stuff on these forums that it would be a massive shame to lose it all.

Secondly, how do we 'claim' our accounts? I'm not that fussed about my posting history (my main concern is still having access to all the great build guides etc people have posted in the forums) but it would be nice to keep if you're happy to set up the functionality.

Thanks again!
  • Fexelea aime ceci

#29
Fexelea

Fexelea
  • Members
  • 1 625 messages
You can make an account with the same name now, or wait for confirmation that we can import everything and make an account then, and all posts will be attributed to that account.

If there are any conflicts, such as repeat usernames or mistakes, we can arrange them on a case by case basis.

I will post an update when I have confirmation on import
  • Jeremiah12LGeek, fraggle et mousestalker1 aiment ceci

#30
Jeremiah12LGeek

Jeremiah12LGeek
  • Members
  • 23 885 messages

4. Confirm how user claiming will work for those who want to keep their posting history

 

I know that staff here have the ability to merge accounts. That's likely a combination of the software used for the forum and the settings they've created for it, but something similar may be possible on yours.

 

My thought is that an inelegant solution might be to crudely create a profile mirroring the one to be associated with the imported posts, and then staff-side, merging the profile with the user's active one (after some confirmation process, of course.)

 

None of which I would know how to do, but I thought I'd mention it as a possibility in case it triggered any ideas for someone else.



#31
Jeremiah12LGeek

Jeremiah12LGeek
  • Members
  • 23 885 messages

Progress update:

 

 

4. For those of you who wish to retain your post history: I am coming to the conclusion that you need to claim your account before the "real" import happens. Otherwise I'll have to make 500,000 accounts and manually activate them on request.

** If there are account conflicts (ie. your username exists or can't work do to symbols or spaces, please let me know)

 

Ah, this modifies what I just said.

 

Perhaps there is a software solution for doing it in batches, but yes, I see how the confirmation system needs to be worked out first.


  • Fexelea aime ceci

#32
fchopin

fchopin
  • Members
  • 5 061 messages

How about a welcome to forum thread so people can introduce themselves and also would be easy for us to know who is registered at which of the available forums?


  • Fexelea aime ceci

#33
PapaCharlie9

PapaCharlie9
  • Members
  • 2 936 messages

4. For those of you who wish to retain your post history: I am coming to the conclusion that you need to claim your account before the "real" import happens. Otherwise I'll have to make 500,000 accounts and manually activate them on request.
** If there are account conflicts (ie. your username exists or can't work do to symbols or spaces, please let me know)


What is the overall strategy for account/user name mapping? As I see it, there are several cases to consider:

1) Non-BSN pre-existing fextra user name that collides with BSN user name

2) BSN user name that is not yet registered in fextra

3) BSN user name that is dormant and will never be registered at fextra (some of these are quite important, like Bioware employees that used to post good stuff)

4) BSN user that is registered at fextra, but their fextra name is not an exact string match with original BSN user name (they might have come to the two sites independently at different points in time, but are the same person).

5) BSN and fextra names are perfect match and are the same person (the happy case)


For the sake of legacy/archival content, I don't think the full profile details of a user are that important. Minimally, we just need to know that post A and reply B were two different people. And when reply C quotes reply B, we know who is being quoted. And that all posts by A can be browsed (currently this can be done from the Profile page). All the rest isn't essential. I wouldn't even expect for number of views or likes to be preserved.

It's also important for the search-by-name function to work across legacy content. Use case: I remember that Patrick Weekes posted something interesting in 2014, but that's all I remember about it. I should be able to search successfully given just that information.
  • Fexelea aime ceci

#34
PapaCharlie9

PapaCharlie9
  • Members
  • 2 936 messages
Separate question, will you automate BBCode conversion during the scrape, or do that as a post-process step?

Fortunately, they are mostly minor edits. Inline images, spoilers, quotes, font sizes and emoticons where the biggest syntax issues that I saw.

Details here:

http://fextralife.co...opy-paste-test/
  • Fexelea aime ceci

#35
Fexelea

Fexelea
  • Members
  • 1 625 messages

Ah, this modifies what I just said.

 

Perhaps there is a software solution for doing it in batches, but yes, I see how the confirmation system needs to be worked out first.

The main issue is that there is no way to notify users that their accounts are somewhere and that they can create a password, because we (of course) don't have access to email addresses. I am thinking that any users created before the import will easily get their posts, and people who notice afterwards we'll have to do manually on-request.

 

How about a welcome to forum thread so people can introduce themselves and also would be easy for us to know who is registered at which of the available forums?

 

There is currently a global announcement across the forum where people are saying hi (http://fextralife.co...bioware-forums/) - I can repurpose it for introductions as well so it's visible on all categories :)

 

 

What is the overall strategy for account/user name mapping? As I see it, there are several cases to consider:

1) Non-BSN pre-existing fextra user name that collides with BSN user name

2) BSN user name that is not yet registered in fextra

3) BSN user name that is dormant and will never be registered at fextra (some of these are quite important, like Bioware employees that used to post good stuff)

4) BSN user that is registered at fextra, but their fextra name is not an exact string match with original BSN user name (they might have come to the two sites independently at different points in time, but are the same person).

5) BSN and fextra names are perfect match and are the same person (the happy case)


For the sake of legacy/archival content, I don't think the full profile details of a user are that important. Minimally, we just need to know that post A and reply B were two different people. And when reply C quotes reply B, we know who is being quoted. And that all posts by A can be browsed (currently this can be done from the Profile page). All the rest isn't essential. I wouldn't even expect for number of views or likes to be preserved.

It's also important for the search-by-name function to work across legacy content. Use case: I remember that Patrick Weekes posted something interesting in 2014, but that's all I remember about it. I should be able to search successfully given just that information.

 

Indeed, this is one of the challenges of importing!

1) We will find these programatically on import, then match and disambiguate these with a symbol. Merge if same person (ie. Me)

2) I'm looking at possibly creating a "ghost account" that isn't really an account but holds the posts of say Person A on Bioware. This person then realizes they want to sign up, do so normally, then message me and confirm their identity, and I assign them all posts of Person A

3) Possibly it will be a simple username display that is not actually an account.

4) We will find these programatically and match them (for example, people with spaces have probably become user_name)

5) Easy peasy

 

Indeed as you say the naming is important for searches, so I'll try and find a good way to make these "ghost" users display something useful.

 

Separate question, will you automate BBCode conversion during the scrape, or do that as a post-process step?

Fortunately, they are mostly minor edits. Inline images, spoilers, quotes, font sizes and emoticons where the biggest syntax issues that I saw.

Details here:

http://fextralife.co...opy-paste-test/

 

Yes I plan on converting everything to bbcode before the import. Will probably use something like html>bbcode which means not everything will display well, but it should save a lot of work.

 

 

 

Progress Update:

 

The Mass Effect forum is enormous. My computer nearly exploded. But it's hanging in there!


  • vertigomez, PapaCharlie9, Jeremiah12LGeek et 1 autre aiment ceci

#36
Jeremiah12LGeek

Jeremiah12LGeek
  • Members
  • 23 885 messages

The main issue is that there is no way to notify users that their accounts are somewhere and that they can create a password, because we (of course) don't have access to email addresses. I am thinking that any users created before the import will easily get their posts, and people who notice afterwards we'll have to do manually on-request.

 

If there turns out to be a way to do it manually, afterwards, that would certainly be great! I expect that in the grand scheme of things, most of the total membership is almost certain to find out the details later in the process, rather than earlier, so here's hoping they'll still have options.



#37
Fexelea

Fexelea
  • Members
  • 1 625 messages

I'm still working on figuring out imports, but if successful I'll be needing everyone's help to spread the word around that users may recover their post history. As it is now I've only been posting on this less-frequented subforum, so I am not sure how many people are even aware of this project xD


  • fraggle aime ceci

#38
fraggle

fraggle
  • Members
  • 1 679 messages

I'm still working on figuring out imports, but if successful I'll be needing everyone's help to spread the word around that users may recover their post history. As it is now I've only been posting on this less-frequented subforum, so I am not sure how many people are even aware of this project xD

 

I already put it in my sig and during the next couple weeks I try to post more in general. Hopefully someone reads it :lol:



#39
Jeremiah12LGeek

Jeremiah12LGeek
  • Members
  • 23 885 messages

I'm still working on figuring out imports, but if successful I'll be needing everyone's help to spread the word around that users may recover their post history. As it is now I've only been posting on this less-frequented subforum, so I am not sure how many people are even aware of this project xD

 

I have a thread updating people on the offsite BSN.

 

There's not much too it, at the moment, but as the testing progresses, I'm going to keep them informed. And as I hear back from Malloc, I will post in here to let people know how things are progressing with his efforts.


  • fraggle aime ceci

#40
Fexelea

Fexelea
  • Members
  • 1 625 messages

I'd like to let you all know that I have successfully created a database with all Mass Effect Andromeda topics, posts, users, posted time. I must now convert this to phpbb. Once that is done, we have green light to import everything and everyone can keep their posts, their history, and have an active board to continue discussions.

 

I also did a smaller run on a small subforum as a proof of concept - I will convert this first, and will be back with a screenshot of the results :)


  • alschemid, The Elder King, vertigomez et 5 autres aiment ceci