BSN is shutting down; files there could be lost.

[moho25]

@Tarshana -- Sounds like we're just crossing wires a bit. My comment about the URL confusion was in relation to this:

 

 

Okay, my attempt to backup the forums with HTTrack has failed with the entertaining error of: 02:21:27 Panic: Too many URLs, giving up..(>100000)

 

So I'll leave the forum backup to the more knowledgeable CreeperLava

 

Giftfish/Moho, can you tell me how you find those extra BSN Projects that don't appear to come up in a usual search? Since the existing 544 or so pages indicates only 5440 projects.

 

Also, do you have the address for the pages that have: the patches and the blogs?

 

I've got to head to work now, but I'll hunt down something else to focus on backing up when I get home.

 

 

Which part of the forums were you on? I've been downloading in chunks, so far I have http://www.mediafire...s - Bioware.zip (bioware forum/bioware/BSN help, Mech, Legacy, Archived and Social groups) .

Edit: Yeah some of those projects are group read only. I'm hoping my request for opening messaging on the BSN legacy will be approved so I can go in there for projects. There are community contest content I can't see at all :sad:

 

Which involves BSN and Projects. At which Creeper responded with this:

 

 

Ok, if you want to help download forums.bioware.com with HTTrack, make sure you have a good bandwidth. There's no use in downloading the same thing multiple times if your download speed is slower than mine. My HTTrack is downloading at about 120 KB/s, Tarshana's 240 KB/s. Instructions to avoid the "too many links" error : <snip>

 

Seemed like there might have been confusion with who was downloading what. As long as everyone is on the same page (which you seem to be), and the problems with downloading from BSN have been solved, then it's all good :)

 

----

 

If we can keep all discussion in this thread from now on -- rather than on the BW forums, BW forum PMs, ME3Explorer forums, and this thread -- that will be helpful. It's very difficult to keep track of discussions in that many places >.<

[CreeperLava]

Tarshana, did you also backup the projects from the ME2/ME3 Texture Mods groups ?

http://social.bioware.com/group/7065/#projects

http://social.bioware.com/group/1547/#projects

 

If not, I'll try to do that tomorrow.

Also, I've contacted CDAMJC. A few of his mods are still only in his BSN thread.

Edited August 4, 2016 by CreeperLava

[Tarshana]

Gitfish; I agree. I'll keep my posts here for now and check on other forums to see if people have questions, but for our Preservation Group Alpha, I will keep relevant info here. CreeperLava and I are on the same page in regards to HTTrack so no worries there. Fexelea is also working on forum scraping and has one set of forums saved on her site.

 

CreeperLava; From what I am seeing in my logs so far every project is being saved, even the NWN modules. Once it is 100% done and I can come comb the logs I will post the info and see if we can use both sets of peepers to make sure I have grabbed everything.

 

@All ; Mediafire is being slow with uploading my Wiki zip file. I'm pretty sure it's because I have a lot of stuff going but I will post the file link in an edit in this message when it is complete. Sometime around 2 pm PST. Completed: ToolKitBackup

Sometime next week, when the downloads should be finished I will create a comprehensive list of what mods will not longer be available on BSN Legacy , or are not uploaded else where, for people to see. I will also have a list of people who have their mods listed on a personal server so their links are not lost as well.

I popped open my Max3DS to work on some tucked hairs. It won't look like tmp's but should serve as an appropriate filler. I hope. fingers crossed!

[hawkethority]

okay, what has NOT been backed up yet? i.e. where can my effort (and my massive hard drive) best be put to use? I have as much storage space as we would possibly need, but I just need some slight clue on what to do; I'm an artist type, not a computer expert, and technology hates me a little bit, so the more advice and suggestions, the better.

 

in addition to httrack, I'm thinking of going through by hand and saving each mod file page for everything in the BSN DAO mod section, since that's what's most important to me (both for nostalgia reasons, and because some of those files are crucial to a current project.) what format do you guys think it would be best to save the pages as? or should I skip it, leave the pages to httrack, and grab the files only?

[Tarshana]

I would like to give you a more definitive answer. How about this: Once I am done downloading what I can from BSN we can compare notes and see if you are part of any project I cannot see. Then we can go from there. If you want to help with wayback machine, it's a simple copy paste. :) So far I have the DA: Toolset GD part of the BW forums saved. Also, grab the mediafile I listed above and go through it on your computer and let me know if anything isn't working properly. I test my HTTrack files on a computer that isn't connected to the internet (that way it doesn't try to target online if it can't find offline). If you see any errors, post them here so I can fix it.

[moho25]

@hawkethority -- I don't think anyone has started to back up the original Mass Effect site (see link in one of my posts on the previous page).

Tarshana, is that correct?

[Tarshana]

Oh snap, you are right. I have not started on the original ME website. That is definitely something we could use help with! Thank you MohoFish

Edit: make sure under Options you have it to configure downloading JPG's to save the wallpapers :)

[hawkethority]

I'm not a mass effect fan so I'd rather focus on saving the DA stuff - I'll just start running the backups and see where it goes; I'm sure I'll have time & room for everything.

[Tarshana]

HTTrack is at 98gigs and still going. I've got the ME site going on my other computer :)

[Shadesofnight]

At last! The weekend has arrived!

Hawkethority: Don't worry too much about who is doing what, I'm not sure most of us know entirely who is doing what either :tongue: Just make sure if you're going to back up a page with HTTrack you visit it while not logged in first. That way you'll see anything like the language redirect thing on the BSN Projects page which you'll have to work around by being logged in or somehow bypassing it. I think Tarshana was working on getting around that.

 

Also see CreeperLava's HTTrack settings here.

 

This is what I think each person is doing (or rather, these are all the people I've seen mention they'd be trying to archive something):
CreeperLava: HTTrack forum backup.
Tarshana: HTTrack everything XD
Fexelea: Working on a scraper/transferring reclaimable forum post histories over to her forums. She mentioned the ME forums.
Baaleos: Made a scraper and got the NWN1 and NWN2 forums.
Mallos: Saw a post mentioning they were working on a scraper.
Henesua: Mentioned IRC talk about people using wget to archive the site.

I'm going to try for the whole forum again with HTTrack, this time with the settings right. I also want to individually save all the mods and patches that I can access. I'ved saved all the pre-Origin DLC I had available.

I have got multiple computers so I'm tempted to set things up on another one (I can spare 900GB on this one and have almost 3TB free on an external) but I think doing multiple at once on the same connection would mean they'd all go slower?

Also CreeperLava: How big is your forum download now? (you mentioned 60GB at 10% which would be an estimated 600GB total?) Do you think me allowing 900GB for it would be enough?

Edited August 5, 2016 by Shadesofnight