Shadesofnight

I'm pretty sure Bioware haven't always acted in their players best interest, so I wouldn't be surprised if we were logged out on the 25th. But since you're sure we won't be Moho, I guess we can go with that assumption. The reason I suggested an easy, simple list of where mods can be found after all this is that I'm guessing most people will have the same reaction as me if they're looking for a mod and are directed to the spreadsheet, ie. "Wow, that's a LOT of info, I have no idea where to even start." A clear and simple list on the other hand you can simply ctrl-f to find the title or author of a mod and easily find out its fate. Actually as Moho is opposed to it I could easily whip up such a list and post it on whatever the most used new forums are after everything is closed and this spreadsheet is done. That way it's no extra work for anyone but myself. But this does actually bring up a point that I'm curious about. Which is... by ordering things by download their order is not static and set in stone, right? In fact if there is only one download number between mods they can actually move around. So if say someone was up to page 54, and overnight a mod got a couple of download hits and moved up to page 53... how would we know? Is there double checking of pages as they're done to be sure? And then there's also the scenario where a mod gets a number of hits overnight and rockets its way up quite a few pages (such as when someone who is a fan of a mod hears about the closedown and tells all their friends to download a certain mod before it's gone). I understand that giving the most used mods priority makes sense, it just seems kinda odd that we're working with a base list that isn't static. Unless there is a plan to do a static search and check it against all the spreadsheet content once we're done to catch any mods that have had moving download counts. And ehhh, sure I can work from the start instead of end if it's a big deal. I'll start when I get home from work.

Juuust got through reading all the instructions, they're a bit daunting! I'll make sure I check back to them frequently so I don't accidentally cause anything catastrophic to happen :D I am also completely unfamiliar with using spreadsheets, so hopefully I won't make entries that look like a cat walked on the keyboard... And having a link to download the BSN legacy stuff would be great Tarshana :) I'm especially interested in seeing if it can be easily opened up and browsed. Because if it can that's an ideal way of being able to access logged-in only visible mods once we go read-only (if it's logged out read-only). And CreeperLava I'm currently downloading at... 30KB/s, that 200GB has taken 12 days to download. So certainly not faster XD I could manually go through with a tab logged in and one logged out to find all the mods that we'll lose access to if we can't log in after the 25th (looks like there are 204 of them), but as I suspect Tarshana's backup will allow us to do that anyway and we're aiming for all mods it seems best that I just jump into helping out with the spreadsheet.

Agh! Sorry it's been a while! Health issues and general chaos... Anyway, I was doing a secondary forum backup in case CreeperLava's one got eaten by a shark or something. It's currently up to 200GB, however since CreeperLava's is up to 450+ and Fexelea is actively importing chunks of the forums over I was wondering if perhaps I might be better off focusing on something else (also because we're moving soon which will mean I won't be able to download 24/7 anymore). My download speeds are not the fastest, but I do have space to just sit on stuff. It also occurred to me that when things go read-only if it's logged out read-only we may very well lose access to anything on the BSN (projects) or forums (group only discussions) which can only be viewed when logged in. I tried asking if it was logged out read-only or logged in read-only but got no response, so I guess it's best to assume the worst. In which case, is there any way to pin down which of the BSN mods are only viewable when logged in so they can be prioritized? Something else I thought of was that, closer to the actual closing date (or rather when/if all mod authors have been contacted) it might be good to have a list of all the BSN mods that are not hosted anywhere else and will be going the way of the Dodo once it goes down. So that people know the mods that they absolutely cannot get elsewhere and need to get copies of now before it's too late. Actually a simple list of all the current BSN mods and where they can be found elsewhere or if they will be unavailable would probably be good. But that's kinda what the spreadsheet is doing from what I gather and a simple form of that could probably be made afterwards. I'll take a look at and try to see if I can help out with the mod author contacting in what free time I can scavenge. Has anyone else started at the end and worked their way backwards? If not I'll do that so I don't duplicate anyone else's work.

Sounds like we have everything pretty much covered then? It also pays to keep in mind that HTTrack as far as I can tell, even when finished can still be set to go back and find any updates to the content that it has archived. I don't think any/many of the old BSN stuff will likely be changing much, but this will be invaluable for the forum backups since there are constantly new posts popping up. And towards the end we'll likely get threads letting people know where the various bits of the community are migrating to. I'm guessing it would pay to make sure HTTrack can look for any updates after things have gone Read-only to catch any last changes. Annndd yeah it does sound like it would be good for you to hold off from the forums Tarshana if you got a warning from your ISP, who knows what they think you're downloading XD

Hmm so just for the sake of it the things that needed to be archived/backed up were... Bioware Forums BSN Projects BSN Blogs Mass Effect 1 page Things we're unclear on but are backing up anyway: DAO Toolset DAO Wiki Pre-Origin DLCs and patches Was there anything else that might be going? Granted we can't backup BSN Projects, Blogs and Social forum group content that we do not have access to.

The closest I've ever come to modding is when I messed with the texture of my Kotor2 Exile so I could turn her blue XD Alas, I would love to try actual modding but I tend to learn through deconstructing things and a) I have no idea how to extract 3D heads out of a game so I can examine them and figure out how they work, and b) I have no 3D modeling software. Saving and hoarding files though? That I can most certainly do :D Speaking of which, I just started going through the BSN Projects, put them in the order by created and I noticed a number have Download greyed out. Am I correct in thinking that these are the ones that are only accessible by certain Groups?

Not to worry, I think we're all learning a lot with this :D And! I know exactly what will happen when your connections goes, because mine disconnects once a day :tongue: It'll all drop to zero, then when your connection is back it will reconnect and continue. Edit: Duh, I didn't even think of it losing the connection because it was being taken away from the connection :P What Tarshana said.

I just downloaded that, or am downloading. It's on the DA2 patch page and is about 1GB by itself, that'll be why you've slowed down to just that. I imagine things will go back to normal once that beast of a thing is downloaded. Speaking of which, I just grabbed all the patches. There are much less than I thought, just the most recent for each game (though that being said, I'm sure I do have a number of older patches downloaded and squirreled away on external hard drives). There doesn't appear to be a patch for ME3 either, though... I think if my memory is correct that ME3 has to run through Origin.

At last! The weekend has arrived! Hawkethority: Don't worry too much about who is doing what, I'm not sure most of us know entirely who is doing what either :tongue: Just make sure if you're going to back up a page with HTTrack you visit it while not logged in first. That way you'll see anything like the language redirect thing on the BSN Projects page which you'll have to work around by being logged in or somehow bypassing it. I think Tarshana was working on getting around that. Also see CreeperLava's HTTrack settings here. This is what I think each person is doing (or rather, these are all the people I've seen mention they'd be trying to archive something): CreeperLava: HTTrack forum backup. Tarshana: HTTrack everything XD Fexelea: Working on a scraper/transferring reclaimable forum post histories over to her forums. She mentioned the ME forums. Baaleos: Made a scraper and got the NWN1 and NWN2 forums. Mallos: Saw a post mentioning they were working on a scraper. Henesua: Mentioned IRC talk about people using wget to archive the site. I'm going to try for the whole forum again with HTTrack, this time with the settings right. I also want to individually save all the mods and patches that I can access. I'ved saved all the pre-Origin DLC I had available. I have got multiple computers so I'm tempted to set things up on another one (I can spare 900GB on this one and have almost 3TB free on an external) but I think doing multiple at once on the same connection would mean they'd all go slower? Also CreeperLava: How big is your forum download now? (you mentioned 60GB at 10% which would be an estimated 600GB total?) Do you think me allowing 900GB for it would be enough?

Well, at least I'm not the only one having the problem, and it's certainly very good that we realized it this early :) I suspect it may have something to do with what you set it to when you first start a new project (The Action: 'Download website(s)' etc bit just above where you put in the URL), but I'm really just guessing and unfortunately can't check what my failed attempt had selected because it's now changed to 'Update existing download'. I can however put down the settings I had for that failed one though (keeping in mind it's the one that got the too many URLs error). I don't think I changed many things at all, but this is what it had: Scan Rules - nothing changed Limits - Everything blank except max transfer rate at 25000 Flow Control - All blank, persistent connections ticked Links - Attempt to get all links, and Get all non-HTML files both ticked Pretty sure I didn't even look at anything else. CreeperLava will likely be our best bet for figuring this out as I'm virtually clueless about how a lot of it works, I'm just used to trying to trouble shoot things. Oh and I'd say it's certainly a good idea to do what I've been doing: When you back something up like say a forum section open it up, go to one of the last pages of topics in that section and randomly select a page to see if it's there, do the same in the middle and start topic pages, repeat for the sub-forums if your section has them. This way with minimal effort you can be fairly sure if it's captured everything or not. And as we now know, check the URL displaying to make sure it's on your computer and not the online version. Also if you find something like I did with the DAO General discussion topics, which is a page of topics all called 'empty', don't panic. Check to see what it looks like on the actual forums. And as you can see here, there are indeed a lot of topics called 'empty' there for some reason, so it isn't a backing up problem.

Aha! After following the instructions, HTTrack is working again! And faster than before too thanks to the bandwidth speed option (I honestly hadn't even noticed the File, Preferences etc at the top... ) I'm not sure what bits everyone else is/is not doing so I decided to start working my way backwards on the Dragon Age sections. I've now done the Keep and Dragon Age Franchise... Onto DAO. Edit: After becoming suspicious about the lightning quick download times I took a look and noticed that although the initial index page I visited was located on my computer, any other pages were in fact simply the online versions (as shown in the address bar), not ones saved. So I must have another setting wrong somewhere. But my previous downloading that failed was saving each page offline. I'll check to see which settings are different.

I'm going to quote some things just to provide reposts of the information. I think mine mostly captured bits of Andromeda before failing. Ohh, it's good to know that you grabbed those bits! I was wondering if segmenting the downloads might work better so we could possibly pick different sections to work on. In terms of the projects that are group read only I think we really need to get word out to those groups so they can allow us access or get one of their own members to backup the content for us. Same goes for the Social group content on the forums, we can't backup anything we can't access, so any groups that want stuff backed up need to allow us to access it or create their own backup of it. Oh and thank you CreeperLava for sharing your settings! <3 Mine was going at about 20 to 40KB most of the time I think :tongue: I'm guessing it's because my connection is used by multiple people, but does HTTrack have a set speed or anything that can be changed?

Okay, my attempt to backup the forums with HTTrack has failed with the entertaining error of: 02:21:27 Panic: Too many URLs, giving up..(>100000) So I'll leave the forum backup to the more knowledgeable CreeperLava Giftfish/Moho, can you tell me how you find those extra BSN Projects that don't appear to come up in a usual search? Since the existing 544 or so pages indicates only 5440 projects. Also, do you have the address for the pages that have: the patches and the blogs? I've got to head to work now, but I'll hunt down something else to focus on backing up when I get home.

Okay! After digging around for a 7 year old password... finding it... then resetting it anyway, I'm here! XD I see some familiar avatars from the BSN boards, does anyone else think this might be an ideal place for those of us archiving/backing up/saving the BSN content to create a temporary base of operations once the BSN forums go read-only? If that's okay with the people here that is. I know people in the NWN community have been working on backing up the NWN1 and 2 forums. Tarshana has mentioned working on the wiki and Projects. CreeperLava and I are both doing HTTracks of the forums. Numerous people have mentioned creating scrapers for the forum content. Fexelea last mentioned something about the Mass Effect section. And I think it was in the NWN section that someone was talking about using wget to do the same thing as HTTrack. I also sent an email to the Wayback Machine asking if they could direct their crawler to just grab the whole Bioware forums instead of having to grab the pages individually, but so far I've only got a standard 'you can save pages by visiting our main page' response directing me to the thing that lets you save one page at a time.