BSN is shutting down; files there could be lost.

[Shadesofnight]

Agh! Sorry it's been a while! Health issues and general chaos...

 

Anyway, I was doing a secondary forum backup in case CreeperLava's one got eaten by a shark or something. It's currently up to 200GB, however since CreeperLava's is up to 450+ and Fexelea is actively importing chunks of the forums over I was wondering if perhaps I might be better off focusing on something else (also because we're moving soon which will mean I won't be able to download 24/7 anymore). My download speeds are not the fastest, but I do have space to just sit on stuff.

 

It also occurred to me that when things go read-only if it's logged out read-only we may very well lose access to anything on the BSN (projects) or forums (group only discussions) which can only be viewed when logged in. I tried asking if it was logged out read-only or logged in read-only but got no response, so I guess it's best to assume the worst. In which case, is there any way to pin down which of the BSN mods are only viewable when logged in so they can be prioritized?

 

Something else I thought of was that, closer to the actual closing date (or rather when/if all mod authors have been contacted) it might be good to have a list of all the BSN mods that are not hosted anywhere else and will be going the way of the Dodo once it goes down. So that people know the mods that they absolutely cannot get elsewhere and need to get copies of now before it's too late.

 

Actually a simple list of all the current BSN mods and where they can be found elsewhere or if they will be unavailable would probably be good. But that's kinda what the spreadsheet is doing from what I gather and a simple form of that could probably be made afterwards.

 

I'll take a look at and try to see if I can help out with the mod author contacting in what free time I can scavenge. Has anyone else started at the end and worked their way backwards? If not I'll do that so I don't duplicate anyone else's work.

[Tarshana]

Hey Shades! I wondered what happened. I was actually composing a message to you when this popped up!

 

MohoFish created a nifty little blogger post: https://whereisbioware.wordpress.com/ If you scroll down to "Inventorying The Projects" you will see our googlespread sheet. It is allowing us to sort through the pages and seeing who's mod is where (if at all).

 

I have all of the BSN legacy stuff downloaded. My .zip file count is at 4.1k (and then some). I have to ultra compress the file and then Creeper has a program to remove anything funky to it. I can send you and MohoFish links to download if you wanted. As of yet I have no idea what mods could be downloaded without logging in as I bypassed that on my htttrack. If you do not wish to keep downloading the forums, then do not worry much on it :) Fexelea just finished another forum scrape and started another.

Right now we are focused on the spreadsheet. I am currently at work so my spreadsheeting is a wee bit slow ><

 

Ehugs :) Happy to see your face again!

[CreeperLava]

Agh! Sorry it's been a while! Health issues and general chaos...

 

Anyway, I was doing a secondary forum backup in case CreeperLava's one got eaten by a shark or something. It's currently up to 200GB, however since CreeperLava's is up to 450+ and Fexelea is actively importing chunks of the forums over I was wondering if perhaps I might be better off focusing on something else (also because we're moving soon which will mean I won't be able to download 24/7 anymore). My download speeds are not the fastest, but I do have space to just sit on stuff.

 

It also occurred to me that when things go read-only if it's logged out read-only we may very well lose access to anything on the BSN (projects) or forums (group only discussions) which can only be viewed when logged in. I tried asking if it was logged out read-only or logged in read-only but got no response, so I guess it's best to assume the worst. In which case, is there any way to pin down which of the BSN mods are only viewable when logged in so they can be prioritized?

 

Something else I thought of was that, closer to the actual closing date (or rather when/if all mod authors have been contacted) it might be good to have a list of all the BSN mods that are not hosted anywhere else and will be going the way of the Dodo once it goes down. So that people know the mods that they absolutely cannot get elsewhere and need to get copies of now before it's too late.

 

Actually a simple list of all the current BSN mods and where they can be found elsewhere or if they will be unavailable would probably be good. But that's kinda what the spreadsheet is doing from what I gather and a simple form of that could probably be made afterwards.

 

I'll take a look at and try to see if I can help out with the mod author contacting in what free time I can scavenge. Has anyone else started at the end and worked their way backwards? If not I'll do that so I don't duplicate anyone else's work.

What speed are you downloading at ? Mine is at about 50 KB/s, so if you download sensibly faster (at least 100KB/s), it will be worth it for you to continue. Given the size of the website, you'll finish faster than me. I *think* the website is putting bandwidth limits per IP, which could explain why the download rate is so low.

 

Good point on the read-only issue. I haven't seen a way to filter private mods. The only way I see is to have 2 tabs open, one logged in, one not logged in (from a proxy for example), and compare page by page. Easiest to backup everything, I think. Unless you can somehow catch all project indexes when logged in and when not logged in with a scrawler, then compare in a spreadsheet. I suppose you could automate this, but I don't have the skills to do it^^.

 

The spreadsheet has instructions to avoid overlapping, see the Inventory Status tab. Also, read carefully the instructions on the first tab :).

[Shadesofnight]

Juuust got through reading all the instructions, they're a bit daunting! I'll make sure I check back to them frequently so I don't accidentally cause anything catastrophic to happen :D I am also completely unfamiliar with using spreadsheets, so hopefully I won't make entries that look like a cat walked on the keyboard...

 

And having a link to download the BSN legacy stuff would be great Tarshana :) I'm especially interested in seeing if it can be easily opened up and browsed. Because if it can that's an ideal way of being able to access logged-in only visible mods once we go read-only (if it's logged out read-only).

 

And CreeperLava I'm currently downloading at... 30KB/s, that 200GB has taken 12 days to download. So certainly not faster XD

 

I could manually go through with a tab logged in and one logged out to find all the mods that we'll lose access to if we can't log in after the 25th (looks like there are 204 of them), but as I suspect Tarshana's backup will allow us to do that anyway and we're aiming for all mods it seems best that I just jump into helping out with the spreadsheet.

[CreeperLava]

Tarshana's backup doesn't include private mods, if I'm not mistaken. However, using the spreadsheet remains the best and most efficient way to solve the issue of losing mods.

[moho25]

@Shades --

 

I would work under the assumption that we're still going to be able to log in.

 

BSN has been read-only for 2 years now, and we can still log in.

 

More importantly, BW has to maintain that log in ability until they implement whatever their plan is for access to DLC and Patch files. My guess is that plan is going to go live on October 25th... and I'm guessing it involves Origin.

 

BW has also said their intention is to give folks time to get their stuff. Many player blogs are set to private and are inaccessible *to that player* unless they are logged in.

 

Removing the ability to log in renders the extra time they're giving us moot. I don't see that happening.

 

---

 

As for mods, visibility, and accessibility, the spreadsheet is the way to go. There's no way to make a quick and easy judgement call (via 4170 mods) as to what we'd lose access to if we can't log in. The inventory is a large job, which is why we need as many people working on it as possible.

 

Spreadsheets aren't hard, it's just entering data into cells. Click & type. If you can do that, you can edit a spreadsheet. The complicated thing is how to determine permissions, contact authors, etc, which is the bulk of the instructions. It's not a simple project, so the instructions aren't simple, either. Unfortunately.

 

The current intent is to not make a second document from the spreadsheet as to mod availability. Folks will just consult the spreadsheet, locate the mod, and click the link to be directed to the page, if present. There's really no need to create another document. Folks can also download the sheet if they want, but they'll of course lack any succeeding updates.

 

 

I'll take a look at and try to see if I can help out with the mod author contacting in what free time I can scavenge. Has anyone else started at the end and worked their way backwards? If not I'll do that so I don't duplicate anyone else's work.

 

Don't do this. We need to make sure we give the mods with the most downloads/views the highest priority. There's a separate sheet where people communicate what page they're working on and it's been working out well so far. Just use that to claim some BSN pages and start plugging along :)

Edited August 17, 2016 by moho25

[Tarshana]

The Spreadsheet is super easy :) I have about 4 tabs open and I start at the Inventory page and claim a page number and then work from there. MohoFish has made the instructions really clear and we have filled enough lines to make it easy to copy info. The little yellow triangles at the top also explain the boxes input :D

[Shadesofnight]

I'm pretty sure Bioware haven't always acted in their players best interest, so I wouldn't be surprised if we were logged out on the 25th. But since you're sure we won't be Moho, I guess we can go with that assumption.

 

The reason I suggested an easy, simple list of where mods can be found after all this is that I'm guessing most people will have the same reaction as me if they're looking for a mod and are directed to the spreadsheet, ie. "Wow, that's a LOT of info, I have no idea where to even start." A clear and simple list on the other hand you can simply ctrl-f to find the title or author of a mod and easily find out its fate. Actually as Moho is opposed to it I could easily whip up such a list and post it on whatever the most used new forums are after everything is closed and this spreadsheet is done. That way it's no extra work for anyone but myself.

 

But this does actually bring up a point that I'm curious about. Which is... by ordering things by download their order is not static and set in stone, right? In fact if there is only one download number between mods they can actually move around. So if say someone was up to page 54, and overnight a mod got a couple of download hits and moved up to page 53... how would we know? Is there double checking of pages as they're done to be sure? And then there's also the scenario where a mod gets a number of hits overnight and rockets its way up quite a few pages (such as when someone who is a fan of a mod hears about the closedown and tells all their friends to download a certain mod before it's gone). I understand that giving the most used mods priority makes sense, it just seems kinda odd that we're working with a base list that isn't static. Unless there is a plan to do a static search and check it against all the spreadsheet content once we're done to catch any mods that have had moving download counts.

 

And ehhh, sure I can work from the start instead of end if it's a big deal. I'll start when I get home from work.

[Tarshana]

When a mod name is inputed into the appropriate box, it would come up if it had been typed in already. A quick "ctrl f" would be used. We have not had any issues so far. I almost double posted on a page I was working on and when the box tried to autofil for me I realize what I was doing so there is little chance we will be double posting mods and even if we do it is no big deal. We can fix it later.

 

There will be files lost, on that there is no way to avoid that. What we are doing is preserving what we can. All mods set to public download is downloaded. We have contacted around 50 mod authors, many of which have hosted their mods somewhere on the interwebs. The spreadsheet has been the only way for us to keep track of who has contacted who and which mods have been inventoried.

 

After all of this is said and done, a person can simply use a search feature for the mod they want help in finding.

 

If a future mod user needs a mod, we have backups, secondary websites, etc. There is no reason not to help them in the future. We will organize the mods as efficiently as possible but right now what we need is more help with the spreadsheet. We need to inventory the projects. Once we have the inventory taken care of we can look at "doubles" if that happened. Can't worry about that now. :)

[moho25]

I'm pretty sure Bioware haven't always acted in their players best interest, so I wouldn't be surprised if we were logged out on the 25th. But since you're sure we won't be Moho, I guess we can go with that assumption.

 

No, they haven't. But, they've specifically laid out a situation saying, "Look, we're going to give you time to get your stuff". If they lock people out so they can't access their blogs, projects, etc, then they might as well just close the sites next week rather than taking them read-only. I'm not saying it won't happen. I'm saying that it doesn't make sense, and I doubt it will happen.

 

But whether it does or doesn't is irrelevant as far as this project goes. The crunch time right now before next week is simply because authors are most easy to contact. Once things go read only -- and I'm assuming PMs won't work at that point -- then it becomes *much* more difficult to reach out to authors.

 

There's a real possibility that we might not be done by late October when the sites are taken down. That's what the backups are for and why Tarsh and I have talked about their integrity, workflow, etc. We'll be working from that come October if we aren't finished yet.

 

 

The reason I suggested an easy, simple list of where mods can be found after all this is that I'm guessing most people will have the same reaction as me if they're looking for a mod and are directed to the spreadsheet, ie. "Wow, that's a LOT of info, I have no idea where to even start." A clear and simple list on the other hand you can simply ctrl-f to find the title or author of a mod and easily find out its fate. Actually as Moho is opposed to it I could easily whip up such a list and post it on whatever the most used new forums are after everything is closed and this spreadsheet is done. That way it's no extra work for anyone but myself.

 

Hey, if you want to create a list, b/c that's easier for you; that's no problem. I'm only saying that it isn't necessary as far as the project itself goes, and the intent is for the gaming community to work off the spreadsheet. Games are organized by tab, which is pretty straightforward. The mod names are clearly listed under a "BSN Project Name" header, Ctrl+F works on a spreadsheet as it does in any other document, and when a link is hosted on Nexus, it's very clearly associated with the mod.

 

Also keep in mind that even once the inventory is completed, links may being updated on an ongoing basis by mod authors and other folks. That's part of the intent. It's a publically-editable document and the community can help keep it up to date. Any alternate document will end up outdated when that happens, unless it's kept up to date with the spreadsheet. Since you're talking about thousands of mods, that's quite a bit of work.

 

 

But this does actually bring up a point that I'm curious about. Which is... by ordering things by download their order is not static and set in stone, right? In fact if there is only one download number between mods they can actually move around. So if say someone was up to page 54, and overnight a mod got a couple of download hits and moved up to page 53... how would we know? Is there double checking of pages as they're done to be sure? And then there's also the scenario where a mod gets a number of hits overnight and rockets its way up quite a few pages (such as when someone who is a fan of a mod hears about the closedown and tells all their friends to download a certain mod before it's gone). I understand that giving the most used mods priority makes sense, it just seems kinda odd that we're working with a base list that isn't static. Unless there is a plan to do a static search and check it against all the spreadsheet content once we're done to catch any mods that have had moving download counts.

 

This really isn't something we need to worry about. BSN isn't Nexus and traffic there is minimal -- even with the closure.

 

1. The goal is to catalogue all 417 pages of mods. We're prioritzing b/c that's the smart thing to do, but the goal is to catalogue every mod that's been released. The chance that a mod will move between pages at the exact time that we're cataloging it is exceedingly slim. I'm not saying it can't happen, but it's very, very unlikely.

 

2. I can tell you that logging out changes the mod count from 4170 to 3964, so if BW does remove the ability to log in come next Thursday those will be the ones we miss. There is no way to know which mods those are and we won't be able to ensure they are part of the inventory.

 

3. A static search afterward won't do much good because we're only cataloguing mods that have been released (which often doesn't correspond to the "released" category). It won't be a simple apples to apples comparison. We also can't use a "missing" project ID to know whether we've missed a mod, since project IDs are apparently not sequential (or there have been a lot of deleted mods). In short, we get one shot at this.

 

I agree that starting with "A" would be the best way to go about it, if we had infinite time and staff. We're short on both, and we're counting on having access to BSN through October. The goal is to be done with the initial inventorying by then. That means we need to prioritize and make sure we account for the most used mods first. Hence ordering by Downloads.

 

And ehhh, sure I can work from the start instead of end if it's a big deal. I'll start when I get home from work.

 

It is important, yes. And, please do :smile: I'm sure Creeper and Tarsh would like the help!

Edited August 17, 2016 by moho25