CDN Download Failure (403 Error) With Everything Green! (See PIC)

[SplashDamage]

Just a quick chime in here... With a quick 'preface', I'm using the free servers and manually downloading files with Chrome.

 

I'm noticing a drastic drop in download speeds... which usually start at 100+ KB/s upwards to somewhere around 600+ KB/s, depending on how many downloads I have going at one time. Mostly it'll range about 100+ KB/s. Which I would expect to see while the server attempts to balance present loads along with my file requests, or if it's under a heavy load from all over the place.

 

Now, after downloading over what appears be a 'set' GB limit per time period, let's say (without further testing the metrics of it) 24 hours and x amout of GB, the download rate will drop around 0 KB/s - 8 KB/s per file, even if there's just two being downloaded. This is independent of file size. If I download, at the same time from, let's say moddb, then the download speeds stays between 1024 KB/s - 780 KB/s. So it's not the port, wires, switches or even the routers here. It seems to be isolated with the Nexus.

 

Pinging cdn.nexusmods.com resolves to... 198.27.85.139 and in one case the first ping returned:

 

----------

 

Pinging cdn.nexusmods.com [198.27.85.139] with 32 bytes of data:

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Request timed out.

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Ping statistics for 198.27.85.139:

Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),

Approximate round trip times in milli-seconds:

Minimum = 62ms, Maximum = 62ms, Average = 62ms

----------

 

Pinging cdn.nexusmods.com [198.27.85.139] with 32 bytes of data:

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Reply from 198.27.85.139: bytes=32 time=67ms TTL=48

Reply from 198.27.85.139: bytes=32 time=63ms TTL=48

Reply from 198.27.85.139: bytes=32 time=67ms TTL=48

----------

... so the ms 'turn-around' isn't too bad. Though the files are still downloading at 0-8KB/s (2 going atm).

If I wait, let's say 30 minutes before downloading another file, the bandwidth will be higher than 0 - 8KB/s (which is bounces back and forth between those numbers. When it hits 0 KB/s for an extended period of time, the download is 'interrupted' completely dropped from the server without being completed)... anyways, as the 24 KB/s continues, it'll certainly drop down to 0 - 8 KB/s.

 

I'm guessing the downloads here (and maybe elsewhere) are being caught in some type of "anti-scraping algorithm 'net'", that tries to limit the amount of data that can be downloaded in a given time, and if exceeding it, the download speeds continue to drop forcing the down loader to wait until the GB/time limit has expired before allowing higher than a 'crawl '. As the download speed appears to continually decrease until a given time has elapsed without downloads occurring; calculating allowed speed versus "cool-down" time remaining.

 

I'm not sure if that's what's happening, but the symptoms seem to match the observation. Not sure if that helps any.

Edited August 6, 2018 by SplashDamage

[MrMason]

Hi SplashDamage,

 

Can you please ping `files.nexus-cdn.com` and let me know what IP address you get ?

 

Also where are you located in the world ?

[SplashDamage]

Sure, here it is:

 

Pinging files.nexus-cdn.com [198.27.85.139] with 32 bytes of data:

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Reply from 198.27.85.139: bytes=32 time=64ms TTL=48

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Reply from 198.27.85.139: bytes=32 time=62ms TTL=48

Ping statistics for 198.27.85.139:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 62ms, Maximum = 64ms, Average = 62ms

My Location is Kentucky, USA.

----------

I still have 2 downloads that's been running for hours with a KB/s now 38 and 24 respectively. We have high speed DSL here (not satellite) and an internet speed test reveals 10.1 Mbps per second download speed (with Netflix streaming also in progress, though I'm not sure how that balances in load wise on this end).

[Ephemeral66]

Having issues trying to download mods. anything from the CDN server. I can download other mods, as long as it's not from that server. Any help would be appreciated.

[MrMason]

Having issues trying to download mods. anything from the CDN server. I can download other mods, as long as it's not from that server. Any help would be appreciated.

 

Can you provide any links with the problem ?

 

@SplashDamage - it looks like the node you were downloading from was having issues and we've traced the problem and I've taken itserver offline for a bit, should be back up tomorrow, in the mean time you should be getting a bit better speeds from another US node.

[SplashDamage]

@SplashDamage - it looks like the node you were downloading from was having issues and we've traced the problem and I've taken itserver offline for a bit, should be back up tomorrow, in the mean time you should be getting a bit better speeds from another US node.

 

 

 

 

Thanks for all the time you've spent helping resolve the issue- look forward to the server coming back up full. If it helps with isolating the cause, I noticed this starting to happen maybe 2+ weeks prior (kinda ball parking it), before that, everything seems to load balance fairly well.

 

Thanks again for the site and the effort everyone there takes to maintain it!

[SplashDamage]

@MrMason:

 

Just a quick update following up with downloads- the server seems to still be 'in a struggle' keeping the downloads consistent. I'm getting 38 KB/s on one, 24KB/s on another with a third hitting 0 every few seconds, then bouncing to 9KB/s, then being dropped completely (which I have to "resume" every few minutes). The third file won't drop out when just 2 downloads are running concurrently.

 

I've been downloading files for quite a few games in the past month or so, with the regular "Skyrim" giving the most apparent problems (which is the mods I'm currently in the process of downloading).

 

There definitely seems to be a massive drop in available bandwidth which causes some files which are 22mb to take 6 hours to complete. It [available bandwidth per download] does depend on how many files are being downloaded simultaneously (which I would expect with load balancing demands for a given ip, but it seems now the amount allocated is so low downloading anything more than 1 file at a time is all but impractical and even then, with really large files your in for a serious long haul).

 

Weeks back it took a given bandwidth, let's say 1MB/s them split that as a "load" across all open downloads coming from a given ip, then seemed to drop over time depending on the GB downloaded (just an observation there), where as now it's taking maybe 1/8-1/4 of that or less, splitting it, which results in any concurrent downloads over 2 to drop to 0 KB/s.

 

This could be due to a limited amount of 'bandwidth' on a "free" server being spread thin during peak times, so I'll try and monitor it for a bit and see if it improves in the early hours of the morning and let you know the results.

[MrMason]

Thanks @SplashDamage,

 

We're trying to get some new nodes online ASAP, they should help and once we've got them up and running we can see if there are still problems.

[SplashDamage]

Anytime; I'm glad to help. Just to give a quick update before I forget...

 

Somewhere around 2:00am - 7:30am (local time- here) there seemed to be an improvement with available bandwidth, but during normal 'daylight' hours up to around 12:30am (night here), the available bandwidth is around 1/10 or ~ 150 KB/s the 1MB/s limit (though it's a capped limit and not necessarily set as a 'norm') to be split between concurrent downloads, web browsing Nexus pages, etc.,. let's say per ip. Which still fairly limits concurrent downloads during certain hours to 3 or less and should definitely improve with additional server bandwidth, or available nodes.

 

The load balancing is interesting in that, if the server settles on a 'balance' between downloads, and I start opening additional pages preparing for the next group of mod files (doing the mods of the month and trending atm, then after that I'll grab the S.T.E.P recommendations), then all but 1 download drops to 0 KB/s and might take up to 30 seconds to attempt to re-balance and rise above 0, as opposed to, let's say, dropping the bandwidth 10 KB/s on each download (or load metric) netting what might be needed to serve the requested web pages; burst 'mode' request wise seems to brings things to a halt fairly quickly.

 

Let's me see if I can cobble together a bandwidth mapping application (something simple) so I can generate results for any 24 hour period and see if that'll help to better show when the things are more than likely getting 'hit' the most. It should be fairly simple, just need a common way to easily map the data without going all out ASCII.