Using Ramdisk + Symbolic Links - Somebody Able To Test?

[FiftyTifty]

Mornin' folks.

 

I remembered about a good ol' thing called a RAMDisk. What these are, is parts of RAM partitioned to function as a temporary storage device; an HDD if you will. The read and write speeds are just leaps and bounds ahead of SSDs, nevermind HDDs.

 

I also learned about a program called "Link Shell Extension". What this does, is it allows you to move files elsewhere without having to copy the data. This is done through Symbolic Links. Basically, it's a shortcut to a file/folder, which functions as if it was actually in a specific place, even though it really isn't.

 

I loaded up Skyrim's Meshes.bsa and Textures.bsa in my 2GB RAMDisk, and the cell-loading hitches reduced considerably. What I'd like to know is if loading all the .bsa files into a RAMDisk would completely remove the cell loading hitches.

 

 

For those with copius amounts of RAM (16GB or more), here's a roughly done guide on how to do this.

 

 

Get Dataram RAMDisk Just get the free version.

Get Radeon RAMDisk (You don't need an AMD anything for this. Don't worry)

Get Link Shell Extension

Set up a 4GB RAMDisk in both Dataram and Radeon RAMDisk

Go to your Data folder, backup these .bsa files "Skyrim - Meshes.bsa, Skyrim - Misc.bsa, Skyrim - Sounds.bsa, Skyrim - Textures.bsa, Skyrim - Voices.bsa, Skyrim - VoicesExtra.bsa", just in case. You never know.

Copy "Skyrim - Meshes.bsa, Skyrim - Misc.bsa, Skyrim - Sounds.bsa, Skyrim - Textures.bsa, Skyrim - VoicesExtra.bsa" into the first RAMDisk.

Move/rename the original .bsa files (Just remove the .bsa extension, or whatever.)

Select the copied files

Right-click the copied files

Click "Pick Link Source"

Go to your Data folder

Right-click anywhere

Click "Drop as Symbolic Link"

For the second RAMDisk, do the same with "Skyrim - Voices.bsa".

Launch Skyrim

 

 

If somebody could report back to me on this, that'd be fantastic. I came into some dosh, and I'm thinking about buying more RAM for this exact purpose, if it works that is.

[Werne]

RAMDisks are pretty good, I turned half of my 2GB RAM into a RAMDisk that holds my /tmp and /proc folders, speeds up the OS performance immensly (would prefer more RAM though). There are some downsides to it though, RAM gets purged when the system shuts down and everything inside is removed which means you have to do it all over again.

 

The upside is speed, and I like it's speed. But to answer your question...

 

What I'd like to know is if loading all the .bsa files into a RAMDisk would completely remove the cell loading hitches.

I don't think so. It will remove the problem of hanging while files are being decompressed from a HDD stored BSA into RAM, but it will not remove the fact that they have to be loaded into VRAM (the graphics card memory) which is not too affected by the RAMDisk stunt.

 

As far as I understand the graphics card concept, the graphics card draws what it needs from RAM, and RAM draws what it needs from the hard drive (or in this case RAMDisk). You only get higher mesh/texture loading speeds into system RAM (HDD => RAM is replaced with RAMFS => RAM) while the graphics loading is intact (RAM => VRAM).

 

And since you already loaded up the two BSAs that matter into RAM, if it's not already working the way you want it, then I don't think it's gonna work the way you want it.

 

Then again, I might be horribly wrong.

[FiftyTifty]

On the BSA's: The "important" files, in terms of usage, are the voice, sound, mesh and texture files. However, I only managed to fit the mesh and texture .bsa files into RAM.

 

The way that things are loaded, goes like so:

 

CPU -> HDD -> CPU -> RAM -> CPU -> VRAM.

 

The culprit there, is the HDD. It's not only the slowest part, it's the slowest by many magnitudes. Case in point; The read/write speeds of RAM is in the GB/s realm. With HDDs, it doesn't go over 180MB/s, at the best of times when using the best of parts.

 

The loading of the .bsa files (and the subsequent decompression) from RAM rather than the HDD should remove this insufferable bottleneck, due to the MUCH lower latencies and MUCH higher read/write speeds.

 

I'd do the testing myself, but as I said, I don't have the hardware to do so.

[baduk]

Hi!

I think it really depens on your system whether its a benefit for using the ramdisk. For me it makes game smoothing without stutter as i travel on horseback or whatever. I can go fast with console commands even like 100 mph and no stutter as i cross map.

So u want use imdisk cause its foss. imdisk lets u mount and format a ramdisk of however much ram u want to use.

rename your skyrim data folder to Data_bu or something and copy that folder onto the ramdisk.

and use console command just use text file and name it .bat extension:

mklink /d (yourinstallpath)\Steam\SteamApps\Common\Skyrim\Data (yourramdiskLetter)\Data_bu

this creates a symbolic link called data in your skyrim folder that behaves like a folder. and contains the contents of data_bu in your ramdisk

Just make sure that when you make changes to the contents of the data folder that u will have to save your data_bu back to hard drive before shutdown or whatever.

My personal experience, i have just hdd and have 32 gigs of ram . I never used a ssd before so although ramdisk is over 10 times as fast as ssd i do not know how much improvement this translates into in practice, u probably dont wnat to go out and buy lots of ram if u dont need for other reasons and u alreayd have ssd. I put the data folder that is 24 gigs on my ram, leaving 8 gigs left for system and skyrim. it is a reasonably lots of mods in my game. results for me are flawless.

[Lord Garon]

I have old HDD's and the ramdisk REALLY helped my cell load stutter. I watched the game in Resource Monitor to see which data files were accessed most often (loose files, HighResTexturePacks, etc) and moved what I could to the default (free) 4GB AMD Ramdisk. Couldn't put everything I wanted to in 4GB.

 

It did NOT fix micro-stutter, but my game is very smooth right now. I'm almost embarassed to say it, but a USB 3.0 Flash Drive actually helped as well; its read speed is faster than my HDD's. I have sounds and voice bsa's on that one. I just use Win 7's mklink and mklink /d to move files or directories around. Works fine.

 

It worked so well for me, that I'm considering a system ram over an SSD upgrade right now. Course, I could probably just get a good, fast HDD and be done with it, but the ramdisk is just ultra-cool.

 

One downside with the AMD Ramdisk; it reads and writes an image file of what you loaded into it on startup and shutdown which takes a LONG time. Especially shutting down, not sure why.

Edited October 22, 2013 by Lord Garon

[FiftyTifty]

Ooh, a necro!

 

Garon, the reason it takes longer at shutdown is due to the hard drive being written to, rather than read from. Write speeds are almost always slower than read speeds.

[baduk]

Hey, Its not necro only a couple months and i havent even logged on since before this thread. all summer actually. But i did accidntaly necro another thread on this cause i got mixed up when posting.

 

Lemme reiterate about using imdisk software because it is free and does not have limit on your ramdisk size.

 

Anyway i also want to add that the ramdisking does reduce disk i/o to almost nothing and this can be great benefit to longevity of your disks.

[Cliven]

So, in the interest of beating a dead horse just that bit deader. . .

 

Total RAMDisk n00b here.

 

Likely will update RAM to 16G (4x4) from present (6G) anyway, but would RAMDisk be *notably* useful > just using a SSD?

 

Allowing 4 G (probably too much) of that 16G to LoseVista64 (err. . WinVista64) and sundry apps, should leave 12 G free for RAMDisk. . . To bad my textures folder alone is presently 16G+. Entire Skyrim folder is 74.5 GB (79,996,510,124 bytes)

 

So. . .

 

Should my scads of hi-res textures be the focus if I go RAMDrive? Should I just try to cram onto RAM as many as I can, picking by what seems to be needed most often? I'm preemtively unloading cells (of course).

 

Looking at my slowdowns on Resource monitor indicated a HUGE drive seek on Sounds.bsa.

 

Also have a ton of SKSE mods in play (I share this install w/a wonderful lass who throws a gazillion mods on it on top of my own), for a floating number of mods ~ 250 w/likely 30+ of them SKSE-based), and script calls are DEFINITELY delayed, even when not moving. Frostfall regularly notifies me that it is halting exposure level updates because the scripting engine is tanking (tho it sometimes still did this w/fewer mods in play for testing purposes), and I actually *see* delays in scripts processing. After a few more SKSE mods joined the fray, it got a lot worse, and took a full minute for TG jobs to proc info, after talking to Delvin and Vex. Should SKSE and scripts be there > tex's? The issues definitely increase with more SKSE-dependant mods, but the CPU is not going past 50% utilization (TESV alone is maxing out around 34% of the CPU).

 

Additionally, will any of the virtual mount methods NOT try to write back to SSD or HDD at shutdown? I d/n need my static resources updated dynamically; I can just reload Windows, and would prefer to avoid excess time spent, and the writing to drive causing wear.

 

Thanks in advance for any advice, guys.

Edited October 31, 2013 by Cliven

[Lord Garon]

Should my scads of hi-res textures be the focus if I go RAMDrive? Should I just try to cram onto RAM as many as I can, picking by what seems to be needed most often? I'm preemtively unloading cells (of course).

 

...

 

Additionally, will any of the virtual mount methods NOT try to write back to SSD or HDD at shutdown? I d/n need my static resources updated dynamically; I can just reload Windows, and would prefer to avoid excess time spent, and the writing to drive causing wear.

 

Thanks in advance for any advice, guys.

 

I used resource monitor to watch disk activity as I ran around outside and had stutter during cell loads. I could see the most accessed files and with my mods, they were my loose files (textures and meshes directories), and the #1 and #2 Hi Res texture packs. I put the loose file directories and the #2 texture pack in ramdisk. Helped a lot on my system, with my old HDD's, and my mod setup. YMMV, especially if you're on an SSD right now. But big texture files would be a prime candidate for a ramdisk. My Sounds bsa is actually on a USB 3.0 Thumb Drive and works fine from there. You may have to experiment with what you put in the ramdisk and see what gives your setup the most, if any, improvement. For me, it reduced load screen times and new cell load stutter outdoors.

 

A ramdisk won't help with script lag, per se; you are CPU bound with that. But if it takes a full minute for script updates at 50% CPU load, something else is wrong. Something is holding up a script thread, or several of them. You might try enabling Papyrus logging (can google for it) and see what it prints out during the delay after getting one of those TG jobs. You can post a bit of the log in the Support forum for help, especially if similar log entries are spamming over and over. SKSE loads before Skyrim and stays in memory; no need to put it in ramdisk unless you are worried about startup times.

 

I use the AMD Ramdisk and yes, you can tell it not to write to disk at shutdown. But, then you will have to copy files to it manually before using it. You will also loose any symlinks to the data in the ramdisk cause files won't be in the same place again. I don't think you can mount the ramdrive to a file, it has to be a directory and then your whole Skyrim data directory would have to fit in it. Either way, symlinks or a virtual mount, a lot of copying will be involved at some point. The data in the ramdisk goes away when you turn off power. Its not like a flash drive or SSD; its volatile storage.

Edited October 31, 2013 by Lord Garon

[Cliven]

 

Should my scads of hi-res textures be the focus if I go RAMDrive? Should I just try to cram onto RAM as many as I can, picking by what seems to be needed most often? I'm preemtively unloading cells (of course).

 

...

 

Additionally, will any of the virtual mount methods NOT try to write back to SSD or HDD at shutdown? I d/n need my static resources updated dynamically; I can just reload Windows, and would prefer to avoid excess time spent, and the writing to drive causing wear.

 

Thanks in advance for any advice, guys.

 

I used resource monitor to watch disk activity as I ran around outside and had stutter during cell loads. I could see the most accessed files and with my mods, they were my loose files (textures and meshes directories), and the #1 and #2 Hi Res texture packs. I put the loose file directories and the #2 texture pack in ramdisk. Helped a lot on my system, with my old HDD's, and my mod setup. YMMV, especially if you're on an SSD right now. But big texture files would be a prime candidate for a ramdisk. My Sounds bsa is actually on a USB 3.0 Thumb Drive and works fine from there. You may have to experiment with what you put in the ramdisk and see what gives your setup the most, if any, improvement. For me, it reduced load screen times and new cell load stutter outdoors.

 

A ramdisk won't help with script lag, per se; you are CPU bound with that. But if it takes a full minute for script updates at 50% CPU load, something else is wrong. Something is holding up a script thread, or several of them. You might try enabling Papyrus logging (can google for it) and see what it prints out during the delay after getting one of those TG jobs. You can post a bit of the log in the Support forum for help, especially if similar log entries are spamming over and over. SKSE loads before Skyrim and stays in memory; no need to put it in ramdisk unless you are worried about startup times.

 

I use the AMD Ramdisk and yes, you can tell it not to write to disk at shutdown. But, then you will have to copy files to it manually before using it. You will also loose any symlinks to the data in the ramdisk cause files won't be in the same place again. I don't think you can mount the ramdrive to a file, it has to be a directory and then your whole Skyrim data directory would have to fit in it. Either way, symlinks or a virtual mount, a lot of copying will be involved at some point. The data in the ramdisk goes away when you turn off power. Its not like a flash drive or SSD; its volatile storage.

 

 

Bother on the loose files nix. I can likely pool certain tex's as multiple virtual drives by extant individual folders, but the entire Texture folder is 16+ G right now, and I think that was already after optimization/compression.

 

Sounds.bsa may very well go in there; it is relatively small but was a huge drive read for some reason.

 

Not on a SSD yet; presently RAID 0 on twin SATA HDDs on Sys/Data volume, and another RAID 0 set for the paging file partition.

 

Fortunately, after a week of looking, it seems that I have finally found the mod conflict (2 different versions of "Birds") that caused my primary script ills. Not sure how I missed that in BOSS 6x, but I did.

 

Also reset the papyrus setup in the ini, and that helped a bit.

 

Unfortunately, Papyrus logging is useless; in 2 months of it being on, it has never once dropped any info other than the stock date/time/min-max RAM allocation.

 

Interstingly, SkyPerfMon thinks the drive seeks are very low, while Vista screams tht they are huge. GPU temp and use are maxing out (thinking this is due to ENBoost), tho, which was invisible to me before; will have to look at that.

 

**TY greatly** for the intel; M'lord Garon; it is going to be a great aid in planning out the hardware setup going forward. May get the SSD 1st, as it looks like that is the more vital component. THAT will be expensive, as it'll need to be a 1T drive.

Edited October 31, 2013 by Cliven