Way to decrease Precombined file size

[Zorkaz]

Whenever I make a worldspace the precombined meshes make up 1 to 2 GB of file size and there isn't much reduction by putting them into archives.

 

Does someone know how to reduce the size?

 

(Because Nuka World and DLCCoast have rather small size in comparison)

[YouDoNotKnowMyName]

I think that the developers had some "better optimization" for the precombined meshes.

And the version of the precombine-generation system we have got with the CK is very "basic" ...

 

So, precombines will always be very large, I don't think we can do anything about that ...

(But I am not the "optimization"-expert, I am satisfied when something works at all, I don't care about a few GBs in my data folder ...)

[Ablaze666]

The good news first:

If set up properly, the Creation Kit can automatically generate all the needed precombioned meshes, PreVis files and compress the meshess by 90% afterwards. Just with a bunch of batch files. Furthermore, the process that is generating PreVis files is optimized for multicore CPUs (I changed it in the CK ini to 16 threads and my CPU gets utilized 100%) and can utilized lots of RAM and VRAM (I've seen usage of 64 GB RAM and up to 20 GB VRAM in tests with my new hardware). In other words, it runs insanly fast compared to processing the files by hand. It is the same method the devs used to build all the precombined files. But there's the catch...

 

Now the bad news:

It's pretty complicated to set up. If done incorrectly, the CK won't start the workload and won't tell you why. And you need to enable Version Controll and work with it or you have a hell of a time with trying to merge the output plugins into a useable esp in xEdit. And even if all steps gets done in the CK, the results can still be bugged ingame (e.g. flickering objects etc).

 

Here's the link to a very detailed explaination of how it works and what you have to do to get started. Make yourself comfortable, get a tea, take a deap breath and jump down the rabbit hole:

 

https://forums.nexusmods.com/index.php?/topic/5522717-fallout-4-optimization-and-performance-systems-explained/page-38&do=findComment&comment=62587836

 

I managed to get the whole thing up and running and the results are worth all the trouble.

[YouDoNotKnowMyName]

But version control doesn't work with more then one master file .....

(That's why I don't use it ...)

[pepperman35]

Complicated indeed. Would be nice if someone made a step by step tutorial to establish the work flow from step A through Z.

[VIitS]

The reason that the linked person had issues with CtDs is because the CK, by default, is setup to generate... I want to say PS4-format precombineds when using commandline. You have to edit a couple hex values in the CK executable, or use Nukem's FO4 CK fixes (only has an initial release, but fixing that hex value is one of the things it does). If you want to edit the hex values, this post lists the values. The offsets will be different for the current version of the CK, but the string that needs to be edited is correct.

For 1.10.162.0 version of the CK, the string to find the first edit is
04 01 E8 63 B3 2B 02 B9 01 00 00 00
and just change the 01 to 00. The second is at
98 7C 03 00 74 11 B9 01 00 00 00
and again, just change the 01 to 00

You can use any hex editor to do this.

If you want to use Nukem's FO4 CK fixes instead, it requires the latest VS2019 x64 redist for it to work.

Also, here is a slightly cleaner list of instructions (from yarrmateys on the xEdit discord) than what was linked earlier:

  Reveal hidden contents

you could probably shorten that if you just run it on desperado itself
but if you use any other mod like bt interiors, those will still break them
you'll have to decide which form to go with. there are two, "clean" and "filtered". clean generates the same type of precombs like vanilla has, filtered generates the same kind creation kit generates. clean is much smaller since it stuffs shared meshes into an indexed file, but you need to redo the whole thing each time you modify one thing.
and also once you're done with precombs, you will need to pack them into a ba2 before generating previs, otherwise it'll crash during generation since ck can't handle loading more than 30k precomb files at once
CreationKit_hexedited.exe -GeneratePrecombined:file.esp clean all
to generate precombs.
then you open CombinedFiles.esp with ck and resave it to fix it. then open the fixed one with xedit and deep copy as override with overwriting back into desperados.
once that's done,
CreationKit.exe -CompressPSG:file.esp
CreationKit.exe -BuildCDX:file.esp
at this point the precombs need to be packed into desperados's ba2 file, and then remove the files from the folder so creation kit can't get them from there and needs to use ba2.
CreationKit.exe -GeneratePreVisData:file.esp clean all
then you do the same with previs.esp as you did with combinedobjects.esp. open in creation kit, save, open in xedit, merge with desperados's esp. then pack the vis files into the same ba2.
that should be it.
the end result will probably be around 1-2 gb
now for the other method
CreationKit_hexedited.exe -GeneratePrecombined:file.esp filtered all
then open combinedobjects.esp in creation kit, save and xedit merge with desperados's esp. then pack the precombineds into ba2 and remove loose files.
CreationKit.exe -GeneratePreVisData:file.esp filtered all
then do the same to previs.esp as you did before and this should be it

Easiest way to merge the previs and precombined .esp plugins that are generated using xEdit is to just use "Copy as override (with overwriting)". If it complains about not being able to add a master, you can first use "Inject forms into master" (right click on the plugin name), but that shouldn't be needed in this case as the plugins generated by using the commandline arguments shouldn't have any new records, just CELL overrides and references to records from the plugin you are generating for.

Just make sure the VC Info 1 data doesn't show "None" on any REFRs. If it does, there is a script included with xEdit for copying VC Info data, but you have to remove the "masterlist must match" requirement for it to be useful first, here is a link with my tweaked version to remove that requirement (with explanation of why it isn't needed).

[YouDoNotKnowMyName]

Ahh ...

That's why after generating preombines, ther is a folder named PS$ in the data folder.

I never knew that!

Interesting stuff ...