572 replies to this topic

[damanding]

I wanted to better understand all the various terms related to optimization/performance improvement and what they mean for mod authors.  So I'm using this topic to summarize information pulled from multiple sources.  I did receive clarification from VlitS on some aspects of this (VlitS being one of the primary experts on this subject matter).  Please feel free to correct anything I've explained incorrectly.  I'm hoping that with community help we can clarify these topics better and maybe include the final draft into the CK wiki.

 

GamerPoets has just released a 4 minute video explaining this in very simple terms for the average mod user. If the rest of this looks super scary and intimidating to you, go watch his excellent video instead! 

 

Basic Definitions

Spoiler

Preculling/Occlusion/Occlusion Culling:

These are basically all words that mean the same thing.  If an object is hidden from the player's view (because it's on the other side of a wall or hedge for example) it won't be rendered.  3D computer programs like video games which need to render complex objects usually use some method of preculling/occlusion.

"The purpose of occlusion culling is to find geometry that has zero effect on the final render buffer because it is behind other geometry, and eliminate that geometry as early in the render process as possible to reduce wasted cpu and gpu time." - http://geck.bethsoft...clusion_Culling

 

Precombines:

This basically combines similar meshes into a combo mesh that is rendered instead of the individual meshes.

"Precombination works by analyzing ref 3D for a cell and merging like geometries across similarly located refs into a different set of NIFs. For example if you had 3 refs to RockWithLichen and 1 ref to RockWithNoLichen tucked together in the corner of a cell, precombination would merge all 4 rock geometries together and all 3 lichen geometries together into a single NIF named for the combination key for that corner of the cell (eg: "Meshes\PreCombined\0000DBBB_01B8656E_OC.NIF"). The cell would store the form IDs for those combined refs so it would know not to load 3D for those NIFs in game since their 3D will already be baked into the combined NIFs." - Ziegfelding (Bethesda staff) 

 

Previs:

As VlitS explained to me via PM, Previs is basically the preculling/occlusion system used by Fallout 4 (versus some type of system used by other games or programs).

"Previs is precomputed visibility data and it is calculated using the precombined NIFs. It builds a visibility graph of which non-mobile uncombined refs and combined geometries are visible from different points which we then use in game for preculling the scene. This data is stored for a 3x3 collection of cells in the UVD files (eg: "Vis\Fallout4.esm\0000DBB9.UVD")." - Ziegfelding (Bethesda staff) 

 

Roombounds/Portals:

This is the older system for optimization used in multiple Bethesda games.  It can still be used in Fallout 4 in addition to the precombined/previs system which is the system for optimization first introduced in Fallout 4.  This only applies to interior cells.

What do Room Bounds do?

A room bound will section off an area to be rendered. If the player is inside of a room bound, nothing outside of the room bound should be rendered. If the player is outside of a room bound, nothing inside of the room bound should be rendered.

What do Portals do?

Portals are visual gateways between room bounds. They should be placed between rooms at any location where the player should be able to see from one room to another. This is generally any location where doors or hallways connect one room bound to another.

To have a properly functioning Portal between two room bounds, it must be connected to both room bounds. - https://www.creation...d_Portal_Basics

In addition to only applying to interior cells, roombounds and portals are apparently not compatible with interior cells which will be made into settlement workshops.  So if you're making an interior settlement don't use roombounds and portals. However non workshop settlement interiors would still benefit from using them.

 

 

What this means for mod authors/users

"Blinking world syndrome", seeing into "the void", and "disappearing objects" are all common descriptions of what happens when precombined meshes are broken or removed but the previs information for a cell is not recalculated or disabled.  This can happen if you are a mod user using a mod that lets you scrap objects not originally intended to be scrapped by the vanilla game such as Spring Cleaning or Scrap Everything.  This can also happen if you are a mod author using the Creation Kit (CK) to disable (remove) vanilla objects like buildings or hedges.  You can often stand in one spot and just rotate your camera view to see objects appear and disappear.

 

Gif by FortuneQuaker to demonstrate this:

Spoiler

Source

 

Mod Users

Spoiler

As a mod author or user you have a few options for how to handle this, each option has positive and negative sides.  The whole point of using the precombined/previs system is to improve performance.  So disabling that system will make your performance worse, you will get an FPS drop.  There are two primary approaches to disabling the system, the first is using an ini file change and second using the CK to  "Clear Affected Cells For PreCombine Data".  Mod users can either apply the ini file change ([General] bUseCombinedObjects=0) or use a mod like FortuneQuaker's "No More Disappearing Act - Cell Level Pre-culling Disabler" which uses the CK to clear previs data just for settlement cells (where scrapping mods are most likely to be used).  Using the ini file change causes a significant performance drop throughout the entire game and is generally NOT recommended.

 

Mod Authors: Recalculating Data

Spoiler

If you are a mod author you have some additional choices.  If your mod disables objects used in precombined meshes, such as the Red Rocket building for example, you have two choices.  You can recalculate the precombined/previs data for the Red Rocket cell -or- you can disable the precombined/previs data for that cell.  Recalculating that data leads to improved performance for the end user but also means your mod will no longer be able to be compatible with any other mod that makes changes in the same area without creating a compatibility patch. Recalculating that data also makes your mod's file size MUCH larger.  Having large file sizes is a problem for Xbox users that have much more limited space. In addition, by recalculating that data your mod now has external files which means it cannot be used by PS4 players. Using the Red Rocket as an example, if you make a mod that cleans the debris and otherwise improves the settlement and you recalculate the precombined/previs data for that cell then another mod author that simply adds a new perk magazine onto the counter inside the Red Rocket building (that was innacurate) disables the counter inside the Red Rocket will now be incompatible with your mod.  Adding new items only does not break existing precombines. Only disabling or moving items will break precombines. Edits to the landscape such as smoothing the terrain or even changing the ground texture will also break precombines, any changes to the LAND records in xEdit are precombine breaks.

 

There is now a script for FO4Edit available which will clean unnecessary precombine data from your mod thanks to Zilav (scripting) and VlitS (hosting/explaining): Precombined Cleaning Script.  This lets you clean out the precombines automatically generated which are actually unchanged, keeping only the precombines actually changed to reduce file size.

 

Mod Authors: Disabling Precombines/Previs

Spoiler

Therefore, depending on location, it is often better to disable the precombined/previs system data for the cell you are editing instead of recalculating it.  If you are editing a cell in downtown Boston then performance will unfortunately be the higher priority.  However if you are editing the Red Rocket cell which is remote and does not have as many objects in it, then disabling the precombined/previs data for that area will have a much smaller performance impact.  In this case greater compatibility between mods is more important and it is a better idea to simply disable precombined/previs data.  This approach is mostly recommended with exterior cells that are not in downtown or otherwise very full.  Disabling precombines/previs in interior cells will default to using the roombounds/portals system instead.  The vanilla game ships with either poorly done or non-existent roombounds/portals for interior cells.  So you may need to repair roombounds and portals for interior cells.  I recommend Darkfox127's tutorial on roombounds/portals for Skyrim which still applies to Fallout 4. 

 

The Beantown Interiors mod by Chucksteel is a good example of why recalculating precombined/previs data is frequently unnecessary.  This mod changes locations all over the map and does not recalculate precombined/previs data.  Because this mod changes so many locations it would become incompatible with hundreds or even thousands of location changing mods if it did recalculate precombined/previs data.  However this mod has a very large user base and seems to have a minor performance impact for most users.

 

How to Recalculate Data

Spoiler

These directions are directly copied from a post by Trosski because he explains it well and also gives some good tips to avoid common mistakes.

1. Open the CK, and Load Fallout4.esm

2. in the Cell View window's world space drop down menu select Commonwealth

3. After the list populates select the cell you wish to edit. *Note: Settlements will often span multiple cells.

4. If you want to remove an object from a settlement, I will use Sanctuary as an example, because it spans multiple cells, then find the central cell (SanctuaryExt) and make the edits you want int that cell(ONLY IN THAT CELL)

    You will know if the object you are selecting is in your current cell, because it will move to and highlight that object in the cells object list.

     It is best to load the cell, leave it in top down veiw, and zoom out to see the whole cell. I have found that this keeps me from crossing into other cells wit the camera, and I select objects from the list instead of in the render window.

5. Select the item you want to remove. Right click on it and select edit. Check both "Initially disabled" and "Hidden From Local Map", then click "ok".

6. Once all of the edits you want to make are complete in that cell, in the World menu at the top click on "Precombine Geometries for Current Cell".

7. Repeat Steps 4 - 6 for each cell you want to make edits to. It is best to note which cells are loaded when you load that first cell, so that you do not stray from the loaded area.

8. Once all of the cells you wish to edit are complete, Load the first cell you edited... In this example it will be SanctuaryExt.

9. In the Visibility Menu at the top click on "Generate *Precombined* Visibility for Loaded Area" *Note Prcombining geometries for each cell, and generating visibility for the loaded area is what fixes the dreaded Cell reset, Sprinting, and object culling bugs.

10. After Generating visibility ***SAVE YOUR PLUGIN!!!*** I save my plugin after I edit each settlement, and then move a copy of the plugin to a separate folder just in case I screw something up (Which I do... A lot!).

11. After you have made all of the changes you want, save one last time. In the File menu click on "Create Archive", and name the Archives I generally make it "Mod name - Main.ba2" for the main, "Mod name - Textures.ba2", for the textures, and so on.

Then just Upload to Beth.net, or Nexus, or where ever else you would like. 

Source

This thread also has good information on Regenerating precombined/previs data

 

If you need to create compatibility patches for multiple mods editing the same cell with precombine/previs data, Trosski also has the following to explain how:

If you have two mods that modify the same cell you would load them both in the CK with none set to active, and just generate new previs data, and save it as a patch.

Source

 

New tutorial video up courtesy of RedRocketTV on how to regenerate precombined/previs data for any mods you use that break them. This video is designed more for mod users.

Older tutorial video up courtesy of Trosski on how to regenerate precombined/previs data with much more technical data than the RedRocketTV tutorial and is designed more for mod authors.

 

How to Disable Precombined/Previs Data

Spoiler

You can right click on an object in the CK and select "Clear Affected Cells For PreCombine Data" however this will clear precombined data for that object in every single cell it's used in throughout the entire game.  You will then want to use FO4Edit to remove that change from all cells other than the one you actually intended to change.  This forum thread on Clearing Affected Cells has much more in-depth information on this topic.

 

Alternatively you can use FO4Edit to remove the VISI, RVIS, PCMB, XCRI, and XPRI info from the Cell record the mod touches (either manually, or using this xEdit script).  GethN7 has developed a new script that removes preculling data from both interior and exterior worldspaces now.

 

General Considerations

Older threads and articles on this topic will mention that altering precombines/previs data in some way will cause the cell reset and sprint bugs.  That was certainly true at one time however Bethesda fixed those bugs in late spring of 2016.

 

If your mod does not disable or otherwise alter any objects in an exterior/interior cell and only adds a new object such as a weapon, clothing item, magazine, etc. then you do not need to worry about the problems associated with the optimization systems in Fallout 4.  Adding new objects (references) will not change precombined/previs data.

 

Source Material

Spoiler

Explanation of precombined/previs system by a Bethesda employee. I recommend reading at least the first page of that thread for a better understanding of why this system is a compatibility problem for mod authors.

Occlusion Culling Fallout 3 GECK Wiki

Occlusion Culling Wikipedia

Article on No More Disappearing Act - Cell Level Pre-culling Disabler mod

Room Bounds and Portal Basics Creation Kit Wiki

Clearing Affected Cells

Regenerating precombined/previs data - Trosski's answer.

Regenerating precombined/previs data - FiftyTifty's answer.

Best practices for adding new items

FO4Edit Script & Explanation Articles for Cleaning Precombines

Detecting Broken Precombines on Consoles

[chucksteel]

Another reason I decided to not rebuild the Pre-Combined and Pre-Vis is simply because if I did it would be so large that it would be the only mod an XB1 player would be able to use if they still wanted it. 

 

I build my mods equally for all platforms (other then Sony's restrictions) 

 

Edit: Beantown does generate Pre-combined and Pre-Vis in two locations "Forest Grove Marsh" and "Nahant", This is because disabling the Pre-Vis breaks the water tiles giving them a glitch where they loose their textures when viewed from some directions. 

[caleb68]

I've noticed with previs/precombines taken care of, you may still have issues with LOD in areas, while the npc is close enough to the cell everything they scrapped will be gone, but if they are two-three cells away, the old LOD kicks in and it looks like the old structures are there.

Even when the Nukaworld DLC was released, heading to the trainstation that sends you off to nukaworld, if  you watch the overpass, you'll notice this same effect take place.  Without rebuilding the LOD, your kinda stuck with this effect far as I can tell.

Edited by caleb68, 09 April 2017 - 04:25 am.

[greekrage]

This  should  be  a  sticky...

 

VERY  interesting  read and  very  informative.

 

As  one  that  highly  edits  locations  i  find  it  to  be  a  serious  "eye opener"  and  makes  me  even  more  scared  to  remove  anything  now  

The  problem  is  that  i  usually  remove  (disable  and hide)  all  most  everything  on  the  surface  of  my  location  overhauls.

 

Is  it  possible  to  "fix"  mods  that  are  all  ready  released  where  all  objects  are disabled  and  sunk ?

[VIitS]

Looks like a pretty good job of bringing all the information together into one place. There are just a few things that I might clarify a bit.

 

1st, I personally only use the "the void" for interior cells, because most interior cells, when you are seeing beyond all the placed objects, is just a blank whiteness, and when looking near those "glimpses into the void", you get the "hall of mirrors" effect, where things leave a sort of trailing afterimage when you move your camera (turning or moving). Not something that really matters, just thought I'd clear that up.

 

2nd, about compatibility: two mods that create new previs/precombineds will always conflict without making a patch for them. But when you have a mod that just adds things and doesn't need to disable/remake previs/precombineds, it will be fine as long as you have it loaded before the one that generated new precombines. And if the second mod chooses not to generate new precombines/previs, it will have the same problems if it is loaded first as they would if the MA had chosen to generate new precombineds/previs (because the small mod that doesn't require new previs will have the vanilla info in the cell record).

 

That's why mods like Scrap Everything say to put them at the end of your load order. It isn't strictly required, there are a number of mods that don't touch the vanilla settlement cells and therefore wouldn't cause problems. But if, for instance, someone had a mod that added an NPC to spawn at the Red Rocket, it would have the vanilla cell info for that location and would essentially disable SE for that location if it loaded after SE. The advantage of straight up disabling precombineds/previs (instead of making new ones) is that you can just recommend your mod goes at the end of the load order, because no mod (as far as I know) can be broken by precombineds/previs being disabled. So in that way, you could consider Scrap Everything to also function as a compatibility patch for any mod that edits the vanilla settlement cells. The only mods that would be affected by loading before a mod that disables precombines (for a given cell) are mods that rely on direct edits to the cell records (which are vanishingly rare). The only disadvantage to just disabling (instead of regenerating) is, of course, performance. As you noted, for most cells (for most players), the performance impact isn't significant.

 

 

This  should  be  a  sticky...

 

VERY  interesting  read and  very  informative.

 

As  one  that  highly  edits  locations  i  find  it  to  be  a  serious  "eye opener"  and  makes  me  even  more  scared  to  remove  anything  now  

The  problem  is  that  i  usually  remove  (disable  and hide)  all  most  everything  on  the  surface  of  my  location  overhauls.

 

Is  it  possible  to  "fix"  mods  that  are  all  ready  released  where  all  objects  are disabled  and  sunk ?

Not really sure what you mean. You can make new previs/precombineds for already released mods and just upload them as new/optional updates. For your mods that you didn't make new previs/precombineds for, unless you are getting complaints about performance, your mods should already disable precombineds/previs (due to how Bethesda made the system work).

 

 

 

Really, the only problem (in exterior cells) you will ever have with previs incompatibilities is if mod A removes certain things that were used in the generation of previs (which means they were in precombined meshes), and mod B either has the vanilla previs/precombineds info or remade previs/precombineds with the removed items active, the things mod A removed will come back. Pretty much the only time you will see occlusion issues is if the bUseCombinedObjects=0 ini edit is used. This is because Bethesda made it so, if a mod touches a record that was part of a precombined mesh, previs and precombineds will automatically be disabled if you don't generate new previs/precombineds for that cell. However (and I have no reason why they did this), using that ini edit will disable the automated workaround, allowing precombined meshes to be removed while leaving the occlusion meshes generated by the previs system. That's why I tend to recommend actively disabling previs/precombineds (in the way damanding notes*), as previs will remain disabled even with the ini edit.

 

The main problem is with interior cells, due to the bad room bounds/portals damanding mentions, and the graphical issues caused by "the void". So if you are editing a vanilla cell, you are practically guaranteed to have problems if you don't either fix the room bounds/portals or regenerate the previs/precombineds.

 

 

*although I will add that the xEdit script he links doesn't actually have the XPRI removal as part of it. Many cells don't have anything in XPRI ("combined physics references", things like lights and water features), and while I am fairly sure they still get disabled (because precombineds sadly seems to be all or nothing), but I like to be thorough.

[VGfallout]

Thanks for this, damanding

 

I greatly appreciate this information; hopefully these founding guidelines should steer folks in the right direction

 

Again, gratitude!!

[Fkemman11]

"The main problem is with interior cells, due to the bad room bounds/portals damanding mentions, and the graphical issues caused by "the void". So if you are editing a vanilla cell, you are practically guaranteed to have problems if you don't either fix the room bounds/portals or regenerate the previs/precombineds."

 

One thing I'm still not completely understanding is why interiors run so much worse than "outside" cells. What exactly is being done in interiors to make them run so much worse? Is it the lighting effects? Is it the mist? Is it simply bugs or piss-poor optimization? Is it a combination of all of these? Is it possible to edit interiors so that they are more efficient and less "buggy"? I ask because I am tired of having to run one mod for interiors that makes it look like s***/. I cannot afford to just go pick up a better system to run this game and should not have to anyway.

This "practice" of making half-finished poorly optimized games that perform erratically is annoying at best. Is this incompetence or just plain laziness on the part of the devs? This reminds me of why some people used to hate Italian sports cars. Has Beth always been guilty of producing games that are seemingly very inefficient?

​Which reminds me why- even on console versions- do some scripted scenes suffer from drops in framerate and lag?

 

Sorry to rant and ask so many questions. But I am trying to dissect this beast with some very blunt tools and do not like stumbling in the dark- so to speak.

 

@ Damanding. I also thank you for starting this thread. Obviously the more the community knows, the better they will be able to work on this. Despite some claims I have heard about FO4 modding to be done, I still think that we are only seeing the beginning of what is possible for this game. And we have a long wait for the next.
 

[zilav]

One thing I'm still not completely understanding is why interiors run so much worse than "outside" cells.

That's what happened

 

 

Instead of using large kit blocks as in previous games, they opted for smaller pieces to achieve more freedom and variety in level design. This didn't add that much strain in exteriors (apart from dense urban cells like Diamond City) because exterior worldspaces have LODs and load only objects in 5x5 cells area around the player. Human height is 128 game units, exterior cell's size is 4096 units, so around 20000x20000 area. Interior cells don't have LODs and load everything at once, the max size of interior cell is around 60000x60000. This means that in the worst case of some very huge interior cells, they can put up to 27x times more strain on the system compared to exteriors because of possible 3x size increase on each axis.

 

In Skyrim they solved that problem for large interiors making them as exterior worldspaces with proper LOD optimizations, for example Blackreach which is actually an exterior worldspace but made to look like a giant cave. But in FO4 they rely on previs system to combine smaller objects back into larger ones. Needless to say that system is overcomplicated and fragile as described above and completely unfriendly to modding.

[Fkemman11]

@ zilav Thanks for the response. I suppose I understand a bit better now. That doesn't mean I like it though. What worked for Skyrim should have been re-used and improved upon. I mean, I don't see any improvements with the what they did for FO4 and I think I understand why mod authors might have more trouble working with it. Will take some time for me to learn more as most of this is very new to me.  

[damanding]

I wish they'd just broken some of the larger dungeons up with at least one more load door.  That would have helped.