Fallout 4 Optimization and Performance Systems Explained

[VIitS]

I think it would work better if the previs system was modular, and the the plugin they gave us for max (or even better, with the CK, the same way they let you make precombined meshes) allowed you to generate an (ultra low-res) occlusion mesh for placed objects. That way you could selectively disable parts of it. I know the game still uses occlusion planes in some places (like the city), though like room bounds/portals they probably only activate if previs is off for the cell.

 

If you do actually have working previs in an interior cell, you don't have to worry about room bounds/portals. It's if you don't have working previs that you have a problem, because Bethesda created them under the previs, but since they only show if it is off they didn't actually put in the work to make it properly. So in places you'll have things popping in and out of view depending on what angle you are looking from, and the biggest problem is when this happens at the borders of a room due to the Void*. Annoyingly, previs for interior cells (or maybe it is just due to the size of the cells for Vault 88/the Mechanist's Lair) is harder to make. The precombined meshes work fine***, though I wonder how much they do for performance. From what I can tell, if they do actually stop certain polys/vertices from rendering when they wouldn't be visible due to intersections, it is only if the entire poly is hidden (which makes sense). Most of the vanilla meshes are fairly low poly, so you don't end up with a ton of non-rendered polys. Though the precombined meshes they generate with their actual dev software (as opposed to the shitty knockoff they gave us) are a lot smaller than the meshes that make them up, so they might render more efficiently even if nothing is hidden.

 

 

Yeah, some people are confused (though they are certain they are right :laugh:) and think that two mods that add things to the same cell (without touching previs/precombineds) will be completely incompatible, but they are fine. Really, the only time you are likely to have problems is when two mods make small changes to a cluttered area that would require new previs/precombineds. Any mod big enough to justify previs/precombineds on its own will probably be incompatible with other mods placing things in that cell just due to the likelihood of intersection, and mods that only touch cells that aren't too "busy" can get by with just disabling it.

 

I'd say the "baked in" functionality is part of vanilla ones as well. Some of the precombined meshes in the Mechanist's Lair included Static Collections that weren't actually in the .ba2 files (it was easy to tell by the weird folder names they were supposedly in). I had thought the data in the precombined mesh might have just pointed them to the original meshes (since they deal individually with the separate parts within a mesh), but I guess it might just be that they have enough data to not need to. It's annoying that the CK generates them in a completely different format than the precombined meshes in the vanilla/DLC .ba2 files, not only are they 5-10 times bigger but (currently, at least) they are completely unreadable by nifskope, so you can't even edit them**.

 

*I have no idea why they made the "background" of interior cells behave like that. It'd be fine if it was just pure white, but it causes a trailing image effect if you look in a direction that would let you see it (and that's for the people it doesn't just cause CtDs for). It's like they are actively f*#@ing with us.

 

**While it is a pain in the ass to figure out, not just how to do it but how to identify what part goes to what, it is possible to edit the shipped-with-vanilla precombineds.

 

***even though they are bloated, and figuring out which ones I can use from vanilla was a pain in the ass since the Cell Record is so big (over 22000 references combined into just under 1600 precombined meshes, editing anything in the cell record took forever). I ended up using FO4Dump to get the Cell data as a text doc, Find and Replace (using wildcards) to trim the info about each record to just Reference ID and precombined mesh, re-organized and sorted it with excel (so the mesh info was on the same line), and used the Compare plugin for Notepad++. It turned out that the CK made 3 new precombineds meshes and randomly shuffled a few references between precombined meshes because reasons. But I was able to figure out I only had to include 109 of the 1593 precombined meshes used in the cell once I un-shuffled the precombineds it messed with needlessly, which requires editing the info in the Cell record. A tip for anyone planning on doing so yourself: save often (either ctrl+S, or using the "save a copy" feature using F5). For some reason, xEdit crashed every 3-5 edits to the reference info in the XCRI field, but once I started saving regularly I didn't crash once.

[Giggilyomeromicon]

I'm pretty sure that occlusion boxes and planes do work with Previs, actually. I've been messing around wit adding occlusion boxes around Postal Square, and there are a couple of spots where culling seems to work better with a combination of both.

 

What I would really like is to find out if there's a way to make occlusion occlusion culling more accurate or more liberal with what it blocks. Right now there are some pretty massive FPS drops around Boston because even objects fully blocked from the player's view continue to render. This is a huge problem on longer streets, as draw calls can get up around 12,000 in certain areas, despite very few meshes actually being visible.

[SMB92]

Thanks for all the info Vlits. As I said, what f*#@ery is the precombined mesh system. In my opinion it doesn't gain that much performance to warrant the regeneration or even use of it. But precis relies on it. The fact the public CK produces 5-10 times larger files explains the small stutter/pause I see when transitioning from an area with custom generated data to vanilla. I would imagine that also the overheads lost in our generations would account some way toward why I don't see reasonable or sometimes even noticeable gain from the meshes, but then as you say it only gened so many new files. IMO it is simply not feasible to do the methods you state in comparing and truncating new combinations even if just for the sake of console modding, I can think of a thousand ways to better spend my time ;)

 

@Giggily - Every time I tried generating any combination of data around where Bethesda left that occ box, the whole area of generation was totaled with occlusion bugs. Maybe its just that unique instance but I have not tested adding occ boxes. Disabling the box and regenerating, everything works as intended and it just makes sense in my mind. Just trying to find the cause of the problem in that area and stumbling upon this adds up to me.

 

12000 draw calls you mention. I record as high as 38000 downtown. Currently 26000 at East City Downs with BTI without new precull data. Some other areas I see 18-20000. Yet my Sanctuary has the lowest fps at 35fps lowest point, 15000 i think it was.

[Giggilyomeromicon]

@Giggily - Every time I tried generating any combination of data around where Bethesda left that occ box, the whole area of generation was totaled with occlusion bugs. Maybe its just that unique instance but I have not tested adding occ boxes. Disabling the box and regenerating, everything works as intended and it just makes sense in my mind. Just trying to find the cause of the problem in that area and stumbling upon this adds up to me.

 

12000 draw calls you mention. I record as high as 38000 downtown. Currently 26000 at East City Downs with BTI without new precull data. Some other areas I see 18-20000. Yet my Sanctuary has the lowest fps at 35fps lowest point, 15000 i think it was.

 

I've also ran into situations where there are massive culling bugs, but they aren't related to the use of occlusion planes/boxes for me. Not sure why you would be having problems with it, but I can't understand half of what happens with the CK anyways.

 

I'm also starting to realize that the elevated highway in the downtown area is a source of massive FPS loss. Looking at a block of buildings with the highway blocked is about 500 draw calls, that same block about a foot to the right, with a chunk of highway visible, is 8000. Not quite sure what to do about that.

[MissingMeshTV]

Something I've been wondering for a while that maybe I could get some insight from others on:

 

Since regenerating precombines and previs will:

1. Increase the final size of a mod's files

2. Require external assets

 

WTF was Bethesda thinking when they decided to get into the mods on consoles business with the system they used in this game? Mods that alter the worldspace that benefit performance-wise from regenerating precombines and previs are a NO for the PS4 from the get go because of the external assets thing. This size of the final mod files can potentially take a large chunk of the Xbone's meager mod storage allotment when multiplied. So...did they not think about this? Is the performance of the game that bad on consoles already that console users don't notice a performance hit when using a bunch of mods that disable the premombine system in various cells?

 

I've always been of the belief that Bethesda announced mods on consoles without actually having tested it or consulted with the console makers...just look at the whole PS4 fiasco and the delays in getting them released for Xbox. Announce it first to sell the game, then figure out how make it happen.

 

Why would anyone sign off on such a f*ed up game engine mechanic as precombines without thinking of the impact on the console use if they were pushing on opening consoles to mods?. Am I missing something?

[chucksteel]

Well I test my mod Beantown Interiors on the XB1 and I don't see any real performance hit with the way the mod is designed. I however don't run more then 30-40 mods on the console and mostly use it for testing and bugfixing my mods.

 

I also don't think the performance of the vanilla game without mods on the console is all that bad, I have never seen any issues that I haven't seen on the PC version. (We all know the trouble spots)

 

That being said I don't think I would ever recommend running more then say 100 mods on the XB1 (I'm being generous here and probably would say far below that number myself.) Also keeping that to only a few system intensive mods.

[Giggilyomeromicon]

I don't see how precombination is a bad idea. The alternative is to just not have occlusion culling at all, and I'll be honest, I'll take occlusion culling that's disabled when a mod changes a cell versus no occlusion culling at all.

[chucksteel]

It's only really bad from a modding point of view. The fact that it makes mod compatibility a tricky subject is part of it. The other thing that is bad would be the funky tools we get that create oversized files, the XB1 having a 150 mod cap with only 2gb space available making me and other choose to just disable the optimization or not share.

[Giggilyomeromicon]

I'd say that the concept is sound, and I really can't think of any alternatives, but the execution has a lot of problems. The CK creating massive filesizes for precombined geometry is one thing, yeah, but the fact that Previs actually needs the precombined geometry files present for the user is what confuses me. The entire point of the Previs system is that it isn't using the precombined geometry's information in real time, that's all done in the editor, so I just don't get why the system wants the user to have the meshes as well. I'm pretty sure you can have generated Previs without the modded precombination files, so it's really just kind of baffling.

[VIitS]

Something I've been wondering for a while that maybe I could get some insight from others on:

 

Since regenerating precombines and previs will:

1. Increase the final size of a mod's files

2. Require external assets

 

WTF was Bethesda thinking when they decided to get into the mods on consoles business with the system they used in this game? Mods that alter the worldspace that benefit performance-wise from regenerating precombines and previs are a NO for the PS4 from the get go because of the external assets thing. This size of the final mod files can potentially take a large chunk of the Xbone's meager mod storage allotment when multiplied. So...did they not think about this? Is the performance of the game that bad on consoles already that console users don't notice a performance hit when using a bunch of mods that disable the premombine system in various cells?

 

I've always been of the belief that Bethesda announced mods on consoles without actually having tested it or consulted with the console makers...just look at the whole PS4 fiasco and the delays in getting them released for Xbox. Announce it first to sell the game, then figure out how make it happen.

 

Why would anyone sign off on such a f*ed up game engine mechanic as precombines without thinking of the impact on the console use if they were pushing on opening consoles to mods?. Am I missing something?

 

Yeah, for all they hyped up modding, they don't actually give a s***. They definitely didn't talk to Microsoft or Sony before claiming people could use mods on consoles. I'd guess they just expected public pressure to force them to give in. And they definitely didn't expect Sony to be so obsessed with their proprietary formats that no one else gives any shits about (beyond cases like this where it keeps them from getting something). Seriously Sony, no one gives a s*** (them thinking their proprietary BS is so amazing is also the reason blu-ray disks are so expensive, compared to what an HD DVD of equivalent capacity would have been).

 

Also, if you look back a little, you'll see that (if the MA is willing to put in the time figuring it out) any precombined mesh that is identical to vanilla* can be left out of the mod, greatly reducing mod size. That doesn't help for PS4, though you could just remove the altered precombineds (and the references relevant to them), and let it still use the rest of the precombined meshes (just make sure the PCMB timestamp is updated). You can even let it use the vanilla previs info as long as you didn't remove.move anything big enough to fully block the player's view.

 

So you could technically still make partial use of the system for the PS4. Of course, it is a lot more work for minimal gain, and most MA's play on PC, so PS4 players are still SoL for the most part, as very few people are gonna put in that work (especially after the ClusterF**k that was the first launch of FO4 mods on consoles). Also, the things I am talking about are ONLY possible using FO4Edit (both figuring out which ones stayed the same, and telling it what to use and not use), the only way you can mess with that stuff in the CK is in bulk (generating/disabling/updating for the entire cell).

 

*as far as what makes it up, it's not actually identical because the vanilla precombined meshes are significantly more optimized than the ones we can make

 

 

On a side note, did Bethesda disable their forums or something? I just tried to update the beth.net thread linked in the first post with some of what I have since learned, and couldn't reply to it. Thinking it might just be locked, I checked and couldn't find any thread I could reply to. I then tried to make a thread asking if they could either give us properly optimized precombined creation tools (so our precombined meshes are the same size/format as vanilla ones) or at least make it so the game doesn't generate new ones when the existing ones would work fine, and every single place is "not available for use".

 

So either they shut down their forums and I missed it (I'm kind leaning towards this, since a quick check failed to show a single post in the last few months, for several games) or I got banned for some reason and they decided not to tell me (not even in my profile info).

Edit:

 

 

are you using the new site? Bethesda.net The old Jive forums have been shut down some time ago.

Yep, that would be the problem. Seems strange how they just left it like that, with no indication in the old forums about the existence of the new ones. So following an old link (like I did) will make you think it's dead.