PyFFI - Why isn't it more popular?

[AllisterHenderson]

Pellape> Yeah I like Nifskope too, but only know a few features, so want to learn a lot more about it :smile:.

 

Pherim> Thank you for clarifying my poor understanding, that it doesn't matter about collision geometry because they are not drawn by default, and that it is visible geometry that we have to be worried about. I was not aware that collision geometry was handled by the CPU, I thought anything mesh or texture related was handled by the graphics card.

 

Thanks for the stern warning regarding using Pyffi 2.2.2 on everything :unsure:.

Okay, so when I eventually finish my mod build, I will:

1. Run Sniff, on my Meshes directory to workout which meshes have NiTriStrips with large numbers of excess strips.

2. Check if mesh isn't a character, creature, animation, armor / clothes model.

3. Run Pyffi 2.2.2 on meshes left that fit above parameters....

 

DrakeTheDragon> Hello sir moderator :smile:

When you refer to Morphed Meshes, do you mean this, where the original mesh's shape is changed. Is there a utility that I can run after I run Sniff, that can determine which meshes have morphs, and thus will leave those meshes alone.....

[Pherim]

Pellape> Yeah I like Nifskope too, but only know a few features, so want to learn a lot more about it :smile:.

 

Pherim> Thank you for clarifying my poor understanding, that it doesn't matter about collision geometry because they are not drawn by default, and that it is visible geometry that we have to be worried about. I was not aware that collision geometry was handled by the CPU, I thought anything mesh or texture related was handled by the graphics card.

The graphics card handles, well, graphics. Yes, modern games can also use the GPU for other things, including Physics calculations, but when Oblivion was released, real-time physics were a pretty new thing in games, and only the CPU was used. Therefore, collision geometry must be as simple as possible, because the GPU is not good at calculating complex geometry. As I said, I suspect that multiple strips in collision geometry are there to split the meshes into even simpler pieces, and only those that actually collide with something have to be calculated. However, that is just a theory and it might be difficult to test it, at least until I found a way to make collision geometry with multiple strips. Strangely enough, the "Stitch strips" setting in the Blender plugin only affects visible geometry, collision geometry is always stitched, as far as I can tell.

 

Thanks for the stern warning regarding using Pyffi 2.2.2 on everything :unsure:.

Okay, so when I eventually finish my mod build, I will:

1. Run Sniff, on my Meshes directory to workout which meshes have NiTriStrips with large numbers of excess strips.

2. Check if mesh isn't a character, creature, animation, armor / clothes model.

3. Run Pyffi 2.2.2 on meshes left that fit above parameters....

Either that or try to convert strips to shapes with Sniff, that might be the safest option, though of course you won't have any additional optimizations Pyffi might do. If you use Sniff, be aware that it copies the files into the specified output folder, and you have to merge them with your mod/data folder (depending on how you install your mods) manually.

[DrakeTheDragon]

DrakeTheDragon> Hello sir moderator :smile:

When you refer to Morphed Meshes, do you mean this, where the original mesh's shape is changed. Is there a utility that I can run after I run Sniff, that can determine which meshes have morphs, and thus will leave those meshes alone.....

Just Drake is absolutely fine here. I'm among friends or family in the Oblivion section.

 

That article could be related. It's just a fixed term in Oblivion modding. Animations can be either bone-based (SK files) or morph-based (EGM/TRI files), while the latter is usually used for faces, expressions and such. The EGM file for example contains all potential shape morphs you can apply onto a head mesh, while every slider in CharGen controls the amount for 1 of them. Similar goes for the TRI file, which contains a morph for every facial expression at its fullest and the facial animations control which is applied how much in a merged result.

 

I don't think there's any automated tool for checking if a NIF has morph files present. Others correct me, if I'm missing one. But it should as well be trivial to make one, considering it's just checking for the presence of "path + (filename - extension) + other extension" in this game ("head.nif" has "head.egm" and "head.tri" in exactly the same folder).

[AllisterHenderson]

DrakeTheDragon + Pherim >

 

Thanks guys for the helpful & valuable knowledge, and trying to improve MY 'slow-brain' understanding :wacko: :laugh: . Both sets of information correlate. Pherim's assessment and discovery (MOO incident above) not to use Pyffi on "character models, creatures & possibly also armor and clothes" correlates with those 'animation' mesh files (.nif) probably also having associated morphs (.egm & .tri).......and it's probably safest to only use Pyffi on static meshes like 'Architecture', 'Dungeons', 'rocks' etc....

 

So in terms of mesh animations..... what is the difference between (.kf) files and bone-based (SK files) / morph-based (EGM/TRI files)?

[Pellape]

What I know is that .kf files are usually used to make one helmet fit different races and sizes of heads.

[DrakeTheDragon]

Ah, my bad. Don't know how "SK" slipped into there. I meant "KF" files for the bone-based animations, of course. Sorry for the confusion.

 

".kf" files is animation through posing armature bones, ".egm"/".tri" is animation through morphs.

 

And sorry, Pekka, but I must correct that slightly. ".egm" files is what makes one helmet fit differently shaped heads in CharGen. ".tri" files would be, if the helmet had movable eye or mouth parts and needs to move them along with your expressions or speaking, or like a woolen or silken face mask for example where you see it when you speak, if you get my drift. Skanti's famous Conformulator, if you recall that name, created EGM and TRI files, not KFs.

".kf" files are mostly only used on actor skeletons or static animated meshes.

 

(Though with the latter I've seen the animation from inside the KF merged into the NIF straight away instead. There are animated NIFs without any accompanying KF files as well. But animating static meshes is something I only dabbled very loosely in myself so far. Elevator like movements of parts of objects or a sarcophagus opening and closing its lid is all I ever did in that regard. And I somehow doubt those bone-based things will have much or any trouble with Pyffi optimizing their NiTriShapes/Strips.)

[Pellape]

Damn, I did it again.... Sorry Drake... ;) ... and everyone else of course

[AllisterHenderson]

Pellape> All good :smile:

 

DrakeTheDragon >

 

Thanks for clearing that up... I was searching all over the internet, wondering what the hell SK files were.. LOL :wacko:

 

I am more familiar with KF animation files, as in I have seen a lot of them in popular mods, just wasn't aware (didn't notice) EGM & TRI animation files,

but thanks for explaining the difference between the different types of animation meshes :wink:.

 

Even though you said KF files should be okay for Pyffi optimizing.... just to be on the safe side if I do any Pyffi optimizing in the future I will only stick to static meshes like 'Architecture', 'Dungeons', 'rocks' etc...

[DrakeTheDragon]

Well, I don't think KF files themselves even can be optimized. Has Pyffi an option for these, too? I meant more like animated static meshes, NIF files with animations from KFs baked "into" them, which I fail to see much potential for Pyffi breaking them.

Only if a fitting EGM or TRI file is lying around in its folders, chances are good the NIF can break all too easily through optimization. Or better yet the NIF and EGM/TRI file structure becoming unfitting is what can crash the game.

[Pherim]

Alright, so I conducted a few tests, and they pretty much confirmed the suspicions I've had since I've been taking a closer look at meshes and their optimization recently. To make sure I'm not limited too much by the CPU I created an empty test cell with a platform for my character to stand on, and a number of instances of one of the most complex meshes in the game, the Leyawiin Mages Guild interior. It has about 35,000 triangles (not counting the collision geometry, which was irrelevant for this test) and causes exactly 15 draw calls, according to ReShade. First of all, it didn't really matter if I put 5, 10, 20 or 40 of them in the cell, with unlocked FPS I didn't even come close to the GPU limit, it seems. The more objects there were in the scene, the lower my GPU load actually was, because the additional draw calls are forcing the game into the CPU limit. With 80 instances rendered I only got about 45% GPU load, as now there were over 1000 draw calls. To get to the actual GPU limit (I've done that in Morrowind, it was several Millions of triangles), one would probably have to make a custom mesh with just one NiTriShape with the maximum possible amount of triangles, which is 65k, I believe (2^16), and place a whole lot of them in the cell. That way, 1 million triangles should be theoretically possible with just 15 draw calls. Maybe I'll try that next.

 

Anyway, what I wanted to know was if NiTriShapes or NiTriStrips with a single strip were faster, and if Pyffi optimization had a noticeable effect on performance. The vanilla meshes mostly use NiTriStrips with a single strip each, and you often read that NiTriStrips are rendered "more efficiently". However, the latest "stable" version of Pyffi (2.2.2) converts them into NiTriShapes by default, which made me wonder if there really was a difference. The answer may surprise you...

 

Alright, no more clickbait :wink:. With the vanilla mesh (NiTriStrips) and 40 instances of the mesh on the screen, I got around 370 FPS in an otherwise vanilla game (so no mod scripts or graphics extenders interfering). Then I used NifSkope to convert the NiTriStrips into NiTriShapes (Mesh->Triangulate), and this actually improved my performance, I was now at 380, give or take. That's about a 3% increase, not much, but it's there. As far as I know, NiTriStrips are mostly more efficient to store, as you don't have to store three vertices per Triangle, because two connected triangles can share two of them. With NiTriShapes, every Triangle has three vertices, and many of them are in the same place, if they belong to adjacent triangles (if these have the same vertex normals, the edges between triangles will be smoothed so it looks like a continuous surface). So, if anything, NiTriStrips might have reduced the file sizes which could have improved loading times back when Oblivion was released, although they were then compressed in the BSAs (and BSA compression is quite efficient for .nif files), so the difference could not have been a lot. And with modern high-speed SSDs, this has become a negligible factor. In fact, the triangulated version is slightly bigger than the stripified one, but not much.

 

Now for Pyffi. First of all, Pyffi usually reduces the file size further than just with NiTriStrips. I don't know what exactly it does, but on vanilla meshes, it can be up to 10%. However, when I used Pyffi on the test model, there was absolutely no performance difference compared to the triangulated one. It may be possible that NifSkope does a few optimizations itself, but as I said, the triangulated version was slightly bigger in file size than the stripified one, and the Pyffi'ed one was smaller than both. Keep in mind that this is a single static mesh and a highly artificial test environment, so it is possible that the results are different in real gameplay. However, I don't believe that it really makes a noticeable difference, especially as modded installations are usually hard in the CPU limit.

 

If you have tried similar things and got the same or different results, please let me know!

 

Edit: My explanation about the difference between TriShapes and TriStrips was not entirely correct, as with TriShapes, adjacent triangles can also share vertices. The main difference seems to be that they are ordered in a certain way that makes them more efficient in memory.

Edited October 14, 2021 by Pherim