Gamebryo Engine - Debate

[JimboUK]

Warning, technical rant follows.

 

What Bethesda really needs it a good lint-picker. The memory leaks and the ever growing hunger for more memory also have an impact on the games performance, impacts the frequency of crashes and freezes and other bugs caused by memory thrashing.

 

What's a memory leak? Simple explanation. A program gets a bit of storage to store a list (let's say your inventory). As you collect more trophies (or just stuff to sell) while wading through a dungeon, the list grows and the original storage reserved for the list is exhausted. More storage is obtained for the growing list. At some point you wind up selling the detritus to a vendor. The list shrinks back to your weapons and armor again. But that second block of storage is not returned to the system and is still there, unused and taking up memory. This is compounded when the game exhausts your original reserved storage a second time, because a new extension is obtained which leaves that first extension allocated and not used.

 

As these unused blocks of memory accumulate, the system allows the game to use more virtual memory. As the needed blocks eventually wind up in different storage sections, these sections must be swapped in and out to continue play, causing the memory manager to thrash. Additionally, these orphaned storage blocks lead to fragmented memory and the game is unable to get a contiguous block of storage of sufficient size to meet the games need (like the game locks up when you open a door).

 

This is why the game seems to bog down the longer a session of play extends and why a save and restart of the game sometimes improves the games performance.

 

A lint-picker is a programs that scans programs and attempts to pair every GetStorage to a FreeStorage in an attempt to ensure that there is a FreeStorage associated with every GetStorage.

 

Yep and If you edit the ini to purge the cell buffers on cell transitions those memory issues evaporate, if their games did that by default they'd be a lot less problematic.

[PoorlyAged]

 

Warning, technical rant follows.

 

What Bethesda really needs it a good lint-picker. The memory leaks and the ever growing hunger for more memory also have an impact on the games performance, impacts the frequency of crashes and freezes and other bugs caused by memory thrashing.

 

What's a memory leak? Simple explanation. A program gets a bit of storage to store a list (let's say your inventory). As you collect more trophies (or just stuff to sell) while wading through a dungeon, the list grows and the original storage reserved for the list is exhausted. More storage is obtained for the growing list. At some point you wind up selling the detritus to a vendor. The list shrinks back to your weapons and armor again. But that second block of storage is not returned to the system and is still there, unused and taking up memory. This is compounded when the game exhausts your original reserved storage a second time, because a new extension is obtained which leaves that first extension allocated and not used.

 

As these unused blocks of memory accumulate, the system allows the game to use more virtual memory. As the needed blocks eventually wind up in different storage sections, these sections must be swapped in and out to continue play, causing the memory manager to thrash. Additionally, these orphaned storage blocks lead to fragmented memory and the game is unable to get a contiguous block of storage of sufficient size to meet the games need (like the game locks up when you open a door).

 

This is why the game seems to bog down the longer a session of play extends and why a save and restart of the game sometimes improves the games performance.

 

A lint-picker is a programs that scans programs and attempts to pair every GetStorage to a FreeStorage in an attempt to ensure that there is a FreeStorage associated with every GetStorage.

 

Yep and If you edit the ini to purge the cell buffers on cell transitions those memory issues evaporate, if their games did that by default they'd be a lot less problematic.

 

 

The cell buffers are a separate issue and purging them does not stop the memory leaks. In the inventory example I used, the inventory survives cell change because you need that inventory in the next and subsequent cells. So the orphaned memory is left around, regardless of whether on not the cell buffers are purged.

[JimboUK]

No but it frees memory including some leaked, I can play NV and FO3 for hours with that edit, without it things start to slow down badly.

[PoorlyAged]

As I said, freeing cell buffers is a separate issue. A storage leak is storage that is gotten and not freed. Purging the cell buffers does not free storage which must survive the cell buffer purge.

 

By way of an example, when you ease your burden by stashing stuff into a chest in a dungeon, you expect it to be there when you return (unless you wait too long and cell reset restores the chest to it's original condition). So the list of the contents of that chest must survive cell buffer purge. Such lists are the type which wind up being orphaned. Clearing cell buffers does not clear or stop the storage leaks.

 

What purging the cells does is free large blocks of contiguous storage and potentially limit the fragmentation of the storage available, but not always.

[JimboUK]

I never said is was the same thing I said it makes the problem go away and it does, anyway Bethesda haven't really had a problem with memory leaks, just poor memory management.

[HeyYou]

I never said is was the same thing I said it makes the problem go away and it does, anyway Bethesda haven't really had a problem with memory leaks, just poor memory management.

"Poor" doesn't even begin to describe it.......

 

I find it absolutely amazing that unpaid modders can dramatically improve the games memory management, but, beth can't seem to figure it out? What's up with that?

[Ethreon]

 

I never said is was the same thing I said it makes the problem go away and it does, anyway Bethesda haven't really had a problem with memory leaks, just poor memory management.

"Poor" doesn't even begin to describe it.......

 

I find it absolutely amazing that unpaid modders can dramatically improve the games memory management, but, beth can't seem to figure it out? What's up with that?

 

Poorly prepared coders. Good coders, with little time/poorly paid. Good coders, but low engine knowledge. And so on..

[JimboUK]

 

 

I never said is was the same thing I said it makes the problem go away and it does, anyway Bethesda haven't really had a problem with memory leaks, just poor memory management.

"Poor" doesn't even begin to describe it.......

 

I find it absolutely amazing that unpaid modders can dramatically improve the games memory management, but, beth can't seem to figure it out? What's up with that?

 

Poorly prepared coders. Good coders, with little time/poorly paid. Good coders, but low engine knowledge. And so on..

 

 

Next to no QA and higher ups with no interest in anything other than the bottom line.

[PoorlyAged]

Memory management is performed by the operating system. Applications do not do "memory management". All programs can do is ask for blocks of memory from the system and return them when they are done.

 

But do not take my word for it. There are some pros on staff here, like bben46. Ask him.

Edited February 6, 2017 by PoorlyAged

[HeyYou]

Basic memory management is handled by the O/S, yes. But, the application itself is what is making requests. If the application 'forgets' it has reserved blocks, the O/S won't clear them until the application is closed. Hence, memory leaks.