[WIPz] TES5Dump/Future TES5Edit

[Sharlikran]

TES5Edit 3.0.23 EXPERIMENTAL is now available in the Downloads section on our Google Code Site.

 

You can download TES5Edit 3.0.23 EXPERIMENTAL at our Google Code site skyrim-plugin-decoding-project in the downloads section.

 

NOTE: This is the actual version of FNVEdit, FO3Edit, and TES4Edit by ElminsterAU being updated for Skyrim. He has allowed us to host both TES5Dump and TES5Edit under the same MPL.

 

TES5Edit 3.0.23 EXPERIMENTAL by ElminsterAU updated for Skyrim by Zilav, and Sharlikran.

 

If possible please post in the Bethesda forum: [RELz] TES5Edit

 

Fo3 Nexus - [WIPz] TES5Dump/Future TES5Edit

FNV Nexus - [WIPz] TES5Dump/Future TES5Edit

Oblivion Nexus - [WIPz] TES5Dump/Future TES5Edit

Skyrim Nexus - [WIPz] TES5Dump/Future TES5Edit

FNVEdit Issue Depot (Bug Reports)

 

This is a an experimental release of TES5Edit. If you don't know what TES5Edit is, you probably shouldn't be using it yet. Although we have put a great deal of work into it be aware of what you are doing. Ask questions in the official [RELz] TES5Edit Plugin Cleaning Emporium. As always backup you plugins, save games, and use with caution. (at your own risk)

 

TES5Edit Cleaning Guide

 

Rename to TES4Edit.exe to work with Oblivion.

Rename to FO3Edit.exe to work with Fallout3.

Rename to FNVEdit.exe to work with New Vegas.

 

Fallout3 Training Manual for FO3Edit (applies to all game versions)

 

Check for new versions here: skyrim-plugin-decoding-project

 

Important for Older Systems:

If you get an error about d3dx9_*.dll not being installed, you need to update your DirectX to the most current version.

 

The most current (March 2008) version can be found here: DirectX End-User Runtime Web Installer

or here: DirectX End-User Runtimes Redistributable (make sure to install it after unpacking it).

 

Version 3.0.23 contains the following changes:

• [TES5] Merged patch updated for Skyrim, currently it merges:
  
• - Leveled NPCs, Items, Spells
  
• - NPC: Items, Spells, Head Parts, Factions
  
• - Race: Hairs, Eyes, Spells
  
• - Containers
  
• - Relations
  
• - Form Lists
  
• - Keywords on: Armor, Weapon, Ammo, NPC
  
• [TES5/FNV/FO3] When performing UDR there will be a warning message for deleted NavMeshes.
  
• [TES5] Updated record definitions.
  
• - BPDT was sorted by a localized name which is inconsistent for different languages (now sorted by a node name)
  
• - NPC's tints are now sorted by tint index for better conflicts detection
  
• - QUST aliases are no longer sorted by index
  
• - QUST stages are now sorted by stage index
  
• - Improved NavMeshes (thanks to Divstator)
  
• [TES5] FaceFX phonemes where empty after copying RACE record.
  
• [TES5] Fixed bugs when copying RACE and QUST (empty model was created for RACE; QUST location aliases were turning into ref ones).
  
• [TES5] Create start-enabled quests sequence SEQ file (you can select several files at once).
  
• [TES5] Improved error checking.
  
• [FNV] CHAL record was missing some challenge types from Dead Money DLC.
  
• [FNV] Disabled DIAL sorting, was causing errors when copying INFO records.
  
• No autosaves when "you know what you are doing".
  
• Compare with external tool option in Edit window.
  
• "Apply Filter for Cleaning" menu. Note that xEdit saves filter settings when you press OK button in filter window, so if you clean plugin and exit the program, your filter settings will be preserved.
  
• Filter option for deleted records.
  
• Saving messages to [TES5/FNV/FO3/TES4]Edit_log.txt upon exit.
  
• Fixed minor bug when Edit can't find game's folder. If your registry settings is invalid due to a Steam bug and Edit is unable to find your game, put it in the game's root folder where the game executable file is (Oblivion.exe, TESV.exe, etc).
  
• - Reason: The game's Registry entry is missing. This happens to a lot of people, because Steam likes to remove it whenever the game is updated or its game cache is verified. Run the game's launcher to add the Registry entry back again.
  

 

Version 3.0.22 contains the following changes:

 

• Skyrim Support.
  
• New exceptions handler.
  
• Optional backups in a separate directory.
  
• Remember position and state of the main window.
  
• Fixed bug when editing with Shift+DblClick.
  
• [All] Wrye Bash ghosted plugins (*.esp.ghost) in plugin selection window.
  
• [ALL] FLST form list is no longer sorted.
  
• [ALL] EDID affects conflict detection.
  
• [TES5] Localization editor, plugin localization/delocalization with optional translation.
  
• [TES5] -l:language command line switch to chose default localization files.
  
• [TES5] Adding missed plugin files from plugins.txt/loadorder.txt to the end of plugins list.
  
• [FNV] WeaponModKit IMOD record support for various subrecords.
  
• [FNV] PLD2 subrecord is no longer removed in package.
  
• [FNV/FO3] Quest stage signed value overflow fix.
  
• [Oblivion] Fixed ownership record order in CELL.
  
• [Oblivion] Snowy weather definition fix (thanks to Arthmoor).
  

 

 

ToDo List:

 

Update Miax’s Tome of FO3Edit for Skyrim.

Update NAVM, NAVI, and wbUnion for VMAD.

Update the TES5Edit Cleaning Guide

 

Known Issues:

 

Issue 59: FNVEdit: "Copy as Override" on some dialogs

Issue 60: FNVEdit: "Copy as Override" on a Base Effect associated to a script.

Issue 63: TES4Edit: 3.0.23 throws an assertion failure when cleaning PGRD records from files.

Issue 69: TES5Edit: Adding items to a leveled list does not increase LLCT count.

 

Current Progress:

 

Some aditional tweaking needs to be done but we are close to having on open beta. About 90-95% of the records are decoded at this time. I have started to update the UESPWiki and I keep a current list on our site in the UESPWikiUpdates.

 

Current Contributors:

 

Sharlikran

Zilav

 

We would love to have more people helping out on the project.

 

Name: Skyrim Plugin Decoding Project

 

Summary: A collaborative project to decode Skyrim's record structure using TES5Dump and provide a working version of TES5Edit.

 

Description: TES5Dump (a.k.a. TES4Dump / FO3Dump / FNVDump) are utilities that dump the contents of a .esp or .esm plugin to a text file, using internal record definitions. By comparing its output with plugins of known contents, the record definitions can be tweaked until the correct structures have been decoded. TES5Dump also forms the plugin parsing backend for the TES5Edit utility. (a.k.a. TES4Edit / FO3Edit / FNVEdit)

 

Skyrim's record structure has not yet been fully decoded, and the Skyrim modding community suffers from the lack of a working TES5Edit. This project's aim is to help the decoding effort by providing TES5Dump and allowing many people to update its definitions using Subversion, and to provide the TES5Edit utility.

 

Its predecessors FNVEdit, FO3Edit, and TES4Edit by ElminsterAU are the best tools to use for exploring overlaps between mods and cleaning mods. TES5Edit will have the same functionality since the compiled release will be made with the same TES4 back engine using the definitions as a result of this project. He would work on it himself, but unfortunately suffers from a severe lack of spare time. He has offered to compile TES5Edit after the necessary supporting files are changed for Skyrim's file format.

 

This utility will let you easily see how multiple mods interact and override each other, as well as giving you a quick view of the changes that a plugin makes. It lets you browse a graphical record tree of your active plugins, revealing the values set by most record types and comparing overlapping changes from each active plugin.

 

When started it will automatically find your Skyrim Data directory. You then get a dialog to select which modules you want to load with the current selection from your plugins.txt as default value. Once you have confirmed that dialog the selected modules will start loading in the background. The program will then work the same as its predecessors.

 

License: Mozilla Public License TES5Dump and TES5Edit.

 

Anyone is welcome to download the source to TES5Dump using the instructions below.

 

Why do we need TES5Dump:

 

.snip. TES5Edit isn't the important thing in this scenario, what's important is getting the records decoded, because a whole load of things (Gecko, Bashed Patch, etc) require that, not just TES5Edit...

.snip. Because so much depends on the record decoding, it's sensible to open up the task to as many people as possible. As such, publicly hosting the definitions and the TES5Dump code/program is best for that. The plugin format has historically been documented at UESP.net for previous games, but I'm of the opinion that only verified decoded records should be documented there, so we still need a central location lots of people can read/write to. A SVN repository would be ideal...

 

How TES5Dump works:

 

TES5Dump Is a command line utility that can take an esm/esp file, decode it, and give a human readable dump to the console. The output can be redirected to a file, so you could use:

 

TES5Dump Skyrim.esm > Skyrim.txt

 

To turn the whole master into a huge text file.

 

What works better when working on the record definitions is to create new .esp’s using the CK with just a single (or a few) records, of a single type, with specific values for all the different fields.

 

The process is then basically:

Create new .esp with example records

Run TES5Dump

Modify record definitions

Repeat till dump is correct

Repeat steps 2-4 until all possible record types and all possible fields are covered

 

What we need for this project:

 

We are looking for volunteers and TES enthusiasts who are willing to contribute their time in decoding Skyrim's files, and making updates to the UESP Wiki.

 

If you would like to participate in this project please send me a PM.

 

Wiki Updaters:

 

Any general knowledge of how to edit html and possibly xml. After you have checked out the files the folder you will work with is UESPWiki. It will contain templates of each record posted on UESPWiki. To make changes to the UESP Wiki site you need to register first. Registration is free and hardly any information is required.

 

Please refer to the on site wiki: PostingToTheUESPWiki

 

Record Decoders:

 

After you have checked out the files the folder you will work with is TES5Dump. The only file you need to edit will be wbDefinitionsTES5.pas.

To compile the source you need a trial version of Delphi found here. embarcadero.com The source is written in Delphi but you only need a basic knowledge with C++ or Java to interpret the Delphi code and figure out the references.

Having knowledge of or, experience in decoding the records in plugins would be extremely helpful as well. The decoding is not programming intensive because all you are really doing is updating the pre existing routines for Skyrim's format.

 

Please refer to the on site wiki: DecodingRecords

 

Getting Set Up:

 

You will need an SVN client like TortoiseSVN to submit your changes. Instructions are listed here on Google Code.

 

Special Thanks:

 

Listed in alphabetical order.

 

AndalayBay: Without her help getting me in touch with Elminster we would all still be waiting for confirmed record definitions, and even longer for a working TES5Edit.

Bethesda: For their awesome program, and for this forum so we can accomplish this task.

Divstator: for his work on Improved NavMeshe Decoding

ElminsterAU: For all his hard work on FO3/FNV/TES4Edit over the years.

Zilav: For all his hard work helping with the project.

Edited October 21, 2012 by Sharlikran

[Sharlikran]

Does anyone have knowledge of known bugs in FNVedit, or any others. Basically the file name makes the version. So once TES5Edit is compiled, if you rename it to FNVEdit it would work for Fallout NV. I ask because any bugs in a previous version will be present in TES5edit and I'd like to get them into the repository once its created.

[Sharlikran]

After reading a post in the Beth thread I decided to add an aditional message. I think people are thinking this is another attempt to reinvent Elminster's work. I can assure you it's not. I may have found a way to compile TES5Dump without Elminster's help. I'm being respectful of his time and making my own attempts to get the code working without the proprietary units that he can not share.

[JustinOther]

This is great news! I've been anxiously awaiting THE TES5Edit. *Edit v3.0.19 EXPERIMENTAL has been working very well for me with TES4, FO3, and FNV. I can't think of any outstanding bugs or problems off the top of my head.

[tox2ik]

You can't release only a binary under GNU :)

I guess I should head over to the mentioned thread for more info and to get involved.

[Sharlikran]

You can't release only a binary under GNU :)I guess I should head over to the mentioned thread for more info and to get involved.

A compiler would still be needed, we just found a free trial version of Delphi. I don't suggest anyone install it since it is a 30 day trial. Once things are in place to host xDump under an MPL we should be able to get under way. As long as I don't run into any obsticles with the trial version compiling xDump.

[Dhegonus]

wow, this would be an absolute must for modders out there.. but a mammoth under taking! i tip my hat to you.. good luck in your adventure, i shall be lookingclosely, unfortunately i cannot assist as i have projects of my own to attend too.. but i wish you all the best :)

[Sharlikran]

The trial version of Delphi does compile TES5Dump and I am currently making changes to display Skyrim's information correctly for use with TES5Edit. Although this is only the start, it will take some time before it is finished.

[rickerhk]

Does anyone have knowledge of known bugs in FNVedit, or any others. Basically the file name makes the version. So once TES5Edit is compiled, if you rename it to FNVEdit it would work for Fallout NV. I ask because any bugs in a previous version will be present in TES5edit and I'd like to get them into the repository once its created.

 

Nice to see that someone is picking up the ball on this wonderful utility.

 

These are the bugs that I know about in FNVEdit:

 

1. Weapon mod references placed in the editor, report an error when checking for errors. This wouldn't be so bad, but these records can't be injected into a master when merging a plugin, because it errors.

 

2. A terminal reference, that is used in a script, also generates an error during an error check, and the script can't be injected when merging your plugin to a master for a big project.

 

3. The most serious bug is one with packages. Certain sub-records are stripped off when saving the plugin with FNVEdit. The one I have been impacted with the most is the 'Trigger Location' of dialog packages. This is the PLD2 subrecord. You can imagine trying to build quest mods with dozens of dialog packages that get broken every time you save the plugin. With the trigger location stripped off, an actor that has low level processing will cross the worldspace in order to force greet you. Right now, the work-around for me is to use version 3.0.15. But that has problems with some of the DLCs, and some performance issues when the tree is expanded.

 

Others are also having serious issues the package records, and there's more detail here. It's not just the dialog packages:

 

Quoting Dandys:

Some of my mod partners and I have identified a problem in a package subrecord that is ruining Dialogue, Eat, Use Weapon, and Ambush packages and costing us a large amount of work that has to be redone every time we update the mod.

We don't know the underlying cause, but this is what happens. In packages, there is the "Locations" subrecord that has "PLDT - Location 1" and "PLDT - Location 2" in it. However, down in the record, the subrecord "PLD2 - Location 2 (again??)" comes up. What we have found is that FNVedit deletes the "PLD2 - Location 2 (again??)" subrecord and moves what was in it to the "PLDT - Location 2" part of the "Locations" subrecord. That is actually not what is supposed to happen, we think, as packages that use a second location with the data in that place act as if they have no second location.

We have found that if we manually move the contents of "PLDT - Location 2" to "PLD2 - Location 2 (again??)" and then remove "PLDT - Location 2" and save, the packages work properly. However, if that mod file is then edited in any way and saved again, FNVedit seems to try to "fix" the packages that in fact we fixed, and moves "PLD2 - Location 2 (again??)" to "PLDT - Location 2" again, breaking the packages.

If FNVedit could be changed to realize that "PLD2 - Location 2 (again??)" is the real, correct place for Location 2 data and save it there every time, a lot of time would be saved.

Is anyone capable of editing FNVedit to fix this bug? 

[Sharlikran]

Does anyone have knowledge of known bugs in FNVedit, or any others. Basically the file name makes the version. So once TES5Edit is compiled, if you rename it to FNVEdit it would work for Fallout NV. I ask because any bugs in a previous version will be present in TES5edit and I'd like to get them into the repository once its created.

 

Nice to see that someone is picking up the ball on this wonderful utility.

 

Others are also having serious issues the package records, and there's more detail here. It's not just the dialog packages:

I started a thread with a different title. Hopefully it ends up being useful.

 

FNVEdit Issue Depot (Bug Reports)