duplicate FormID numbers in recent mods - many comflicts in-game

[Gaal]

Alright, I'm gonna try to make sense of all this despite the fact that I'm tired, frustrated, and ... well... really tired.

 

So, I've been installing mods like mad and enjoying most of them. However, upon the release of the GECK, I discovered recently that some of the mods I'd been playing with were beginning to malfunction in some really odd ways. I discovered, after a lot of attempts to fix the issue in GECK, TESsnip, and FO3Edit, that the issue was due to duplicate Form IDs. This would usually not be a problem due to the way Oblivion, and thus Fallout 3, handles this problem. I discovered a post about 4 or 5 pages down the list in this forum that the first two numbers of a duplicate Form ID are replaced with the number corresponding to the plugin's order-number in plugins.txt.

 

Here's where it gets rather confusing. There's a mod I have been using for some time that adds a bunch of scripts that mimic the functionality of the Ghoul Mask that is in the default game. This allows you to be disguised as a raider when wearing raider armor, meaning the raiders will not attack you when wearing their armors and will even stop attacking you if you put the armor on after thay've already spotted you. However, if you attack one it will still fight back, stopping that defensive attack if you stop attacking it. Yes, I know it feels a bit like cheating, but it's the best I've seen so far. It works great on it's own, but then another mod came about that added a new weapon with it's own script. The weapon itself had a unique Form ID, but the script ID was the exact same as the ID for the script handling the raider armor in the other mod. Somehow, due to the order of loading, suddenly the weapon was working fine, but the raiders were no longer responding to the armor. Oddly enough, a third mod which added a Raider Friend Perk was causing the raider's dogs to see me as a friend, but the raiders themselves were still attacking me while I wore their armor type.

 

Still with me? Okay, so I had to conclude that when Fallout 3 modifies the Form ID of the duplicate item, it does not check for references to it, or from it, and make the proper adjustments to their corresponding Form ID references. This is where it gets confusing to me, because, so far as I know, there's no way to tell a piece of armor to reference a certain script Form ID while allowing for a certain number of possibly random digits caused by plugin load order. I did a bunch more investigating and found a number of cases where modded items were referencing FLST items that were actually the IDs for scripts, and several script items that were referencing ARMA IDs. I even found a few container items that turned out to be the IDs for a MESG item. These conflicts are present BEFORE being loaded into Fallout 3, and after my attempts at fixing them and testing them, the results were even more broken in-game due to the reassignment of the first two ID digits, which I didn't even known about until after those tests when I decided to check these forums.

 

So, now that you're all totally convinced of my insanity after reading and re-reading this post several times, and beginning to doubt your own insanity, I pose this question...

 

How the HECK are we going to resolve this issue?

 

In other games that used hex-based IDs for mod items, the community had to develope scripts, and eventually whole programs, which could open the mod's files (sometimes as simple as XML-based scripts/lists) and check them for ID ranges that had been registered for use by a certain registered modder. The mods were then either filtered and denied posting on the website if their IDs were out of bounds, or they would flag them as Safe or Not Safe on the download page.

 

I've also known a few cases, though I can't exactly remember them working very reliably, where programs were developed to scan for these types of conflicts in a wide number of mods, and then provide either a patch-mod which would be loaded AFTER the conflicting mods containing only the conflicting items and their adjusted IDs, as well as any objects referencing them, or the programs would edit the mod's own files and re-package them with the fixes items so that every ID was unique.

 

However, we're not jsut making new items here, we're modifying original items too, and if the above fixes were to be developed we'd have to make sure we're not fixing things that are supposed to be a duplicate of a master-entry. Those Form IDs are supposed to be the same, or it's no longer a replacemement.

 

And finally, with this system of Form IDs, eventually you'll end up with too many unique IDs for brand new items than the 8-digit hex IDs will no run out of possible numbers (unless you can have Form IDs longer than 8 digits.) so, when the last mod comes out with a Form ID of FFFFFFFF (8 Fs) then we're all done and royally screwed.

 

As my eyes become rather blurry and I loose my ability to type with some level of accuracy... not to mention high winds starting outside threatening my power... I will leave you all to contemplate our fate however grim it may be. We're gonna need some sort of fix, or solution program, eventually.

 

Good Luck.

 

 

 

Gaal

[BanjoOz]

I thought I had the hang of all his when I was struggling to hex edit with FOMM. Then the GECK came along and although the process of mod creation is infinitely easier, I've had no end of trouble just trying to recreate my old mods in it, what with the way it seems to refuse to allow Master files, how it seems to get confused over FormIDs, and all sorts... so I feel your pain!

 

Nobody seems to know (or want to say, though I suspect the former) how to make a mod that mods a mod, either! I.E. to solve the HUGE problem of modding vendor and loot lists, my "solution" was that modders could make two esps: one that was the actual new items/guns/npcs/whatever, and one that altered "major" leveled loot lists (eg rare small guns) so that users could make their own "combined" loot list esps for use with multiple mods (eg have machetes, desert eagles, FN FALS, etc all for sale). BUT this fell flat with the GECK's way of doing things, because it only allows you to make a plugin that relies on other plugins (besides fallout3.esm) except ONCE, when you make it... go back to edit, and boom! Conflicts galore,, because it faisl to remember anything, it seems!

 

Does this mean, if I want to make a simple mod that has stores sell both my lever rifles and Vash's Desert Eagles, I HAVE to get permission to reupload his whole mod combined with mine, just for one little loot table record???

 

The whole idea of "shifting" FormIDs confused me at first, but now I see the problem that if it changes the first two digits, how can mods that rely on other mods ever work, since they'd only work if the mod load order was always the same (another flaw seems to be the FORCED load order in everything bar FOMM... having to change file dates just to cahnge load order seems awfully ham-fisted!).

 

Besides, at least before the GECK modders could at least make their mods compatible with their other own mods, even if they couldn't predict what FormIDs other users were using. Now, the GECK auto-assigns uneditable (in GECK, anyway) FormIDs that are almost always the same as the last mod you just made... fine if used in isolation (thanks to those shifting load order digits) but seemingly useless if you ever want to merge or combine two mods, as you'd have to change the IDs once they were part of the same esp OR in a master/plugin situation.

 

Like the OP, I'm confused and a bit fed up right now... not knowing how to do something is a challenge, as it was when I first started tinkering with FOMM and notepad as my tools! Now it's more that I know exactly what I WANT to do, but there seems no provision to do it!

 

Like the OP, I am concerned that the whole FormID situation is going to make mods pretty incompatible... then again, I've not really been part of Oblivion scene - modding or otherwise - and given that games similarities and massive fan base, I would suspect somebody "solved" these issues there, and that the same solutions might apply eventually to F3.

[jaysus]

the form ids never really were an issue in any former beth game so it seems to work...

 

you can create esms... just download tes4gecko and use that... the only downside so far is that you loose your newly created radio stations but these usually have nothing to do in an esm anyway...

 

esps cant change other esps...

[Gaal]

I am at least a teeny bit comforted by the fact that someone out there is feeling similar pains.

 

I will clarify that I don't know a whole lot about the engine itself and the way the Form IDs are handled. I'm also a bit confused over the fact that when using the console you pretty much have to use the ID to reference items in the game, but within the scripts I'm finding numerous places where the same commands are being issued but the items are being referenced by their Editor ID which is plain text.

 

The last Bethesda game I played (outside of the game-testing world that is) was Morrowind. I'm sure that underneath all the plastic-coating there were similar hex-based conflicts arising in that environment, but the modders/players only had to reference items by their unique text-based Editor ID, and sometimes even their full not-completely-unique in-world name. It also seemed to have a capacity for a near-limitless number of separate and/or merged mods with a near limitless number of new and unique objects of any type defined. At least, I never ran into a limit and I still have a backup of over 200 separate esp files that I was loading into Morrowind on a daily basis. That's wothout tallying up the mods that required MGE and MWSE to run properly.

 

Oblivion I never touched, mainly due to my computer not being powerful enough upon release and hearing that, compared to Morrowind, it looked amazing but wasn't nevessarily a replacement for the massive and open world that Morrowind provided, especially given that many of the graphical upgrades of Oblivion were making their way into Morrowind through 3rd party enhancements.

 

It's quite possible that all this is really very easy to fix, or avoid completely, and we're only being confused by our total lack of experience with the tools available. I'm also quite sure that the designers of this system are well aware of these potential problems and couldn't have foreseen it would be this messy out here due to the fact that they were never adding to a completed product from several different sources. It was all one, controlled, project. We're not so lucky as the inferior beings that we are. Simple, one-celled, possibly soul-less, mod-makers. No offense intended.

 

I did, however, in my experiements, discover that the Geck does help in some ways with the Form IDs being identical in the same esp file. I conclude that this would work in multiple esp files as well, but wouldn't be easily saved to those individual files due to the fact that there is always only one active file. The selected active esp, or the temporary work environment that can be saved as a new esp. That aside, it did detect that some items in my mod were supposed to override items in the master due to the fact that the esp referenced fallout3.esm as a master dependancy, and left those alone. It also found many cases of two non-replacing items having the same Form ID and changed one of them to a unique ID. However, it did not check for cross-references to the changed items and fix anything, so I had a lot of repairing to do on my own to make things work. Also, since I am merely human, I can't easily find things by their Form IDs, so I had to search by hand/eye to find items by their Editor ID which was very tedious and tended to leave me with blanks due to the extreme differences between some Editor IDs and their in-game names. I believe I was doing a lot of switching between Geck and TESsnip (via FOMM) with the same esp files loaded in both, in order to make sense of what I was seeing and what was actually there. Unfortunately after fixing everything, Geck crashed on me before I was able to save my work, which was rather rude.

 

 

I could probably go on typing about my various discoveries as a simple, one-celled, barely intellectually trained, neophyte modder, but I might as well give someone else a chance to say "hello" and let my fingers rest. I do want to clarify however that these posts are not rants, as they call them. My intention is to cause people out there to turn on their brains, swap out the batteries, and simply be aware that though we may have a "Garden of Eden Creation Kit," we're a long way off from being able to produce the "Garden of Eden" with one push of a button. Without proper handling, we're more likely to blow ourselves up before we can say "oh crud, I hit the wrong button!"

 

 

 

Gaal

[jsnider193]

Be nice if there was a registry of already used FormIDs somewhere so folks know what is in use. I keep running into this as go along, with me more of an issue cause I don't like 100s of small esps running around so try to combine into one larger grp (Weapons, Places, Items, Armor etc). Then use either the Geck or one of the others to change dups ....

 

Just takes more work but hey my choice.

[BanjoOz]

I'm curious how Oblivion handled shops and loot lists, since to me that seems to be the biggest problem here. If I have two mods that change to 10mm Pistol, or two mods that alter Oasis, I expect only one of them to work. But if I download two new guns, I would like to think there's a way to have shops sell both of them, without both modders having to combine the two mods and reupload them (potentially hundreds of combinations). I've no experience with Oblivion, but know how popular modding is for it, which is what makes me wonder if it's at all possible. Or is every Oblivion "new weapon" pack (of which there are tons) really incompatible with each other, unless they just add to a chest or something somewhere?

 

As for keeping track of "used FormIDs"... that would be great and what I thought was going to be how all this worked (seen similar things for other games) except for F3 changing them based on load order... AND the fact that the GECK strangely does not allow you to edit FormIDs (sure, FOMM does, but if you start in GECK, it's a heck of a job to go in and change every single reference!).

 

Is it perhaps just a case that for Oblivion, there were other 3rd party tools (I keep hearing about something called WyreBash as well as Gecko) that helped solve these problems, and we just have to wait/hope until these are updated to support F3?

 

Jaysus, if I use Gecko, will the GECK (geez, memories of Fallout 2!) actually allow me to load a second esm (after changing that one line in the GECK ini file, as per the wiki) during editing? I tried using FO3Edit to convert an esp to an esm, but couldn't get it to load... wonder if I was just doing something wrong.

[jaysus]

i dunno if fo3edit works for esm conversion... i used tes4gecko for my commonwealth mod and it works like a charm, you can load i in the geck, use it as a base for esps and i dunno what else... only the radio station broke during conversion for some unknown reason...

 

as to form ids

the geck i assume assigns them ids starting with 01xxxx whichmakes them random on loading... i used this in a pre geck mod and ingame the ids were different with each 1st load of the mod... you couldnt cheat that way but definately prevent any conflicts... apart from that you can try to calculate the amount fo possible ids... 6 digits, each has 6letters and 10 numbers to choose from = (10+6) x 6 x 5 x 4 x 3 x 2 x 1 = dunno lol a high number (or im wrong lol... quite some years since my last stochastik lesson - could also be like: 16! x 6! which would be even crazier 16x15x14...x1 x 6 x 5 x.... 1)

 

@loot lists and merchant containers...

thats the real conflict problem imo... and always was... wyre bash was the tool of choice for oblivion to merge those lists... or now for fallout the commonwealth of modders till its released where the modders get their very own merchant + lists...

[Khet]

Maybe I'm wrong but I'm not seeing where the conflicts come in. Are you saying that two mods are using the same exact formIDs (Mod A adds MyWeap01 and mod B adds MyWeap01?) If so then yeah I can see a conflict. All that would be needed however is one of the two IDs to be changed.

 

I don't know why it started with Fallout 3 but people really need to STOP calling up items as their Hex in scripts and the like with the GECK. Typing in FissionBattery works just as well (if not BETTER) than typing in player.AddItem 0033bc6 1 in a script. This would prevent load order issue, and also conflicts would probably be reduced.

 

If it's scripts, again I don't know what has happened between Oblivion and Fallout 3 but people must have stopped using the 'tag line' added before a script name, item name, interior, whatever. Instead of having an item name KTMMyWeap01 people are just using MyWeap01?

 

 

Like I said maybe I'm misunderstanding everything here but to me it sounds more like bad modding choices rather than actual conflicts. The fix for this is simple too, load up each mod in the GECK and give everything added a UNIQUE FormID and you're all set.

 

But, things could have internally changed between Oblivion and Fallout 3, or the GECK might be the culprit. But I used a good 280 or so mods for Oblivion, quite a few of them armor mods/weapon mods/item mods and none of them ever conflicted.

[BanjoOz]

Maybe I'm wrong but I'm not seeing where the conflicts come in. Are you saying that two mods are using the same exact formIDs (Mod A adds MyWeap01 and mod B adds MyWeap01?) If so then yeah I can see a conflict. All that would be needed however is one of the two IDs to be changed.

 

Aside from the afore-mentioned MAJOR issue of loot tables, the only problems I was having before the GECK with FormIDs was that they are randomly changed if they start with "01", hence other mods can't refer to them (this seemed to be fixed by using "00", so long as it was a unique FormID not used by the original game). After the GECK, however, when I tried recreating my mods, it seems to assign it's own "preferred" FormIDs to each new item you make... the problem being that if you make two new mods in GECK, it's very likely both will use the same FormIDs. Not a problem "in game" because of the auto-changing of the first two digits (01, 02, etc.) but a big new problem for making mods that refer to other mods, since you now have the double problem of a) the first two digits may change according to load order AND b) even if not, several items will have the same FormID as far as GECK is concerned (since it doesn't change those first two digits, AFAIK).

 

The fix for the latter? Yes, go and change the FormIDs for all your new items, etc. to new ones that you know won't be used by GECK, but that seems an awful lot of work unless it's just one misc item (all the changes needed to reflect textures, etc.) when the GECK is supposed to make things easier! I don't mind doing all the work (as in FOMM) but it just seems there should be an easier way, and that it's the GECK just making life harder than it has to be (heck, I've spent days trying to figure out how to combine multiple mods with it!).

 

One question, though... HOW do you change FormIDs in GECK? It took me long enough to find them in that collapsed column... can they be edited (as EditorIDs can be) in GECK, or do I need to use another program?

[KFK]

So if I follow this thread correctly, I just ran into this issue while trying to modify Tailor Maid to fit Welcome to the Wasteland. I loaded Fallout.esm, and Tailor Maid.esp, then went and removed the Object Effects (ENCHantments), the changes I saved as KFK_Tailor_mod.esp. All this accomplished was two wasted hours, and a series of DUPLICATE000X entries with different FormIDs instead of a mod for Tailor Maid. I had to modify Tailor Maid directly in order to get things to work.

 

*Edit* I did not realize this thread was over 6 months old when I bumped it. Sorry.