wghost81

About shadow maps. While working on deserialization of map objects, I discovered several interesting facts. First, although Components are based upon Object class, their data are serialized before NetIndex. Second, there is a class named DominantDirectionalLightComponent, which has an entire ShadowMap inside, serialized at the beginning of the object. There is exactly one DominantDirectionalLight object and exactly one DominantDirectionalLightComponent object per each map.

Well, it looks like it was figured out long before me. :smile: I was thinking and I can't think of anything better than good old escape characters to guard special symbols. It'll make some of the older mods incompatible, but it should eliminate all the problems. I believe guarding inner " characters with backslash symbol will be enough. And of course, the backslash itself should also be guarded: "\"" and "\\".

Offsets are described in "UPK Texture2D.pdf" file by Drakous79, which is inside CustomTFC.7z archive. So, the problem is that max NetIndex value prevents you from changing indexes for EU objects? This means, we need to find the Max NetIndex and change it, I guess.

Yes, // is most probably treated as a comment, so the end of the line is removed and all the code up to the next text token ends up treated as a text. I tried to do without escape characters, but it's getting increasingly hard as mods are getting increasingly complex. :) I'll think of something, so stay tuned. ;)

I'm not sure that adding new names is a good idea in this particular case. There could be tons of low-res textures embedded directly into the map package and those are stored as BulkMirror data, which operate on absolute file offset basis. By inserting new names we will shift the data and effectively make all the offsets invalid.

LW general feedback: http://forums.nexusmods.com/index.php?/topic/2765009-beta-15c-general-feedback/ LW bug reports: http://forums.nexusmods.com/index.php?/topic/2764974-beta-15c-bug-reports/ LW players helping players: http://forums.nexusmods.com/index.php?/topic/2228599-players-helping-players/ LW strategy: http://forums.nexusmods.com/index.php?/topic/1082425-long-war-strategy/

We don't have that many Mac users in LW modding team. Personally, I play under Windows and Linux, but I have no access to Mac. Since both Linux and Mac are similar in many ways, I can give some general advices on how it should work, but I can't point you directly to the right folder or make a dmg file. Sorry. :( All the discussions on LW under Mac have happened here, in this topic, so you can look it up and see if there is any helpful information. I can say only that you need to find at least three folders: the main game folder where all the game files are stored, feral overrides folder for Linux/Mac specific files and local user folder for saves and config files. One way is to search your system for every file you can find under install folder and replace all the relevant occurrences. Note that under Linux and Mac Defauilt*.ini files have to be copied under local user config folder and renamed to XCom*.ini files.

After doing some research I can say that 4 bytes of every export object serialized data are indeed a NetIndex. I tried to understand the NetIndex by looking closely at map packages. That's what I found. TheWorld.PersistentLevel objects have negative NetIndex values. This is probably related to the fact that all PersistentLevel objects are instances in the World. Objects of the same Type have identical NetIndex values. Objects outside TheWorld.PersistentLevel have ForcedExport flag set. Those appear to be a collection of various game assets, cooked directly into the package to decrease its loading time. All of them can be traced back to parent object of Package type, which has NetIndex=-1 and also has non-zero package GUID and NetObjectCount=1. NetObjects[0] appears to hold some flags and 4 bytes in between GUID and NetObjects[0] are different for each package, but I can't understand what exactly they represent. Child objects of each Package object appear to have unique NetIndex values. I.e. different objects in one upk file can have the same NetIndex values, but only if they belong to different packages. For example, in URB_GasStation.upk objects FX_Trash.Blowing.Trash_A and FX_Vehicle_Destruction.P_Car_Explosion.ParticleModuleSize_9 share the same NetIndex=1090, but they belong to different packages: FX_Trash and FX_Vehicle_Destruction respectively. This explains the fact that all the objects inside script packages have NetIndex=ObjRef-1. Since both URB_GasStation.upk:FX_Trash.Blowing.Trash_A and Weapon_FragGrenade_SF.upk:FX_Trash.Trash_Explosion.Trash_Debris_B belong to the same package FX_Trash, I assume that those indeed have to have different NetIndex values. I googled some bits and pieces of related unreal code and came to the conclusion that NetIndex is assigned by Linker and is unique to the package (not upk file, but uncooked package which name can be obtained by looking for objects of Package type inside upk file). While loading packages, NetIndex is taken directly from the upk file, if the file is cooked. Net Replication can also assign NetIndex to an object, but I haven't looked into this closely. BTW, I wonder what'll happen if we set all the duplicated NetIndexes to -1.

I have an interesting piece of news here. While looking into NetIndex problem I discovered a very interesting thing: this number is what we call PrevObjRef. It occupies the very first 4 bytes of serialized data. Script packages have this number equal to the Export Table Index of previous export object, so we decided that this was some internal linker info, some kind of enumerator local to the package. But now it turns out that this number is in fact the NetIndex and that it is unique between all the packages. Crazy. I know where to find all the packages list, but I don't know where to find NetIndexes table or even if it exists or how to find max used NetIndex value. But I'll keep on searching. So, LiQuiD911, if you know the name of the object, you can change its NetIndex by writing at the beginning of its data: OBJECT=FX_Trash.Blowing.Trash_A UNSIGNED=1090 // 4 bytes of NetIndex UPD: After doing some more research I can't say for sure if NetIndex is globally unique. But it sure is unique for some of the object. It also gets assigned on Net Replication, so it can be dynamic.

Dubiousintent, "\ " is a space symbol, guarded with the escape character. Looks perfectly normal under Linux/Mac, but I recommend just enclosing the whole path into "" instead. Tomarsh, USERFILES is basically the folder where your game keep its saves and INSTALLDIR is the main directory where all the game files are located. Search for some internal folder like "XComGame" or "CookedPCConsole", which should be named identically under all systems, and then use the path to the root game directory as INSTALLDIR path.

No, they have to be renamed and copied. We can potentially rewrite all the offsets in EU map files to point to the new EW textures, but this will take ages, because there are tons of textures and we'll need to find the offsets first. Renaming the textures looks like the least painful way now.

LW is intended for use with Steam version, so, again, I'm not sure if it works with AppStore version at all. Try reading this topic from the start, there are notes on specific files and where to find those. You could try locating the corresponding files manually on your system and replacing them with LW files.

I'm not sure LW is compatible with non-Steam versions. Try searching this forum for AppStore version, I believe there were several users with the same problem here, but I can't recall if they were able to solve it.

LiQuiD911, thanks, I'll look into those objects then.

LiQuiD911, I was talking about binary data in Texture2D objects. Judging by its definition, NetIndex is native private constant initialized somewhere in c++ code most probably when cooking. "editconst" means that is is visible in Editor but not changeable and noexport means that it is defined manually in some c++ header file. Can you point me directly to two or thee objects (upk files and full names) with the known NetIndex value?

That is indeed interesting. Since it's native, it should be serialized directly via c++ code, not via Default Properties. There is a file in CustomTFC.7z archive called "UPK Texture2D.pdf" - a description of binary part of Texture2D object made by Drakous79. I suspect those 4 bytes he was unable to identify can be the NextIndex you seek. Can you confirm it?

I compared all the files you listed and they are identical under Linux and Windows.

What are the Shader Caches? Are they stored in different files or in upk files? Forgive my ignorance, I really have almost no knowledge about graphics. :smile:

I thought it won't be the problem here, as EU maps and textures are still XCOM's maps and textures. Drakous79 was talking about entirely new objects, created with UDK, which have licensee number set to zero. So I'm not sure it's a licensee problem. I can test it under linux, but not right now. BTW, all the packages under windows and linux are binary identical, so they haven't recooked anything. We can insert new default properties into existing ones, we just need to be very careful with any bulk data objects like embedded low-res textures. I've also noticed a lot of GUID references inside textures/materials/meshes. Some of the objects on those old maps do load all the textures, so, again, I don't think it's licensee related. But it's possible that some of the map-specific objects got unused when they deleted those maps from EW and as a result some of the references became invalid.

Well, I linked everything I have: http://forums.nexusmods.com/index.php?/topic/1104408-r-d-xcom-map-alterations/?p=24119134 I helped Drakous79 with testing and lzo packing/unpacking, but I'm afraid my knowledge of XCOM graphics (and graphics in general) isn't quite good. :smile:

This was like RENAME=Textures:Yextures and it didn't work, because it also "renamed" all the references, which use Textures in their names, like Texture2D'GEN_MaterialParents.Textures.Default_Normal', for example. So your patch is better. :smile: Any ideas about those bad tiles? There are also materials and meshes there, as far as I know, probably, some parameters there also need to be adjusted?

Ghostbusters flag by Drakous79: https://drive.google.com/file/d/0B5MAcyqYBx4dMDJweGRTVnRKSmM/view?usp=sharing Note that this was made for an earlier patcher version which had no pseudo-code and no replace all functionality. It contains both low-res embedded textures and links to hi-res textures in TEN.tfc If you look at the Texture2D object's default properties, you'll see the one named TextureFileCacheName. I believe, that's the one that needs to be replaced (example for WP_PistolModern.Textures.T_PistolModern_DIF): NameIdx: 0x000008FF (Index) 0x00000000 (Numeric) -> TextureFileCacheName TypeIdx: 0x000005A6 (Index) 0x00000000 (Numeric) -> NameProperty PropertySize: 0x00000008 ArrayIdx: 0x00000000 Name: 0x00000914 (Index) 0x00000000 (Numeric) = Textures As I understand, Drakous79 replaced it with a name reference to TEN - a random name from name table. I thought of just renaming Textures name table entry, but it is used by other objects, which don't need to be changed. I just remembered that I even tried the renaming idea myself shortly after Drakous79 made his discovery, but failed, probably, because I broke the other objects. So, forget what I said earlier about renaming textures to eutextrs, it was a bad idea. :smile: Just pick some name from the name table and replace all the TextureFileCacheName properties value with it while renaming EU's Textures.tfc accordingly. UPD: just found another example by Drakous79: https://drive.google.com/file/d/0B5MAcyqYBx4ddXQyVEI4QlZ3TTQ/view?usp=sharing

Well... it's just a theory. Drakous79 was able to replace flag textures by replacing old texture file name with a new one (and by changing bulk data offsets, but this won't be necessary here), but as I recall it needs to be done for each Texture2D object in the package. Since the replacement code is similar, it should be possible to use replace all patcher functionality.

Still no luck running UEE with mono, but I was able to finally run it with wine: http://i.imgur.com/JHkEgao.png There are some visual glitches and it works really slow, especially with big packages, but still it works. :smile: There were several problems involved. First, to run 32-bit windows application under 64-bit linux you need wine32 installed, which conflicts with wine64 (at least on my distro). Second, you don't need to install wine mono libs, but use winetricks to install dotnet40 instead. If you allowed wine to download and install mono libs before installing dotnet40, it will fail to uninstall mono properly and dotnet40 won't work. In this case you need to download ".NET Framework Cleaner Tool" to clean up all your mono and possible previous dotnet versions and then install dotnet40 again.

You can rename EU's texture.tfc into something else like eutextr.tfc, copy it into EW cooked folder, unpack those maps and replace texture references with eutextr.