Question: Native Code

[Krazyguy75]

Hello, been modding here for a while, and I constantly hear native code mentioned. I know what native code is and vaguely what it does, but I was wondering, where exactly is all the hex for native code located, and why can't we work on finding variable values in that (by guess and check if need be)?

 

It's probably a very simple answer, but I never understood why it is such a problem to edit.

[johnnylump]

We believe native functions reside in xcomgame.exe, which we think was coded in C++. My understanding is C++ generally can't be decompiled with any reliability -- we've never been able to even locate or change constants in the exe, much less edit actual functions. About all we've done is edit some file paths, which are strings.

[Amineri]

My understanding of this (which I am sure is far from complete) is that 'native' code is so-called because it is actually compiled to work on a particular host processor. Most programs work this way (with the exception of Java / Actionscript or other languages that are designed to run through an interpreter or some form of virtual machine). So the native code in the executable is compiled to machine-hardware-specific bytecodes (e.g. x86, or Cell for the PS3). If it has to make calls to the OS (which is likely) then either those code libraries are compiled in or (more likely for PC) added through dlls.

 

Hence on PC you have the XEW/Binaries/Win32 folder containing everything that is compiled to bytecode to be executed directly on the intel/AMD/etc x86+win32 host architecture. This includes XComEW.exe but also a lot of dlls to perform various operations (libogg.dll, libvorbis.dll, bin2w32.dll, steam_api.dll, SteamQuery.dll).

 

Whereas Java is designed so that a program can run complete within the Virtual Machine environment, the Unreal Engine is a more hybrid design. This is probably due to both historical design requirements, the performance requirements of running 3D modeling/rendering, and various interfacing requirements.

 

Unrealscript is compiled into unrealhex bytecodes that are designed to be run on the Unreal Engine virtual machine.

 

As far as I can tell the entry point of every Unreal Engine game is always an executable. The Unreal Engine provides for linking between native code functions and unreal functions so that "thread of execution" can pass between the unreal bytecode part of the game and the native bytecode part of the game.

[dreadylein]

Amineri gave a good explanation :)

 

@johnnylump

 

The biggest difference between compiled nativ code and bytecode that runs in any sort of interpreter/vm is:

 

1. Bytecode has normaly more highlevel informations

2. Bytecode has less optimations as most of it is done by optimizing the VM code.

 

This leads to some problems for the modding community:

 

Reversing Binary Code needs a good understanding of the host system and a quiet big skillset.

If i remember right here a quiet some people able todo it, but it is always realy timeconsuming.

 

There is no way to get highlevel Code out of a Binary, as many informations are striped by the compiler(as example variable names), additional optimazations done by the compiler make the code we can see even more unreadable(the same codepart in highlevel can translate to two quiet different parts in the binary, as example the code calls function a on two different points, but the compilere decides it would be faster to directly inline them)

now the binary code would change from:

 

Function b:

bla

call Function a

bla

 

Function c:

bla

call Function a

bla

 

to

 

Function b:

bla1

code of Function a

bla2

 

Function c:

bla3

code of Function a

bla4

 

In the next step of the opimization the Compiler looks at the Code and mix him for getting a better perfomance out of it ,backing together the whole code of Function b and c

 

In the End it would look like:

 

Function b:

bla5

 

Function c:

bla6

 

And there are tons of codepart where such things can happen.

Additional the Work must be done on every different platform, as different compileres for different platforms give totaly different code(see Amineri`s Response).

 

Reversing is the art of rebuilding a cow out of hamburgers ;)

 

Are there any nativ Functions which are absolutly needed to be moddable ?

[Krazyguy75]

Are there any nativ Functions which are absolutly needed to be moddable ?

 

No, it's just a matter of not liking being told that I can't do something without knowing why. Now that I know why, I'm fine with it. Just a little pet peeve. Though, to be fair, I really wish I could see the ability code.