Reverse Lighting Issues

[einherjrar]

I've been having some trouble with some of the weapon models I've included in 20thCenturyWeapons mod.

 

I've had to mirror some of the models from left to right to make them anatomically correct (ie; shell ejection port on right hand side, safety switches on left hand side, ect); The trouble is, the lighting maps ('normal' maps?) remain oriented for the un-mirrored model, so some of the models will now display reverse lighting, which can look pretty ugly.

 

Does anyone know how I can fix this?

 

I'm fairly competent with Milkshape, Nifskope, and the GECK, and I think I may have seen a setting or option to reverse lighting, but I can't remember where the option was and have begun to wonder if I imagined seeing this setting/option at all.

 

I also have Autodesk3ds, and Adobe Elements (but I'm not that great at using them).

 

Can anyone help me out? It's been a long time problem but is really starting to bother me.

 

Thanks in advance,

-ein

[SpeedyB64]

in nifskope right click on the object and select Mesh -> Face normals

 

That should work.

[Baelkin]

If the above does not work, you can also manually fix the normals in 3ds max although that can be quite time consuming. I don't really know anything about Milkshape, but it's likely to have comparative tools for flipping the normals to their proper position, although I have no idea what it'd be called.

[einherjrar]

Thanks for the info;

 

I haven't had a chance to try it out yet, but hopefully I can get it done in Nifskope with the Mesh > Face normals command.

 

As to 3ds, I wouldn't have any idea how to get it done without flipping the entire model back to the anatomically incorrect configuration or having the faces create a transparent window effect on the model, since only one side of the faces apply any texture.

 

-ein

[Skree000]

OK before you go any further,

 

 

are you talking about the Normal Map being mirrored is displaying texture detail insideout/upside down, ie. Bumps appear to be 'dented in'?

 

-OR-

 

do you mean the normal map is being mirroed, so the texture detail is being Lit Backwards, ie. When a light shines down upon it, the mirrored part seems to look like its lit from below instead of above.

 

-OR-

 

do you mean parts of your model appear transparent.

 

-OR-

 

are you getting strange (usually 'blobby') shadows or shading on certain parts of your object?

 

 

in the first case, its your normal map having some parts on either red or green channel need to be flipped

 

in the second case, its because of how normal maps contain directional lighting information. One channel is Up/Down, the other is Left/Right. If you flip something on the model that is assigned to a place on the texture with a specifical directional lighting info, it will appear wrong on the model. (this is commonly referred to as 'mirrored normals problem'.

It depends on the game engine, i dont know offhand if there is a setting in nifskope to fix that. I believe it has to do with the renderer.

btw. 'Normals' are the direction a triangle is facing. A 'normal map' is a fake map of the directions the user wants light to behave in, so as to hint at details that arent really there. Ie. a flat surface with normal information of something complex will LIGHT like its complex, yet the details arent really there.

 

 

in the third case, it just means some parts of your model have 'flipped normals', meaning, some polygons are flipped inside-out (the 'normal' of the triangle points inward into the model). This can be fixed in max by opening your model, right clicking on it, in the Properties box, check off 'Cull Backfacing'. This will 'hide' any back-facing polygons, to fix them, click them (in polygon mode) then hit 'Flip' on the side bar (also available from right click menu)

Typically on a model you want all the normals to face outward so players can see them. If your texture is not a 'double sided' material, it will look invisible from the 'wrong' side of the polygons 'normals'.

NOTE: Vertices cannot be welded together from one triangle facing one way, and another triangle facing the opposite way. Ie. if you have a flat surface with one triangle facing inwards, that triangle will never be able to attach to the flat surface, until its normals are flipped to match the surrounding normals. You can never have a surface facing up, directly WELDED to a surface facing down. There has to be some kind of transitional triangle that is half way bent, to accomodate the change in direction. (or just flip the normals on the anomalous tri)

You can ATTACH multiple triangles facing all kinds of directions together, but when you want to weld them, they have to have matching face-normals, or they wont weld.

 

 

in the fourth case, (smoothing groups problem)

 

Smoothing groups are artificial relationships between multiple triangles to instruct the 3d renderer to 'treat this as one surface'. So any bunch of triangles on the same smoothing group will appear smooth, and rounded.

Triangles not on the same smoothing group will visibly show their edges, and appear 'faceted' (like a crystal, where each side is clearly visible, looking 'chiselled out')

 

In nifskope, the right click option 'Face normals' will indeed do something to your normals, but not what you expect. 'Face normals' basically breaks all smoothing-groups.

You only really need to do this if you are having 'Smoothing group' problems. (usually denoted by shading errors, ie. Weird shadows on your model that look very wrong and usually 'blobby'.)

setting 'face normals' in nifskope is the exact same as in max, selecting all polygons, then hitting 'Clear groups' in the smoothing groups window. It will make everything look faceted.

 

This is not likely what you want. If your object has parts that you want to remain rounded-looking, those parts need to be selected (by hand) and their smoothing values/groups need to be setup.

 

Smoothing Tolerance, or angle, is a generic 'across-the-whole-model' angle in degrees that will determine whether or not surfaces should be smoothed (soft edges) or given hard edges.

Its really lazy, and usually never looks 100% right, but it works in a pinch, and can save a ton of time.

 

Doing smoothing groups by hand, triangle by triangle is a pain in the ass, but the only REAL way to get the model to look 100% perfect.

[einherjrar]

OK before you go any further,

 

 

are you talking about the Normal Map being mirrored is displaying texture detail insideout/upside down, ie. Bumps appear to be 'dented in'?

 

I doubt this is the problem, I haven't identified anything being 'dented in'.

 

-OR-

 

do you mean the normal map is being mirroed, so the texture detail is being Lit Backwards, ie. When a light shines down upon it, the mirrored part seems to look like its lit from below instead of above.

 

I'm not sure if this is exactly what I am experiencing, but it sounds very much like something similar to what the problem is. Whichever side is facing a light source will be shaded, and the opposite side of the model will be lighted as though it is facing the light source.

 

As I mentioned in the above message(s), the model comes built with the parts facing the wrong direction (ie; shell ejection port is facing the left, when it should be facing the right side of the model from the shooters point of view).

To correct this, I select the entire model (or model faces) and in MilkShape go to 'Vertex > Flip front/back' or 'Vertex > Flip left/right' depending on which direction it has to be flipped to be anatomically correct to a real world version of the weapon.

 

This method effectively creates a mirror image of the model, with all the components being flipped in the direction that is needed, but I think is the main reason that I am experiencing this lighting issue.

 

-OR-

 

do you mean parts of your model appear transparent.

 

I have experienced transparency of the model when I first tried to flip the models in my first attempts at modding, but I got past this long ago. I was selecting vertices and not faces when I was flipping the model (which caused this); Transparency was solved by selecting faces instead of vertices and using milkshapes 'flip front>back' or 'flip left>right'.

 

Transparency is definitely not the problem.

 

-OR-

 

are you getting strange (usually 'blobby') shadows or shading on certain parts of your object?

 

Not that I can see.

 

in the second case, its because of how normal maps contain directional lighting information. One channel is Up/Down, the other is Left/Right. If you flip something on the model that is assigned to a place on the texture with a specifical directional lighting info, it will appear wrong on the model. (this is commonly referred to as 'mirrored normals problem'.

It depends on the game engine, i dont know offhand if there is a setting in nifskope to fix that. I believe it has to do with the renderer.

btw. 'Normals' are the direction a triangle is facing. A 'normal map' is a fake map of the directions the user wants light to behave in, so as to hint at details that arent really there. Ie. a flat surface with normal information of something complex will LIGHT like its complex, yet the details arent really there.

 

<snip>

 

I was under the impression (mistakenly it seems) that normal maps were just texture files, I was unaware that they are built into the mesh; but if they are part of the mesh that makes sense too.

 

In nifskope, the right click option 'Face normals' will indeed do something to your normals, but not what you expect. 'Face normals' basically breaks all smoothing-groups.

You only really need to do this if you are having 'Smoothing group' problems. (usually denoted by shading errors, ie. Weird shadows on your model that look very wrong and usually 'blobby'.)

setting 'face normals' in nifskope is the exact same as in max, selecting all polygons, then hitting 'Clear groups' in the smoothing groups window. It will make everything look faceted.

 

This is not likely what you want. If your object has parts that you want to remain rounded-looking, those parts need to be selected (by hand) and their smoothing values/groups need to be setup.

 

Smoothing Tolerance, or angle, is a generic 'across-the-whole-model' angle in degrees that will determine whether or not surfaces should be smoothed (soft edges) or given hard edges.

Its really lazy, and usually never looks 100% right, but it works in a pinch, and can save a ton of time.

 

Doing smoothing groups by hand, triangle by triangle is a pain in the ass, but the only REAL way to get the model to look 100% perfect.

 

If I understand this correctly, there is no quick fix and I would have to correct each model face by face?

 

That gives me a headache just thinking about doing face by face corrections to even a single model; most of these models have hundreds of faces, and at least 40 of the models I've included in the mod are experiencing this.

 

Have I understood this right?

 

Is there a quick and dirty method that might not be perfect, but will be expedient?

 

-ein

[Skree000]

ok before i can be 100% sure of your exact situation, if you can post a screenshot of your problem it would be most helpful.

 

 

Normal maps arent part of the mesh, they are just a texture, however they are a special texture that has directional lighting information stored in grayscale, in 3 different channels, RGB.

Specular maps are also an informational texture, white being shiney/reflective, black being matte (nonreflective)

so think of a normal map as more of an informational texture.

Diffuse is the only texture that actually contains stuff we want to see. The others are just for giving instructions to the renderer.

 

When a renderer goes to light a model, it takes a look at the polygons, the smoothing groups, the surface normals of the triangles (taking into account smoothing groups) and the light source, and makes some calculations then starts to shade it as best it can.

 

However, if it sees a Normal map plugged in, depending on its intensity, it will disregard the actual polygons of the model, and instead gather all the information for lighting from the normal map texture.

 

If a part of the texture says, 'this area has ridges', but is uv-mapped to a perfectly flat part of the texture, the renderer will light that flat area like it DOES have ridges, and you will see cool shadows and lighting in that area, despite the fact there isnt really anything there.

 

 

IF the problem is mirrored UV's with things looking upside-down or backwards, then you may have a big problem.

 

On the weapons im working on, i mirror the geometry all the time from one side to the other, sometimes even top to bottom to piece it together. I dont really notice any bad normal-related lighting issues (too much)... from what i can tell Fallout3 seems to treat all normal maps equally whether they are mirrored or not.

 

Again, if you can post a pic of your issues (perhaps in 2 different lighting conditions so we can compare whats going on) id be able to provide more help. (and a solution)

 

 

There are solutions to everything :) some are just less desireable solutions lol

[einherjrar]

ok before i can be 100% sure of your exact situation, if you can post a screenshot of your problem it would be most helpful.

 

 

Normal maps arent part of the mesh, they are just a texture, however they are a special texture that has directional lighting information stored in grayscale, in 3 different channels, RGB.

Specular maps are also an informational texture, white being shiney/reflective, black being matte (nonreflective)

so think of a normal map as more of an informational texture.

Diffuse is the only texture that actually contains stuff we want to see. The others are just for giving instructions to the renderer.

 

When a renderer goes to light a model, it takes a look at the polygons, the smoothing groups, the surface normals of the triangles (taking into account smoothing groups) and the light source, and makes some calculations then starts to shade it as best it can.

 

However, if it sees a Normal map plugged in, depending on its intensity, it will disregard the actual polygons of the model, and instead gather all the information for lighting from the normal map texture.

 

If a part of the texture says, 'this area has ridges', but is uv-mapped to a perfectly flat part of the texture, the renderer will light that flat area like it DOES have ridges, and you will see cool shadows and lighting in that area, despite the fact there isnt really anything there.

 

 

IF the problem is mirrored UV's with things looking upside-down or backwards, then you may have a big problem.

 

On the weapons im working on, i mirror the geometry all the time from one side to the other, sometimes even top to bottom to piece it together. I dont really notice any bad normal-related lighting issues (too much)... from what i can tell Fallout3 seems to treat all normal maps equally whether they are mirrored or not.

 

Again, if you can post a pic of your issues (perhaps in 2 different lighting conditions so we can compare whats going on) id be able to provide more help. (and a solution)

 

 

There are solutions to everything :) some are just less desireable solutions lol

 

I'll give one more shot at explaining before I'll try and produce screenshots of the problem.

 

In nifskope, under:

BSFadeNode > NiTriShapes (or strips) > BSShaderPPLightingProperty > BSShaderTextureSet;

 

Under block details (expand textures branch with '+' sign) I see:

6 entry fields for textures.

 

If I understand right, the Diffuse Texture Map will occupy the top-most entry field, and the Normal Texture Map will occupy the second top-most entry field. I have no real idea what the other entry fields are for, but the last two slots seem to be for 'shiny' effects.

 

The diffuse map (top field of entry) displays the correct facing straight from directly from export, and any text that will be displayed on the model is oriented correctly.

 

 

So, if I am having trouble with reverse lighting, maybe I should leave the second field of entry blank and just let the game attempt to render the shadows?

 

...That may actually explain why the game renders some of the flipped models with correct shading, and some without. I never really payed attention to which had the normal maps and which didn't.

 

The second field of entry graphic files I'm referring to are usually marked with a '_n', '_norm', or '_normal' tacked onto the end of their texture file, and tend to either be a psychedelic or very dark greyscale image of the diffuse map.

 

So, if I'm understanding this correct, the light blue/green maps are normal, the greyscale are specular, and full textured images are diffuse.

 

If removing the normal maps from the model doesn't seem to do anything for the lighting situation, I'll take some screenshots of each side facing the light and facing away from the light and post them here sometime tomorrow (I'm not near my actual PC ATM).

 

I really appreciate the help. =)

 

-ein

[Skree000]

yeah texture set is like this

1. Diffuse/Color

2. Normal (_n) with Specular in the Alpha of that texture

3. Glowmap (_g)

4. (not sure)

5. Cubemap/Environment Map

6. Cubemap/Environment Map Mask

 

 

a cube map or environment map is a texture of a picture that you would see 'reflected' across whatever surfaces are 'masked out' by the texture in 6.

 

5. can be a full color texture of anything, usually its best to make a proper cubemap for those, but a blurry or bizarre random shape can often produce interesting results too.

6. is a grayscale map that defines what parts of your mesh will show the 'cubemape/enviromap'. Black wont show it, white will. Grey and middle values are gradual blends.

 

If the texture is a 32 bit TGA, or a DX5-compressed DDS, it will have an alpha channel, which is basically an additional channel, (in addition to red, green and blue).

 

In a normal map, since red and green contain updown and leftright light information, (blue is usually blank white in Tangentspace normal maps), the alpha channel is free, and Bethesda likes to stash their Specular maps in there. That way it saves memory since theres only 1 texture used for both purposes :)

[einherjrar]

Okay,

 

Thanks to this info I've managed to solve the mystery of my reverse lighting problems;

 

I haven't been able to test if this is the case on all the models I've used, but at least on a few of the models I've tested I've been able to fix the problem;

 

I had applied the diffuse map on the first AND second texture file slot in NifSkope for any model that didn't have a normal map included; which I guess is a big no-no.

 

The models that came with an actual normal map light correctly; but if there wasn't a normal map, and the diffuse map was applied to the normal map (2nd slot), the lighting displays reversed.

 

Since I assumed the problem was that the normal map had somehow become reversed when I created a mirror image of the model, it appears the problem was actually the opposite of what I thought and can be solved by simply leaving the 2nd texture slot blank.

 

Easy solution, but I couldn't have done it without the information provided here,

 

Thanks again!

-ein