SSE Normal Map Woes

[mcchuggernaut]

I know this topic has been discussed before, but upon scouring the internet I couldn't find anything that worked for me. Hopefully someone here has ideas.

 

 

The problem is that Skyrim uses the alpha channel of normal map files to control the "shininess" of textures. Whenever I make a normal map in Gimp, the alpha channel defaults to white which makes everything ultra-shiny, but conversely when you force gimp to use a greyscale or black alpha channel (Holy s#*! did it take me forever trying to get Gimp to do that. Gimp REALLY makes it difficult.) I am getting a very odd black sheen on the objects instead. Has anyone ever finally solved how to deal with reflective textures once and for all?

If I should be using a different program or am going about doing this all wrong, please let me know. I have been at this for a full 2 days and it's driving me crazy, so I figured asking where all the other mod creators discuss these things was the best option I had left.

[Di0nysys]

You could always just turn off the specular setting in the BSLightingShader Property of your mesh. It also allows you to control the gloss through it's settings as well if u wanna retain some gloss. I suspect the black spots are due to incorrect mipmaps generated for ur normal. Either switch over and use the intel dds plugin for photoshop or save the normal without mipmaps.

[thumbincubation]

I don't understand why it's defaulting to white for you. This isn't my strength, but here's what I usually do.

 

Open the diffuse map in Gimp, but uncheck "load mipmaps" first.

 

When open, layer > transparency > remove alpha channel

 

Filters > map > normal map

 

I check "wrap," and give it a filter setting (4 sample, 3x3, 5x5, etc.) and set the scale, usually to about 7ish, but more if I'm doing a rough texture.

 

Now it should be a blue with a bit of pink in it.

Layer > transparency > add alpha channel

 

Select the eraser tool with a double click, to bring up the tool settings. It should have a blurry dot for the brush. Set that to a solid, so it will fill evenly, and drag the size up until the circle completely covers the entire texture.

 

Set the opacity to whatever you want it to be, with higher numbers being more dull. (I have read in a tutorial, the name of which now escapes me, that anything 95 and over can cause problems, so I stick to 94% as my "most dull."

If I'm doing shiny metal, I'll usually start around 75% with the understanding that I may have to redo the normal a few times, at 80%, 85%, etc., to get the shine I want, without it being too much.

 

Once set, center over the map and click once to erase. It should give you the checkerboard with just a hint of the details of your pic visible.

 

Export as > give it the same name as the diffuse but with _n afterward, and set the export settings to at least BC2/DXT3, if not a bit higher. I tend to muddle around in vanilla-level graphics, so I don't have any experience with the super high definition compressions. Set it to generate mipmaps and then open advanced options in that same window. Tell it to preserve alpha test coverage.

 

That *should* give you a working normal map. When I need things to be at different scales, I end up using the lasso tool to outline them and creating a normal map for each selection, separately.

 

That being said, a graphics pro will probably have a much quicker, more efficient way to do this. I just have had success with it in my tinkering, and hopefully it will help you.

[mcchuggernaut]

 

Select the eraser tool with a double click, to bring up the tool settings. It should have a blurry dot for the brush. Set that to a solid, so it will fill evenly, and drag the size up until the circle completely covers the entire texture.

 

Set the opacity to whatever you want it to be, with higher numbers being more dull. (I have read in a tutorial, the name of which now escapes me, that anything 95 and over can cause problems, so I stick to 94% as my "most dull."

If I'm doing shiny metal, I'll usually start around 75% with the understanding that I may have to redo the normal a few times, at 80%, 85%, etc., to get the shine I want, without it being too much.

 

Once set, center over the map and click once to erase. It should give you the checkerboard with just a hint of the details of your pic visible.

 

Holy crap this worked! Thank you so much! I have been beating my head in frustration over this for days!

 

Yes, for some strange reason Gimp was defaulting my alpha channel to white. I tried messing around with layer masks and all kinds of other nonsense, and when I did manage to darken the alpha channel I got weird reverse shadows and other errors. BUT for some strange, incomprehensible cosmic mystery of a reason, after I hit the map with the eraser tool the alpha channel works correctly, and can be darkened or lightened depending on the magnitude to which I set the eraser! That lets me precisely control the level of shininess!

 

Bless you kind soul! I supremely appreciate you taking the time to help me with this vexing problem!

Edited March 13, 2017 by mcchuggernaut

[thumbincubation]

I'm glad to help. :)

[montky]

apologies for a bit of benign-thread necro.

one good deed though deserves another, as this thread has answered a couple of questions I didn't even know I had

 

 

 

thumbincubation's answer here was recently linked elsewhere,

I don't check Skyrim etc as much as other threads, so hence my delay hehe.

It's really awesome, and I hope it'll be cross-posted in the GIMP Script-Fu

code-stack-exchange and thisweekintech guides.

 

It may be possible, with Python-Fu or Script-Fu,

to automate some or all of that pipeline to a stage to save on some modding-time,

less time manually loading stuff etc, and more time iterating.

Perhaps, some of you have dabbled in that also and might tentatively have some script-fu snippets to share?

 

each of those commands in Script-Fu can be automated to an extent,

made to open the texture source files from an index or directory,

and you could say, get it to perform the 'basic operations',

assuming you keep an exegetical note on how you achieved your effects.

that directory itself could be populated by other search commands...

say, as a for-instance, from photo-ref 'webcrawlers' from prop databases, which have share-alike ISO view photoref etc.

this comes in handy if you're making lenticular things,

or converting stats in a database into their visual stat-card form.

 

if you have a lot of things you wanted to consistently texture,

you could keep an exegetical 'recipe', and update your script-fu from the exegesis.

 

or, say you have iso-view sketches, or scan pen-and-paper from your scanner/photo-archive.

one script-fu script can clean those up, turning them effectively into "OHT's".

another can then convert the images produced into useful textures.