Texturing Metals Not Working in FO4 (cubemaps)

[Searcherz64]

I'm trying to texture metals so they look proper and shiny, but I just cannot get this to work. I read after much research that metals get their magic from cubemaps (https://www.nexusmods.com/fallout4/mods/27631/). But it just isn't working - I would love to get input from people who've managed to get this working:

 

My specular (_s):

 

 

My diffuse (_d):

 

 

My BGSM:

 

\

The cubemaps are from https://www.nexusmods.com/fallout4/mods/27631/

 

And the model:

 

 

 

I swear I got this working yesterday and switching out different cube maps gave me different reflective metals but now lo and behold it doesn't work anymore, and switching cubemaps have 0 effect on the textures

 

Edit: For some reason after I closed out all my windows and tried again, the reflections seems to work now in the preview:

 

 

However it's still pitch black in game:

Edited July 28, 2018 by Searcherz64

[dikr]

I recently learned that Normal maps are highly influential on the rendering of cubemaps. Not sure how you save and open those as I only see spec and diffuse maps but it is worth experimenting with normals. When I work with existing normals I open them with PS with the "open as" option and pick D3D/DDS then tweak the most left channel in Levels to 120 /128 to get normal (pun not intended) results. When saving them I use the D3D/DDS option and pick 3Dc. Good luck!

[Searcherz64]

Ah I read somewhere that normals weren't that influential on metals here, so I didn't bother posting it, but here it is:

 

 

I've also tried inverting the red channel as shown in the screenshot there (as instructed in https://forums.nexusmods.com/index.php?/topic/4700175-fallout-4-texture-system/?p=40825650) but it didn't seem to affect anything. How would I be able to change the channel to 120/128? (not too experienced at this stuff)

 

I updated my original post with some new information where the reflections show in the preview window but not in-game:

 

[ehtyeci]

What happens when you put the default cubemap in the material? Eg. Shared/Cubemaps/mipblur_DefaultOutside1.dds

Does it show up in-game?

If yes, something's wrong with the custom cubemap.

If no, the problem is likely elsewhere.

And it's the green channel you may need to invert on the normal map if you baked normals with +Y swizzle.

Neutral normals are 128/128/255 RGB respectively, although blue channel doesn't matter since it gets computed in the shader.

[dikr]

The normal map looks ok to me.

 

Other ideas:

 

You could try saving the Normal map as 3Dc. I have better results with that than BC5 sometimes.

 

Also I do recall reading about the orientation of textures and that they can be accidentally flipped inside out in nifskope and yielding similar dark results. Not sure how to flip it back though but that is a common problem and should yield some info in google. You could also try playing with the Two Sided flags in the material file and/or the nif. Maybe that helps with mapping the texture properly.

[Searcherz64]

The normal is baked with Substance Painter and outputted with a custom preset (like what they did in https://forums.nexusmods.com/index.php?/topic/4700175-fallout-4-texture-system/):

 

 

 

Original normal baked with painter: (ignore the problematic mesh designs below hehe, I just need to nail down how to get metal reflecting)

 

I'm not sure how to tell if the default cubemap works but here are screenshots I took:

 

Without flipping anything on the normal, with black and white diffuse (ignore the ugly mesh)

 

With red flipped:

 

Flipped green:

 

To be clear the texture isn't always black - the diffuse texture shows fine, it's just the metal not reflecting properly with the use of cubemaps. Thank you guys for the help so far. I'll try converting it to 3DC with nvidia's plugin instead and report back

Edited July 28, 2018 by Searcherz64

[ehtyeci]

Judging by the pictures it looks like the specular isn't being applied - only diffuse and glossiness.

Looking at your export settings, I think I see an issue.

Painter treats Specular as RGB (true PBR spec/gloss uses RGB for specular channels) while as FO4 uses grayscale (one channel).

In the export configuration, you need to assign the gray channel from the converted Specular into the red channel of your $textureSet_s. Gray means RGB values added together into one value. If you only export the red channel of the converted Specular, you will only get a third of the luma value you're supposed to have. You can even see in the first image of your post, the red channel is basically dark gray. Specular needs to be much higher to show the metallic properties, especially when the diffuse is pitch black.

Edit: Another thing is that you're exporting an OpenGL normal map, which is wrong for FO4. It uses DirectX. Which means you probably also have the Project Configuration set to OpenGL, which means you'll need to set it to DirectX, re-bake, and pick the Normal DirectX for the export configuration. This way you don't have to fuss over flipping any channels.

[Searcherz64]

 

Judging by the pictures it looks like the specular isn't being applied - only diffuse and glossiness.

Looking at your export settings, I think I see an issue.

Painter treats Specular as RGB (true PBR spec/gloss uses RGB for specular channels) while as FO4 uses grayscale (one channel).

In the export configuration, you need to assign the gray channel from the converted Specular into the red channel of your $textureSet_s. Gray means RGB values added together into one value. If you only export the red channel of the converted Specular, you will only get a third of the luma value you're supposed to have. You can even see in the first image of your post, the red channel is basically dark gray. Specular needs to be much higher to show the metallic properties, especially when the diffuse is pitch black.

Edit: Another thing is that you're exporting an OpenGL normal map, which is wrong for FO4. It uses DirectX. Which means you probably also have the Project Configuration set to OpenGL, which means you'll need to set it to DirectX, re-bake, and pick the Normal DirectX for the export configuration. This way you don't have to fuss over flipping any channels.

 

 

Ah got it. Changed from OpenGL to DirectX.

 

Sorry, could you clarify this bit? "assign the gray channel from the converted Specular into the red channel of your $textureSet_s". The Converted Specular is already in the red slot , and glossiness is in gray slot for $textureSet_s. Here is an updated pic of my current export config, with Normal DirectX:

 

 

I've also tried using nvidia's photoshop plugin to convert to 3DS for only the specular map and it seems to have a dramatic change (keep in mind, this is with one of the default cube maps):

 

BGSM:

 

 

Converting the spec map:

 

 

Results with Intel-converted black-colored diffuse, normal, and specular (left) vs Intel-converted black-colored diffuse, normal, and nvidia-converted specular (right)

 

 

I'm not sure if this is a success or not, but it's better than pitch black, so I tried with the nvidia-converted specular with one of the custom cube maps by that mod author (https://www.nexusmods.com/fallout4/mods/27631/) and I get this result. Not sure if it's right since from what I've read, the cube map should determine the metal color (this was with Emerald color)

 

Edited July 28, 2018 by Searcherz64

[ehtyeci]

Your config is set to export only the red channel from Specular as indicated by the red bar below the "R".

This means that values from green and blue channels of Specular (Painter's specular) is not factored in when exporting, so your output texture only contains a third of the specular value you used for texturing.

When assigning a channel onto a texture set in the export config, you get a dropdown menu to choose which channel from the converted maps you want to export.

Here you need to select Gray, because it is the sum of RGB from Painter's converted specular. When doing so, the bar under the "R" will change from red to gray, which is the correct one.

[Searcherz64]

Ah I see now. Alright here are the results:

 

Config:

 

 

Specular:

 

 

with intel-converted spec dds vs nvidia-converted spec dds (3Dc):

 

 

 

It doesn't look like it's changed anything unfortunately, unless I did something wrong