Please avoid using BC7 compression for normal maps

[EinBlonderTraum]

Normally I download the largest size textures and resize down as appropriate in Photoshop with a little bit of filtering thrown in (eg sharpening after a reduction step). At first I bought into the hype surrounding BC7 and was automatically saving all normal maps I had resized as BC7 with the Normal quality settings (the highest two settings in the Nvidia Dds plugin take forever). Also diffuse textures with alphas I was saving as BC7 (those without as BC1 since it is half the size).

 

Everyone talked about how superior the quality of BC7 was, especially for preventing Normal map artifacts. Everything seemed to be okay. Then one day I reopened a normal map I had altered and glanced at the alpha map channel in Photoshop. I was horrified at how low res and pixellated it had become, was though the 256 colour greyscale had been rudely truncated down to a palette of 8 or fewer shades. I initially assumed it was a bug in Photoshop or the Nvidia plugin, but I am increasingly thinking that this is unlikely. I downloaded retexture mods that had both Oldrim and SE releases (the later using BC7 compression), such as the excellent Rustic Clothing. I compared the normal maps from both versions. There was a very slight quality loss in the RGB channels of the Oldrim DXT compressed version, but the alpha channel of the Oldrim versions (the specularity information) was far superior in quality.

 

BC7 without an alpha channel is the same size as BC7 with an alpha channel, whereas with old DXT formats alpha channels double the file size (which seems excessive since you are adding one channel to the existing 3). Obviously BC7 allocates very little bandwidth to the alpha channel, hence the pixellation and degradation. If an alpha channel has little detail (mostly blocks with the same brightness values), BC7 compression woll not cause much harm, but for textures with highly detailed speculars where every pixel is different, the quality will be totally trashed. I am currently going through Mod Organizer and redoing normal maps that I had saved as BC7; now they are saved as ARGB 8888 in Photoshop and converted to DXT5 using Ordenador since it is less likely to artifact normal maps.

 

Most programs display normal maps with transparency applied, so it not surprisingly is hard for people to see the degradation in the alpha channel of the normal maps. The only way I know of to view the isolated alpha (spec) channel for BC7 Dds is using Photoshop, but you have to select the option to load transparency in a separate alpha channel when you first run the plugin (I assume it saves the setting in the registry). I am including a link to the Nvidia plugin since it is a hassle to download it directly from Nvidia: https://www.mediafire.com/file/g9waxu5a2uh9pe2/Nvidia_Texture_Tools.zip/file

Install it by extracting to C:\Program Files\Common Files\Adobe\Plug-Ins\CC\Nvidia2020