So after an absolute ton of testing, I finally found the culprit, and no wonder I had such a hard time figuring out what mod it was. The mod in question is a texture mod but not a landscape texture mod which is surprising. The culprit was Prewar walls for Sanctuary by DKnightwolfe. I'm not sure if the mod conflicts with some other similar mods I'm using like Paint that House, Patch Job, or Repaired Sanctuary Roofs, there were some overwrite requests in testing but either way disabling that mod fixed the problem entirely.