I've noticed that removing all ENB, ReShade and ShadowBoost mods (I mean completely remove their DLL's) helps immensely. I have reference Sapphire RX 480, 1080p monitor, all settings on Ultra (including God Rays, HBAO+, TAA, 16 sample Anisotropic Filtering, max draw distance, ultra shadows). Getting dips to 40 FPS around GoodNeighbour and in Diamond City when looking from Mayor's office. But usually, I get solid 60 FPS (have not tested very thoroughly with fresh updates). AMD Drivers are 16.7.3, Fallout 4 - 1.7.9 with F4SE 0.2.1 rel 1. I play with around 40 mods, some of them impact graphics - like Fallout 4 Seasons - Summer mod. Screenshots of GPU taxing areas. EDIT: I see you are using an ultrawide-HD monitor, so my setup comparison is not exactly apples to apples. But my main point still is - try removing ENB's.
