I think I found the problem here.
I deleted the plugins.txt file and restored files from steam, copied that one as a backup, then let Vortex install mods and noticed that Vortex seems to add two extra lines to the Plugins.txt folder that should not be there. Then I tried disabling all the mods in Vortex and those two extra lines are still there?
When I delete those two lines the game loads just fine with all the mods.
Go into the Plugins.txt file and delete these lines:
AltarGymNavigation.esp
TamrielLeveledRegion.esp