Are you downloading mods from the Skyrim SE Nexus Mods domain? If so, you cannot stop Vortex from placing the archives into the downloads folder for skyrimse. This is intended behavior by Vortex. That is the domain in the mod's metadata, so that is where the downloaded archive is stored.
You can still use the mod just fine. Why are you so concerned with where the downloaded archive is stored? Skyrim VR does not have it's own domain on Nexus Mods, so all mod metadata will be for Skyrim SE.