Whether Norse legends hold elves or not isn't the issue. The Nords are not from Norway, they're from Skyrim, a province of Tamriel, which sits on the celestial body (planet) of Nirn, some ways west of a continent that hosts an ongoing war between bestial races of snake-people and brutish tiger people, who are headed up by an obscenely powerful Tiger Dragon, the first among them to have somehow reached the status and power of a dragon. If you continue attempting to ground Skyrim firmly in reality, I'm afraid you've missed the point entirely that Norse mythology hardly holds a place in Tamriel, and you're not really in a good position to decide what is or is not in the best interests of the game's lore or canon.