Theory time boys!
So why did the Dwemer just up and vanish? I'm going to take some liberty and assume you already know the ins and outs about the battle of red mountains and jump from there.
My theory hinges on a journal/book found somewhere in Skyrim but for the life of me I can't remember it's title, only the plot. I'm only going to need bits and pieces from it so I'll take what I need when we get there. Now we should all get ourselves familiarized with the concept of CHIM and GODHEAD. Yeah, reality is a dream and all that. Go learn about those then come back here because they're important too.
Here we go
So, in Morrowind we get to meet the last living Dwemer, Yagrum Bagarn. He tells us that he has no idea where the Dwemer went but he does tell us that he was in an outer realm when the incident happened. I'm going to take a leap here and say that the reason the Dwemer vanished had to do absolutely with The Calling, the Dwemer's ability to telepathically communicate over pretty much any distance, but probably not between realms. Now we get into the idea of Godhead and the journal I found. I'll explain how I came to this conclusion later but any mortal being in TES cannot comprehend the idea of Godhead without ceasing to exist. Not death, just non-existence. If one were to fully understand the concept they would simply do what the Dwemer did, vanish into thin air. My main theory here is that when Kagrenac used his tools on the heart he instantly gained an absolute understanding of reality and in turn of Godhead. Because that's the equivalent of a dream knowing it's a dream, he would have stopped existing, but through the Calling he let every other Dwarf on the face of Nirn understand what he was understanding and in an instant, they all vanished.
Quick break to break down the plot of the journal: some idiots got hired by mage to go into dwemer ruins, many die, solid few sentences about some nord girl's tits, find green lexicon, mage touches it, vanishes. merc dude touches it, spews black s*** from his eyes and mouth and goes insane and mentions godhead right before vanishing.
Now back to the theory, how do I know the Calling is responsible for everyone vanishing instead of just Kagrenac? Well the Dwemer aren't dumbasses who sit around all day, so it's safe to assume they would've created devices to harness the Calling, like leaving a voicemail. Well we've never seen a green lexicon before so I'm making the leap to say that the lexicon those idiots found had the full understanding of Godhead on it, received from Kagrenac himself when he used his tools on the heart. If that lexicon did, in fact, have that understanding engraved on it, that would explain why the mage and the merc vanished after coming into contact with it. The mage having more knowledge than a simple merc would've grasped the concept faster hence his immediate vanishing vs that of the merc who took a hot minute to understand what he had learned before disappearing as well.
There it is. Full theory. No idea if anyone else ever came to this conclusion before but hey leave opinions. Peace.