I ended up using an Oldrim mod called Extensible Follower Framework, and went with the newest version that didn't use SKSE and used loose files instead of a BSA. So far, I've had no problems and I've probably logged about 20 hours since installing. I avoid using any of the follower commands it provides, so I'm essentially just using it to have more than one follower. I'm also using Inigo, so mod followers seem fine. But, yea, use at your own risk. Lol