Well not sure how to describe it better (even in my own language)
I wouldn't mind if it was with vanilla followers only - but I got some followers mods that actually have (According to mod anyway) from 200 to 500 lines of dialogue.
The problem is with entering new area, quick travel or quitting the game (f5 > f9 not, apparently) follower idles seems to "reset" and actually if I don't travel a long time in one "cell" (so for example running on foot from Whiterun to Solitude) I can hear 2-3 dialogues. The same ones. Over and over again. It's annoying not to mention I won't hear all the "idles".
I tested it with game on (and read a book in the meantime)- during 20 minutes I heard at least 15 dialogue lines, when I fast traveled I heard the "first" line again.
I thought it's only with vanilla followers (every one of them told me 2, up to 3 lines) but it's not.
Also dialogues are with white font, not dark gray - as if I haven't talked with them at all. It doesn't include "follow me/leave" options.
It's the same with vanilla followers and unique-voiced ones from different mods.
Other than that no problems
Load order, there's at least few mods that have followers
Interesting NPC, Inconsequential npcs, Anna NPC...