For what it's worth, I use the balanced nuts Tormach sells for ER20 holders and they do seem to make a difference vibration wise.

I don't see that they sell the same for ER32.