Zero, most of my machines are direct drive.
I do have 2 machines that are mechanically geared down to almost slow speeds. 1 of those is actually being refitted with pulleys and belts now. I plan on changing ratio for more speed. Though I will be running much more modern electronics and steppers, larger too. So far all of my machines have been able to easily overload the tool or spindle, causing breakage of the part or tool. Even on my Sherlines. Torque really doesn't seem to be an issue in my experience.
Gib adjustment and getting it right changes everything.
A lazy man does it twice.