You pretty much *need* a lathe to bore the pulleys out properly. I was in the same situation you are with just a mill. I used a coaxial indicator to dial in the exact center and then a boring head to bore it out. Mine was still offcentered a little. I used it as the excuse to buy the lathe. The one I bored out on the lathe was absolutely perfect.

You could save the old pulleys by boring them larger on a lathe and then sleeving them but I would just buy another set of pulleys. See, if you'd bought the lathe already you could have saved the $100 in wasted pulleys, that's the way I explained it to my wife.