Hope you don't mind, but here is a FEA of a similar spindle body made of 6061 Al. The bearing load for this test is set at 6000lbs. Note the safety factor. Almost 2x. Displacement set to 0.0001 and in Z, is near nil.
I also did a test with stainless and it near triples the aluminum.
But, being that you already have the 303 SS, I would use it, and as you said find other way to mount it?
Nothing wrong with that strategy if you can be that flexible.
Like I said, you are doing awesome work here for a guy using nothing more than his brains and a lathe.