If your o-ring is too stout, your part may not sit very solid on your fixture, basically allowing it to roll around on the o-ring. Also, the only thing holding your part in place is friction. It looks like the contact area is very small so you might not get a lot of friction. It might work better to have a small waffle pattern inside your oring so you have some grooves to make sure the vacuum gets to the whole surface but also some more support for it to help hold it still.

It is not possible to get any more holding force. atmospheric pressure is ~14.7psi so that times the area of your part inside the o-ring is the max force. Looks like maybe 10lbs in your case.