Yes, the largest diameter of the cutter in use takes precedence, because that is the part that is going to overheat if you run it over rated speed. Once you have determined that, then of course, the number of teeth, and their rated chipload, belonging to that cutter, controls the maximum feedrate. However, other cutters in the same gang that cannot handle such a high chipload will have priority over chipload as well.
The biggest controls the speed, and the weakest controls the feed
First you get good, then you get fast. Then grouchiness sets in.
(Note: The opinions expressed in this post are my own and are not necessarily those of CNCzone and its management)