Of course the annular ring should be at least as large as the minimum annular ring specified by the PCB house, but generally you'll want to make it much larger, especially on heavy parts or parts (like your switch) that are exposed to force.
At least 10 mils (0.25mm) annular ring is better for manufacturability regardless of any strength considerations, and a bit bigger than that is even better.
If the board has plated-through holes, exceeding a 1mm annular ring is probably not of much benefit for pins up to 1.5mm or so.
If the board is single-sided, the adhesive under the pad has to provide a lot of strength, so you probably want to go as high as 2mm annular ring or more (bigger is better for strength) on a part requiring strength such as a switch, terminal block, coil, etc. If there is not enough room to get appropriate clearances you can use a non-round (oblong or rectangular) pad and still get some of the benefit.
I don't know of a specific formula or rule of thumb.