I'm not sure if you refer to the quality of the PCB or the quality of the soldering, or both.
So, is this normal with PCbs in general, and with SOD-323 diodes in particular?
A properly done solder joint provides both a good mechanical and electrical connection. So in regards of soldering, you should not be able to do that by hand unless you use a lot of force. As in, if you press firmly against the component with a finger it shouldn't move, but if you do something extreme like press for all that you are worth, you'll probably damage things even with a proper solder joint. Note that SOD323 isn't the most rugged of components - the legs are relatively small in comparison with the component body. It isn't meant to take up force - most components are not.
You should be able notice the culprit if the component comes off: is there a square pattern in the solder where the component pad used to sit? Then the cause was bad soldering, most likely caused by a cold joint, oxidated component legs or bad paste. Did you tear off traces or damaged the component itself? Then you simply used excessive force or maybe the PCB is bad.
Them "well-known Chinese manufacturer" tend to have diverse quality depending on mood and manufacturer. I was soldering one such board just the other day and noticed that the solder mask started to come off close to the pads just from heating the pad during normal hand soldering. That's a bad quality PCB.
Similarly, you shouldn't be able to easily peel off pads from the PCB given that they are attached to traces/vias/copper pour (single "dummy" pads not connected to anything tend to come off and that's no fault of the PCB manufacturer). Some boards used for volume production are of low quality and can for example not stand desoldering/component replacement well at all. Doing such is an easy informal way to test the PCB quality.
What is the recommended pad size for something like a SOD-323 diode?
This should be mentioned when reading the friendly datasheet, or otherwise from manufacturer general recommendations. Apparently 0.6 x 0.6mm is a common land pattern recommendation for reflow soldering. So 0.65 x 0.55 sounds reasonable.
Note the difference between pad size and land pattern - pad size is slightly smaller, in this case 0.45 x 0.40. You should CAD according to land pattern.
In addition there will be a recommendation for paste stencils, in this case 0.5 x 0.5mm. That's an important measurement for the PCBA assembly/whoever orders the stencils. Get this wrong and that could explain bad soldering with wrong amount of paste.
What's a reasonable level of "physical resilience" that a PCB is expected to have?
I assume you mean a mounted PCBA or otherwise I don't know the answer. Well, it depends... it is subjective. Some components like crystals, LEDs or other optics, inductors, potentiometers/trim caps etc are sensitive to physical damage and if you drop a board with any of those on-board it might be broken now. While resistors and other passives can usually take quite a beating.
If you expect a rough treatment of the board, there's various tricks like only place components in a direction that the board will not be expected to bend in, or better yet use through-hole components all over. In general we shouldn't mechanically place the PCBA in such a way that it will take the blow from dropping the product etc.
There's standards for shock and vibration depending on what the end application is - design accordingly.