If you can decide how long (=Tp) is the inductive pulse which recharges the output capacitor and the switching cycle period (= T = 1/operating frequency), you can calculate the output capacitor:
C = Iout * (T-Tp)/Vr where Iout is the max. load current, Vr is the allowed max. output voltage sving, the ripple. This is actually the general current and voltage law of capacitors fitted to your case and its already presented in the other answer.
The length of the inductive pulse Tp = 2 * T * Iout/Ip where Ip is the peak value of the inductive pulse current. This is actually the charge balance, inductive current pulses must supply the whole output current. The pulses are assumed to be triangular; sharp beginning, linear deacay. This is ok if resistive losses are negilble.
Ip and Tp are tied by the induction law (voltage = inductance * current changing rate):
Uout + Vd = Ls * Ip/Tp where Ls = the inductance of the flyback transformer secondary winding and Vd is the voltage drop in the diode which prevents the output capacitor to discharge back to the transformer. You can use this to determine the needed Ls. If Ls is given, this is the other needed equation to solve both Tp and Ip. The first one was the charge balance equation.
These equations assume that there's not continuous current in the primary switch (=all inductive energy is outputted to the capacitor before new switching cycle), the transformer losses are neglible and the output capacitor works ideally.
To make designs you need still one equation, one which binds the available input DC voltage, primary inductance, allowed max switch current and the inductive energy accumulating period length (equal or less than T-Tp). Obviously you can write it with induction law.
ADD: Less than ideal transformer, switch and capacitor make the case complex. The application note in the other answer is useful as a start for practical designs with real components.