How to calculate the instant flux density of a flyback transformer
with airgap?
There is a fundamental equation that relates flux density (\$B\$), core permeability (\$\mu\$) and magnetic field strength (\$H\$). That equation is: -
$$B = \mu H$$
You may have seen that relationship expressed as a graph (known as a BH curve): -
Image from here.
The curve also indicates the magnetic hysteresis associated with many magnetic materials. The dotted line is when beginning the plot (no remnant of magnetism in the core).
Note that \$H\$ is the driving force of magnetism. It accounts for current, the number of turns and, the distance of the flux path. So, you need to know the cores magnetic permeability if you are to find flux density.
I am trying to understand airgap is enough or not?
If you know the permeability of the ungapped core and, you know the effective mean length of the core (as per the diagram below) you can calculate the effect of an air-gap: -
The mean length is that shown by the dotted green line and is detailed in the data sheet for the core you are considering. Image from here.
So, the formula the relates the gapped and ungapped permeabilities is this: -
$$\mu_{gapped} = \dfrac{\mu_{ungapped}}{\frac{\ell_{gap}}{\ell_{mean}}\cdot\mu_{ungapped}+1}$$
And, this tells you the gapped permeability (also called effective permeability). Then you use this new value in the first formula to calculate the gapped value of flux density.
A good choice for peak flux density is around 200 mT for most ferrite materials. Also note that once you have lowered the permeability by introducing a gap, you will probably need to restore the inductance by adding more turns. This increases the H-field as per my previous answer to you entitled Why is the air gap important in a flyback transformer.
how to calculate \$N_{primary}\$ of the transformer with respect to
airgap?
The core data sheet will give you a value for \$A_L\$ and, \$A_L\$ multiplied by \$N^2\$ tells you the value of primary inductance. So, if the gap reduces the permeability by (say) ten, then \$A_L\$ also reduces by a factor of ten.
So, you know the current in your coil and, you know how many turns you need (ungapped and gapped) and, you know the mean length of the magnetic core so, you can calculate what value the H-field is. It's just a tiny bit of number crunching to find the peak flux density.
\$\ell_{gap}\$
\$\endgroup\$