Suppose we have a non-ideal transformer such that secondary load is fixed and we change the voltage applied to the primary side. How does efficiency change with respect to primary voltage? I know there is a relation for the change of efficiency with load but I don't know what is the answer for the primary voltage and when the efficiency is maximum? By efficiency I mean the ratio of powers in the secondary and primary circuit.
As you increase the supply voltage, you start to reach H (magnetic field strength) field levels that cause the magnetic core to saturate more.
The effect of saturating the core more is to open up or expand the BH curve and, the impact of doing so means that you lose more energy (disproportionately) in what is called hysteresis loss. Here is a picture that might help you understand: -
At moderate H fields (the blue curve above), the area enclosed is quite small but, as you increase the AC supply voltage, saturation effects cause the area enclosed to become disproportionately bigger and this means more significant losses.
Hysteresis loss is worse at higher supply voltages because you expend more energy in reversing the magnetic field each AC cycle. It's all down to what is called the remanance magnetic field - that is the field remaining in the ferromagnetic core when H field is backed down to zero. This remanance is the value on the Y axis when H is zero and, as you should be able to see on the picture above, is usually quite small for moderate H values.
See also Magnetic Coercivity; this wiki page also shows the widening and broadening effect of the BH curve as greater fields are demanded (by higher primary voltages): -
Efficiency is not affected by insulation , voltage rating or primary voltage.
Efficiency is affected by winding resistance and Leakage Inductance or its equivalent Coupling ratio with primary inductance.
There are also eddy current losses.
So it has more to do with copper losses and magnetic coupling ratio which becomes more important with higher power (MVA) so the design of those tends to be > 99%.
Since Excitation current is mostly reactive, it is not true power lost . This VAR excitation current is needed to coupling the magnetic fluxes in primary and secondary and may be as much as 10% primary current. This has some primary loss with no load.