It sounds like you are either using the H bridge to produce a sinewave input to the transformer in a linear fashion or, you are using PWM to create an equivalent sinewave. Either way will be fraught with inefficiencies; either the H-bridge will get warm (linear sinewave generation) or the transformer will get warm (PWM).
You never mentioned the H-bridge getting warm so I'll concentrate on the PWM problems.
If you use high-speed PWM switching to create a pseudo sinewave, the high frequency harmonics will play havoc with a regular power transformer. You'll get significant eddy-current losses due to the laminations simply being to fat. Ferrite (as used in HF transformers) do not conduct and therefore have very low eddy current losses.
My solution would be to step-up the 40 volts to something like 320 volts DC. I'd use a high-frequency transformer and a high-frequency driver to do this. So now, you have an isolated 320 volts DC rail and, to finish the job off, I'd use a H-bridge to generate the sinewave using PWM techniques and, add a low-pass filter on the output to return the generated signal to something close to a decent sinewave: -