Yes, ask the fab. If they don't have an answer, check with another, etc. (If this means going with an expensive local or fully-featured fab instead of a budget/proto service, that's well worth the peace of mind -- and the UL/CSA/CE/etc. rating, if applicable.)
Laminate has excellent voltage handling. Consider a standard product such as Isola 370HR: 54 kV/mm breakdown. You might want to be safely under this, say by 4x or more, but that's still only 0.3 mm (400 VAC rounds up to 600 VAC class, category II requires 4 kV; CAT III might also be required for an industrial application).
Standards such as IEC/UL 60950-1* consider inner layers a "cemented joint", at no creepage penalty (or, creepage equals clearance), and clearance determined by breakdown of the material. I don't recall offhand if a safety factor is suggested.
*Deprecated. An industrial product probably follows other standards, anyway; this is just an example I'm somewhat familiar with.
Also, ah, found a copy of 60950-1; there is a specific clause for PCBs (2.10.6), which is a bit more nuanced than my recollection. For example, sheets of prepreg are considered insulating layers, sort of, but evidently not as confidently as for film/sheet insulators (Table 2R: two layers and routine tests for electrical strength (i.e., hi-pot testing), or three or more layers with tests not required; compare with "reinforced", which only requires protection equivalent to two layers of insulation).
From my own experience, I've seen a protoboard, accidentally made with default (~0.18mm) clearances, withstand rectified 480 VAC (i.e. ~650 VDC). There was one failure among four tested pieces, I think. And that was probably a lateral failure (e.g. between hole and inner plane) -- mind that there can be gaps or bubbles in the lay-up, or delamination can occur after soldering or handling. (Some defects can be detected, such as by hi-pot and partial-discharge tests.)
On an unrelated note, mind that the power dissipation of inner layers is poorer than surface copper: the ampacity is generally lower. Ensure adequate conductor width, use multiple layers in parallel, or use heavier copper.