As proposed, it will function, but badly enough to have usability problems.
The inverter has got to go
The weak point is the inverter. You should be staying at native 12V (or 24V DC, if distance is a factor) for several reasons.
First, it takes the inverter's parasitic losses out of the picture. Inverters are not magic; they require energy to sit there "spun up". In fact, it would be wise to picture an inverter as a M-G set that is actually spun up. You wouldn't leave that spinning for no reason, would you?
Second, your battery is way, way, way too small to burn that light any length of time. I get where the battery says "18AH" but that is only relevant to extremely rare use e.g. in a UPS. Lead-acid batteries have a serious problem being deep discharged. If you regularly dip them to 50% it will greatly shorten their life. For a reasonable life in a frequent-use senario, lead-acid better dip only to 70% full. So you now have a 5.4AH battery, giving 64.8 watt-hours, so your pack will barely make it through the night carrying the inverter's parasitic load, let alone any time of lighting.
- If you're serious, we really should have a conversation about battery packs.
Therefore the only way this dog hunts is if you use a motion sensor to minimize lights-on time. DC motion sensors are half the price of AC motion sensors. Simply because motion sensors are low-voltage critters, and AC motion sensors need to have an internal 230->12 power supply in them, and that's not free.
- Further, simpler AC motion sensors (i.e. series-wired) do not play well with inverted power.
In any case, you would need to keep the inverter spun up all night to keep the motion sensor powered up, which is nuts.
Motion sensors also should not have their power cut off, ever. That is because they also contain day-night sensors. They are not human eyes; they do not auto-adjust; they see true solarization. That, at any given spot, can vary dramatically depending on shade, moon, and night lighting. Therefore the sensors need to be powered up for at least 1 24-hour period so they can calibrate to the difference between day and night. Otherwise they will work in broad daylight, or refuse to turn on under a full moon/street lighting /etc.
Low voltage DC motion sensors are the way to go, therefore. Their parasitic load is negligible. Most sensors that accept 12V are also labeled for 24V.
About that relay
Why don't you want this light coming on when the power is on? Does your yard not need lighting then? I bet it does, and I bet there's another light already. That is silly, there don't need to be 2 lights unless you want to do a high/low thing. But then you mentioned an "On when power on" relay! That's a good use for that relay.
Other than that, I'd skip the relay altogether, reduce system complexity and simply allow the light to work even when the power is on. Make sure your power supply is big enough to both top up the battery and carry this load. Which, to your original question, is fine, as long as it's capable of 3-stage charging.
If you really want the relay to knock out the light with AC power on, then put the relay after the DC motion sensor, i.e. interrupting power to the light (NOT the motion sensor) if AC mains is on. At that point, post-relay, if you really, really want to have the inverter power a 120VAC light, then that is fine. The inverter only "spins up" when the motion sensor commands the light on, so you aren't wasting energy on its parasitic load the rest of the time.
If carrying 12V long distances is an issue, use 24V, or put the whole package out there at the light.
About those lights
It's hard to find bright, useful 12V yard lighting in the normal places you look for yard lighting. You have to look elsewhere. I've found happy choices in automotive / off-road. I use a 42W 8-degree spotlight (12V of course) that is as bright as a locomotive headlight. They also make them in 30 degree and 60 degree spot. So work out your angles and see what you really need; often, far less than you think.
The most important thing about yard lighting with LED is aiming it properly. Usually people are replacing metal-halide lights, which cast light in a sphere - with reflectors they cast light in a very wide wedge of almost 180 degrees. We had seven of them lighting up a yard; they burned 3300 watts. Seriously. And they mostly lit up the sky... trees... our neighbor's bedrooms... I redesigned the lighting system to use LED spot lighting, and cut the draw down to 300 watts simply by aiming the lights properly. Mind you, metal halide is already near 100 lumens/watt so there was no lumen efficiency gain. It's all about the aiming.
For a wall-pack light, the presence of the wall is not a surprise, so there is simply zero excuse for throwing any light at the wall. LED fixtures should be lensed so all the light misses the wall and most gets thrown beyond the here-overbright sidewalk. This involves about a 45 degree wedge of light, but that is easy for LED. These here 40W lights could be 15W and cover the yard as well.
