Table 1. Extract from the MOC3081 series datasheet.
The opto-triac has a very low holding current. It's likely that you're getting enough leakage through U1 and R13.
I note that you seem to be triggering only every second half-cycle.
From the comments:
Now, what exactly is a Holding Current on a opto-triac?
Basic triac theory tells us that once triggered an SCR (silicon controlled rectifier) - of which the triac is a bi-directional type - will remain on until the current through it falls below the holding or hold-on value. We use this feature on AC dimmer circuits, etc., to our advantage by giving a short pulse to the triac gate when we want to turn it on and let it turn itself off at the next zero-cross.
Figure 1. The upper trace shows the trigger delayed close to the end of the cycle. The resultant effective voltage is low. The lower trace shows the trigger close to the start of the cycle. This will result in close to full voltage. The relationship between phase angle delay and resultant RMS voltage is graphed on the right. Source: Opto-triacs, solid-state relays (SSR), zero-cross and how they work,
For larger triacs the holding current might be 50 mA or so. (It's a long time since I've checked.) For your teeny opto-triac it's only 0.25 mA. Since U5's opto-LED will draw more than that it holds the triac of U1 on once triggered until the next zero-cross. The linked article (by me) may be worth reading.
But I'm lost now, how can I achieve the burnt bulb detection?
- Sense the current while the lamp is supposed to be on. This will be a bit bulky and will require some sort of current sensor - maybe a Hall effect type.
- Monitor the voltage across your triac, D5, while the triac is off. If the lamp is OK you'll have full mains voltage. If the lamp is open-circuit you will have none (provided you remove your other monitoring circuit). To do this you can add an opto-isolated zero-cross monitoring circuit (of which there are very many on the web and on this site - and not all of them are good) across the D5. You'll have to figure out the logic of testing and, for example, if you need to monitor the lamps while on then you'll have to switch them off periodically for a half-cycle or two to check them.