There are several potential safety issues with mains-connected circuits. I don't know about Australian legal issues, so I have no comment on that.
- shock, electrical burn or electrocution hazard
- fire hazard
- energy related injuries from arcing etc. (damage to vision, burns)
- damage to other equipment caused by a circuit being completed through connected devices
- indirect safety hazards (for example, an oscilloscope probe or case becoming electrically 'hot' due to connection to a mains circuit, harm from falling off a ladder or whatever due to a shock, etc.)
Your simple circuit is unlikely to present an energy related hazard with a 250mA fuse (assuming you don't do something pathological like using a 5x20mm fuse on an industrial circuit). In tests I was able to make glass fuses literally explode, projecting a cloud of glass shards and molten metal for a fair radius with no more than 240VAC (they were rated 250VAC), but a large interrupting current (well beyond their rating).
Shock hazard on your simple circuit- it might be an issue. The LED is not safety agency approved for mains connection- and it will stick through the case, I'd imagine. Probably you'll see an additional bit of transparent plastic in front of any mains-connected LED. They're worried about possible damage to the LED, maybe a molding short-shot that could expose wires, what a kid could do poking at the LED with a fork or whatever.
You mention the capacitor is mains-rated. Do you mean it's safety agency listed, or just rated for what you think is sufficient voltage? An "X-rated" (or Y-rated) capacitor should be okay.
As far as the mechanics of prototyping mains circuits, usually what I do is to hack the power stuff together on a perf board or a proto PCB with everything very, very neat and clear and visible isolation gaps between high voltage connections etc. If there is complex control stuff that will not be isolated eventually from the line, I'll add adequate galvanic isolation for development. Most people will recommend using an isolation transformer, which means you have to do two things wrong rather than one to be electrocuted in most (but not all) cases, and makes test equipment damage less likely. An RCD (GFCI)-protected circuit should be a bit safer.
Everything gets buttoned down, secured and mounted on a non-conductive base before power is applied.
Stuff that gets connected- if the inputs are earthed (most oscilloscopes, computers etc.) then even a momentary accidental mains connection can blow the heck out of stuff, so galvanic isolation and keeping everything clean and neat is vital. A stray cut lead or a wire that comes off and goes a-sparking can be big trouble, as can a probe or screwdriver that slips with the power on. I've seen every single chip in a computer blown by a stray ground strap that touched the 'hot'. If it's not grounded, the attached device itself can become a shock, burn or fire hazard.
An appropriately-rated fuse is an excellent idea- sometimes a series incandescent lamp works even better (it's effectively a PTC resistor).
Arcing can cause eye damage- if everything is not adequately enclosed for whatever could happen, safety glasses at least, and perhaps a full face shield are good. Really high power stuff requires precautions against arc flash. Even low voltages can be hazardous if potential fault currents are high- spraying molten metal in your face, for example, or heating a ring on your finger up to red heat in a fraction of a second.
One frequently stated bit of advice is to work with (at least, LOL) one hand in your pocket, which avoids the possibility of current flowing from one hand to the other (through the heart), but still leaves the possibility of shock and burn injuries.
You should treat working with mains voltage like any other potentially hazardous activity. Educate yourself on safety precautions, consult with local experts if there is any doubt, work slowly and deliberately, don't do it when you're tired or otherwise off your game, and if you find yourself making small mistakes, take a break before you make a big one.