The LED is in backwards or should be connected to 0V instead of 5V: The anode should be connected to 5V for a high on pin 1 to light the LED. If you connect the cathode to 0V then a 0V on pin 1 will light the LED.
The red wire looks like it might accidentaly short to the resistor lead.
You have not tied off the unused inputs. I can't say for sure that this is responsible for your problem but Note 3 in the datasheet says, "NOTE 3. All unused inputs of the device must be held at VCC or GND to ensure proper device operation. Refer to the TI application report: Implications of Slow or Floating CMOS Inputs, literature number SCBA004."
The 74LS04 can sink 8mA, but source 0.4 mA. It is better to connect the anode to 5V cathode to the resistor, then calculate the resitor value for <=8mA. These inverters can be paralleled to get higher current.
The 7404 can sink 16 mA per output.
The OP indicates that ensuring the unused inputs are tied off solved the problem. Also mentioned in a comment the problem changed from always on to always off.
This indicates a latch is created by cross-coupling through the power rails.
A decoupling capacitor from pin 14 directly across the chip to pin 7 must be used to decouple the internal inverters from each other.
A comment by Hearth reminded me of the time that we received counterfeit LS chips from a local shop. They tested as HCT chips.
So I zoomed in on the state of Texas logo on the chip in the OP’s photo. Looking closely reveals a mermaid folded to look like the state of Texas. This is a counterfeit that looks like a TI chip to a glancing eye.
The 74HCT04 data sheet may be a better fit. I had to measure the input characteristics to be certain.
The CMOS inputs will be more sensitive when not tied off, so further explaining the OP’s problem.