You say you don't want to add any ICs but then you say you are planning on adding a flip-flop, so I am not sure what you really mean by IC. But since you are willing to consider a flip flop I have a suggestion.
If I were you I would consider designing a logic circuit from 4000B series CMOS logic. They are cheap chips so you can buy lots of them for as many of these things as you are inclined to build. It also has all the useful functions that you will need: flip-flops, counters, and all the other useful operators. The other good thing about 4000B series is that it can take +15V (or more) power rails. Designing from CMOS logic will give you a more robust and reliable circuit than trying to use only transistors and passives, (I promise.)
You will probably want to use the 4000B series CMOS to drive bjt's that switch the led because the output current drive of the logic isn't great. This isn't a problem.
I think your control logic may get a little complicated at first pass, but it is a fun problem (at least I think so.) It will give you an opportunity to get clever and get a working circuit! Check out: CMOS Cookbook; Author ‑ DON LANCASTER for a good reference. Good luck.