Short Answer: Tuck Tape.
I prefer Tape that holds like a Bulldog in any temperature but allows the LED to be removed with a little effort.
Since my LED's have smooth lead frames the gap between is 2.05mm which would fit a 2.0mm thick coin cell loosely or a 2.5mm thick cell with perfect contact tension and not excessive stress on whisker gold bondwire. Therefore I recommend CR2025 cells and TUCK TAPE ( made in Montreal) used in construction industry for holding anything 1000x times its own weight.
Since the ESR of Energizer and Panasonic CR2025's are perfectly matched to 5mm LEDs it limits the current to 20mA typically, thus Red and Yellow drop from 3.3Vopen to 2.2Vload. Blue and White having a threshold 1V higher than Red,Yellow will cause them to last long but draw much less current but if you have LEDs with 16~30Cd@20mA , but operating at 5~10mA, it is still very bright. All my LEDs range from 10Cd to 30Cd in many colours. I don't make any profit on these custom parts any more as I am retired from that, but if you want them in 250~500 pc/bag, just ask. Compare your specs with 16Cd min.
Most tapes use elastic glue which gradually releases, unlike TUCK tape, which only needs 5 to 10mm wide strip to hold both DMM leads and the LED leads in photo above for a White LED 15 ohm ESR thus I can estimate the current to be 5mA which if you derate the CR2025 to 100mAh yields 20h but of course as current drops with voltage, will glow for months or a year after 20h. Thus Red Yellow will draw the right amount of current. White are better suited to CR123A which far greater capacity and only $1 ea online, albeit bigger.