What is the something that you want to turn on? You said LED in your question, but then said something in the question. :) I only ask because an LED vs. a solenoid can have very different power requirements. Is there a budget in mind? Do you want something turnkey?
I was going to recommend trying the TI Launchpad, which gives you a micro with a handful of I/O, plus a built-in USB-to-serial adapter, and is Linux-compatible, for a reasonable $5. The price is so good, it's unbelievable. And the tools are free. But if you want to make anything that requires more current than an LED turn on with a serial command sent via cron through Linux, then you'll have to wire up additional components to make it work.
If you don't have much of a budget, and want something that looks like it would work out of the box, take a look at this module from Phidgets to see if it's easy enough to get working in Linux. They provide plenty of source code, and have Linux drivers available.
There is also the obvious other option, Arduino, which from what I've read works on Linux. I'm just not a fan of the tools, nor the price when compared to the Launchpad.
EDIT -- a totally unrelated comment that I just had to share -- today I wanted to get more of the female headers for the Launchpad from Digikey. Well, the two headers cost $1 apiece. I ended up buying another Launchpad, because I would get both headers, as well as a USB cable, the dev board, and an extra MSP430G2 for an extra 3 bucks! Seriously, TI? :)
EDIT (regarding USB-to-serial adapter) -- I am actually having a hard time getting information that this works on the Launchpad. From this post, it sounds like it's possible, but I'm a little confused because some responses talk about the ez430, instead of the Launchpad. I definitely have the MSP430 application UART appear in my Device Manager, but haven't tried to use it in any way. I'll try to play with this next week to see if it really does work.