What are my options if I need to deal with relatively large amounts of data on an embedded processor? I've found storage around ~2-4GB with an SPI controller, but I'm thinking about something in the 10s to 100s of GB.
For that range of sizes, your best option is probably SD. It's reasonably easy to interface with, readily available in large capacities (up to 256 GB now!), and can be easily read from a computer if need be.
Depending on your exact needs, eMMC may also be an option. It's essentially an embedded variant of SD, with some slight modifications.
NAND flash is also an option, but I wouldn't recommend it. Raw NAND flash doesn't include any translation layer, so you end up having to manage write leveling and bad blocks yourself. It's a huge pain.
Depending on what you want to do and what processors you can use, I have successfully used, for a few years, up to 64Gb sdCards on a variety of ARM based controllers including
- Raspberry Pi
- STM32F from the 100, 300 and 400 ranges.
- Various Olimex STM32 dev boards with SDCard built in
- Various Android devices e.g. Samsung Galaxy, Nexus
The ST libraries include sdCard drivers, and you can use either an SDIO peripheral or SPI bus. You can use 1 bit (SPI or SDIO) or 4 bit(SDIO)
Since you are building an embedded device, the address range can be anything , it just depends on how many bits you send, and the SD requires in its address.
You can even use more than one SD card if you control the select line yourself, or use multiple SPI peripherals.
I have never found any of these to be flaky.
You cannot boot Raspberry from > 32Gb devices without reading and applying the patches on the Raspberry web site. This is an OS limitation not a CPU funny. (Haven't done this with Beagle).
The Raspberry will also take a USB flash stick of 64Gb. Any of the OTG capable controllers can do this as well.
The 32 bit PICs should also be capable of using SDcard and or USB flash drives.
Android opens up a whole new realm where lots of functionality is built in, I only need to provide a Bluetooth, wifi, USB or NFC channel. Guess you could go IOS as well.