What are the standard ways of data storage in small consumer electronics (routers, phones, etc.)? For simple devices I guess microcontrolles with in-system programmable flash are enough, but for example on a smartphone, would there be an internally mounted micro-SD card or similar? Where would OS, etc. be stored?
For items such as you use as examples, there is usually a NAND flash chip (or more than one) hanging off the main processor.
For smaller items (say microwaves), the code will either be in internal flash on the microcontroller, or in very cost-sensitive high-volume applications will be in ROM, again usually on the same die as the micro.
There are three typical choices: internal flash, external NAND flash and micro-SD cards. Flash memory evolved from early EEPROM devices and has pretty much replaced them, being more dense (and thus cheaper per byte). Very high volume devices might use ROM, which I won't cover here since I don't have either pricing or specs for it. Functionally, it would act as either internal or external flash memory, except for their being read-only and have no endurance issues.
NAND flash IC's are usually quite a bit more expensive than the comparatively-sized micro-SD card, just because the latter are manufactured in such large quantities the economies of scale bring the price down. For example, an 8 GB NAND flash (64 Gb) chip costs around $20, and an 8 GB SD card costs $6. (I couldn't even find an 8 GB EEPROM IC for comparison.) And of course internal flash memory almost comes for free, except microcontroller cost does increase with flash memory size.
The access time for the SD-card though will be slower than the NAND flash, since the SD card is accessed using either a 2-wire (R/W) SPI interface, or a 4-wire proprietary interface, while the NAND flash chips typically have an 8-bit parallel interface. Internal flash will be the fastest, since no external interface is required.
Since SD cards can be formatted in a PC, they often are set up with a file system such as FAT-32, which makes it easy to copy file fro the PC and then access them from a microcontroller containing the necessary FAT-32 library. (And of course the microcontroller can also write new files, create directories etc too if it wants.)
Another difference is wear leveling. Brand-name SD cards typically include wear-leveling (all SD cards have a processor inside them), while NAND flash chips do not and your application needs to do wear-leveling to prolong the life of the device. (Erasing/writing is much slower than reading for all these devices.)
SD cards will typically have the longest endurance in terms of erase/write cycles (typically 100,000). Internal flash and NAND flash chips typically have significantly fewer erase/write cycles, often measured in the thousands or tens of thousands. Thus they are not good substitutes for a hard drive.
SD cards can only be used for storing data; you cannot run code from them. However if the processor can execute code out of RAM (Von Neumann architecture), one could load programs as needed off the SD card into RAM and execute it there. (Note: most microcontrollers are Harvard architecture and cannot execute code from RAM.)
External flash can be used to execute code only if the processor has an external memory interface that specifically allows this, such as many ARM microprocessors. If the access is just via a general purpose parallel port, then the external flash can only be used for data.
Internal flash is just the other way around -- it is usually used for storing and executing program code, and most Harvard architecture microcontrollers have to have special means (such as a data "window") that allows access to the internal flash as a data area (not part of a program const area).
To answer your last question, an OS would almost always be wholly contained in the internal program flash area of the microcontroller. But it could be contained on an external flash if the processor allows this. If the OS needs to load drivers on an as needed basis, these could be stored on an SD card (or external flash), and either loaded into RAM if there is enough room and the processor can execute out of RAM, or burned into an area of internal flash reserved for drivers.