5
\$\begingroup\$

I'm considering an application where a BeagleBone Black, preferably running one of the recommended Debian images loaded in eMMC, is powered by a 5V source that can be removed at any moment. I want to prevent catastrophic data loss in this event, including if a write to file is underway. At least, I want the BBB to still be able to reboot from eMMC on return of power, repair the (probably ext4) file system, and run my application. If I can have the insurance that any file that was properly flushed is consistent, that's ideal.

The problem I fear is that, when power is lost while a write in underway in the eMMC, there could be data loss in an unpredictable sector, because a recent sector write handled by the eMMC triggered a Flash page erase, and the other sectors in that page are lost, and (because of wear leveling and sector reallocation internal to the eMMC) they belong to files or volumes that have not been written recently, including perhaps in read-only file systems or files. That problem is mentioned here:

a power cycle at the wrong time can destroy data anywhere on the (SD or eMMC) card, no matter where you THINK you're writing

and that could justify the warning in the BBB manual and here that

[powering down using the power button] will also help prevent contamination of the SD card or the eMMC.

as well at the terse warning made by what seems to be the primary eMMC source for the BBB

Avoid power-down during WRITE and ERASE operations.


To what degree can such data loss occur? What is the duration of the vulnerability window, if any?

Are there standard solutions? Recommendations?

Is there any built-in software mechanism against this, and how does it work? In particular, is the TPS65217C PMIC programmed to generate an NMI on loss of VDD_5V, aka AC, as the hardware allows, and is it handled in a way that stops ongoing eMMC write activity? If not, how can I at least add that?

Is stopping any eMMC-related activity good enough for the eMMC to enter a safe state by itself after some delay? Which delay?

What about a capacitor or Supercap (like Murata DMT3N4R2U224M3DTA0: 0.22F, 4.2V, 0.3Ω) on the battery connector to provide some power reserve?

Are there modes/settings in the eMMC to configure a write performance/safety compromise? What are the defaults, are they adjustable?


Important update: I have located authoritative information in the March 2010 eMMC specification about reliability when power is lost during write:

In general, an interruption to a write process should not cause corruption in existing data at any other address. However, the risk of power being removed during a write operation is different in different applications. Also, for some technologies used to implement eMMC, there is a tradeoff between protecting existing data (e.g., data written by the previous completed write operations), during a power failure, and write performance.

Lots of valuable information follows in that source, including description of how the eMMC declares that it promises (or not) to protect previously written data from haphazard erasure in case of power loss, and what seems to be truly write-protected areas.


Addition: a comment mentions that mobile phones use read-only file systems in eMMC, and are not known to experience corruption of that. However, mobile phones have a large battery, and that practically ensures they do not experience abrupt power loss in normal operation, thus are not concerned by the eMMC corruption mechanism that I fear.

\$\endgroup\$
  • 1
    \$\begingroup\$ At a minimum, you should keep system and data on distinct filesystems (as android does). System should be mounted RO. \$\endgroup\$ – Sean Houlihane May 18 '16 at 13:15
  • 1
    \$\begingroup\$ That's an interesting question. However, I am not sure someone can come up with an already tested and guaranteed solution. I'm afraid you'll have to make this up by yourself... If you do, please post an answer to your own question... \$\endgroup\$ – dim May 18 '16 at 13:18
  • \$\begingroup\$ @Sean Houlihane: I see no reason why using a distinct read-only filesytem would insure that a block belonging to that file system can't be lost by the mechanism that I describe; I'm not even able to justify that it would significantly reduce the odds of such loss (for lack of a model of how the eMMC maps logical block to Flash pages better than "at random"). If you have a reasoning or experience justifying that a read-only filesytem has significantly reduced odds to be corrupted, please make an answer on that interesting point. \$\endgroup\$ – fgrieu May 18 '16 at 13:45
  • \$\begingroup\$ Best ask Google about justifying that. So far as I'm concerned, this is common practice, and has been for years. Phones typically use a single eMMC with multiple partitions for boot, emergency boot, system, user-data. Never heard of a phone hard-bricked due to FS corruption by this method you describe. \$\endgroup\$ – Sean Houlihane May 18 '16 at 14:05
1
\$\begingroup\$

All layers of the stack need to cooperate here.

The eMMC storage device has an integrated load balancer that needs to handle power loss situations gracefully. I'd expect it to do that, by pretending that incompletely written blocks were not written at all.

The file system needs to be able to handle power loss between block writes. The ext3 and ext4 file systems fulfill this, as they are journaled and enforce the ordering of writes so the journal accesses are kept separate from actual file system modifications.

The weak point in your setup is the package manager, which doesn't offer a good solution if the power fails during a package upgrade. Usually you should still be able to boot unless the error happened precisely during renaming files after an installation, and the files being renamed were important system files.

For almost all purposes, that window is sufficiently small that you can ignore the problem. You can shrink it more by running dpkg --configure -a and dpkg -iGROEB /var/cache/apt/archives during each boot -- these should get you back into a sane state from an interrupted update, at the expense of a bit of time, and they cannot handle cases where dpkg itself or the init system were damaged as a result.

Last but not least, you can set up a rescue system in an initrd that will check and try to repair and/or reinstall the system if needed, and anchor this in the bootloader.

\$\endgroup\$
  • \$\begingroup\$ Do you have something comforting that when powered down during a write or (internal) erase, the eMMC will be "pretending that incompletely written blocks were not written at all", rather than "destroy data anywhere"? The best I have towards that is that WR_REL_SET "can be set to 1Fh to enable protection on previously written data if power failure occurs during a WRITE operation" but that insurance is rather vague, and not from the supplier of the eMMC actually used; which on the contrary states: "Avoid power-down during WRITE and ERASE operations"; something requiring a power reserve. \$\endgroup\$ – fgrieu May 19 '16 at 5:55
  • 1
    \$\begingroup\$ Internally, the data is written as an update record to a new block to avoid overwriting the same flash block multiple times, and the newly written block has a checksum and bits that are "written last", so interrupted writes can be detected and the update records ignored after reboot. \$\endgroup\$ – Simon Richter May 19 '16 at 13:03
0
\$\begingroup\$

schematic

simulate this circuit – Schematic created using CircuitLab

To compare intupt voltage and to do some action until cap has still charge

\$\endgroup\$
  • \$\begingroup\$ I get the idea (although that schematic won't work as is; output is always high). On the BeagleBone Black there is a PMIC that integrates this, and seems to have the capability to generate a NMI, that could be made to back-up data and sleep. But Linux, and the eMMC, are complex beasts; I do not even know how much time it takes for the eMMC to safely finish the last write! \$\endgroup\$ – fgrieu May 18 '16 at 11:36
  • 1
    \$\begingroup\$ @fgrieu It was just an idea, I am just looking at TPS65217C. The entire kernel should be patched to finish the actual write procedure in eMMC and then sleep, but also when it wakes up. A NMI is more usefull to wakeup (IMO), rather than the signal of supply outage - this can be queried each scan and change action, like disable some periphereals and wait until write is complete. \$\endgroup\$ – Marko Buršič May 18 '16 at 11:54
0
\$\begingroup\$

If you are powering your Beagle Bone from an AC supply you can control, you can connect the non-smoothed, rectified AC with a small timer cap to one of the GPIOs. When power fails, you have sufficient time to call an fsync or perhaps even a kernel halt, or if nothing else works, a kernel panic. Anything to prevent cutting power during eMMC write/erase cycles; any pure software interruption is dealt with by the file system journal.

I'm looking into this method for a project I'm working on.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.