2
\$\begingroup\$

I am using an ESP32-S3 chip with 8 MB internal flash. Out of which 4 MB is for the FAT32 file system. I read and write files every 30 min into the file system. I am also using the wear leveling library that is available in the default ESP32-S3 libraries.

Wear leveling is to prolong their lifespan and improve performance. Flash memory has a limited number of write and erase cycles before individual memory cells can no longer reliably store data. Wear leveling helps distribute these write-and-erase cycles evenly across the memory cells, preventing some cells from wearing out prematurely while others remain unused.

Now, In some rare cases even though the file system is not having many files, the 4 MB is getting full. I even ran Defragger to check if there was any fragmentation issue, but it says there are 0 fragments.

Now I don't know what exactly is occupying the space here and how to check it.

Please note: we are using 1000's of devices with the same code, and the issue is piling up only in 10-15 devices and these devices used to work fine previously for months.

Below is the partition table I'm using. I'm not using any extended storage.

# Name, Type, SubType, Offset, Size,
nvs, 0x01, nvs, 0x11000, 0x8000,
otadata, 0x01, ota, 0x19000, 0x2000,
phy_init, 0x01, phy, 0x21000, 0x3000,
ota_0, 0x00, ota_0, 0x110000, 0x140000,
ota_1, 0x00, ota_1, 0x250000, 0x140000,
fatfs_storage, 0x01, fat, , 4M,

I have looked at the volume ID sector. I have used analyzer tools on it, it shows that the files occupy the expected space inside the volume but it does not show what data is filling the rest of the memory.

Let me know if any more information is required.

Below is the code for the FAT32 file system initialization

bool FatFS_initialize(bool make_fs) { FRESULT fr_result; BYTE pdrv; FATFS *fs;

esp_err_t esp_result;

static bool first_call = true; // just to make sure FS registration isn't done even if this function is called another time
if (first_call)
{
    const esp_partition_t *partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_FAT, "fatfs_storage");
    if (partition == NULL)
    {
        ESP_LOGE("FatFS", "Failed to find FATFS partition (type='data', subtype='fat', partition_label='%s'). Check the partition table.", "fatfs_storage");
        return false;
    }
    esp_err_t ret = esp_vfs_fat_register(fatfs_base_path, "", 5, &fs);
    if (ret != ESP_OK)
    {
        ESP_LOGE("FatFS", "VFS registration error(%u)", ret);
        return false;
    }

    wl_handle_t wl_handle;
    esp_result = wl_mount(partition, &wl_handle);
    if (esp_result != ESP_OK)
        ESP_LOGE("FatFS", "wl_mount error");

    // Get a physical drive
    esp_result = ff_diskio_get_drive(&pdrv);
    if (esp_result != ESP_OK)
        ESP_LOGE("FatFS", "get drive error");

    // Register physical drive as wear-levelled partition
    esp_result = ff_diskio_register_wl_partition(pdrv, wl_handle);
    if (esp_result != ESP_OK)
        ESP_LOGE("FatFS", "disk register error");

    if (make_fs)
    {
        BYTE work_area[FF_MAX_SS];
        fr_result = f_mkfs("", FS_FAT32, 0, work_area, FF_MAX_SS); // Use default volume
        if (fr_result != FR_OK)
        {
            ESP_LOGE("FatFS", "f_mkfs error : %d", fr_result);
            return false;
        }
        else
        {
            ESP_LOGI("FatFS", "File system created successfully");
        }
    }

    first_call = false;
}
return true;

}

\$\endgroup\$
9
  • \$\begingroup\$ You stated only a few devices are exhibiting a particular behavior and I think you want us to take the implication that it's not a software problem and are hoping for a hardware suggestion. Could be. But software remains an open question, I think. For the failed devices, you need to closely examine the MBR to start (I see nothing in your question about it.) Are you using only the usual four partition structures? Or extended ones? Have you looked at the volume ID sector? I don't know what your wear-leveling library does to all this either. I think you need to write more. What's your process? \$\endgroup\$ Commented Sep 6, 2023 at 10:12
  • \$\begingroup\$ I have added some details to the question, please check. Also, could you give more details about this MBR? When checking with ChatGPT it says "The ESP32-S3, like many embedded systems, does not have a traditional Master Boot Record (MBR) in the same way that a computer's hard drive or SSD would have. Instead, it typically relies on a simpler boot ROM or bootloader that is responsible for initializing the system and loading the firmware from a specified location in memory." \$\endgroup\$ Commented Sep 6, 2023 at 11:37
  • 2
    \$\begingroup\$ Are you doing any sort of error checking/correcting on the system? FAT32 is absurdly brittle - if you had a power loss/partial write event while updating the FAT table it can cause huge problems that will continue to propagate until you format. \$\endgroup\$
    – Jon
    Commented Sep 6, 2023 at 11:49
  • \$\begingroup\$ You could always try to raw read your entire internal flash and investigate. Since you have thousands, comparing working to not working statistically is likely to yield some results. \$\endgroup\$
    – Abel
    Commented Sep 6, 2023 at 11:50
  • 3
    \$\begingroup\$ @cyber122010 ChatGPT is an inveterate liar. But the point that the MBR isn't used the same way makes sense, because there is code on the MBR and that code would make no sense in your situation. (Back in the day, the BIOS would always load the MBR into memory and then jump into it. In this case, there's no point to any of that.) So it should be ignored. But the MBR should be there, if this is really a FAT32. It's just that only a small part of it would be used. \$\endgroup\$ Commented Sep 6, 2023 at 11:59

1 Answer 1

1
\$\begingroup\$

I even ran Defragger to check if there was any fragmentation issue, but it says there are 0 fragments.

That's not a problem that increases the size of anything, though. Fragmentation is a problem on mechanical media. On flash storage it has a small performance impact only in memory-constrained systems that can't use better performing algorithms for allocating blocks in FAT.

What you should run instead is plain old chkdsk. I bet you there are lost chains in the FAT - allocated blocks that have no associated directory entry. There may also be chains that are much longer than the directory entries indicate. So a DIR may show a file as e.g. 100kb long, but it may be actually 1MB long on disk, but you don't see it in DIR. Chkdsk will clear it all up.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.