How to enable Secure Boot v2 AND Flash Encryption (+ NVS encryption) on the ESP32-S3 with user-generated keys?

Question

General issue report

I am trying to figure out how to enable both security features on my ESP32-S3 (N8R8). I bought 5x boards, but I have burned through 4 of them. I have ONE fresh board left, which I don't want to brick.

Chip

ESP32-S3 (N8R8)

IDF Version

IDF 5.1 (stable)

Host Machine

Ubuntu 22.04 + vsCode

Problem

I have successfully tested both features independently on separate boards. Flash encryption works when I have just flash encryption enabled and configured. Secure Boot V2 works when I have just secure boot v2 enabled and configured. When I try to enable both on a single board, the device bricks because anything I flash no longer boots.

Procedure followed for Flash Encryption

1. We burn the XTS-AES key into the ESP32's eFUSE. Run the following command to burn the flash encryption key to the eFUSE:

    $ espefuse.py burn_key --show-sensitive-info BLOCK_KEY0 ./encryption/flash_encryption_key.bin XTS_AES_128_KEY
   

   • The --show-sensitive-info option will make the burned key human-readable in the prompt that follows.
   • The key will be burned to BLOCK_KEY0 which is BLOCK4 of the eFUSE.
   • The flash encryption key is in XTS_AES_128 format.
   • Type BURN in all capitals and press enter to continue.

2. Now, flash encryption must be enabled in the project configuration. Open menuconfig and navigate to Security Features. Check the following box:

    [ ] Enable flash encryption on boot (READ DOCS FIRST)
   

   • When you enable this option, additional options will show up

3. Under Security Features, set the Enable usage mode to Development (NOT SECURE)

4. Save and exit menuconfig.

5. Use the following command to flash the encrypted version of just the app image:

    $ idf.py encrypted-app-flash monitor
   

6. Use the following command to flash all data in encrypted format:

    $ idf.py encrypted-flash monitor
   

When I follow the instructions above, the flash encryption is configured upon the first boot. I added the encrypted keyword to 3 extra partitions in the partition table, i.e. partitions 3, 5 and 8. The partition encryption can be seen in the logs below:

I (58) boot: Partition Table:
I (62) boot: ## Label            Usage          Type ST Offset   Length
I (69) boot:  0 otadata          OTA data         01 00 00011000 00002000
I (77) boot:  1 phy_init         RF data          01 01 00013000 00001000
I (84) boot:  2 coredump         Unknown data     01 03 00014000 00010000
I (92) boot:  3 nvs_keys         NVS keys         01 04 00024000 00001000
I (99) boot:  4 nvs              WiFi data        01 02 00025000 000d0000
I (106) boot:  5 storage          Unknown data     01 ff 000f5000 00001000
I (114) boot:  6 ota_0            OTA app          00 10 00100000 00200000
I (122) boot:  7 ota_1            OTA app          00 11 00300000 00200000
I (129) boot:  8 logs             WiFi data        01 02 00500000 00100000
I (137) boot:  9 files            Unknown data     01 ff 00600000 00200000
I (144) boot: End of partition table
I (149) boot: No factory image, trying OTA 0
I (153) esp_image: segment 0: paddr=00100020 vaddr=3c020020 size=0b018h ( 45080) map
I (170) esp_image: segment 1: paddr=0010b040 vaddr=3fc91e00 size=029fch ( 10748) load
I (173) esp_image: segment 2: paddr=0010da44 vaddr=40374000 size=025d4h (  9684) load
I (181) esp_image: segment 3: paddr=00110020 vaddr=42000020 size=1cba4h (117668) map
I (208) esp_image: segment 4: paddr=0012cbcc vaddr=403765d4 size=0b760h ( 46944) load
I (225) boot: Loaded app from partition at offset 0x100000
I (272) boot: Set actual ota_seq=1 in otadata[0]
I (272) boot: Checking flash encryption...
I (273) efuse: Batch mode of writing fields is enabled
I (277) flash_encrypt: Using pre-loaded flash encryption key in efuse
I (284) flash_encrypt: Disable UART bootloader encryption...
I (290) flash_encrypt: Disable UART bootloader cache...
I (296) flash_encrypt: Disable JTAG...
I (301) efuse: BURN BLOCK0
I (306) efuse: BURN BLOCK0 - OK (all write block bits are set)
I (311) efuse: Batch mode. Prepared fields are committed
I (316) esp_image: segment 0: paddr=00000020 vaddr=3fce3980 size=02b0ch ( 11020) 
I (326) esp_image: segment 1: paddr=00002b34 vaddr=403c9700 size=00004h (     4) 
I (333) esp_image: segment 2: paddr=00002b40 vaddr=403c9704 size=00c08h (  3080) 
I (341) esp_image: segment 3: paddr=00003750 vaddr=403cc700 size=045c8h ( 17864) 
I (773) flash_encrypt: bootloader encrypted successfully
I (823) flash_encrypt: partition table encrypted and loaded successfully
I (823) flash_encrypt: Encrypting partition 0 at offset 0x11000 (length 0x2000)...
I (931) flash_encrypt: Done encrypting
I (931) flash_encrypt: Encrypting partition 3 at offset 0x24000 (length 0x1000)...
I (982) flash_encrypt: Done encrypting
I (982) flash_encrypt: Encrypting partition 5 at offset 0xf5000 (length 0x1000)...
I (1036) flash_encrypt: Done encrypting
I (1036) esp_image: segment 0: paddr=00100020 vaddr=3c020020 size=0b018h ( 45080) map
I (1046) esp_image: segment 1: paddr=0010b040 vaddr=3fc91e00 size=029fch ( 10748) 
I (1048) esp_image: segment 2: paddr=0010da44 vaddr=40374000 size=025d4h (  9684) 
I (1056) esp_image: segment 3: paddr=00110020 vaddr=42000020 size=1cba4h (117668) map
I (1084) esp_image: segment 4: paddr=0012cbcc vaddr=403765d4 size=0b760h ( 46944) 
I (1093) flash_encrypt: Encrypting partition 6 at offset 0x100000 (length 0x200000)...
I (28053) flash_encrypt: Done encrypting
E (28054) esp_image: image at 0x300000 has invalid magic byte (nothing flashed here?)
I (28055) flash_encrypt: Encrypting partition 8 at offset 0x500000 (length 0x100000)...
I (40657) flash_encrypt: Done encrypting
I (40657) flash_encrypt: Setting CRYPT_CNT for permanent encryption
I (40657) efuse: BURN BLOCK0
I (40662) efuse: BURN BLOCK0 - OK (all write block bits are set)
I (40667) flash_encrypt: Flash encryption completed
I (40673) boot: Resetting with flash encryption enabled...

If you want to flash images manually, follow steps 7 to 11

7. Run the following command to encrypt the partition table using the earlier decrypted flash encryption key:

    $ espsecure.py encrypt_flash_data --aes_xts --keyfile ./encryption/flash_encryption_key.bin --address 0x10000 --output encrypted_table.bin build/partition_table/partition-table.bin
   

   • This command will encrypt the generated partition table binary and put it in a file called encrypted_table.bin in the root directory

8. Run the following command to encrypt the application binary using the earlier decrypted flash encryption key:

    $ espsecure.py encrypt_flash_data --aes_xts --keyfile ./encryption/flash_encryption_key.bin --address 0x100000 --output encrypted_firmware.bin build/embed-reader-firmware-2.bin
   

   • This command will encrypt the generated firmware binary and put it in a file called encrypted_firmware.bin in the root directory

9. Run the following command to encrypt the bootloader using the earlier decrypted flash encryption key:

    $ espsecure.py encrypt_flash_data --aes_xts --keyfile ./encryption/flash_encryption_key.bin --address 0x0 --output encrypted_bootloader.bin build/bootloader/bootloader.bin
   

   • This command will encrypt the generated bootloader binary and put it in a file called encrypted_bootloader.bin in the root directory

10. Run the following command to encrypt the OTA data using the earlier decrypted flash encryption key:

    $ espsecure.py encrypt_flash_data --aes_xts --keyfile ./encryption/flash_encryption_key.bin --address 0x11000 --output encrypted_ota.bin build/ota_data_initial.bin
    

    • This command will encrypt the generated OTA binary and put it in a file called encrypted_ota.bin in the root directory

11. Run the following command to flash the encrypted files in their respective addresses:

    $ esptool.py -b 921600 --before default_reset --after no_reset --chip esp32s3  write_flash --force --flash_mode dio --flash_size 8MB --flash_freq 80m 0x10000 ./encrypted_table.bin 0x100000 ./encrypted_firmware.bin 0x0 ./encrypted_bootloader.bin 0x11000 ./encrypted_ota.bin
    

When I follow the steps above, everything works, and flash encryption is enabled, allowing me to only flash encrypted images of data.

Procedure followed for Secure Boot V2

Navigate to menuconfig > Security features of your IDF project and proceed through the following steps:

1. Check the box for Enable hardware Secure Boot in bootloader (READ DOCS FIRST). This will toggle further configurable options under Security features.

2. Navigate to the section App Signing Scheme and select RSA as the scheme. Since ESP32-S3 can only run secure boot version 2, RSA will be the only option to choose.

3. Navigate to Select secure boot version and select Enable Secure Boot version 2

4. Under Select secure boot version, check the box for Sign binaries during build

5. In the Secure boot private signing key section, provide the path to your private signing key (relative to the current working directory).

6. Check the box for Allow potentially insecure options under Secure boot private signing key

   • This will prevent IDF from automatically disabling the read and write options for the eFUSE. This should be left unchecked only for flashing to a production unit.
   • If unchecked and flashed, no more data can be written to the eFUSE. Only the ESP32 encryption hardware can read this data (non-human-readable).

7. Check the box for Leave unused digest slots available (not revoke)

   • This will prevent IDF from destroying the remaining available slots for storing keys in the eFUSE. This should be left unchecked only for flashing to a production unit.
   • If unchecked and flashed, no additional keys can be stored in the key blocks of the eFUSE.

8. Navigate to UART ROM download mode under Security Features and select UART ROM download mode (Enabled).

   • For development, this must be enabled.
   • If disabled, it will no longer be possible to flash firmware images to the ESP32.

9. We extract the public key from the private key. Run the following command to extract the public key from the private key:

    $ espsecure.py extract_public_key --version 2 --keyfile ./encryption/signing_key_v2.pem ./encryption/signing_key_v2_public.pem
   

10. We must use the espefuse.py tool to burn data to the eFUSE. The following command will generate a SHA256 hash (digest) of the supplied RSA public key and burn it to a key block in the chip's eFUSE:

    $ espefuse.py burn_key_digest --show-sensitive-info BLOCK_KEY0 ./encryption/signing_key_v2.pem SECURE_BOOT_DIGEST0
    

    • The public key file needs to be in PEM format.
    • This command will also burn the SECURE_BOOT_EN bit inside the eFUSE's BLOCK0, so there is no need to explicitly call espefuse.py burn_bit for enabling secure boot.
    • The --show-sensitive-info is useful for developers. It will print to console a summary of the operation, revealing the data you are burning to the eFUSE.

11. We must flash a signed version of the 2nd stage bootloader. Build a signed version of the bootloader with the following command:

    $ idf.py bootloader
    

    • You will be able to see in the build logs if a signature was found and if the compiled binary is signed or not.

12. If the previous command executed without problems, run the following command to flash the signed bootloader:

    $ idf.py bootloader-flash
    

    • If this command fails, then the bootloader needs to be force-flashed.
    • Scroll to the bottom of the failure message and copy the failed command. Paste it into the terminal.
    • Navigate to the function call write_flash and type ;--force; as demonstrated in the following point.
    • Here is the command for reference. It will vary from system to system:

        $ cd /home/xxxxx/esp/esp-idf/components/bootloader && /usr/bin/cmake -D IDF_PATH=/home/xxxxx/esp/esp-idf -D "SERIAL_TOOL=/home/xxxxx/.espressif/python_env/idf5.1_py3.10_env/bin/python;;/home/xxxxx/esp/esp-idf/components/esptool_py/esptool/esptool.py;--chip;esp32s3" -D "SERIAL_TOOL_ARGS=--before=default_reset;--after=no_reset;--no-stub;write_flash;--force;@bootloader-flash_args" -D WORKING_DIRECTORY=/home/xxxxx/esp/project_dir/build -P /home/xxxxx/esp/esp-idf/components/esptool_py/run_serial_tool.cmake

13. Since we enabled binary signing in menuconfig, we can simply proceed as usual to build application binaries using the following command: $ idf.py build

    • You will be able to see in the build logs if a signature was found and if the compiled binary is signed or not.

14. If the previous command executed without problems, run the following command to flash the signed application binary: $ idf.py flash monitor

After following the above steps, I am able to build and flash signed binaries to the chip. If I flash unsigned binaries, the image fails to boot, which is expected. I don't have the logs to show anymore (because I bricked it trying to also enable flash encryption).

I would appreciate it if someone could point out any mistakes in my procedure. I have written the whole procedure myself after following Espressif's documentation for IDF 5.1 (stable) and ESP32-S3. The documentation does not explain how to enable both features together. I know the procedure is not perfect, so please let me know the things I missed. I want to know the correct order of steps from the procedures above so that I can have both features enabled on my board.

TIA

Answer 1

In Procedure followed for Secure Boot V2 - Step 9, you have extracted the public key from the private key by using

 $ espsecure.py extract_public_key --version 2 --keyfile ./encryption/signing_key_v2.pem ./encryption/signing_key_v2_public.pem

The public key is stored in ./encryption/signing_key_v2_public.pem.

In the following step (step 10) it seems that you do not burn the extracted public key but use the private key ./encryption/signing_key_v2.pem instead.

$ espefuse.py burn_key_digest --show-sensitive-info BLOCK_KEY0 ./encryption/signing_key_v2.pem SECURE_BOOT_DIGEST0