QEMU SPI NOR flash with SWUpdate recovery

In the QEMU Custom SPI peripheral post the support for SPI controller for Cubieboard was introduced. A custom SPI peripheral was developed to demonstrate basic functionality.

QEMU comes with out-of-the-box support for several different SPI peripherals, one of which is the SPI NOR flash memory, modeled after M25P80 chip, but supports different variants, from various manufacturers and memory sizes.

The NOR memory can be combined with the SWUpdate for Cubieboard Yocto setup and Yocto Initramfs, to introduce a recovery option in case the eMMC memory gets corrupted.

In this post the steps for making a SWUpdate recovery initramfs image with Yocto and loading it from SPI NOR flash memory attached to QEMU Cubieboard will be shown. Once loaded, the initramfs image will format and partition the eMMC memory and start SWUpdate installation with a file from a USB drive.

The following items will be covered

• Yocto recovery image
  • SWupdate support
    • Bootloader upgrade
    • Deploying U-Boot environment
  • Disk management utilities
  • Init script updates
• FitImage
  • Determining size
• Adding SPI NOR flash to Cubieboard
  • Accessing from U-Boot
  • Loading to flash memory
• Testing
  • Preparation
    • Building images
    • Preparing USB image
    • Preparing SPI NOR image
    • Preparing SD card image
  • Recovery run
  • eMMC installed run
• Summary

Yocto recovery image

In order to implement the recovery image, the initramfs image used in the initramfs post needs to be extended with several utilities.

SWupdate support

The SWupdate support needs to be added, but in this case there is no need for the web server support. Instead, the client utility can be added since the installation will be started using an update file stored on the USB drive.

Therefore, the following two items should be added to the image definition

PACKAGE_INSTALL += "\
 swupdate \
 swupdate-client \
"

Bootloader upgrade

Since the recovery image is supposed to completely recover the eMMC, the update image should be extended to also write the bootloader to the appropriate location in the eMMC memory.

The SWupdate sw-description is updated to also hold information on the bootloader and the bootscript, so those files can be deployed with every upgrade.

images:(
  ...,
  {
    filename = "u-boot-sunxi-with-spl.bin";
    sha256 = "$swupdate_get_sha256(u-boot-sunxi-with-spl.bin)";
    device = "/dev/mmcblk0";
    offset = "8K";
});
files: ({
  filename = "boot.scr";
  path = "/boot.scr";
  device = "/dev/mmcblk0p1";
  filesystem = "vfat";
  sha256 = "$swupdate_get_sha256(boot.scr)";
});

This also means that the recipe for the update file generation needs to be extended to depend on these files being present, so following lines must be updated in the update-image.bb recipe

IMAGE_DEPENDS = "\
    virtual/bootloader \
    mistra-swupdate \
"

SWUPDATE_IMAGES = "\
    ${SPL_BINARY} \
    boot.scr \
    mistra-swupdate-cubieboard-ng.rootfs \
"

Deploying U-Boot environment

In order for SWUpdate to work, it expects certain information to be provided by U-Boot. That information is passed through the U-Boot environment.

The default configuration for Cubieboard is to have environment stored in the /boot partition of the eMMC. Since eMMC should be clean in the scenario that is considered for this post, then the U-Boot environment will not be present.

In order to circumvent this, we can deploy the default U-Boot environment in the initramfs and use it before running the update command.

This is done by adding a ROOTFS_POSTPROCESS_COMMAND in the initramfs image recipe. The command will copy the initial environment into a file in the /etc/u-boot-initial-env, which is the default location libubootenv tools (fw_printenv and fw_setenv) use.

add_uboot_initial_env() {
    install -d ${IMAGE_ROOTFS}${sysconfdir}
    install -m 0644 ${DEPLOY_DIR_IMAGE}/u-boot-initial-env ${IMAGE_ROOTFS}${sysconfdir}
}
ROOTFS_POSTPROCESS_COMMAND += "add_uboot_initial_env; "

Disk management utilities

Once the initramfs is started, the eMMC memory needs to be partitioned and formatted.

The eMMC will be partitioned using sfdisk and mke2fs will be used to format the ext4 partitions.

The sfdisk and mkfs packages are added in the image recipe using

PACKAGE_INSTALL += "\
    util-linux-sfdisk \
    e2fsprogs-mke2fs \
"

Partitioning eMMC in the initramfs init script will be performed as

sfdisk /dev/mmcblk0 <<EOF
unit: sectors
sector-size: 512
start=    4096, size=  81920, type=c, bootable
start=   86016, size= 681574, type=L
start=  770048, size= 681574, type=L
start= 1454080, size=       , type=L
EOF

The values for the start and size sectors for partitions are dumped from the eMMC image that was generated with the Yocto build system, so the partitioning of eMMC from initramfs applies the exact same configuration.

After the eMMC is partitioned, formatting of the boot and data partitions is done as

mkfs.vfat /dev/mmcblk0p1
mkfs.ext4 -F -E lazy_itable_init /dev/mmcblk0p4

Init script updates

While most of the configuration is kept from the Yocto Initramfs, a new init script is created to fulfill the desired scenario requirements.

The script performs several actions

1. Partition and format the eMMC according to the steps shown above
2. Create uboot.env file in the /boot/ partition to hold the U-Boot environment, needed for SWUpdate to function properly
3. Start SWUpdate main process
4. Mount USB drive which holds the update file
5. Run the update using swupdate-client

After swupdate-client is done with the update, a reboot will be performed.

FitImage

One of the requirements set at the top is that the SPI NOR memory is used to store the recovery image. This means that the Linux kernel binary, device tree and the bundled initramfs should be stored there and fetched via U-Boot at boot time.

U-Boot has command sf read that can be used to read desired amount of bytes from SPI NOR memory into the RAM memory. However, in order to be able to invoke that command, the U-Boot somehow needs to know how many bytes should be loaded. Therefore, we need to find a way to obtain that information for both kernel image (with bundled initramfs) and device tree blob.

In order to simplify things, we will use fitImage format to bundle all three items into one binary file. This way we will have to get information on size and load only one item instead of three. The additional benefit will be that the fitImage header will hold the size of the complete bundle, so we will be able to determine the size of the binary at runtime.

The fitImage generation requires creating a fitImage description, and then running mkimage command on it. The description is in the form of device tree description, containing information about the binaries that need to be included, as well as configurations that are supported.

KERNEL_CLASSES += "kernel-fitimage"
KERNEL_IMAGETYPES += "fitImage"

The fitImage description for our use-case (auto-generated by Yocto) is

/dts-v1/;

/ {
  description = "Kernel fitImage for Mistra Recovery/6.6.28/cubieboard-ng";
  #address-cells = <1>;

  images {
    kernel-1 {
      description = "Linux kernel";
      data = /incbin/("linux.bin");
      type = "kernel";
      arch = "arm";
      os = "linux";
      compression = "none";
      load = <0x40008000>;
      entry = <0x40008000>;
      hash-1 {
        algo = "sha256";
      };
    };
    fdt-sun4i-a10-cubieboard.dtb {
      description = "Flattened Device Tree blob";
      data = /incbin/("arch/arm/boot/dts/allwinner/sun4i-a10-cubieboard.dtb");
      type = "flat_dt";
      arch = "arm";
      compression = "none";

      hash-1 {
        algo = "sha256";
      };
    };
  };

  configurations {
    default = "conf-sun4i-a10-cubieboard.dtb";
    conf-sun4i-a10-cubieboard.dtb {
      description = "1 Linux kernel, FDT blob";

      kernel = "kernel-1";
      fdt = "fdt-sun4i-a10-cubieboard.dtb";

      hash-1 {
        algo = "sha256";
      };
    };
  };
};

Running the mkimage command will generate the following output, indicating that the inidividual binaries were included

FIT description: Kernel fitImage for Mistra Recovery/6.6.28/cubieboard-ng
Created:         Wed Apr 17 09:19:38 2024
 Image 0 (kernel-1)
  Description:  Linux kernel
  Created:      Wed Apr 17 09:19:38 2024
  Type:         Kernel Image
  Compression:  uncompressed
  Data Size:    5343968 Bytes = 5218.72 KiB = 5.10 MiB
  Architecture: ARM
  OS:           Linux
  Load Address: 0x40008000
  Entry Point:  0x40008000
  Hash algo:    sha256
  Hash value:   76b08f9eca78d0a8f8985d85a900dcad012206232a73323149ce9f1d44b3f0cf
 Image 1 (fdt-sun4i-a10-cubieboard.dtb)
  Description:  Flattened Device Tree blob
  Created:      Wed Apr 17 09:19:38 2024
  Type:         Flat Device Tree
  Compression:  uncompressed
  Data Size:    23462 Bytes = 22.91 KiB = 0.02 MiB
  Architecture: ARM
  Hash algo:    sha256
  Hash value:   42e9f15c500146fb23e7fac3f9e46fee9d67b5326c367eaac80a2813e4fdb68c
 Default Configuration: 'conf-sun4i-a10-cubieboard.dtb'
 Configuration 0 (conf-sun4i-a10-cubieboard.dtb)
  Description:  1 Linux kernel, FDT blob
  Kernel:       kernel-1
  FDT:          fdt-sun4i-a10-cubieboard.dtb
  Hash algo:    sha256
  Hash value:   unavailable

Determining size

The fitImage header is essentially the Flattened Device Tree (FDT) header, used by U-Boot to get information on loading binaries. The header structure is

struct fdt_header {
    uint32_t magic;
    uint32_t totalsize;
    uint32_t off_dt_struct;
    uint32_t off_dt_strings;
    uint32_t off_mem_rsvmap;
    uint32_t version;
    uint32_t last_comp_version;
    uint32_t boot_cpuid_phys;
    uint32_t size_dt_strings;
    uint32_t size_dt_struct;
};

From the structure definition, the size of the image is the second word, or 4 bytes from the start of the image.

To check if the value is there and that it can be used, the size of the fitImage that is generated for this post is 10676900 bytes, or 0xA2EAA4 in hex format. Checking the 2nd word of the binary with xxd returns

xxd -s 4 -p fitImage--6.6.28-r0-cubieboard-ng-20250501190148.bin | head -c 8

00a2eaa4

Adding SPI NOR flash to Cubieboard

In order to add the SPI NOR flash memory to Cubiebboard in QEMU, some code changes are required. The patch with the code changes can be used on QEMU v10.0.0.

What the patch does is:

• instantiate the SPI NOR memory
• connect it to MTD interface (important for later)
• connects the CS (Chip select) pin of the memory with the chip select pin 0 (CS0) of the SPI controller

With these changes, the QEMU would create an SPI NOR memory on startup and we could write and read back the written contents.

However, the idea in this post is to put the fitImage with recovery image in this memory, so the QEMU invocation command will have an addition, which will connect a binary file on the host, representing the flash memory, with the MTD interface component generated in the code. That way, every read and write associated with the emulated SPI NOR flash memory will be done with that binary file on the host.

When the file is generated, we should fill it with ones (0xFFs), since that would correspond to an erased flash memory. That can be achieved by using

dd if=/dev/zero ibs=1 count=$((16*1024*1024)) | LC_ALL=C tr '\000' '\377' >flash.bin

In order to start QEMU with host-backed flash memory storage, following command can be used

qemu-system-arm \
  -M cubieboard \
  -m 1G \
  -kernel ${UBOOT} \
  -nographic \
  -drive if=mtd,file=flash.bin,format=raw

Accessing from U-Boot

Before the memory can be accessed from U-Boot (or Linux kernel, but that is not the focus of this post):

• configuration update is needed, in order to integrate drivers for SPI flash and ISSI memory,
• a device tree update is needed, since U-Boot (Linux kernel) get information about available SPI devices from it.

Following config options need to be added to the U-Boot configuration

CONFIG_SPI=y
CONFIG_SPI_FLASH=y
CONFIG_SPI_FLASH_ISSI=y
CONFIG_CMD_SF=y
CONFIG_CMD_SF_TEST=y

The following addition is needed in the spi0 definition of the sun4i-a10-cubieboard.dts description (patched file)

 flash: m25p80@0 {
   compatible = "issi,is25wp128", "jedec,spi-nor";
   spi-max-frequency = <20000000>;
   reg = <0>;
 };

Once U-Boot is started, we can check that the SPI NOR is available and do several tests.

First we can probe that the memory is recognized using

=> sf probe

SF: Detected is25wp128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB

We can also try to write some content and the read it back. Since the sf write and sf read commands can only move data from RAM to flash and back, first we need to populate RAM with some data.

First we write one word (0x12345678) to an address in RAM (0x40000000)

=> mw 40000000 12345678

Then write the value from that address in RAM (0x40000000) to start of flash memory (0x0). The 4 in the command is number of bytes to transfer (one word)

=> sf write 40000000 0 4

Reading the value from memory to RAM (different location now, 0x40001000) and printing it is done in the following way

=> sf read 40001000 0 4
=> md 40001000 1

40001000: 12345678                             xV4.

which confirms that the value is written.

We can also check on the host that the value is written to the flash.bin using

xxd -p flash.bin | head -c 4

78563412

This shows that the data is written, but the representation of bytes in the NOR flash has different endianess.

One more test that can be performed is the built-in U-Boot NOR memory test (enabled with CONFIG_CMD_SF_TEST=y). Command accepts offset and length of bytes to test, so running test across the whole memory would be invoked using

=> sf test 0 1000000

SPI flash test:
0 erase: 1 ticks, 16384000 KiB/s 131072.000 Mbps
1 check: 5354 ticks, 3060 KiB/s 24.480 Mbps
2 write: 9475 ticks, 1729 KiB/s 13.832 Mbps
3 read: 5284 ticks, 3100 KiB/s 24.800 Mbps
Test passed
0 erase: 1 ticks, 16384000 KiB/s 131072.000 Mbps
1 check: 5354 ticks, 3060 KiB/s 24.480 Mbps
2 write: 9475 ticks, 1729 KiB/s 13.832 Mbps
3 read: 5284 ticks, 3100 KiB/s 24.800 Mbps

Loading to flash memory

Besides loading the Flash memory with all ones, we can load a custom value into the memory.

For instance, load other binary files using dd. Let’s create a helper file with the 0x12345678 contents

echo -ne '\x78\x56\x34\x12' > helper.bin

Then, we can write contents of helper.bin to the beginning of flash.bin using

dd if=helper.bin of=flash.bin conv=notrunc

Now, when the first 4 bytes are read from the SPI NOR flash in QEMU, it will show

=> sf read 40000000 0 4
=> md 40000000 1

40000000: 12345678                             xV4.

Testing

The following test scenario will be used:

• QEMU will be started with U-Boot as kernel parameter, with flash memory loaded with fitImage, USB drive with SWUpdate update file and an empty SD card image attached to it.
• fitImage initramfs will partition and format the eMMC/SD card, install update from USB and reboot; after reboot the QEMU can be closed in order to start it again from the SD card
• QEMU will be started with SD card parameter, to demonstrate that the eMMC/SD card has been created properly.

Preparation

The preparation for testing will involve several steps.

First, the needed images should be built using Yocto. Then, those images should be put into appropriate locations so they can be used.

Building images

The recovery image is built using

. ./setup-environment build_recovery
DISTRO=mistra-recovery MACHINE=cubieboard-ng bitbake mistra-swupdate-recovery

The bootloader that will be used to run the loading from SPI NOR flash is built using

. ./setup-environment build_recovery
DISTRO=mistra-recovery MACHINE=cubieboard-ng bitbake virtual/bootloader

The SWUpdate update image is built using

. ./setup-environment build_fb
DISTRO=mistra-framebuffer MACHINE=cubieboard-ng bitbake update-image

Once all images are built, preparation of other binary files can be done.

Preparing USB image

The USB drive should have one FAT partition and the update image should be copied to it.

The following commands create a 1GB image for the USB drive and one FAT partition on it.

dd if=/dev/zero of=usb.img bs=1M count=1024
sfdisk usb.img <<EOF
,,c
EOF

The kpartx utility can be used to make the partition available to the system, so it can be formatted.

sudo kpartx -av ./usb.img

add map loop0p1 (252:0): 0 2095104 linear 7:0 2048

Based on the output, the partition to use is /dev/mapper/loop0p1, so we can format it using

sudo mkfs.vfat /dev/mapper/loop0p1

In order to copy the files, partition needs to be mounted

sudo mkdir -p /run/mount/usb
sudo mount /dev/mapper/loop0p1 /run/mount/usb
sudo cp update-image-cubieboard-ng.rootfs.swu /run/mount/usb

Now that the file is in place, cleanup can be performed

sudo umount /run/mount/usb
sudo kpartx -d ./usb.img

Preparing SPI NOR image

The SPI NOR image should be created as it was done initially, by writing all FF’s to it

dd if=/dev/zero ibs=1 count=$((16*1024*1024)) | LC_ALL=C tr '\000' '\377' >flash.bin

After that, the fitImage binary should be copied directly

dd if=fitImage-cubieboard-ng.bin of=flash.bin conv=notrunc

Preparing SD card image

The SD card/eMMC image should be created empty, since the recovery image should partition and format it later. A 1GB image is enough and it can be created using qemu-img tool

qemu-img create sd.img 1G

Recovery run

After all images are prepared, the QEMU can be started with

qemu-system-arm \
    -M cubieboard \
    -m 1G \
    -kernel /work/cubieboard_ng-mistra-linux-gnueabi/u-boot/2024.01/build/u-boot \
    -usb \
    -device usb-storage,bus=usb-bus.1,drive=stick \
    -drive if=none,id=stick,file=usb.img \
    -sd sd.img \
    -drive if=mtd,file=flash.bin,format=raw \
    -nographic

U-Boot 2024.01-g (Jan 08 2024 - 15:37:48 +0000) Allwinner Technology

CPU:   Allwinner A10 (SUN4I)
Model: Cubietech Cubieboard
DRAM:  1 GiB
Core:  76 devices, 26 uclasses, devicetree: embed
WDT:   Not starting watchdog@1c20c90
MMC:   mmc@1c0f000: 0
Loading Environment from FAT... Unable to use mmc 0:0...
Unknown monitor
Unknown monitor
In:    serial,usbkbd
Out:   serial,vidconsole
Err:   serial,vidconsole
Net:
Error: ethernet@1c0b000 No valid MAC address found.
No ethernet found.

starting USB...
Bus usb@1c14000: USB EHCI 1.00
Bus usb@1c14400: USB OHCI 1.0
Bus usb@1c1c000: USB EHCI 1.00
Bus usb@1c1c400: USB OHCI 1.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found
scanning bus usb@1c14400 for devices... 1 USB Device(s) found
scanning bus usb@1c1c000 for devices... 2 USB Device(s) found
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 1 Storage Device(s) found
Hit any key to stop autoboot:  0

Using U-Boot prompt we need to load the fitImage from the SPI NOR flash memory. But before we can do it, we need to get information on the fitImage size from the header.

The simplest way is to load the first 8 bytes into RAM, and then reorder bytes in order to get the correct value.

=> sf probe

SF: Detected is25wp128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB

=> sf read 40000000 0 8

device 0 offset 0x0, size 0x8
SF: 8 bytes @ 0x0 Read: OK

=> setexpr.b fit_size_msb *40000004
=> setexpr.b fit_size_3 *40000005
=> setexpr.b fit_size_2 *40000006
=> setexpr.b fit_size_lsb *40000007
=> setenv fit_size ${fit_size_msb}${fit_size_3}${fit_size_2}${fit_size_lsb}

Now we can load the fitImage into RAM with

=> sf read ${loadaddr} 0 ${fit_size}

device 0 offset 0x0, size 0xa2eaa4
SF: 10676900 bytes @ 0x0 Read: OK

The image can now be executed using

=> bootm

The kernel will now start and perform the scenario as we defined it.

After the update is done, the system will reboot.

eMMC installed run

Now that the recovery is complete, QEMU can be started in the following manner in order to boot from SD card/eMMC

qemu-system-arm \
    -M cubieboard \
    -m 1G \
    -net nic \
    -net tap,ifname=qemu-tap0,script=no,downscript=no \
    -usb \
    -sd sd.img \
    -nographic

Mistra FrameBuffer 4.0 cubieboard-ng ttyS0

  cubieboard-ng login:

Summary

This was a rather complex post, combining use of USB, SPI NOR and SD card to implement recovery procedure for Cubieboard QEMU.

The SPI NOR and USB can be an interesting addition to the interfaces, and various scenarios can be tested.