UEFI Boot for Mochabin

Photo: Jakub Dziubak

MOCHAbin is a pretty capable ARM board - it has a quad core ARMv8 Cortex-A72 @ 1400MHz, 8GB of RAM, 16GB of onboard eMMC, not to mention a bunch of Ethernet connectivity (1x 10Gb SFP+ cage, 1x 1Gb SFP cage, a WAN RJ45 port with PoE in, and 4x LAN ports connected to an onboard switch chip).

The main downside for me, however, was the boot firmware. Out of the box, it ships with a pretty ancient build of U-Boot, which fails to properly support UEFI.

There are other options: there are references to people using what I can only assume to be some variant of Marvell's fork of EDK II to provide UEFI support. However, I went down the route of trying to get a more modern version of U-Boot working.

To cut a long story short, I got Tow-Boot, a user-friendly distribution of U-Boot with a pretty decent build system, to boot properly including proper UEFI support.

What works

  • Environment storage in SPI flash
  • Reading (and booting) from:
    • eMMC
    • USB
  • EFI boot

What's not tested

  • The SFP+/SFP cages
  • The WAN NIC
  • SATA
  • PCIe

What doesn't work

  • Any of the 4 "LAN" ports -- I think they require bringup of the switch chip, and I haven't tried configuring that properly yet
  • Using the actual hw_info SPI block that's supposed to hold the Ethernet MAC addresses and PCB serial number.

Building an image

If you want to build an image yourself, you'll need a Linux system with a working install of Nix. This doesn't need to be NixOS, Nix on Debian should do just fine. You can even (probably) use Nix inside a Docker container if you'd like.

Got that together? Alright:

  1. Grab https://github.com/lukegb/Tow-Boot. The lukegb/globalscale-mochabin branch should be the default, and it's what you'll need.
  2. Run nix-build -A globalscale-mochabin-8gb (if you have the 8GB RAM variant), or nix-build -A globalscale-mochabin-4gb (if you have the 4GB variant).
  3. Wait.
  4. You should now have a result symlink that points to a directory with some files in it. The one you probably want is result/binaries/Tow-Boot.spi.bin, which is the build of Tow-Boot that uses the SPI flash for storing the U-Boot environment.

At this point, you can use the instructions in "Recovering" below to use mvebu64boot to just boot this once, to see what it's like, or continue below to flash it as your main bootloader.

Flashing

Apologies in advance: many of these steps should be automated, but aren't. There are various steps where you should take notes or backups of things so you can restore your device to a working state later. Please do that, and copy them somewhere safe!

You will need:

  • a USB stick that you can wipe
  • a micro-USB cable to use to connect to the MOCHAbin via USB-serial

Getting set up

  1. Wipe your memory stick. Put a ext2* filesystem on it, and copy result/binaries/Tow-Boot.spi.bin to it.
  2. Turn off the MOCHAbin.
  3. Unplug any other USB devices from the MOCHAbin, and connect your memory stick.
  4. Start your terminal emulator. The MOCHAbin runs at 115200 bps.
  5. Turn on the MOCHAbin. When prompted to interrupt boot, press a key.
  6. You should now have a Marvell>> prompt.

Making sure everything is in place

  1. Run usb start. You should get some output ending in scanning usb for storage devices... 1 Storage Device(s) found.
  2. Run usb part. You should see your single, ext2 filesystem.
  3. Run ext2ls usb 0:1. You should see the content of your filesystem, including Tow-Boot.spi.bin.

Backing up the SPI flash

  1. Run sf probe. This should report SF: Detected w25q32bv with page size 256 Bytes, erase size 4 KiB, total 4 MiB. This is the information about your SPI flash. If your SPI flash is not 4MiB, stop!
  2. Run sf read $kernel_addr_r 0 0x400000. This should report device 0 whole chip and then hang for 10s or so.
  3. Run ext4write usb 0:1 $kernel_addr_r /backupspi.img 0x400000. This should report File System is consistent, and then complete after between 6 and 15s.

Backing up the environment block

  1. Run env export -t $kernel_addr_r. This will appear to do nothing.
  2. Run ext4write usb 0:1 $kernel_addr_r /backupenv.txt $filesize. This should again report File System is consistent, then complete after a few seconds.

Noting down key environment variables

  1. Run env print ethaddr eth1addr eth2addr pcb_sn.
  2. Copy and save the output somewhere. You'll need this later.

Flashing the new image

  1. Erase your SPI flash: run sf erase 0 0x400000. This ensures you don't have any of the existing U-Boot environment lingering around. This will appear to hang for a few tens of seconds, then report SF: 4194304 bytes @ 0x0 Erased: OK.
  2. Flash the new image: run bubt Tow-Boot.spi.bin spi usb.

Rebooting and setting the environment again

  1. Turn the power off, then on again
  2. When prompted to interrupt boot, hit Escape or press Ctrl-C.
  3. You'll now be presented with the Tow-Boot menu. Scroll down to Firmware Console using the arrow keys and hit enter.
  4. Now instead of Marvell>> , you should get a => prompt.
  5. For each of ethaddr, eth1addr, eth2addr and pcb_sn, run env set $VARIABLE_NAME $THE_VARIABLE_VALUE_YOU_NOTED_DOWN_EARLIER
  6. Run env save. This should eventually print OK.
  7. Turn the power off, then on again.

Congratulations!

You now have Tow-Boot. If you have a UEFI-compatible boot medium (either on the eMMC or over USB, for instance), it will Just Work(TM).

  1. Take the memory stick and back up backupspi.img and backupenv.txt somewhere safe.

* Technically this could be ext4. I suggest ext2 because modern Linux distros will enable some ext4 extensions that won't work by default, and this is very temporary.

Using an up-to-date DTB

U-Boot will automatically discover new DTBs from the boot partition. I have the following setup (my EFI System Partition is mounted at /boot; adjust appropriately if yours is /boot/efi):

/boot
  /boot/dtb
    /boot/dtb/marvell
      /boot/dtb/marvell/armada-7040-mochabin.dtb

I copy this file from the DTBs shipped with my Linux kernel build on update; if you don't do this then you will (I think?) inherit the DTB used by U-Boot.

Recovering

Oops! Something went wrong and you need to recover. No sweat. The Marvell BootROM supports sending it an image via X-Modem, and you can do that over the USB-Serial interface.

The tool I've been using for doing this is mvebu64boot, which is super simple. If you have Nix, then this is available in recent nixpkgs-unstable, but this is super trivial to build without Nix: just clone that somewhere and use make.

Make sure you don't have any terminal emulators open on the port, and run:

$ mvebu64boot -t -b path/to/flash-image.img /dev/ttyUSB0

# for instance:
$ mvebu64boot -t -b backupspi.img /dev/ttyUSB0

# or:
$ mvebu64boot -t -b result/binaries/Tow-Boot.spi.bin /dev/ttyUSB0

# or, using the bootloader images available from Globalscale's FTP (ftp://76.80.10.5/Downloads/Mochabin/bootloader_for_mochabin_hw-rev-1-5-0_20220905/):
$ mvebu64boot -t -b mochabin-bootloader-ddr4-8g-mvddr-41927ee-atf-277d4b6b-uboot-b794de0054-20220905-rel.bin /dev/ttyUSB0

(where ttyUSB0 is replaced with whatever device name you have for the flash image you're booting)

Once this prints Sending boot pattern..., powercycle the MOCHAbin. You should then get BootROM is ready for image file transfer, followed by a rapidly-increasing percentage. This will take a few minutes - first the image prolog needs to be sent, then there's a Waiting for BootROM... stage, then the image itself. You'll then be presented with a serial terminal. Use Ctrl-\ c to exit.

This works for booting the Tow-Boot images you built, or the SPI flash backup image I repeatedly reminded you to make.

To return to the previous image, boot to the U-Boot console (either the built-in Marvell one or a Tow-Boot one), insert a memory stick containing the image to flash from (I suggest ext2 again, for the same reasons as above) and run bubt backupspi.img spi usb.

Getting things working

NOTE: if you're just interested in building your own image, you can ignore everything from this point on.

Getting things working was all relatively simple: upstream U-Boot 2022.07 and 2023.07 have pretty much all the hardware support that's required to boot this board. The main thing missing is a device tree.

Device trees

Device trees are effectively a description of what hardware is where: this avoids some of the automatic configuration, and simplifies a bunch of customization required to get things booting. The Linux kernel has a bunch of these - and we're in luck, because Globalscale have been paying Sartura to upstream a bunch of the hardware enablement required to get the MOCHAbin to boot well (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/marvell/armada-7040-mochabin.dts).

There's one problem here, however: this DTS doesn't work inside the U-Boot tree. U-Boot tries to be compatible with whatever Linux is doing so that device trees are as portable as possible from the Linux source to the U-Boot tree, but the specific naming of the compatible arguments is important for drivers, and some of the includes are in different places.

Instead, I opted to take the device tree from their version of U-Boot, and hack it together until it worked on a more recent version.

Porting their device tree forwards

To give Globalscale credit, this mostly worked without a hitch. There are a few differences (phy-mode = "sfi"; needed to become phy-mode = "10gbase-r";), but for the most part this... just worked.

I haven't tested a bunch of the functionality that I didn't need: in particular, I don't plan on PXE booting or using any network functionality at startup. I also don't have any drives connected over the internal M.2 SATA interface, so I couldn't test that either.

Future work

I'm hoping to, eventually:

  • Try to get the Linux DTS working on U-Boot and upstream it.
  • Get the 4 RJ45 LAN ports working.
  • Check that the SATA M.2 port actually works.
  • Get writable EFI variables working?
  • Upstream my build to Tow-Boot proper.
  • Getting the flashing steps automated in the Tow-Boot installer.

So stay tuned. Maybe.