I am a hoarder of SBCs, I had the first Raspberry Pi on pre-order from 2 different suppliers in fear I might not get to have one at launch. Seems intensely silly now but at the time I was fascinated by what a Raspbery Pi was and could do.
A while later I discovered however the SBCs I really liked: ODROIDs.
I have a C1,C1+,C2,2xXU4s and an HC1(oh and an odroid-go). And I will probably get more in the future.
As you can image I messed around with them for years and tried all sorts of OSes. At times there was a feature that I needed in the kernel and could not get in the official image and that’s when things got interesting.
ODROIDs and many other SBCs use bootloader called U-Boot. Unlike the all might Grub(anyone remember Lilo? I don’t miss you…) this a less sophisticated bootloader meant for loading just one OS and only supports FAT32 for initial booting which means you must have a separate boot partition(not unlike EFI systems but so much simpler).
This is the reason why all Raspberry PI disk images and most other SBCs have a boot FAT32 partition alongside the rootfs partition but we’ll get to that in a minute.
Now the configuration of the bootloader happens in one file boot.ini which is on the boot partition(duh!). Some more sophistcated systems also use some assorted .txt files to inject linux boot command line params and such. Armbian for example specifically asks you to make any changes to u-boot via armbianEnv.txt so that you do not destroy your boot configuration… to much.
For ODROIDs however things are kept simple and boot.ini is your friend.
BTW, one a live system there are 2 boot folders:
- /boot: this contains the initramfs image, we’ll get to it.
- /media/boot: this is the FAT32 partition I was talking about which contains the files required to boot the board, including our precious boot.ini
Just for your entertainment this what my boot.ini looks at the moment:
ODROIDXU-UBOOT-CONFIG
# U-Boot Parameters
setenv initrd_high "0xffffffff"
setenv fdt_high "0xffffffff"
# Mac address configuration
setenv macaddr "get your own, man!"
...OMITTED for brevity....
# --- HDMI / DVI Mode Selection ---
# ------------------------------------------
# - HDMI Mode
setenv vout "hdmi"
# - DVI Mode (disables sound over HDMI as per DVI compat)
# setenv vout "dvi"
# --- HDMI CEC Configuration ---
# ------------------------------------------
setenv cecenable "false" # false or true
# set to true to enable HDMI CEC
# Enable/Disable ODROID-VU7 Touchsreen
setenv disable_vu7 "false" # false
# DRAM Frequency
# Sets the LPDDR3 memory frequency
# Supported values: 933 825 728 633 (MHZ)
setenv ddr_freq 825
# External watchdog board enable
setenv external_watchdog "false"
# debounce time set to 3 ~ 10 sec, default 3 sec
setenv external_watchdog_debounce "3"
#------------------------------------------------------------------------------
#
# HDMI Hot Plug detection
#
#------------------------------------------------------------------------------
#
# Forces the HDMI subsystem to ignore the check if the cable is connected or
# not.
# false : disable the detection and force it as connected.
# true : let cable, board and monitor decide the connection status.
#
# default: true
#
#------------------------------------------------------------------------------
setenv HPD "true"
#------------------------------------------------------------------------------------------------------
# Basic Ubuntu Setup. Don't touch unless you know what you are doing.
# --------------------------------
# SD card: UUID=e139ce78-9841-40fe-8823-96a304a09859
# HDD: UUID=54155bab-e84a-4948-9569-86e8147b1f2f
setenv bootrootfs "console=tty1 console=ttySAC2,115200n8 root=/dev/mapper/vg_storage-lv_rootfs rootwait ro fsck.repair=yes net.ifnames=0"
# Load kernel, initrd and dtb in that sequence
fatload mmc 0:1 0x40008000 zImage_kvm
fatload mmc 0:1 0x42000000 uInitrd
setenv fdtloaded "false"
if test "x${board_name}" = "x"; then setenv board_name "xu4"; fi
if test "${board_name}" = "xu4"; then fatload mmc 0:1 0x44000000 exynos5422-odroidxu4-kvm.dtb; setenv fdtloaded "true"; fi
if test "${board_name}" = "xu3"; then fatload mmc 0:1 0x44000000 exynos5422-odroidxu3.dtb; setenv fdtloaded "true"; fi
if test "${board_name}" = "xu3l"; then fatload mmc 0:1 0x44000000 exynos5422-odroidxu3-lite.dtb; setenv fdtloaded "true"; fi
if test "${fdtloaded}" = "false"; then fatload mmc 0:1 0x44000000 exynos5422-odroidxu4.dtb; setenv fdtloaded "true"; fi
fdt addr 0x44000000
setenv hdmi_phy_control "HPD=${HPD} vout=${vout}"
if test "${cecenable}" = "false"; then fdt rm /soc/cec@101B0000; fi
if test "${disable_vu7}" = "false"; then setenv hid_quirks "usbhid.quirks=0x0eef:0x0005:0x0004"; fi
if test "${external_watchdog}" = "true"; then setenv external_watchdog "external_watchdog=${external_watchdog} external_watchdog_debounce=${external_watchdog_debounce}"; fi
# final boot args
setenv bootargs "${bootrootfs} ${videoconfig} ${hdmi_phy_control} ${hid_quirks} smsc95xx.macaddr=${macaddr} ${external_watchdog}"
# set DDR frequency
dmc ${ddr_freq}
# Boot the board
bootz 0x40008000 0x42000000 0x44000000
Most settings in boot.ini happen via “setenv” similar to how you would set a bash environment variable.
A few important such args to note:
- bootargs: additional kernel boot params, like passing extra module options, similar to what you would put in a modprobe config on a regular linux
- bootrootfs: linux kernel boot args including the vital root= param which tells the kernel where the party is at i.e. where the root partition which contains the next stage and the userspace stuff(after initramfs has been loaded though – this will be an ext3/4 partition and needs those sweet filesystem drivers)
- macaddr – that’s right, set whatever mac address you want, this Linux, baby
Also important are the fatload commands that use the built-in support for FAT32 to load 2 vital components:
- The compressed kernel image: usually named zImage, this lives on the boot partition
- The initramfs, usually named uInitrd, this also lives on the boot partition
- The Device Tree file which something particular to Linux on ARM devices.
So, to recap, u-boot loads the compressed linux kernel image(and device tree) and initramfs from the FAT32 boot partition.
Side note: parameters for hardware devices can sometimes be modified by editing the Device Tree file(the dtb) by decompiling it to .dts, editing, and re-compiling it to .dtb. An example includes changing the clock rate on i2c bus, assuming the running kernel allows for it.
Next, it runs the linux kernel and passes a vital piece of information: the root filesystem, the partition where the userspace stuff lives, this is normally the only other partition on an SD card made for booting an SBC and it is formatted as ext3/ext4(probably ext4).
As a side note, my root partition is in a LVM volume(because why not! this is possible because my initramfs contains the drivers for LVM) and from the SD card I use only the boot FAT32 partition because the Odroid is hardwired to boot from either an eMMC or an SD card, no way to boot via USB or any other hardware.
If your kernel booted successfully – as in you passed it a rootfs it could read and mount(!!) you are then in business. I should say that /etc/fstab also plays a part once the kernel is booted in making the root(i.e. the / folder) accessible to userspace programs. Now I think you could have an empty /etc/fstab and still boot but I did not test that theory, yet.
Finally the systemd process is started(or init for other distros) which starts all the Linux services.
As you can imagine I made a lot of mistakes messing around with different kernels and boot.ini settings, luckily there is a serial interface and you can almost always see an error message there that tells you where you screwed up.
Most of the information in this post has been gleaned from my own failures to understand the boot process so if you are curious, go out there and fail too!