4. Software Guide¶
This chapter provides instructions for compiling and deploying the BSP (Board Support Package) software to the Q7 module.
4.1. Architecture Overview¶
The BSP consists of several parts. They run on different parts of the CPU and each play their role in the boot process. Because the CPU contains cores running different instruction sets (ARMv6-M and ARMv8-A), two different compilers are needed. The table below list the parts and their instruction set.
BSP Part | Architecture |
---|---|
Cortex-M0 power management firmware | ARMv6-M |
ATF (ARM Trusted Firmware) | ARMv8-A |
U-Boot bootloader | ARMv8-A |
The Linux kernel | ARMv8-A |
Debian user-space | ARMv8-A |
The next section explains how to install suitable cross-compilers for both instruction sets.
The section “Compiling Linux Applications” provides guidance for compiling user-space applications for the RK3399.
4.2. Prerequisites¶
You need a recent x86_64 Linux installation to run the cross-compiler on and at least 10GB of disk space. The cross-compiler requires libc version 2.2.5. Distributions shipping this version are, among others:
- Ubuntu 16.04 “Xenial”
- Debian 8 “Jessie”
- Debian 9 “Stretch”
We recommend Debian 9 “Stretch” or Ubuntu 16.04 “Xenial”. Please install the following packages to set up the common build infrastructure:
sudo apt install device-tree-compiler u-boot-tools build-essential git bc debootstrap qemu-user-static libssl-dev
4.2.1. ARMv6-M Compiler¶
The “GNU Embedded Toolchain for ARM” is suitable for compiling the Cortex-M0 power management firmware. It can be downloaded from https://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads .
For this manual, it is assumed that version 6-2017-q1
is used. Direct link to the file:
Extract the tar.bz2 archive to /opt:
sudo tar -xf gcc-arm-none-eabi-6-2017-q1-update-linux.tar.bz2 -C /opt
4.2.2. ARMv8-A Compiler¶
The Linaro aarch64-linux-gnu toolchain is suitable for compiling all other parts of the BSP. It is also suitable for compiling user-space applications. You can download ready-to-use binaries from Linaro: https://releases.linaro.org/components/toolchain/binaries/6.3-2017.02/aarch64-linux-gnu/ .
Direct link to the file:
Extract the tar.xz archive to /opt:
sudo tar -xf gcc-linaro-6.3.1-2017.02-x86_64_aarch64-linux-gnu.tar.xz -C /opt
4.3. Compile the Cortex-M0 power management firmware¶
The Cortex-M0 firmware runs inside a microcontroller embedded in the CPU IC. It implements power-management functionality and helpers (e.g. DRAM frequency switching support).
Set up environment variables to make use of the ARMv6-M compiler, then download the source code and compile:
export ARCH=arm64
export CROSS_COMPILE=/opt/gcc-arm-none-eabi-6-2017-q1-update/bin/arm-none-eabi-
git clone https://git.theobroma-systems.com/rk3399-cortex-m0.git
cd rk3399-cortex-m0
make
cd ..
4.3.1. Optional: Compile the cross-compiler¶
As an alternative to using a ready-made compiler, the firmware repository has a mechanism to compile the ARMv6-M-compiler as a part of the build process. This is called “internal toolchain”.
If you want to use the internal toolchain instead you will also need the following packages:
sudo apt install libssl-dev autoconf gperf bison flex texinfo help2man gawk libncurses5-dev
Then to use the internal toolchain, specify “USE_INTERNAL_TOOLCHAIN=1” as part of your invocation to make.:
make USE_INTERNAL_TOOLCHAIN=1
4.4. Compile the ATF¶
Download the source code and compile using:
export CROSS_COMPILE=/opt/gcc-linaro-6.3.1-2017.02-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
git clone https://git.theobroma-systems.com/arm-trusted-firmware.git
cd arm-trusted-firmware
make PLAT=rk3399 bl31
cd ..
4.5. Compile U-Boot¶
U-Boot is used as the bootloader on the RK3399-Q7 module.
Download the source code using:
git clone https://git.theobroma-systems.com/puma-u-boot.git
The U-Boot build process uses the files generated in the previous steps Copy the previously generated
files rk3399m0.bin
from the Cortex-M0 firmware and bl31.bin
from the ATF to the puma-u-boot
directory. Recent U-Boot releases expect the bl31.bin
file under the name bl31-rk3399.bin
. To
support all variants, the file is copied twice:
cp rk3399-cortex-m0/rk3399m0.bin puma-u-boot
cp arm-trusted-firmware/build/rk3399/release/bl31.bin puma-u-boot/bl31.bin
cp arm-trusted-firmware/build/rk3399/release/bl31.bin puma-u-boot/bl31-rk3399.bin
Then you are ready to compile U-Boot:
cd puma-u-boot
export ARCH=arm64
export CROSS_COMPILE=/opt/gcc-linaro-6.3.1-2017.02-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
make puma-rk3399_defconfig
make -j4
tools/mkimage -n rk3399 -T rksd -d spl/u-boot-spl.bin spl_sd.img
tools/mkimage -n rk3399 -T rkspi -d spl/u-boot-spl.bin spl_spi.img
make u-boot.itb
cd ..
The resulting bootloader consists of three files: spl_sd.img
, spl_spi.img
and u-boot.itb
,
but only one of the spl
files is used at a time.
The file spl_sd.img
is only used when booting from SD-card or eMMC, while spl_spi.img
is only
used when booting from SPI NOR flash.
4.6. Compile the Boot Script¶
The U-Boot boot sequence is controlled by a file called boot.scr
. This file
is generated from a plain-text file called boot.cmd
.
Download the repository and generate boot.scr
using:
git clone https://git.theobroma-systems.com/som-tools.git
cd som-tools
make -C boot-script
cd ..
4.7. Compile the Linux Kernel¶
The kernel source code can be cloned with:
git clone https://git.theobroma-systems.com/puma-linux.git
Compile using:
cd puma-linux
export ARCH=arm64
export CROSS_COMPILE=/opt/gcc-linaro-6.3.1-2017.02-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-
make puma-rk3399_defconfig
make -j4 rockchip/rk3399-puma.dtb Image
This will create the two files needed for booting with U-Boot (paths are
relative to the puma-linux
directory):
- The device tree:
arch/arm64/boot/dts/rockchip/rk3399-puma.dtb
- The kernel image:
arch/arm64/boot/Image
4.8. Building the root filesystem¶
A filesystem can be created using Debootstrap, specifying arm64 as architecture in the command line.
Supposing the target dir is called rk3399-rootfs
and the chosen distribution is
Debian 9 “Stretch” (recommended):
export targetdir=/opt/rk3399-rootfs
sudo mkdir -p $targetdir
sudo debootstrap --arch=arm64 --foreign stretch $targetdir http://deb.debian.org/debian/
Next, copy the qemu-arm-static binary into the right place for the binfmt packages to find it and copy the resolv.conf file from the host system:
sudo cp /usr/bin/qemu-aarch64-static $targetdir/usr/bin/
sudo cp /etc/resolv.conf $targetdir/etc
This will provide a very basic arm64 rootfs in the targetdir. For the next
stages, we chroot
to the target dir:
sudo chroot $targetdir
Second stage of debootstrap inside the new root dir:
/debootstrap/debootstrap --second-stage
Set up the apt package sources:
cat << EOT > /etc/apt/sources.list
deb http://deb.debian.org/debian stretch main contrib non-free
deb http://deb.debian.org/debian stretch-updates main contrib non-free
deb http://security.debian.org/ stretch/updates main contrib non-free
EOT
We can now pull the latest apt database from the Debian mirrors and install locales:
apt update
apt install locales
echo "en_US.UTF-8 UTF-8" > /etc/locale.gen
locale-gen
Install any additional packages inside the chroot. An ssh server and sudo are recommended:
apt install openssh-server sudo
Set the root password for logging in via serial console:
passwd
To log in over ssh, create another user besides root (root login over ssh is not permitted by default):
adduser user
Add the new user to the sudo
group so you can switch to root privileges if needed:
adduser user sudo
Set up a basic network configuration file with DHCP via eth0 and enable automatic DNS configuration through systemd-resolved:
cat << EOT > /etc/systemd/network/eth0.network
[Match]
Name=eth0
[Network]
DHCP=yes
EOT
systemctl enable systemd-networkd
systemctl enable systemd-resolved
ln -f -s /lib/systemd/resolv.conf /etc/resolv.conf
When executing systemctl enable
you may get the message qemu: Unsupported syscall: 278
.
The operation still succeeds and the message can be safely ignored.
Set the hostname and exit from the chroot:
echo rk3399-q7 > /etc/hostname
exit
We now have a root filesystem which can be deployed to the SD card.
4.9. Deploy on SD Card¶
4.9.1. Partition Setup¶
Both U-Boot and Linux will be located on the same SD card. The layout of the card after setup is as follows:
Offset | Contents | Files |
---|---|---|
0 | Partition table | |
32kiB | U-Boot SPL | spl_sd.img |
240kiB | U-Boot environment | |
256kiB | U-Boot + ATF + Cortex-M0 firmware | u-boot.itb |
2MiB | Partition 1 (ext4 - Linux root fs) | boot.scr, Image, rk3399-puma.dtb, defaultEnv.txt and rootfs |
To setup a SD card for booting you first need to create partitions.
Partitions can be created using fdisk (assuming the SD card is mapped to /dev/sd``X`` , where X
should be replaced with your corresponding device-letter) and has no partitions (this can be checked using the p
command):
sudo fdisk /dev/sdX
> p
This should show an empty partition table, for example:
Disk /dev/sdX: 3980 MB, 3980394496 bytes
123 heads, 62 sectors/track, 1019 cylinders, total 7774208 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xdbbd45c7
Device Boot Start End Blocks Id System
If there are partitions on the sdcard, they can be deleted with o
.
The required partition can be created with the command n
,
then accepting the defaults, except for First sector
, where we use 4096:
> n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): <ENTER>
Partition number (1-4, default 1): <ENTER>
First sector (2048-7774207, default 2048): 4096
Last sector, +sectors or +size{K,M,G} (...): <ENTER>
This will create a primary partition at offset 2MiB.
Enter w
to write the new partition table to the disk:
> w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Now we format the partition as ext4:
sudo /sbin/mkfs.ext4 -E lazy_itable_init=0 /dev/sdX1
The option lazy_itable_init=0
speeds up the first boot because it
initializes the inode tables in advance.
The SD card is now ready to have the U-Boot bootloader and Linux deployed.
4.9.2. Deploy U-Boot¶
The U-Boot images spl_sd.img
and u-boot.itb
are written to the SD
card. Assuming the SD card is mapped to /dev/sdX:
sudo dd if=puma-u-boot/spl_sd.img of=/dev/sdX bs=1k seek=32 conv=nocreat
sudo dd if=puma-u-boot/u-boot.itb of=/dev/sdX bs=1k seek=256 conv=nocreat
4.9.3. Deploy the Linux Kernel and the Root Filesystem¶
Mount the SD card partition and copy the rootfs (assuming that the rootfs is located
at /opt/rk3399-rootfs
and the sd card at /dev/sdX1
):
sudo mkdir -p /mnt/sdcard
sudo mount /dev/sdX1 /mnt/sdcard
sudo cp -av /opt/rk3399-rootfs/* /mnt/sdcard
Copy kernel image, device tree and boot script into the boot directory:
sudo cp -r som-tools/boot-script/boot/{boot.scr,puma_rk3399} /mnt/sdcard/boot
sudo cp puma-linux/arch/arm64/boot/dts/rockchip/rk3399-puma.dtb /mnt/sdcard/boot/puma_rk3399
sudo cp puma-linux/arch/arm64/boot/Image /mnt/sdcard/boot/puma_rk3399
Finally, unmount the SD card:
sudo umount /mnt/sdcard
The SD card is ready for booting.
4.9.4. U-Boot Customization¶
The boot script /boot/boot.scr
handles the boot sequence.
Unless you want to customize the sequence, no further action is required.
If you want to step through the sequence manually or customize it, you can execute the following commands on the U-Boot prompt:
setenv bootargs root=/dev/mmcblk0p1 rw rootwait
ext4load mmc 1:1 $kernel_addr_r boot/puma_rk3399/Image
ext4load mmc 1:1 $fdt_addr_r boot/puma_rk3399/rk3399-puma.dtb
booti $kernel_addr_r - $fdt_addr_r
Optionally, these commands can be compiled together in a single command and saved so it is performed on every subsequent boot:
setenv bootargs root=/dev/mmcblk0p1 rw rootwait
setenv boot_sd "ext4load mmc 1:1 $kernel_addr_r boot/puma_rk3399/Image && \
ext4load mmc 1:1 $fdt_addr_r boot/puma_rk3399/rk3399-puma.dtb && \
booti $kernel_addr_r - $fdt_addr_r"
setenv bootcmd run boot_sd
saveenv
To reset the U-Boot settings to default, execute:
env default -f -a
saveenv
Note
root=/dev/mmcblk0p1
and ext4load mmc 1:1
refer to the SD Card.
Use root=/dev/mmcblk1p1
and ext4load mmc 0:1
instead to boot from
eMMC.
4.10. Deploy on SPI NOR-flash¶
To have a reliable boot sequence even if eMMC and/or SD-card fail, U-Boot can be written in
to the onboard NOR-flash. While u-boot.itb
is used for both SD-card and SPI-flash boot,
spl_spi.img
is used instead of spl_sd.img
.
From U-boot:
sf probe
load mmc 1 $kernel_addr_r root/spl_spi.img
sf erase 0 +$filesize
sf write $kernel_addr_r 0 $filesize
load mmc 1 $kernel_addr_r root/u-boot.itb
sf erase 0x40000 +$filesize
sf write $kernel_addr_r 0x40000 $filesize
From Linux:
dd if=spl_spi.img of=/dev/mtdblock0 bs=256k seek=0 conv=nocreat
dd if=u-boot.itb of=/dev/mtdblock0 bs=256k seek=1 conv=nocreat
4.11. Deploy on On-Board eMMC storage¶
As the eMMC storage is only accessible from the module itself, you must first boot the RK3399-Q7 from SD card.
Partition and format the eMMC storage as described in 4.9.1 Partition Setup, but using the
device /dev/mmcblk1
.
Mount the eMMC partition and copy the contents of the SD card to the eMMC storage. The copy process will take about 30 seconds:
sudo mkdir -p /mnt/emmc
sudo mount /dev/mmcblk1p1 /mnt/emmc
sudo cp -ax / /mnt/emmc
sudo umount /mnt/emmc
The final step is copying the bootloader to the eMMC:
dd if=/boot/puma_rk3399/spl_sd.img of=/dev/mmcblk1 bs=1k seek=32 conv=nocreat
dd if=/boot/puma_rk3399/u-boot.itb of=/dev/mmcblk1 bs=1k seek=256 conv=nocreat
Shut down the board (poweroff
command) and remove the SD card. Make sure the boot selector switch is set to “Normal Boot
”.
The next boot will run U-Boot off the internal eMMC storage.
4.12. Compiling Linux Applications¶
The easiest option is to compile your applications directly on a running RK3399-Q7 module. Just install the gcc package and related utilities and you are good to go:
sudo apt-get install build-essential
The second option is to cross-compile your applications. The ARMv8-A compiler that was installed earlier is suitable to compile applications for the RK3399-Q7.
4.13. Serial Number¶
Each RK3399-Q7 module has a unique serial number that can be read by software.
In U-Boot, the serial number is contained in the environment variable serial#
.
You can print it using the command:
printenv serial#
Under Linux, it is represented by a simple text file in /sys
:
cat /sys/firmware/devicetree/base/serial-number
The serial number is fixed in hardware (derived from the SoC CPU ID) and cannot be modified.
4.14. MAC Address¶
By default, the MAC address of each RK3399-Q7 module is a random value derived from the serial number. The properties of this default MAC address are:
- It is a Locally Administered Address: The U/L bit of the MAC address is set to 1
- It is not guaranteed to be globally unique
- The address is fixed for each RK3399-Q7 module. It stays constant across reboots as it is deterministically derived from the serial number
To set your own Universally Administered Address, you overwrite the U-Boot environment variable ethaddr
.
On the U-Boot prompt, with XX:XX:XX:XX:XX:XX replaced by your MAC address:
setenv ethaddr XX:XX:XX:XX:XX:XX
saveenv
The MAC address can be queried from the U-Boot prompt using:
printenv ethaddr
To reset the MAC address to the default value, run:
env delete ethaddr
saveenv