Canonical Voices

Posts tagged with 'u-boot'

abeato

Ubuntu Core (UC) is Canonical’s take in the IoT space. There are pre-built images for officially supported devices, like Raspberry Pi or Intel NUCs, but if we have something else and there is no community port, we need to create the UC image ourselves. High level instructions on how to do this are found in the official docs. The process is straightforward once we have two critical components: the kernel and the gadget snap.

Creating these snaps is not necessarily complex, but there can be bumps in the road if you are new to the task. In this post I explain how I created them for the Jetson TX1 developer kit board, and how they were used to create a UC image for said device, hoping this will provide new tricks to hackers working on ports for other devices. All the sources for the snaps and the build scripts are available in github:
https://github.com/alfonsosanchezbeato/jetson-kernel-snap
https://github.com/alfonsosanchezbeato/jetson-gadget-snap
https://github.com/alfonsosanchezbeato/jetson-ubuntu-core

So, let’s start with…

The kernel snap

The Linux kernel that we will use needs some kernel configuration options to be activated, and it is also especially important that it has a modern version of apparmor so snaps can be properly confined. The official Jetson kernel is the 4.4 release, which is quite old, but fortunately Canonical has a reference 4.4 kernel with all the needed patches for snaps backported. Knowing this, we are a git format-patch command away to obtain the patches we will use on top of the nvidia kernel. The patches include also files with the configuration options that we need for snaps, plus some changes so the snap could be successfully compiled on Ubuntu 18.04 desktop.

Once we have the sources, we need, of course, to create a snapcraft.yaml file that will describe how to build the kernel snap. We will walk through it, highlighting the parts more specific to the Jetson device.

Starting with the kernel part, it turns out that we cannot use easily the kernel plugin, due to the special way in which the kernel needs to be built: nvidia distributes part of the needed drivers as separate repositories to the one used by the main kernel tree. Therefore, I resorted to using the nil plugin so I could hand-write the commands to do the build.

The pull stage that resulted is

override-pull: |
  snapcraftctl pull
  # Get kernel sources, which are distributed across different repos
  ./source_sync.sh -k tegra-l4t-r28.2.1
  # Apply canonical patches - apparmor stuff essentially
  cd sources/kernel/display
  git am ../../../patch-display/*
  cd -
  cd sources/kernel/kernel-4.4
  git am ../../../patch/*

which runs a script to retrieve the sources (I pulled this script from nvidia Linux for Tegra -L4T- distribution) and applies Canonical patches.

The build stage is a few more lines, so I decided to use an external script to implement it. We will analyze now parts of it. For the kernel configuration we add all the necessary Ubuntu bits:

make "$JETSON_KERNEL_CONFIG" \
    snappy/containers.config \
    snappy/generic.config \
    snappy/security.config \
    snappy/snappy.config \
    snappy/systemd.config

Then, to do the build we run

make -j"$num_cpu" Image modules dtbs

An interesting catch here is that zImage files are not supported due to lack of a decompressor implementation in the arm64 kernel. So we have to build an uncompressed Image instead.

After some code that stages the built files so they are included in the snap later, we retrieve the initramfs from the core snap. This step is usually hidden from us by the kernel plugin, but this time we have to code it ourselves:

# Get initramfs from core snap, which we need to download
core_url=$(curl -s -H "X-Ubuntu-Series: 16" -H "X-Ubuntu-Architecture: arm64" \
                "https://search.apps.ubuntu.com/api/v1/snaps/details/core?channel=stable" \
               | jq -r ".anon_download_url")
curl -L "$core_url" > core.snap
# Glob so we get both link and regular file
unsquashfs core.snap "boot/initrd.img-core*"
cp squashfs-root/boot/initrd.img-core "$SNAPCRAFT_PART_INSTALL"/initrd.img
ln "$SNAPCRAFT_PART_INSTALL"/initrd.img "$SNAPCRAFT_PART_INSTALL"/initrd-"$KERNEL_RELEASE".img

Moving back to the snapcraft recipe we also have an initramfs part, which takes care of doing some changes to the default initramfs shipped by UC:

initramfs:
  after: [ kernel ]
  plugin: nil
  source: ../initramfs
  override-build: |
    find . | cpio --quiet -o -H newc | lzma >> "$SNAPCRAFT_STAGE"/initrd.img

Here we are taking advantage of the fact that the initramfs can be built as a concatenation of compressed cpio archives. When the kernel decompresses it, the files included in the later archives overwrite the files from the first ones, which allows us to modify easily files in the initramfs without having to change the one shipped with core. The change that we are doing here is a modification to the resize script that allows UC to get all the free space in the disk on first boot. The modification makes sure this happens in the case when the partition is already taken all available space but the filesystem does not. We could remove this modification when these changes reach the core snap, thing that will happen eventually.

The last part of this snap is the firmware part:

firmware:
  plugin: nil
  override-build: |
    set -xe
    wget https://developer.nvidia.com/embedded/dlc/l4t-jetson-tx1-driver-package-28-2-ga -O Tegra210_Linux_R28.2.0_aarch64.tbz2
    tar xf Tegra210_Linux_R28.2.0_aarch64.tbz2 Linux_for_Tegra/nv_tegra/nvidia_drivers.tbz2
    tar xf Linux_for_Tegra/nv_tegra/nvidia_drivers.tbz2 lib/firmware/
    cd lib; cp -r firmware/ "$SNAPCRAFT_PART_INSTALL"
    mkdir -p "$SNAPCRAFT_PART_INSTALL"/firmware/gm20b
    cd "$SNAPCRAFT_PART_INSTALL"/firmware/gm20b
    ln -sf "../tegra21x/acr_ucode.bin" "acr_ucode.bin"
    ln -sf "../tegra21x/gpmu_ucode.bin" "gpmu_ucode.bin"
    ln -sf "../tegra21x/gpmu_ucode_desc.bin" "gpmu_ucode_desc.bin"
    ln -sf "../tegra21x/gpmu_ucode_image.bin" "gpmu_ucode_image.bin"
    ln -sf "../tegra21x/gpu2cde.bin" "gpu2cde.bin"
    ln -sf "../tegra21x/NETB_img.bin" "NETB_img.bin"
    ln -sf "../tegra21x/fecs_sig.bin" "fecs_sig.bin"
    ln -sf "../tegra21x/pmu_sig.bin" "pmu_sig.bin"
    ln -sf "../tegra21x/pmu_bl.bin" "pmu_bl.bin"
    ln -sf "../tegra21x/fecs.bin" "fecs.bin"
    ln -sf "../tegra21x/gpccs.bin" "gpccs.bin"

Here we download some files so we can add firmware blobs to the snap. These files come separate from nvidia kernel sources.

So this is it for the kernel snap, now you just need to follow the instructions to get it built.

The gadget snap

Time now to take a look at the gadget snap. First, I recommend to start by reading great ogra’s post on gadget snaps for devices with u-boot bootloader before going through this section. Now, same as for the kernel one, we will go through the different parts that are defined in the snapcraft.yaml file. The first one builds the u-boot binary:

uboot:
  plugin: nil
  source: git://nv-tegra.nvidia.com/3rdparty/u-boot.git
  source-type: git
  source-tag: tegra-l4t-r28.2
  override-pull: |
    snapcraftctl pull
    # Apply UC patches + bug fixes
    git am ../../../uboot-patch/*.patch
  override-build: |
    export ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
    make p2371-2180_defconfig
    nice make -j$(nproc)
    cp "$SNAPCRAFT_PART_BUILD"/u-boot.bin $SNAPCRAFT_PART_INSTALL"/

We decided again to use the nil plugin as we need to do some special quirks. The sources are pulled from nvidia’s u-boot repository, but we apply some patches on top. These patches, along with the uboot environment, provide

  • Support for loading the UC kernel and initramfs from disk
  • Support for the revert functionality in case a core or kernel snap installation goes wrong
  • Bug fixes for u-boot’s ext4 subsystem – required because the just mentioned revert functionality needs to call u-boot’s command saveenv, which happened to be broken for ext4 filesystems in tegra’s u-boot

More information on the specifics of u-boot patches for UC can be found in this great blog post.

The only other part that the snap has is uboot-env:

uboot-env:
  plugin: nil
  source: uboot-env
  override-build: |
    mkenvimage -r -s 131072 -o uboot.env uboot.env.in
    cp "$SNAPCRAFT_PART_BUILD"/uboot.env "$SNAPCRAFT_PART_INSTALL"/
    # Link needed for ubuntu-image to work properly
    cd "$SNAPCRAFT_PART_INSTALL"/; ln -s uboot.env uboot.conf
  build-packages:
    - u-boot-tools

This simply encodes the uboot.env.in file into a format that is readable by u-boot. The resulting file, uboot.env, is included in the snap.

This environment is where most of the support for UC is encoded. I will not delve too much into the details, but just want to mention that the variables that need to be edited usually for new devices are

  • devnum, partition, and devtype to set the system boot partition, from which we load the kernel and initramfs
  • fdtfile, fdt_addr_r, and fdt_high to determine the name of the device tree and where in memory it should be loaded
  • ramdisk_addr_r and initrd_high to set the loading location for the initramfs
  • kernel_addr_r to set where the kernel needs to be loaded
  • args contains kernel arguments and needs to be adapted to the device specifics
  • Finally, for this device, snappy_boot was changed so it used booti instead of bootz, as we could not use a compressed kernel as explained above

Besides the snapcraft recipe, the other mandatory file when defining a gadget snap is the gadget.yaml file. This file defines, among other things, the image partitioning layout. There is more to it, but in this case, partitioning is the only thing we have defined:

volumes:
  jetson:
    bootloader: u-boot
    schema: gpt
    structure:
      - name: system-boot
        role: system-boot
        type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
        filesystem: ext4
        filesystem-label: system-boot
        offset: 17408
        size: 67108864
      - name: TBC
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 2097152
      - name: EBT
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 4194304
      - name: BPF
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 2097152
      - name: WB0
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 6291456
      - name: RP1
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 4194304
      - name: TOS
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 6291456
      - name: EKS
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 2097152
      - name: FX
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 2097152
      - name: BMP
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 134217728
      - name: SOS
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 20971520
      - name: EXI
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 67108864
      - name: LNX
        type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
        size: 67108864
        content:
          - image: u-boot.bin
      - name: DTB
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 4194304
      - name: NXT
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 2097152
      - name: MXB
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 6291456
      - name: MXP
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
        size: 6291456
      - name: USP
        type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
size: 2097152

The Jetson TX1 has a complex partitioning layout, with many partitions being allocated for the first stage bootloader, and many others that are undocumented. So, to minimize the risk of touching a critical partition, I preferred to keep most of them untouched and do just the minor amount of changes to fit UC into the device. Therefore, the gadget.yaml volumes entry mainly describes the TX1 defaults, with the main differences comparing to the original being:

  1. The APP partition is renamed to system-boot and reduced to only 64MB. It will contain the uboot environment file plus the kernel and initramfs, as usual in UC systems with u-boot bootloader.
  2. The LNX partition will contain our u-boot binary
  3. If a partition with role: system-data is not defined explicitly (which is the case here), a partition which such role and with label “writable” is implicitly defined at the end of the volume. This will take all the available space left aside by the reduction of the APP partition, and will contain the UC root filesystem. This will replace the UDA partition that is the last in nvidia partitioning scheme.

Now, it is time to build the gadget snap by following the repository instructions.

Building & flashing the image

Now that we have the snaps, it is time to build the image. There is not much to it, you just need an Ubuntu One account and to follow the instructions to create a key to be able to sign a model assertion. With that just follow the README.md file in the jetson-ubuntu-core repository. You can also download the latest tarball from the repository if you prefer.

The build script will generate not only a full image file, but also a tarball that will contain separate files for each partition that needs to be flashed in the device. This is needed because unfortunately there is no way we can fully flash the Jetson device with a GPT image, instead we can flash only individual partitions with the tools nvidia provides.

Once the build finishes, we can take the resulting tarball and follow the instructions to get the necessary partitions flashed. As can be read there, we have to download the nvidia L4T package. Also, note that to be able to change the partition sizes and files to flash, a couple of patches have to be applied on top of the L4T scripts.

Summary

After this, you should have a working Ubuntu Core 18 device. You can use the serial port or an external monitor to configure it with your launchpad account so you can ssh into it. Enjoy!

Read more
rsalveti

During the end of October and beginning of November we had the last Linaro Connect for the year. This time we also had it together with the Ubuntu Developer Summit, giving us the opportunity to better discuss the roadmap with both Linaro and the Ubuntu team.

From the Developer Platform team perspective, we had a quite nice week, with demos happening at Monday and Friday (showing people what we’ve been working on), and also sharing some great news with the Ubuntu team, now that Mark Shuttleworth announced that Ubuntu will go to Tablets, TVs and Phones (and ARM for sure will be a huge part of that).

Some nice links and videos of what happened during that week (related with our team):
* Sessions related with the Developer Platform Team (Ubuntu)
* Linaro Demo: Ubuntu Unity with OpenGL ES on Pandaboard
* Linaro Developer Platform Tech Lead Ricardo Salveti Interview at Linaro Connect
* Linaro Connect Q4.11 – Ubuntu LEB tutorial
* Linaro Connect Q4.11 – Interview with Marcin Juszkiewicz

Linaro 11.11 Release

Another quite good achievement for us during November was the 11.11 release.

During this release we had a quite a few great highlights, including some that we were planning for quite a while already:
* Ability to cross build Firefox using Multiarch
* OMAP4 SPL USB Booting, enabling USB boot at Pandaboard
* ARM DS-5 support for the 5.8 release
* CI Builds for Linaro GCC both for cross and native
* And a lot of bug fixes

Now it’s time to get ready to develop the blueprints we’re planning for 11.12, to also make December another great and solid month :-) (will do another post about the 11.12 planning later this one).


Read more
rsalveti

Over the past month I’ve being working with John Rigby to integrate the SMSC95XX and OMAP4 EHCI patches into Linaro U-Boot, so we could deliver the network booting feature for people using Pandaboards.

Those patches are published at the U-Boot mailing list, but still as a working in progress. While we work helping the original developers to get the patches accepted upstream, we also want to deliver the functionality for our users, so all those patches are now integrated at the Linaro U-Boot tree.

You can check the patches by going at http://git.linaro.org/gitweb?p=boot/u-boot-linaro-stable.git;a=shortlog.

Testing with Pandaboard

To make it work properly, besides using Linaro U-Boot you’ll also need to use the upstream X-Loader tree, with one additional patch that’s not yet merged. You can clone the upstream tree from http://gitorious.org/x-loader/x-loader, then just apply the patch http://people.canonical.com/~rsalveti/pxe/0001-omap4-pandaboard-ehci-fref_clkout-per-board-revision.patch and build for the Pandaboard target.

If you just want to test without building your own X-Loader and U-Boot, you can just grab both files from  http://people.canonical.com/~rsalveti:

Building your TFTP + DHCP server for PXE

To build your TFTP + DCHP server just follow the instructions described at https://help.ubuntu.com/community/Desktop/PXE. Don’t worry about the ‘filename “pxelinux.0”;’ line at the dhcpd.conf file, you can remove it.

Then just create your PXE config file at the right place:

$ cat /tftpboot/pxelinux.cfg/0A2A2B0A
default panda-natty
prompt 0
timeout 3

label panda-natty
kernel panda/uImage
append console=ttyO2,115200n8 root=/dev/mmcblk0p2 ro fixrtc vram=48M omapfb.vram=0:24M mem=1G@0x80000000 text earlyprintk=ttyO2
initrd panda/uInitrd

PXE Booting

With the proper X-Loader and U-Boot files in place (at your first SD card partition), and with the TFTP + DHCP server also properly installed, you can just jump and try TFTP/PXE boot.

Stop the U-Boot autoload and call the following commands:

  • setenv pxecfg_ram 0x88000000: location in RAM to load the pxecfg file
  • setenv kernel_ram 0x80000000: location in RAM to load the kernel
  • setenv initrd_ram 0x81600000: location in RAM to load the initrd
  • setenv autoload no: disable autoload while calling bootp (so you can just set up your network without autoboot)
  • usb start: start USB and enables the SMSC95xx ethernet interface
  • bootp: initialize the network, probing the ip address settings from your DHCP server
  • pxecfg get: probe the pxecfg config file
  • pxecfg boot: boot :-)

You should get a similar output as:

Texas Instruments X-Loader 1.5.0 (Jul 11 2011 – 07:52:49)
Reading boot sector
Loading u-boot.bin from mmc

U-Boot 2011.06 (Jul 11 2011 – 02:49:51)

CPU : OMAP4430
Board: OMAP4 Panda
I2C: ready
DRAM: 1 GiB
MMC: OMAP SD/MMC: 0
Using default environment

In: serial
Out: serial
Err: serial
Net: No ethernet found.
Hit any key to stop autoboot: 0
Panda # setenv pxecfg_ram 0x88000000
Panda # setenv kernel_ram 0x80000000
Panda # setenv initrd_ram 0x81600000
Panda # setenv autoload no
Panda # usb start
(Re)start USB…
USB: Register 1313 NbrPorts 3
USB EHCI 1.00
scanning bus for devices… The request port(2) is not configured
EHCI timed out on TD – token=0x80008c80
The request port(2) is not configured
4 USB Device(s) found
scanning bus for storage devices… 0 Storage Device(s) found
scanning bus for ethernet devices… 1 Ethernet Device(s) found
Panda # bootp
Waiting for Ethernet connection… done.
BOOTP broadcast 1
DHCP client bound to address 10.42.43.10
Panda # pxecfg get
missing environment variable: pxeuuid
missing environment variable: ethaddr
Retreiving file: pxelinux.cfg/0A2A2B0A
Waiting for Ethernet connection… done.
Using sms0 device
TFTP from server 10.42.43.1; our IP address is 10.42.43.10
Filename ‘pxelinux.cfg/0A2A2B0A’.
Load address: 0x88000000
Loading: #
done
Bytes transferred = 239 (ef hex)
Config file found
Panda # pxecfg boot
Hit any key to stop autoboot: 0
Label: panda-natty
kernel: panda/uImage
append: console=ttyO2,115200n8 root=/dev/mmcblk0p2 ro fixrtc vram=48M omapfb.vram=0:24M mem=1G@0x80000000 text earlyprintk=ttyO2
initrd: panda/uInitrd
Retreiving file: panda/uInitrd
Waiting for Ethernet connection… done.
Using sms0 device
TFTP from server 10.42.43.1; our IP address is 10.42.43.10
Filename ‘panda/uInitrd’.
Load address: 0x81600000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
############
done
Bytes transferred = 3982715 (3cc57b hex)
Retreiving file: panda/uImage
Waiting for Ethernet connection… done.
Using sms0 device
TFTP from server 10.42.43.1; our IP address is 10.42.43.10
Filename ‘panda/uImage’.
Load address: 0x80000000
Loading: #################################################################
#################################################################
#################################################################
#################################################################
#########################
done
Bytes transferred = 4174480 (3fb290 hex)
## Booting kernel from Legacy Image at 80000000 …
Image Name: Ubuntu Kernel
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 4174416 Bytes = 4 MiB
Load Address: 80008000
Entry Point: 80008000
Verifying Checksum … OK
## Loading init Ramdisk from Legacy Image at 81600000 …
Image Name: Ubuntu Initrd
Image Type: ARM Linux RAMDisk Image (uncompressed)
Data Size: 3982651 Bytes = 3.8 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum … OK
Loading Kernel Image … OK
OK

Starting kernel …

Uncompressing Linux… done, booting the kernel.
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu

This should be enough for you to get your Pandaboard booting with PXE. You can also script these commands at your boot.scr file that U-Boot loads automatically from your SD card, so you don’t have to call them by hand every time you reboot your board.

In case it doesn’t work for you, just ping me (rsalveti) at #linaro on freenode :-)


Read more