Raspberry PI 3 — Compile your own 64-bit Linux Kernel

I recently downloaded latest Raspbian image (2017-03-02-raspbian-jessie.img) from there website and updated it on RPI3. While i was working on the board i realized that this is my first 64-bit ARM machine and i immediately checked the kernel version.  To my absolute surprise the device was running a 32-bit kernel instead of a 64-bit kernel. It was no hindrance to the work i did and should be same for most of the people who are using the Raspberry PI 3. But i kinda felt that i not doing justice to the hardware or at least take advantage of the extra address space that is given by hardware.

So I decided to update at-least the kernel to 64-bit version without changing the File system and bootloader (as of now).

Kernel Code base .. Not vanilla

Normally i always choose vanilla stable kernels due to personal preference (check my other blogs related to kernel as all will be based on vanilla kernels only) and professionally too this is much more helpful when talking to customers but here i was made to pull the code from raspberry pi repository. For quite good reasons (explained below) i needed to pull the code from their official repository.

Pls look at the end for details about vanilla kernel.

Getting the kernel code base

kasi@Vostro ~/rpi $ git clone https://github.com/raspberrypi/linux.git

We can clone the latest code base from the RPI repository using the above command. Let us move into the directory and check a few things.

kasi@Vostro ~/rpi $ cd linux
kasi@Vostro ~/rpi/linux $
kasi@Vostro ~/rpi/linux $ git branch
* rpi-4.9.y
kasi@Vostro ~/rpi/linux $

The git branch command shows that the current branch is 4.9.y. The repo has a lot of remote branches. Since the latest vanilla kernel version is around 4.10.x I wanted to checkout the 4.10.y branch and then try to build the kernel using that branch.

kasi@Vostro ~/rpi/linux $ git checkout rpi-4.10.y
Checking out files: 100% (11532/11532), done.
Branch rpi-4.10.y set up to track remote branch rpi-4.10.y from origin.
Switched to a new branch ‘rpi-4.10.y‘

kasi@Vostro ~/rpi/linux $ git branch
* rpi-4.10.y
rpi-4.9.y
kasi@Vostro ~/rpi/linux $

Setup the cross tool chain path

In my previous blog post i have created a custom cross toolchain which is capable of creating 64-bit binaries. 64-bit kernel compilation for RPI 3 was the main motivation for creating the tool chain in the first place. Pls check here to know more.

I am going to export the path of the cross tool chain and then use it to compile the kernel which has been pulled. This is for pure experimentation only. You can always use the toolchain provided by various vendors like linaro, Codesourcery or others and follow the exact same procedure without any issues.

kasi@Vostro ~/rpi/linux $ export PATH=/opt/aarch64/bin/:$PATH

Let us also set the ARCH and CROSS_COMPILE environment variables.

kasi@Vostro ~/rpi/linux $ export CROSS_COMPILE=aarch64-linux-gnu-
kasi@Vostro ~/rpi/linux $ export ARCH=arm64

Since the environment is set up, the kernel be configured and then compiled to create the 64-bit kernel image.

Time to Configure the Big boss

As with other packages Linux kernel also needs to configured. When i searched the latest vanilla kernel (4.10.9 at the time of this writing) there was no specific config file for RPI3 but a default one for 64-bit machines was available. But the kernel pulled from raspberry PI repository had the config file which was specific for RPI3. So let us use it configure the kernel. This is very similar to the way in which i have configured kernels in my previous blogs.

Only two config files are available in the configs directory and we need to use the “bcmrpi3_defconfig” file to configure the kernel..

kasi@Vostro ~/rpi/linux $ ls -l arch/arm64/configs/
total 44
-rw-r–r– 1 kasi kasi 30895 Apr 8 17:21 bcmrpi3_defconfig
-rw-r–r– 1 kasi kasi 10397 Apr 8 17:21 defconfig
kasi@Vostro ~/rpi/linux $ make bcmrpi3_defconfig

The above make command creates the .config file. If you want to customize the kernel you can always do it using the “make menuconfig” command and then save the changes to the .config file. But as of now I want to build the kernel using the default config.

Make and wait

kasi@Vostro ~/rpi/linux $ make -j 8

The compilation completed successfully and both kernel image and dtb files were created at the expected locations. Pls note as of now I am not going to install the compiled kernel modules.

Location of the kernel image

kasi@Vostro ~/rpi/linux $ ls -l arch/arm64/boot/
total 16928
drwxr-xr-x 26 kasi kasi 4096 Apr 9 12:26 dts
-rw-r–r– 1 kasi kasi 12509696 Apr 9 12:43 Image
-rw-r–r– 1 kasi kasi 4941919 Apr 9 12:43 Image.gz
-rw-r–r– 1 kasi kasi 1562 Apr 7 01:16 install.sh
-rw-r–r– 1 kasi kasi 1159 Apr 7 01:16 Makefile
kasi@Vostro ~/rpi/linux $

Location of the dtbs

kasi@Vostro ~/rpi/linux $ ls -l arch/arm64/boot/dts/broadcom/*.dtb
-rw-r–r– 1 kasi kasi 23897 Apr 9 12:25 arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b.dtb
-rw-r–r– 1 kasi kasi 16178 Apr 9 12:25 arch/arm64/boot/dts/broadcom/bcm2837-rpi-3-b.dtb
kasi@Vostro ~/rpi/linux $

We need to copy these files into the boot directory of the already partitioned SD Card. The already loaded raspbian image has created the partitions in the Micro SD card for us.

Mounted Micro SD card

dev/mmcblk0p1 on /media/kasi/boot type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)

/dev/mmcblk0p2 on /media/kasi/139ca52c-d45e-44ca-8b8c-12c0af0c9cf5 type ext4 (rw,nosuid,nodev,relatime,data=ordered,uhelper=udisks2)

Pls check your mount directories in your machine as it will be different. The above capture is just for reference.

Image and DTB copy

Now let us copy the kernel image and the corresponding dtbs into the boot directory of the Micro SD card.

kasi@Vostro ~/rpi/linux $ cp arch/arm64/boot/Image /media/kasi/boot/kernel8.img
kasi@Vostro ~/rpi/linux $ cp arch/arm64/boot/dts/broadcom/bcm2837-rpi-3-b.dtb /media/kasi/boot/
kasi@Vostro ~/rpi/linux $ cp -r arch/arm64/boot/dts/overlays /media/kasi/boot/
kasi@Vostro ~/rpi/linux $ sync

After copying kernel image, dtbs and overlay directory we need to make one more change that is to edit the “config.txt” file in boot partition.

kasi@Vostro ~/rpi/linux $ sudo vim /media/kasi/boot/config.txt

In this file, add “kernel=kernel8.img” entry if not already available. Else edit the kernel parameter to the image name which is copied in the Micro SD card.

After making all the above said changes I booted the RPI3 but it didn’t boot up properly. I was not able verify it using the serial cable as i didn’t connect any. So i did some guessing and found out that the device tree file which i copied was not the proper one which is interesting. Only two dtb files were available in the dts directory, one i have already tried and it didn’t work as expected. The other one i need to try. So let us copy the other dtb file into the Micro SD card.

kasi@Vostro ~/rpi/linux $ cp arch/arm64/boot/dts/broadcom/bcm2710-rpi-3-b.dtb /media/kasi/boot/

Now when the RPI3 was booted i was able to access it through SSH. See below for complete details.

SSH Issues

Initially even with the raspbian image i was not able to access RPI3 through SSH. But i edited the “rc.local” file in /etc directory of the Micro SD card and then added /etc/init.d/ssh start before the exit 0 (at the end) in that file. This made sure that SSH server was getting started on boot up and RPI3 was listening on port 22.

kasi@Vostro ~/rpi/linux $ ssh pi@192.168.10.10
pi@192.168.10.10’s password:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Mar 3 16:27:16 2017

SSH is enabled and the default password for the ‘pi’ user has not been changed.
This is a security risk – please login as the ‘pi’ user and type ‘passwd’ to set a new password.

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.10.8-v8+ #1 SMP PREEMPT Sun Apr 9 12:43:39 IST 2017 aarch64 GNU/Linux
pi@raspberrypi:~ $ cat /proc/cpuinfo
processor : 0
BogoMIPS : 38.40
Features : fp asimd evtstrm crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4

processor : 1
BogoMIPS : 38.40
Features : fp asimd evtstrm crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4

processor : 2
BogoMIPS : 38.40
Features : fp asimd evtstrm crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4

processor : 3
BogoMIPS : 38.40
Features : fp asimd evtstrm crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4

pi@raspberrypi:~ $

The kernel version is 4.10.8 and it is 64-bit. The stable vanilla kernel version at the time of this writing is 4.10.9. So this kernel is pretty much close and more importantly this is 64-bits.

Default kernel running in the box

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux

More info about vanilla kernel for RPI3

When I tried to check the vanilla kernel (4.10.8) i kinda found out that the support was not there and below are the details which i found out.

root@kasi-desktop:/Sourcefiles/rpi/linux-4.10.8# ls arch/arm64/configs/ -l
total 12
-rw-rw-r– 1 root root 11739 Mar 31 14:03 defconfig
root@kasi-desktop:/Sourcefiles/rpi/linux-4.10.8#

Anyway i successfully compiled the vanilla kernel using the defconfig available for 64-bit ARM machines and then loaded it into the RPI3. But RPI3 didn’t boot up. As i have already shared i didn’t have serial cable attached to the RPI3 so i was not able to debug more. I shall connect the serial cable to debug and find out more and share it here.

Updated Kernel — 4.11.0-rc6

I switched to the rpi-4.11-y branch of the RPI repository and then compiled the latest kernel code.

root@kasi-desktop:/Sourcefiles/rpi/linux# git branch
* rpi-4.11.y
rpi-4.8.y
root@kasi-desktop:/Sourcefiles/rpi/linux#

The steps are the same except for some additional commands to compile the modules, install them and then copy them to the RPI3 file-system. If you are compiling on top the already compiled kernel (4.10.8) then you need to give “make distclean” so that no residue files are there and we can absolutely make sure that the build is starting from scratch.

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcmrpi3_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j4
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j4 modules

Below i have given a specific directory for the installation of the kernel modules.

make INSTALL_MOD_PATH=../modules/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j4 modules_install

After executing the above commands the kernel modules were copied to the “modules” directory (which i have created exclusively for this).

Now you can copy the “lib/modules/4.11.0-rc6-v8+/” directory to the Micro SD card’s file-system partition. This makes the compiled kernel modules available in case they need to be loaded dynamically.

cp -r /Sourcefiles/rpi/modules/lib/modules/4.11.0-rc6-v8+/ /media/kasi/139ca52c-d45e-44ca-8b8c-12c0af0c9cf5/lib/modules

The procedure to copy the kernel image, dtbs and overlay remains the same. Since we have already edited the “config.txt” file we don’t need to redo it again.

Let us login through SSH and check it

kasi@intel:~/git/iso$ ssh pi@192.168.1.132
pi@192.168.1.132’s password:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Apr 11 06:17:09 2017

SSH is enabled and the default password for the ‘pi’ user has not been changed.
This is a security risk – please login as the ‘pi’ user and type ‘passwd’ to set a new password.

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.11.0-rc6-v8+ #1 SMP PREEMPT Tue Apr 11 11:47:17 IST 2017 aarch64 GNU/Linux

Few points

Pls note that the commands which are available in the Micro SD cards file-system partition are all 32-bit binaries. For example, “ls” command is 32-bit ELF file which is perfectly fine to use in a machine which is running a 64-bit kernel.

pi@raspberrypi:~ $ file /bin/ls
/bin/ls: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=5e052e8de057d379ab51d4af510ad9318fe77b46, stripped
pi@raspberrypi:~ $

Others ways to get 64-bit kernel in your RPI3.