Software is most often developed on PC (the host), with the x86-63 architecture. But if it should run on a target devices with the ARM architecture, how can one build it on the host for the target? The usual solution is cross compiling. Many systems are developed to do cross compiling. May be Yocto is the most famous, but it has its own disadvantages:
- Yocto is complicated and needs special knowledge.
- It needs administration.
- It is under constant development, so it is likely that a Yocot recipe that you write today, is not compatible with Yocto of 10 years later.
- Your project will be dependent on Yocto. To use a new version of some package, you should wait till Yocto supports it.
- Most software packages are not tested against cross compiling.
An alternative is to build the software natively. This idea is foremost adhered by Rob Landley with his mkroot project. Watch his excellent talk about it. The target architecture is simulated with QEMU. One installs Linux together with the tool-chain on QEMU. Reboot QEMU to linux, and build your own software. finally, copy the image of the simulated system to your device and reboot the device. That’s it! To speed up the build, we use distcc. It distributes the precompilation on the local network, including the host. The advantages of this method are:
- None of the disadvantages of Cross Compiling.
- It is much like software development on PC.
- You can create a system with as few packages as needed. This is important for safety critical systems.
I have implemented this method for my application: Debian-Linux together with the GNU-toolchain are installed on QEMU. On Debain, I built the Linux from Boundary Devices for the i.MX6 microcontrollers, and rebooted QEMU to that Linux, and built my own software on it. Finally, I copied the image of the simulated system to my i.MX6 device, and it runs! I now explain this procedure, so you can adapt it to your application.
$apt-get install qemu
We use the QEMU machine „virt“, which is a general machine with for the ARM architecture. QEMU also has three „machines“ for boards ARM-boards from Freescale (now NXP).
Installing Debian on QEMU
This article explains how to install Debian on QEMU.
Note. The i.MX6 (and all Cortex-a9 chips) have an FPU unit for floating-point calculations. The Debian port to support that is named armhf (otherwise it is called armel). I guess armel could also run on Cortex-a9.
alternative solution (not recommended)
Download a Debian fertige Image-Dateien für Qemu and follow the instructions.
$qemu-system-arm -M vexpress-a9 -m 1G -redir tcp:5555::22 -kernel vmlinuz-3.2.0-4-vexpress -initrd initrd.img-3.2.0-4-vexpress -drive if=sd,file=debian_wheezy_armhf_standard.qcow2 -append "root=/dev/mmcblk0p2 console=ttyAMA0" -nographic
One gets Debian Wheezy on QEMU with command prompt.
In the file /etc/ssh/sshd_config change the corresponding line:
On the Host
qemu-system-arm -M virt -m 1024 -kernel bd-zImage -initrd initrd.img-4.9.0-4-armmp-lpae -append 'root=/dev/vda2' -drive if=none,file=hda.qcow2,format=qcow2,id=hd -device virtio-blk-device,drive=hd -netdev user,id=mynet,hostfwd=tcp::5555-:22 -device virtio-net-device,netdev=mynet -nographic
ssh root@localhost -p 5555
Installing and Configuring distcc
To speedup the build process on the QEMU-simulated system, we use distcc. distcc distributes the precompilation on the local network, including the Host. For more information look up #man distcc
On the Host:
#sudo apt-get install distcc
To keep distccd (the server) from trying to make TCP cork on the SSH connection (see here):
mv /usr/bin/distccd /usr/bin/distccd-bin
2) Create a new file /usr/bin/distccd, with contents like this:
#!/bin/bash export DISTCC_TCP_CORK=0 exec /usr/bin/distccd-bin $@
chmod 755 /usr/bin/distccd
On the QEMU-Simulated Debian
#sudo apt-get install distcc
Configure distcc as described in „Debian distcc(1) Manpage„.
MASQUERADING, and with shell scripts (do not forget: chmod +x).
But because we want to cross-compile, we should write the complete name of the compiler, so on Host distcc will run the cross compiler and not the native x86 compiler. For example in /usr/lib/distcc/bin/cc:
#!/usr/bin/env bash distcc /usr/bin/arm-linux-gnueabihf-gcc-6 "$@"
Here we assume that on both Host and QEMU distcc is installed in /usr/bin (see the Section SEARCH PATHS).
On /etc/distcc/hosts: email@example.com:/usr/bin/distccd
Then set the SSH Keys on Host (as SSH Server) and QEMU (as SSH client).
To the test the whole thing create a simple c file: ~/temp/test.c
#cd /temp #export DISTCC_VERBOSE=1 #gcc -c temp.c ... compile a.c on firstname.lastname@example.org:/usr/bin/distccd completed ok ... # objdump -p a.o a.o: file format elf32-littlearm private flags = 5000000: [Version5 EABI]
How to do a native build of the Kernel
qemu-debian$git clone https://github.com/boundarydevices/linux-imx6.git bd qemu-debian$ cd bd qemu-debian$git checkout boundary-imx_4.9.x_1.0.0_ga qemu-debian$cp ../defconfig arch/arm/configs/
Some parameters should be added at the end of the existing .defconfig file, so the compiled kernel can be run on the QEMU-simulated Machine.
The needed packages, according to kernel.org, should be installed on the QEMU-simulated Debian
apt-get install gcc make binutils util-linux kmod e2fsprogs jfsutils reiserfsprogs xfsprogs squashfs-tools btrfs-progs pcmciautils quota ppp nfs-common procps udev grub2-common iptables openssl libcrypto++6 bc sphinx-common isdnutils-base apt-get install libncurses5-dev
To get the .config file from the existing _defconfig file (see Kernel Readme by Torvalds):
export ARCH=arm PLATFORM=... make -j 8 defconfig
At the end, to compile:
make -j 8
Note. According to Debian 8.6. Compiling a New Kernel “Loadable module support” should be selected in menuconfig (“Kernel module loader” too, but in new kernel, it is by default selected (see this question).
Running the image on the i.MX6 device
On the Host:
qemu-img convert -O raw hda.qcow2 hda.raw sudo fdisk -l hda.raw Disk hda.raw: 10 GiB, 10737418240 bytes, 20971520 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x5a6f7563 Device Boot Start End Sectors Size Id Type hda.raw1 * 2048 499711 497664 243M 83 Linux hda.raw2 499712 18966527 18466816 8.8G 83 Linux hda.raw3 18968574 20969471 2000898 977M 5 Extended hda.raw5 18968576 20969471 2000896 977M 82 Linux swap / Solaris #499712*512=255852544 sudo mount -o loop,offset=255852544 hda.raw /mnt sudo echo "/dev/mmcblk2p2 / auto defaults,discard 1 1" > /mnt/etc/fstab #/mnt/boot ist schon leer, aber zu sicherstellen. rm /mnt/boot/* sudo cp uImage imx6q-himx0294-ivap.dtb /mnt/boot sudo chmod +rwx /mnt/boot/* cd /mnt tar -cpf ~/bd/rootfs.tar bin boot etc home lib root sbin usr var/tmp var/lib/exim4/berkeleydbvers.txt var/lib/dbus/machine-id umount -l /mnt cd ~/bd scp rootfs.tar root@_device_
One should not use scp to transfer the root file system to the divece, because it messes up the symlinks. cp -a can handel symlinks, but it does not work between two machines.
On the device:
mount /dev/mmcblk2p2 /mnt cd /mnt tar -xpf ~/rootfs.tar -C . mkdir dev lost+found media mnt opt proc run srv sys tmp rm rootfs.tar umount -l /mnt
Native build of the own packages (on QEMU)
apt-get install build-essential linux-headers-4.9.0-3-armmp-lpae autoconf autotools-dev gawk pkg-config valac libtool apt-get install gupnp-tools libupnp-dev libjansson-dev libgpgme-dev libreadline-dev
Note. In preparing the build on Debian, install the libXXX-dev packages instead of the libXXX packages.
autoreconf -ifv ./configure --prefix=/usr make make install
That compiles drtopold, libdrtrace, libdrhip and drbcc.
drbcc --cmd 'heartbeat 65535' --dev=/dev/ttymxc2,921600 drtopold -s1