Native build on QEMU instead of Yocto

Software is most often developed on PC (the host), with the x86-63 architecture. But if it should run on a target devices with the ARM architecture, how can one build it on the host for the target? The usual solution is cross compiling. Many systems are developed to do cross compiling. May be  Yocto is the most famous, but it has its own disadvantages:

  • Yocto is complicated and needs special knowledge.
  • It needs administration.
  • It is under constant development, so it is likely that a Yocot recipe that you write today, is not compatible with Yocto of 10 years later.
  • Your project will be dependent on Yocto. To use a new version of some package, you should wait till Yocto supports it.
  • Most software packages are not tested against cross compiling.

An alternative is to build the software natively. This idea is foremost adhered by Rob Landley with his mkroot project. Watch his excellent talk about it. The target architecture is simulated with QEMU. One installs Linux together with the tool-chain on QEMU. Reboot QEMU to linux, and build your own software. finally, copy the image of the simulated system to your device and reboot the device. That’s it! To speed up the build, we use distcc. It distributes the precompilation on the local network, including the host. The advantages of this method are:

  • None of the disadvantages of Cross Compiling.
  • It is much like software development on PC.
  • You can create a system with as few packages as needed. This is important for safety critical systems.

I have implemented this method for my application: Debian-Linux together with the GNU-toolchain are installed on QEMU. On Debain, I built the Linux from Boundary Devices for the i.MX6 microcontrollers, and rebooted QEMU to that Linux, and built my own software on it. Finally, I copied the image of the simulated system to my i.MX6 device, and it runs! I now explain this procedure, so you can adapt it to your application.

Installing QEMU

$apt-get install qemu

We use the QEMU machine „virt“, which is a general machine with for the ARM architecture. QEMU also has three „machines“ for boards ARM-boards from Freescale (now NXP).

Installing Debian on QEMU

This article explains how to install Debian on QEMU.

Note. The i.MX6 (and all Cortex-a9 chips) have an FPU unit for floating-point calculations. The Debian port to support that is named armhf (otherwise it is called armel). I guess armel could also run on  Cortex-a9.

alternative solution (not recommended)

Download a  Debian fertige Image-Dateien für Qemu and follow the instructions.

$qemu-system-arm  -M vexpress-a9 -m 1G -redir tcp:5555::22 -kernel vmlinuz-3.2.0-4-vexpress -initrd initrd.img-3.2.0-4-vexpress -drive if=sd,file=debian_wheezy_armhf_standard.qcow2 -append "root=/dev/mmcblk0p2 console=ttyAMA0" -nographic

One gets Debian Wheezy on QEMU with command prompt.

SSH Configuration

This article

On QUME

In the file /etc/ssh/sshd_config change the corresponding line:

PermitRootLogin yes

On the Host

qemu-system-arm -M virt -m 1024   -kernel bd-zImage -initrd initrd.img-4.9.0-4-armmp-lpae   -append 'root=/dev/vda2' -drive if=none,file=hda.qcow2,format=qcow2,id=hd   -device virtio-blk-device,drive=hd -netdev user,id=mynet,hostfwd=tcp::5555-:22  -device virtio-net-device,netdev=mynet -nographic

and then

ssh root@localhost -p 5555

Installing and Configuring distcc

To speedup the build process on the QEMU-simulated system, we use distcc. distcc distributes the precompilation on the local network, including the Host.  For more information look up #man distcc

On the Host:

#sudo apt-get install distcc

To keep distccd (the server) from trying to make TCP cork on the SSH connection (see here):

1)

mv /usr/bin/distccd /usr/bin/distccd-bin

2) Create a new file /usr/bin/distccd, with contents like this:

#!/bin/bash

export DISTCC_TCP_CORK=0

exec /usr/bin/distccd-bin $@

3)

chmod 755 /usr/bin/distccd

On the QEMU-Simulated Debian

#sudo apt-get install distcc

Configure distcc as described in „Debian distcc(1) Manpage„.

MASQUERADING, and with shell scripts (do not forget: chmod +x).

But because we want to cross-compile, we should write the complete name of the compiler, so on Host distcc will run the cross compiler and not the native x86 compiler. For example in /usr/lib/distcc/bin/cc:

#!/usr/bin/env bash
distcc /usr/bin/arm-linux-gnueabihf-gcc-6 "$@"

Here we assume that on both Host and QEMU distcc is installed in /usr/bin (see the Section SEARCH PATHS).

On host

On /etc/distcc/hosts: ruhollah@172.29.23.111:/usr/bin/distccd


Then set the SSH Keys on Host (as SSH Server) and QEMU (as SSH client).

To the test the whole thing create a simple c file: ~/temp/test.c

#cd /temp
#export DISTCC_VERBOSE=1
#gcc -c temp.c
...
compile a.c on ruhollah@172.29.23.111:/usr/bin/distccd completed ok
...
# objdump -p a.o
a.o:     file format elf32-littlearm
private flags = 5000000: [Version5 EABI]

How to do a native build of the Kernel

qemu-debian$git clone  https://github.com/boundarydevices/linux-imx6.git bd
qemu-debian$ cd bd
qemu-debian$git checkout boundary-imx_4.9.x_1.0.0_ga
qemu-debian$cp ../defconfig arch/arm/configs/

Some parameters should be added at the end of the existing .defconfig file, so the compiled kernel can be run on the QEMU-simulated Machine.

The needed packages, according to kernel.org, should be installed on the QEMU-simulated Debian

apt-get install gcc make binutils util-linux kmod  e2fsprogs jfsutils reiserfsprogs xfsprogs squashfs-tools btrfs-progs  pcmciautils quota ppp nfs-common procps udev grub2-common iptables  openssl libcrypto++6 bc sphinx-common isdnutils-base
apt-get install libncurses5-dev

To get the .config file from the existing _defconfig file (see Kernel Readme by Torvalds):

export ARCH=arm PLATFORM=...
make -j 8 defconfig

At the end, to compile:

make -j 8

Note. According to Debian 8.6. Compiling a New Kernel “Loadable module support” should be selected in menuconfig (“Kernel module loader” too, but in new kernel, it is by default selected (see this question).

SSH connection from Host to Qemu (see here. See also qemu.org).

Running the image on the i.MX6 device

On the  Host:

qemu-img convert -O raw hda.qcow2 hda.raw

sudo fdisk -l hda.raw
Disk hda.raw: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5a6f7563

Device     Boot Start      End Sectors Size Id Type
hda.raw1   * 2048   499711 497664 243M 83 Linux
hda.raw2          499712 18966527 18466816  8.8G 83 Linux
hda.raw3        18968574 20969471  2000898 977M 5 Extended
hda.raw5        18968576 20969471  2000896 977M 82 Linux swap / Solaris

#499712*512=255852544
sudo mount -o loop,offset=255852544 hda.raw  /mnt
sudo echo "/dev/mmcblk2p2        / auto defaults,discard      1 1" > /mnt/etc/fstab
#/mnt/boot ist schon leer, aber zu sicherstellen.
rm /mnt/boot/*
sudo cp uImage imx6q-himx0294-ivap.dtb /mnt/boot
sudo chmod +rwx /mnt/boot/*

cd /mnt
tar -cpf ~/bd/rootfs.tar bin boot etc home lib root sbin usr var/tmp var/lib/exim4/berkeleydbvers.txt var/lib/dbus/machine-id
umount -l  /mnt

cd ~/bd
scp rootfs.tar root@_device_

One should not use scp to transfer the root file system to the divece, because it messes up the symlinks.  cp -a can handel symlinks, but it does not work between two machines.

On the device:

mount /dev/mmcblk2p2 /mnt
cd /mnt
tar -xpf ~/rootfs.tar -C .
mkdir dev   lost+found media  mnt opt proc run   srv sys tmp
rm rootfs.tar
umount -l /mnt

Native build of the own packages (on QEMU)

apt-get install build-essential linux-headers-4.9.0-3-armmp-lpae autoconf autotools-dev gawk pkg-config valac libtool
apt-get install gupnp-tools libupnp-dev libjansson-dev libgpgme-dev libreadline-dev

Note. In preparing the build on Debian, install the libXXX-dev  packages instead of the libXXX packages.

autoreconf -ifv
./configure --prefix=/usr
make
make install

That compiles drtopold, libdrtrace, libdrhip  and drbcc.

drbcc --cmd 'heartbeat 65535' --dev=/dev/ttymxc2,921600
drtopold -s1

How to build the Debian Packages

Documentation, Tutorial

Werbeanzeigen

Die Autofalle!

IMG_20181102_100632
Anfang des November 2019

Jeden Monat ereignet sich einen schwierigen Unfall an der Einmündung

Rudower Chaussee/Wegedornstraße in Berlin-Adlershof, in der Nähe meiner Uni. Und das läuft seit Jahren! Deshalb halte ich die Einmündung als eine Autofalle!

Das Foto zeigt den letzten davon. Schau mal wie ein Auto gegen die Ampel gefahren ist. Obwohl beide Autos  erheblich geschändet und vielleicht nicht mehr Verbrauchbar sind, zum Glück gab es keinen Opfer.

Es liegt auf der Hand dass es bei dem Entwurf  der Einmündung ein Problem gibt.
Was ist die (etwa fachlich) Beschreibung des Problems?
Wie viel Opfer oder Schwerverletzte insgesamt gab es da bis Heute?
Was sind die andere „Autofallen“ in Berlin?
Können die betroffene solche Autofalle, unter anderem die Autoversicherungen, das Land Berlin als Schuld halten (zumindest wegen Untätigkeit) und einen Schadenersatz fordern?
Ich hoffe die zuständige Behörde vor dem nächstem Unfall was tut!

Was ist das?

Wann ein Flugzeug über uns in Himmel mal flog, und man sieht viel davon in Berlin, ich zeigte es dem Schams (meinem Sohn, damals 18 Monat alt), und fragte ihn in einem übertreibend bewunderten Ton „Was ist das?“. Er wunderte sich, mit ausgestrecktem Arm wies er auf dem Ding im Himmel und fragte „Was ist das?“.
Nun, Schams hatte seinen zweiten Geburtstag im September. Jemand hat ihm die Sache erklärt. Seitdem blickt er mal hin und erwidert die Frage mit „Flugzeug“ und spielt weiter.
Ist ein Flugzeug wirklich so selbstverständlich?

FlugplatzJohannisthalAmStartplatz
Am Startplatz des Flugplatzes Johannisthal, nach Februar 1912