Using Gentoo to build embedded devices

Introduction

This is my guide on using Gentoo to build an x86-based embedded system. I found Gentoo very good for this because it is source based: installing programs to the embedded system is not much harder than installing on the PC itself (you get to use portage for the embedded system).

Step 1: Build the development environment

First thing we do is build the environment (from where we will build our embedded system). Get yourself a copy of the Gentoo uclibc stage 3 tarball, unpack it into a directory (I call it "embEnv", but you can call it anything):

mkdir ./embEnv
tar -jxvpf ./stage3-x86-uclibc*.tar.bz2 -C ./embEnv
cd ./embEnv
mount --bind /proc ./proc
mount --bind /dev/ ./dev
cp /etc/resolv.conf ./etc/

Now you have an option, you can either use the development system's portage tree (if it is running Gentoo), by doing the following:

mount --bind /usr/portage ./usr/portage

..before chrooting. Alternatively, chroot (see below), then emerge --sync to build your own portage tree in the chroot environment. I used this method because the other one was causing problems (something to do with virtual packages) but note that it does increase the size of your development environment considerably. This method has the advantage that you do not in fact have to be running a Gentoo host to use it (I have built and used this system on Ubuntu Linux with no problems).

We then chroot into our new system:

chroot ./
env-update
source /etc/profile

And we are done, onto the next step.

Step 2: setting up the embedded system

First thing we do is edit our make.conf file to decide what options we want:

# nano -w /etc/make.conf

USE="bitmap-fonts minimal truetype-fonts"
CHOST="i386-gentoo-linux-uclibc"
CFLAGS="-march=i586 -Os -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
FEATURES="buildpkg"

INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="vesa"
UCLIBC_CPU="586"

In this example I am building an embedded system with an Xserver (for use as a thin-client), so I added font support in the USE flags, as well as "INPUT_DEVICES" and "VIDEO_CARDS". If you will not have any graphics, you can omit these. The UCLIBC_CPU option is set to 586 for me as I am building this for a P1 75MHz, you can pick one of the others, consisting of:

Once this is set, you are ready to build your embedded system.

Step 3: Building the embedded system

Now we create a working directory where we will store the embedded system, I will call it "rootfs"

mkdir ./rootfs

Once this is done, we proceed to emerge the basics which the OS needs to run. We use the $ROOT variable to tell portage where to install the packages:

ROOT=./rootfs emerge baselayout-lite uclibc busybox -av

The "-av" command lets you view what you are installing, and what use flags you are using, so that if you need to make any changes, you can pause the emerge. and either add them to the make.conf file, or set them using the $USE variable, like so:

USE="+X -mmx floppyboot make-symlinks" ROOT=./rootfs emerge baselayout-lite uclibc busybox

uclibc is merged because it is required, busybox provides the common tools (cp,mv,sh,etc...) and baselayout sets up the basic filestructure (usr,bin,proc,dev,etc...). the "make-symlinks" option simplifies the building, as it will automatically add symlinks in your system (for busybox, which is a single binary that provides all the normal programs). The advantage is that it is faster and easier to do, while the disadvantage is that you lose control over what is linked.

Once it is done emerging, chroot into it and setup usernames and passwords:

chroot ./rootfs /bin/sh
passwd root

#add a normal user if you wish
useradd username
mkdir /home/username
chown username /home/username
passwd /home/username

Note: Remember to leave the rootfs environment before continuing (if you executed the command above).

Once this is done and everything is ok, we proceed with installing the software we need.

First we remove bits from the rootfs which are not needed:

rm -Rf ./rootfs/var/db/pkg/*
rm -Rf  ./rootfs/var/lib/portage/

Now we emerge the kernel (not to the rootfs):

emerge vanilla-sources

We go to /usr/src/linux, and configure the kernel (usual make menuconfig, etc...). I decided to not have the ability to load modules (all static), as this both reduces the (overall) size (the kernel itself does end up bigger) and complexity of the system (no need to emerge and configure module-init-tools for example). I also removed everything that was not needed to make the kernel as small as possible. Once done, save your configuration (but do not install yet).

Now we can install what applications we need:

KERNEL_DIR=/usr/src/linux USE="-X " ROOT=./rootfs PORTAGE_STRIP_FLAGS=" --strip-unneeded -R .comment" emerge pcmcia-cs -av

In this example I am emerging pcmcia services as my thin client is a laptop with no built in network card. The PORTAGE_STRIP_FLAGS tells portage what should not be installed from the package, this reduces space wastage from unrequired package components.
The "KERNEL_DIR" is needed for those packages that need to be built against the kernel sources (such as pcmcia-cs), this is why we configured and prepared the kernel beforehand.

Repeat the command above for all programs you wish to install. Also, I recommend that you edit any configuration files (e.g Xorg.conf in my case) at this point.

Step 4: Setup init scripts

Now you have installed and configured everything, it's time for your OS's initialisation. Linux works by booting the kernel, which then mounts the root filesystem, and executes the contents of a file known as inittab (in /etc). This file launches all the programs needed by the OS either in the form of rc/other scripts or direct execution.

First thing we need to do is setup our rc-scripts. As this system has pcmcia support, we do have a pcmcia rc script. We link it to a number like so:

ln /etc/init.d/pcmcia /etc/init.d/S00.sh

The number indicates in which order to run the scripts, so in this case pcmcia will be run first. If you look, you will notice that there are in fact two rc-script categories, SXX (as above) and KXX. The S-series deals with bootup, while the K-series for shutdown. So if we want our system to be cleanly shutdown we do the following:

ln /etc/init.d/pcmcia /etc/init.d/K00.sh

This means that pcmcia will be the first to be shutdown.

I also added a line to the /etc/inittab file, after:

#now run any rc-scripts
::sysinit:/etc/init.d/rcS

consisting of:

#execute custom script
::sysinit:/etc/initscript.sh

Of course, you must create this file (and set it to be executable), so :

touch /etc/initscript.sh
chmod a+rx /etc/initscript.sh

I use this file to setup the rest of the system (In this case, it will provide a menu to let the user choose what kind of X-system he wants (local, remote broadcast or remote query).

Step 5: Set Paritions

Every Linux system needs to know what to mount, and how. This is defined in a file called /etc/fstab. You need to edit this according to your setup. We have a single root partition, so this fstab looks like this:

 

Step 6: Set it to boot

From here on you get to make a choice. How do you want to boot your system?

The options are as follows:

Disk based booting

This is the way most general Linux distributions boot: you have a bootloader (e.g. Grub or LiLo) installed in the disks MBR, and the rootfs in a bootable partition.

First thing you need to do is emerge your bootloader to the development system. I will use Grub in my example:

emerge -av grub

Then emerge a copy to the rootfs:

ROOT=/rootfs/ emerge -av grub

The next thing you need to do is compile the kernel that you configured earlier on, so:

cd /usr/src/linux
make bzImage
And wait for it to finish. Once done, we need to copy over the Grub files to the bootable partition of your embedded system:
cp /boot/ /rootfs/ -R

And then we edit grub.conf:

nano -w /rootfs/boot/grub/menu.lst

One thing to realise is that on most computers, the device that is being booted from is shown as the primary device (i.e. root(0,0) ) but this is not always the case, you may find you need to fiddle with the root(hdX,X) and root=/dev/XXX variables to get it to boot. I used the following:

# Which listing to boot as default. 0 is the first, 1 the second etc.
default 0

# How many seconds to wait before the default listing is booted.
timeout 0

title=kernel 2.4.32
# Partition where the kernel image (or operating system) is located
root (hd0,0)
kernel /boot/bzImage root=/dev/hda4

And now copy over the compiled kernel:

cp /usr/src/linux/arch/i386/boot/bzImage /rootfs/boot/

And the final thing, install grub into the mbr:

grub-install --root-directory=/rootfs/ /dev/sdb4

...where "sdb4" is the partition on your drive which you selected as "bootable".

Floppy based booting

Before you start, make sure you emerged the syslinux package. The first thing you will need is a formatted and known-good floppy disk, once you have one of these, format it with msdos:

mkfs.msdos /dev/fd0

(Replace /dev/fd0 with your path to the floppy drive). Once this is done, mount your floppy, and copy over the kernel you want booted:

mount /dev/fd0 /mnt/floppy
cp /path/to/bzImage /mnt/floppy

Then we create a configuration file called SYSLINUX.CFG, and place it in the root of the floppy disk:

nano -w /mnt/floppy/SYSLINUX.CFG

...where we will set our configuration options. My SYSLINUX.CFG looks like this:

LABEL kernel_2.4.32

#The default kernel and options, is ran if SYSLINUX boots automatically.
DEFAULT bzImage


#APPEND allows to add options to the kernel for both auto and manual operation, acts like "APPEND" in lilo
APPEND root=/dev/sda4

More information is available at the SYSLINUX faq. We then install the syslinux bootloader:

syslinux -s /dev/fd0