0x5ca1ab1e: July 2013

This post has been a long one. I have to confess that I jumped into the deep end on this one, which was the plan, but I didn't know how deep this would go.

In my quest to downsize several older components on my home network, I came to the firewall/router. Currently I am running a dual Zeon Gentoo Linux server to handle all my server needs in one system, including routing. Having moved all the services off of the server, all that was left was it's routing/firewall setup. I decided the Gumstix based COM devices would make a good fit with significantly less heat issues and minimal power requirements. This would also be a good learning experience, being my first real workings with embedded ARM.

So here's the configuration I'm going with. The Janus module was a later add-on, and isn't required. Add in the power adapter and a 8GB MicroSD card and you'll total about $200.

The first major road block that I ran into was the lack of documentation. Originally I was planning on loading the system with either a Gentoo based system, or a Buildroot system. The online documentation mentions that Buildroot is no longer supported, but no details are given for why. It also mentions that Yocto is the only supported build environment.

The Overo systems come with some NAND Flash pre-loaded with a working distribution, unfortunately there's very little online for what capabilities exist in the built-in system. Since the core COM board is designed to be attached to very different expansion modules, I understand why it didn't start the network on boot. Since I didn't start with a Janus module, I was relying on getting the network cards up and accessing the system through there. So I was starting out running blind.

The next option was to follow the Gumstix Getting Started Guide to get a bootable MicroSD Card using their prebuilt images. Even this caused issues due to the documentation. If you follow their guide, you will notice they refer to u-boot.bin, but the prebuilt images contain a u-boot.img, if you copy u-boot.img to u-boot.bin then the system won't boot. I'm assuming the "u-boot.bin" is just a typo, but it appears in the guide 6 times, maybe a copy/paste mistake. So after fixing that little issue, the system boots and I see dhcp requests from a "00:15:C9" MAC prefix. So a quick power off to copy some ssh authorized keys, and a boot later and I can ssh into the system.

Now that I'm in the system, I can dig around a bit and see what this thing can do. Unfortunately the prebuilt images don't provide what I need in a router. The kernel is missing some networking features that I need, IPsec support, IPv6 is only partially implemented, and no NAT. The system is also missing some packages that I need, iproute2, ipsec-tools being the most important. The other thing that I really didn't like is the full dbus/udev/systemd setup. Personally, I think this is extreme overkill for such a small system. So now I need a new kernel, and a new rootfs.

This is where the bulk of my failures accumulated. Between the documentation being misleading, or just out of date and the same steps don't work anymore, I believe I have 8-12 half dead builds of various kernel/rootfs images lying around. I'm going to skip over the failures and show what worked. First we will start with the getting the Gumstix Yocto build system up and running. Most of the steps are from the Gumstix Wiki and the Gumstix Github Repo. I highly recommend running all this in both screen and script so you don't miss anything.

First we need the repo command. This appears to be a git wrapper of sorts.

Get repo...

curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > repo
chmod a+x repo
sudo mv repo /usr/local/bin
repo --help

Now we can use the repo command to clone the Yocto build environment.

Get Yocto...

mkdir yocto
cd yocto
repo init -u git://github.com/gumstix/Gumstix-YoctoProject-Repo.git
repo sync

Now we can build the default kernel. This step will take a while, and depending on your build host software it can fail at several points. I recommend going over to the Yocto Project Reference Manual and make sure you have the required build packages installed before continuing. Another warning, other packages may also cause issues. I'm using Gentoo as my build host, and I ended up running into an issue with sys-devel/make-3.82-r4, and I had to downgrade to <sys-devel/make-3.82 to complete the kernel build.

TEMPLATECONF="meta-gumstix-extras/conf" source ./poky/oe-init-build-env
bitbake virtual/kernel

It's pretty much clear sailing from here. The next part is just configuring the kernel and re-building. I've uploaded my Gumstix kernel .config file so you can use that as a reference. The trick to this part is the documentation on the Wiki is using an older 2.6 kernel, where in my case it's a 3.5 kernel. So if the kernel changes, you may need to change where you copy your custom .config file for the re-build. Use the provided find command to narrow down your search if the location changes.

Reconfigure and build the new kernel

bitbake -c menuconfig virtual/kernel
#Find your new custom .config file
find ./tmp/work/ -path "*/git/.config"
#Find your new custom .config file
find ../poky/meta-gumstix/recipes-kernel -name "defconfig"
#Copy the .config over to the defconfig
cp ./tmp/work/overo-poky-linux-gnueabi/linux-gumstix-3.5-r0/git/.config ../poky/meta-gumstix/recipes-kernel/linux/linux-gumstix-3.5/overo/defconfig
bitbake -c clean virtual/kernel
bitbake virtual/kernel

Your new kernel will be ./tmp/deploy/images/uImage-*. I recommend making a backup of the kernel and the .config for it so you have it for reference later in case you need to update or change kernel features later on. Also, make sure you compare your kernel to the live running system with the prebuilt kernel. In my case, I made the feature changes that I needed, but then also did a significant amount of cleanup and removal of any features that I can't use. I also enabled CONFIG_DEVTMPFS since I don't really have any way to modify the hardware once the system is booted. The live system wasn't running any modules in the kernel, so I also removed all modules and module support. The benefit for this is now my kernel is completely independent of my rootfs.

Now to follow Gumstix Create a Bootable MicroSD Card guide to setup the MicroSD card.

Setup the MicroSD card

sudo mkfs.vfat -F 32 /dev/sdd1 -n boot
sudo mke2fs -j -L rootfs /dev/sdd2
mkdir boot rootfs
sudo mount /dev/sdd1 boot
sudo mount /dev/sdd2 rootfs
#Download the prebuilt images
wget http://yocto.s3-website-us-west-2.amazonaws.com/Releases/2013-06-16/overo/dev/MLO-overo-2012.10
wget http://yocto.s3-website-us-west-2.amazonaws.com/Releases/2013-06-16/overo/dev/u-boot-overo-2012.10-r0.img
#According to the guide: "For Overo COMs only: MLO (the second-stage bootloader binary) must be copied first"
sudo cp ./MLO-overo-2012.10 boot/MLO
sudo cp ./u-boot-overo-2012.10-r0.img boot/u-boot.img
sudo cp ./tmp/deploy/images/uImage boot/uImage

I've worked with Buildroot already, so I'm sticking with it for the rootfs. This will help to produce a much more streamlined system. I've detailed out the important parts here, but I've also uploaded my buildroot .config file so you can use that as a reference. Here's the major config options that need to be set

BR2_ARCH="arm"
BR2_ENDIAN="LITTLE"
BR2_GCC_TARGET_TUNE="cortex-a8"
BR2_GCC_TARGET_ARCH="armv7-a"
BR2_TARGET_GENERIC_GETTY_PORT="ttyO1"
BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_DEVTMPFS=y

The rest is mostly package selection, which you will need openssh, iproute2, and iptables. I would recommend radvd, openntpd, dhcp server and client. Some form of VPN support would be good too, I usually go with ipsec-tools, but openvpn is also good. The other packages I will leave up to you as they are mostly user preference.

Configure buildroot

tar xjf buildroot-2013.02.tar.bz2
cd buildroot-2013.02
make menuconfig
make
tar xf output/images/rootfs.tar -C ../rootfs
cd ..

Before we umount the rootfs, lets make some changes to how things will run. We will need to start the network interfaces, and setup some firewall routing rules. I'll only be going through the minimal firewall setup for a gateway router, you should tailor it to your needs. You will also need to change the IPs, I'm using rfc 5737/3849 addresses for documentation purposes, so they will need to be changed for your network.

Filesystem Changes

#First thing is to add your ssh public key to the /root/.ssh/authorized_keys
cp ~/.ssh/id_rsa.pub rootfs/root/.ssh/authorized_key
#We will need to set up the network interfaces, so lets change the startup script
vi rootfs/etc/init.d/S40network # see contents below
#We will also setup a simple forwarding firewall, I recommend tailoring this for your own needs
vi rootfs/etc/init.d/S35firewall # see contents below
sudo umount boot
sudo umount rootfs

Insert the flash card into the Gumstix COM, connect up the network cables, and plug in the power. You should be up and running with a nice and efficient router. Depending on your package selection, this could be a good security monitoring device, or a man in the middle device on a pen-test. You should be able to run it off a battery as well.

The following are copies of configurations and scripts that were used above for reference.

/etc/init.d/S40network Contents