11.1 Embedded Linux Core

What is Embedded Linux?


First things first:


There is no such thing! It’s just Linux on an Embedded System.


Saying that, the term “Embedded Linux” is in broad and common use, so it will probably become a standard description in the longer term. The word embedded is used to describe “Embedded System”, which can be loosely explained as a system that has software integrated into a hardware system that is designed for a quite specific application. This contrasts somewhat to a Personal Computer (PC) which is a general purpose device, used for many applications, from web browsing, to word processing to playing games. Embedded Systems tend to have specific and dedicated applications.


Some general characteristics of Embedded Systems:

  • Resources such as memory, CPU are often limited
  • They are not self contained and are generally part of a larger system
  • They often have a role where reliability is critical (e.g. controls in cars, airplanes, hospital monitoring equipment etc.)
  • In many cases they work in real-time, where outputs are directly related to current inputs.

They are everywhere in everyday life: vending machines, kitchen appliances, phones/smart phones, manufacturing/assembly line, TV, games consoles, cars (power steering, security, engine management, tyre pressure, reversing sensors …), network switches, routers, wireless access points, sound systems, medical monitoring equipment, printers, building access controls, parking meters, smart energy/water meters, watches, building tools, digital cameras, monitors, tablets, e-readers, anything robotic, smart card payment/access systems and many, many more...


Those who work with embedded systems (A massive generalisation!)?

  • Electronic Engineers - typically design and build embedded systems (design at the physical layer, use VHDL/Verilog to design the actual processor models, work with companies such as Intel, Samsung etc. to fabricate devices)
  • Computer Engineers - typically those who are ‘close to the hardware’ and would be concerned with the design of the embedded systems and the programming of these devices (from hardware to software - perhaps write bootloaders, operating-systems, device drivers etc.)
  • Software Engineers/Computer Scientists - Typically work at the layer above the operating system/bootloader, e.g. writing the applications that meet client demands or integrating applications with web technologies.

So, how is programming on embedded systems different than desktop computers? Well first:

  • Always write the clearest and cleanest code that is as maintainable as possible, just like you would on a desktop PC.
  • Don’t optimise your code until you are certain that it is complete.
  • You typically have to be more aware of how you are consuming resources than you were before when we were looking at C++/Java programming on the desktop computer. The size of our types matters, passing by value vs. constant reference really matters! You have to be concerned with your memory availability, your file system size, data communication availability/bandwidth etc.
  • You typically have to learn about the underlying hardware platform - how does it handle the connected hardware, what data buses are available, how do we interface with the operating system and low-level libraries?


Embedded Linux

There has been huge growth recently of embedded Linux devices, thanks in part to the rapid evolution of smartphone technology that has dramatically driven down the price of ARM-based processors. ARM’s model of licensing its CPU model for a royalty of about 1.0% to 2.0% of the price of the processor means that if Texas Instruments (AM3359 the processor in the Beaglebone) sells for $5 that it only has to pay a royalty fee of between 5 cents and 10 cents on each processor sold. Despite this low price point, ARM Holdings (a UK company) earned about $1 Billion in 2012. Amazing, considering ARM don’t manufacture chips! As of 2012 there were 320 companies licensed to produce ARM cores in their chips.


Intel have always been involved in high-end embedded, devices for high-end manufacturing, mobile phone base stations, high-end switches etc. In fact, Intel (Shannon) is core to Intel’s strategy in those segments within the embedded space. Intel (Lexlip) recently designed Galileo, which is a micro-controller board based on the Intel Quark SoC (a 32-bit  Pentium class SoC). The board is pin-compatible with the Arduino Uno board. It is also possible to run Linux on this board (pretty raw at the moment).

Why Embedded Linux?

There are many embedded platforms, each with its own advantages and disadvantages - at the low end there are PIC and AVR micro-controllers, Texas Sitara, and many, many more - Almost all are programmed in C/C++ and some Assembly language. Generally you require specific knowledge of each of those types of platforms before you can develop useful applications. An alternative platform is embedded Linux...


Here are some of the reasons that embedded Linux has seen such growth:

  • Linux is efficient and scaleable, from consumer-oriented devices to large-scale servers. It has evolved over the years, starting when desktop computers were far less powerful than today.
  • There is a huge number of open-source applications already developed that can be deployed in an embedded environment, from web-servers, SSH servers to GUI based applications.
  • There is excellent open-source support for many different peripherals and devices, from network adapters to displays.
  • It is open-source and does not require a fee for its use.
  • There is an active community of developers on many different distributions, from Angstrom that we have by default on the BeagleBone to Ubuntu, which has even greater device support and can be installed on the BeagleBone.

The downside is that Linux is not by default set up for real-time applications. So, for high precision, fast-response applications, Linux may not be the perfect solution. Not yet! there are many current developments such as RTOS (Real-Time Operating System) Linux Embedded Distributions - Many are expensive, such as LynuxWorks, which is used in military, avionics, medical and security applications.

Open Source? Free?

Linux is released under the GNU GPL (General Public License) which grants users the freedom to modify to modify its code in any way - it does not mean that every distribution does not have a cost. In fact, some of the most expensive Linux distributions are currently those for embedded architectures. There is a quick guide to the GPLv3 at www.gnu.org, which lists the four freedoms that every user should have:

  • the freedom to use the software for any purpose,
  • the freedom to change the software to suit your needs,
  • the freedom to share the software with your friends and neighbors, and
  • the freedom to share the changes you make.
So, is it free? No, even if you are using a free distribution, it can and will cost you significant effort to tailor libraries, device drivers, to suit the particular component set and digital blocks that you want to use in your product development.


Where is my BIOS and time?

The first thing you see when you boot a desktop PC is the BIOS screen, also known as the System BIOS. The BIOS on a PC tests the hardware components and then loads the operating system, typically from the hard drive. So, when the power is turned on:

  • The BIOS takes control of the processor.
  • It initialises and tests the hardware components such as the system memory.
  • It loads the operating system off the hard drive.

The BIOS provides an abstraction layer for the operating system to interact with the display and other input/output peripherals, such as the mouse, keyboard etc. The BIOS settings are battery backed and if you look at a PC motherboard you will see a small 'watch style' battery that is used to store BIOS settings and keep the time correct on the system clock.


The Beaglebone does not have a BIOS and so uses a combination of bootloaders instead. A bootloader carries out the exact same functionality as the BIOS, but it is a custom program that was developed for each and every embedded system board. The main Linux bootloader that we use on the Beaglebone is called U-Boot. For example, for this bootloader, the source code for u-boot is available from:


git clone git://git.denx.de/u-boot.git


and the patches for the Beaglebone are at:



To build it:


make ARCH=arm CROSS_COMPILE=${CC} am335x_evm_config

So, you can see how we might take the latest code for u-boot and apply patches for it to run on the BeagleBone hardware. The AM3359 is the processor on the Beaglebone board. So, you can see that this bootloader is built using the standard u-boot distribution that has been patched. When it is built you get a file called u-boot.img. In fact you can see this in place when you plug your Beaglebone into your USB port on your desktop PC, it is a file called u-boot.img and it is about 371KB. The MLO, u-boot.img and uEnv.txt files are vital to your Beaglebone booting. See these in Figure 1 below.


Figure 1. The FAT Partition of the Beaglebone Black



The uEnv.txt file sets the boot parameters for your Beaglebone black. By default it contains:


optargs=quiet drm.debug=7

but if you wanted to you could add much more complex settings.


Figure 2. Calling ifconfig to see the network adapters


Booting the Beaglebone

So, the Bootloader (u-boot in our case) performs the following critical functions, linking the specific hardware of your board to the Linux operating system:

  • Initialises the controllers (memory, graphics, I/O)
  • Prepares system memory for the operating system - allocates it as required
  • Finds the operating system and provides the facility for loading it
  • Loads the operating system and passes control to it


But U-Boot is only one step in the boot chain - here is the full set:




Power is applied or the CPU is invokes the reset vector to start program counter at a particular location in the Boot ROM.


Texas Instruments Boot ROM (inside AM3359)

Internal/First Stage Bootloader

(enough knowledge to access the SD card/eMMC/Uart to find MLO)

Fixed at manufacture by TI

Performs minimal peripheral configuration, searches for booting image, loads x-loader


The x-loader (MLO on FAT partition)

Second Stage Bootloader

Provided by Texas Instruments

Sets up the pin muxing, initialises clocks and memory, loads U-Boot


U-Boot (u-boot.img on FAT partition)

Third Stage Bootloader

Specifies the root file system, uses uEnv.txt configuration, additional platform initialisation

Loads and passes Control to the Linux Kernel


Linux Kernel (on a different Ext3/4 partition on your SDCard/eMMC)

Angstrom by default

Decompresses Kernel into memory, sets up peripherals USB, I2C, HDMI etc. mounts the file system that contains all of the Linux applications


Calls first User-Space Process - init

Moved from Kernel context to User context




Kernel context is when the kernel owns all of the system memory and has complete control of the memory. It then passes control to init which is in user space context. The user space has restricted access to the system and makes requests of the the kernel to access resources. This is where your programs live - the big advantage of this structure is that one user-space program cannot damage the memory space of other programs in user-space.


On the Beaglebone init calls a set of programs as described in /etc/init.d, which can be seen in Figure 3.


Figure 3. The Initial user-space processes




U-boot uses a board configuration file called a device tree (also called a device tree binary) that contains board-specific information that the kernel requires to boot the board. This file contains all of the information needed to describe the memory size, clock speeds, on-board devices etc.This device tree binary or dtb (binary) is created from a dts (source) file using a compiler called a device tree compiler (dtc)


The code looks like this, which describes the 4 user LED pins and the 2 I2C buses on the beaglebone. The full description for the Beaglebone Black for Linux 3.8.13 is available at: https://github.com/derekmolloy/boneDeviceTree/tree/master/DTSource3.8.13

am33xx_pinmux: pinmux@44e10800 {

      pinctrl-names = "default";

      pinctrl-0 = <&userled_pins>;


      userled_pins: pinmux_userled_pins {

          pinctrl-single,pins = <

               0x54 0x7        /* gpmc_a5.gpio1_21, OUTPUT | MODE7 */

               0x58 0x17        /* gpmc_a6.gpio1_22, OUTPUT_PULLUP | MODE7 */

               0x5c 0x7        /* gpmc_a7.gpio1_23, OUTPUT | MODE7 */

               0x60 0x17        /* gpmc_a8.gpio1_24, OUTPUT_PULLUP | MODE7 */

          >;

      };

      i2c0_pins: pinmux_i2c0_pins {

           pinctrl-single,pins = <

                0x188 0x70       /* i2c0_sda, SLEWCTRL_SLOW | INPUT_PULLUP */

                0x18c 0x70        /* i2c0_scl, SLEWCTRL_SLOW | INPUT_PULLUP */

          >;

      };

      i2c2_pins: pinmux_i2c2_pins {

          pinctrl-single,pins = <

               0x178 0x73         /* uart1_ctsn.i2c2_sda, SLEWCTRL_SLOW */

                0x17c 0x73        /* uart1_rtsn.i2c2_scl, SLEWCTRL_SLOW */

          >;

      };

};

We are able to develop our own additions and modifications to the description using Device Tree Overlays (DTOs) which can be installed in the /lib/firmware directory of our Linux distributions.


Linux Top-Level Directories

If I go to the top level directory on the Beaglebone and type ls -al I get the top-level directory, which looks as follows:


root@beaglebone:/# ls -al

total 64

drwxr-xr-x  18 root  root   4096 Mar 18  2013 .

drwxr-xr-x  18 root  root   4096 Mar 18  2013 ..

drwxr-xr-x   2 root  root   4096 Mar 18  2013 bin

drwx------   2 xuser xuser  4096 Mar 18  2013 boot

drwxr-xr-x  13 root  root   4020 Jan  1 00:02 dev

drwxr-xr-x  65 root  root   4096 Jan  1 01:45 etc

drwxr-sr-x   4 root  root   4096 Mar 18  2013 home

drwxr-xr-x   9 xuser xuser  4096 Mar 18  2013 lib

drwx------   2 root  root  16384 Mar 18  2013 lost+found

drwxr-xr-x  11 root  root   4096 Jan  1 00:02 media

drwxr-xr-x   2 root  root   4096 Mar 18  2013 mnt

dr-xr-xr-x 110 root  root      0 Jan  1  1970 proc

drwxr-xr-x   7 root  root    160 Jan  1 00:02 run

drwxr-xr-x   2 root  root   4096 Mar 18  2013 sbin

dr-xr-xr-x  12 root  root      0 Jan  1 00:02 sys

drwxrwxrwt  11 root  root    240 Jan  1 03:11 tmp

drwxr-xr-x  13 root  root   4096 Nov 15  2013 usr

drwxr-xr-x  13 root  root   4096 Mar 18  2013 var

root@beaglebone:/#


The key directories are:


bin   contains the binary executables used by all of the users

boot contains the device tree binaries for the BeagleBone

dev  contains the device nodes (device drivers)

etc   configuration files for the local system

home   the user home directories (e.g. /home/root is the root user home)

lib    all of the standard system libraries

media/mnt can mount your sd card here by default

proc  come and go, related to processes running on the BeagleBone (for example if you go into /proc and type cat iomem you can see the mapping addresses as in Figure 4 below)

run   information about the running system since the last boot

sbin contains the binary executables for the root user (superuser)

sys  a directory containing a virtual file system that describes the system sysfs - I will discuss this below.

tmp temporary files location

usr  contains application programs for all of the users

var contains variable files such as system logs



Figure 4. The /proc directory on the BeagleBone Black



We can see the processes that are running on the Beaglebone using the top command. See Figure 5.

Figure 5. Using top to describe the processes that are running on the BeagleBone (use CTRL-C to exit)

Figure 6. The /sys directory on the BeagleBone Black



This brings us on to sysfs - the system file system!

Sysfs

Sysfs was first introduced into the Linux 2.6 kernel. It allows kernel code to export information to the user space through an in-memory file system, which is available for the user space user to view and manipulate. You will find this in your Beaglebone at the directory /sys off the root directory. The directory hierarchy is strict and maps to the kernel data structures as follows:

  • Kernel Objects   -> mapped to Directories
  • Kernel Object attributes -> mapped to regular files in those directories
  • Kernel Object relationships -> mapped using symbolic links

This provides with a well-described interface to kernel infrastructure which is relatively easy to interact with. The top-level directory structure for /sys usually looks like:


/sys/

block - block devices, like memory, eMMC/SD addresses

bus  - bus types registered in the kernel, each sub directory has a devices and drivers. The devices contains the flat listing of every device discovered on that type of bus and the drivers contains a directory for every device driver registered with the bus type. Within the drivers directory you can modify driver parameters. See Figure 7 below.


Figure 7. The /sys/bus directory on the BeagleBone Black



class - contains representations of every type of device class that is registered with the kernel. Each device class contains sub-directories for each class object that has been allocated and registered with that device class. For most device classes the directories contain symbolic links to the device and driver directories associated with that class of object. See Figure 8 below.


Figure 8. The /sys/class directory on the BeagleBone Black


        devices - contains the global device hierarchy, which contains every physical device that has been discovered by the bus types registered with the kernel. You can see in the beaglebone there is a 44e10800.pinmux. This is the base address of the pin multiplexer that allows us to configure the GPIOs. See Figure 9 below.

Figure 9. The /sys/devices directory on the BeagleBone Black


        firmware - contains the interfaces for viewing and manipulating firmware specific objects and attributes. This is more related to x86 bios and is empty on the BeagleBone


module - this contains sub-directories for every module that is loaded into the kernel. The kernel in Linux allows pieces of code, called modules, to be loaded and unloaded into the kernel on demand, without needing to reboot the system. For example, when a USB device is plugged in, it would be possible to load a kernel module to control that device on demand. See the output in Figure 10. Loadable modules are discussed in the next section


Figure 10. The /sys/module directory on the BeagleBone Black


        power - represents the power system, which allows us to control the methods by which the system will suspend to disk. See Figure 11 below.


Figure 11. The /sys/power directory on the BeagleBone Black


The directories represent the main subsystems that are registered with sysfs. These directories are created at startup and when the subsystems register themselves with the kernel core they discovered objects appear on the sysfs directory tree.


So for example, to export (make available in user space) a GPIO pin, we can use the sysfs tree at the location /sys/class/gpio. You can see in Figure 12 that we can export a GPIO by echo'ing the GPIO pin number to export. Then a new directory will appear for that GPIO. If we go into this directory, you can see that you can set up the direction, edge type etc. and we can read the 0/1 state from the value file/value.


Figure 12. Exporting a GPIO on the Beaglebone

We can controlling the on-board LEDs from the /sys/class/leds/ directory. There is a full discussion on this and how to write C++/Java code to manipulate the LEDs in this section of the notes: Flashing the LEDs using C++ and Flashing the LEDs using Java


Figure 13. Using the Beaglebone on-board LEDs.



So, sysfs is a file system that allows kernel subsystems to export kernel objects and their attributes and relationships to the user space. It provides a key step in building flexible device and system management tools.


For more information on sysfs, see: https://www.kernel.org/pub/linux/kernel/people/mochel/doc/papers/ols-2005/mochel.pdf

Loadable Modules

Linux lets you add and remove kernel components at run-time (after boot time). It is useful for the user and device driver developer. You can insert and remove the device driver from a running kernel at run-time, instead of rebooting the system to test a change. Loadable modules are important to embedded applications as they allow better field upgrade possibilities. 

Device driver architecture: There are typically two types, character devices (that can be thought of as serial streams, e.g. a mouse) and block devices (can read/write blocks of random data to an addressable medium e.g. flash device)

Interacting with Loadable Modules - Module Utilities:

lsmod: Is a command on Linux which prints the contents of /proc/modules. It shows which loadable kernel modules are currently loaded. You can see this used below on the BeagleBone Black in Figure 14.

Figure 14. Using lsmod and modinfo on the BeagleBone, with no USB devices attached


Modprobe: is a Linux program to add or remove a loadable kernel module to the Linux kernel. It has the ability to add a blacklist where two or more modules support the same devices.

Modinfo: Extracts information from the Linux kernel modules given on the command line, listing each attribute of the module.



Flash Memory

Flash memory is like a very small hard-disk and we are starting to see SSDs used in laptops and desktop PCs. They are less prone to mechanical failure, especially due to environmental knocks/bangs than regular hard-disks.


Flash memory is divided into large erasable units (erase blocks). In a typical NOR flash memory chip data can be changed from a 1 to a 0 under software control, but to change a bit from a 0 to a 1 requires the entire erase block to be cleared. These are large blocks, on a typical 8MB flash chip you would have only 128 blocks with a size of 64KB each. So, even if only 1 byte needs to be changed, the entire block needs to be cleared and re-written. As the lifespan is only about 10,000 to 100,000 cycles per block you have to be careful in writing applications that you do not have code that constantly clears and rewrites blocks (even through a coding bug). Early flash memory cards were all NOR flash based; however, the newer microSD cards are all NAND flash based.


NAND flash is a newer technology that offers smaller block sizes, allowing for more efficient writes. They are typically accessed serially and present a model that is similar to a hard disk. The write cycle lifetime for NAND flash is many times greater than for NOR flash (1,000,000+ with Error Correction Code).


The Beaglebone Black has a 2GB 8-bit eMMC (embedded Multi-Media Card), by Micron Semi. It has a density of 2GB on a 153-pin Ball Grid Array in a WFBGA package (WF means 0.8mm thick) . It is a NAND flash-based device but all internal transitions are handled by the device itself. On the Beaglebone Black it is the rectangular chip near the processor to the user LED side. NAND devices are currently more reliable and less prone to bit errors. 


Figure 15. A WFBGA Package

The advantage of the eMMC over the SD card is that it allows you to boot directly from the board and because it is a fast interface, the BeagleBone boots very quickly. It is a little limited at 2GB, but you still can mount a microSD card as a drive. The downside is that when you want to ‘burn’ a new image to the BeagleBone black it can take about 40 minutes.


Regular Memory Space

On the Beaglebone Black we are using a 16 bit wide 512MB DDR3 RAM chip at 800MHz. What do we need this memory for? Well, this a large, flat memory space that runs from the physical address 0x00000000 to 0x1fffffff; however all systems use a virtual memory map that internally maps to physical memory. For example, on the Beaglebone Black the GPIOs begin at 0x4804C000. We can write a short program to get some information about our user-space memory locations. For example:


#include<stdio.h>


int global_var;

int int_global_var = 1;


int main(int argc, char **argv){

  void *local_stack_var;


  local_stack_var = (void*)main;


  printf("The main function is executing at address: %p.\n",                 local_stack_var);

  printf("The address space of our local stack frame includes %p.\n",           &local_stack_var);

  printf("The uninitialized global has address %p\n", &global_var);

  printf("The initialized data has address %p\n", &int_global_var);

  return 0;

}


We can compile this by typing: gcc memory.c -o memory and then executing it gives:


root@beaglebone:~/temp# gcc memory.c -o memory

root@beaglebone:~/temp# ./memory

The main function is executing at address: 0x8400.

The address space of our local stack frame includes 0xbedd2c0c.

The uninitialized global has address 0x10700

The initialized data has address 0x106f8

root@beaglebone:~/temp#


So, you can see our user-space memory is right down at the lower addresses of our virtual memory map. The advantage of a virtual memory map is that while it make it appears that there is more memory there than what is physically present, the kernel can enforce access rights to prevent one process from accessing memory that belong to other processes or to the kernel itself. 


Figure 16. A General Linux Virtual Memory Model




Linux Distributions

There are many different distributions of Linux, including expensive proprietary versions for real-time programming. At heart, they all use Linux, but the different distributions contain different default tools and configuration that give very different look-and-feels. The main open source distributions used on embedded systems include:

Ångström http://www.angstrom-distribution.org/

Ångström was started by a small group of people who worked on efforts to make a stable and user-friendly distribution for embedded devices like handheld devices, set top boxes and network-attached storage devices. Ångström can scale down to devices with only 4MB of flash storage. Angstrom uses Busybox for many key utilities, which has both pros and cons.

Busybox (See Figure 17 below) contains compact versions of many command-line utilities that are commonly found on Linux systems. The advantages include requiring less storage space and a smaller memory footprint for many common utilities, which also improves system startup time and performance. The main disadvantages stem from those utilities not mirroring exactly their full-size counterparts. These differences can be annoying if it breaks shell scripts that rely on portable functionality. Angstrom uses connman for network connection management. Angstrom does not overdo user account control and makes a good platform for beginners and this is why it is used in this module.

Arch Linux

Arch Linux for the BeagleBone is a version of the Arch Linux ARM distribution. This carries forward the Arch Linux philosophy of simplicity and user-centrism, targeting and accommodating competent Linux users by giving them complete control and responsibility over the system. Instructions are provided to assist in navigating the nuances of installation on the varied ARM platforms; however, the system itself will offer little assistance to the user. It’s lack of support beginners makes it unsuitable for this module.

Ubuntu

Ubuntu aims to provide free software that is available free of charge to everybody on the same terms, and funded through a portfolio of services provided by Canonical. The first version of Ubuntu was based on the GNOME desktop, but has since added a KDE edition, Kubuntu, and a server edition. All of the editions of Ubuntu share common infrastructure and software. In recent years, special emphasis has been placed on net books for lightweight, connected, mobile computing, and on the cloud as a new architecture for data centres. While Ubuntu probably has the best device support, it is not an embedded Linux focused distribution and enforces the same user account control settings that you would have on a larger scale server. It can be frustrating to keep switching between the user account and superuser when you want to make changes to the system.


Figure 17. Busybox
Comments