A Voyage to the Kernel, Day 11

By Aasis Vinayak PG on May 1, 2009 in Coding, Columns http://www.linuxforu.com/2009/05/a-voyage-to-the-kernel-day-11/ Were entering a new phase in our journeykernel programming. In the first part, well cover a broad introduction to the Linux platform for newbies, along with some history.

The Linux platform

The UNIX platform showed us the power of the multi-layer security architecture, and the beauty of a well-organised structure in terms of subsystems and layers. This is perhaps one aspect that inspired Linus Torvalds, who was then studying computer science at the University of Helsinki, to develop a free operating system. (It should be noted that there is still an ongoing project to remove the non-free portion of the kernel code.) In 1990 he wrote: From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds) Newsgroups: comp.os.minix Subject: Gcc-1.40 and a posix-question Message-ID: <1991Jul3.100050.9886@klaava.Helsinki.FI> Date: 3 Jul 91 10:00:50 GMT Hello netlanders, Due to a project Im working on (in minix), Im interested in the posix standard definition. Could somebody please point me to a (preferably) machine-readable format of the latest posix rules? Ftp-sites would be nice. The 90s home PCs were powerful enough to run a full-blown UNIX OS. So Linus thought it would be a good idea to come out with a freely available academic version of UNIX. He based his project on Minix. To quote him again: Date: Aug 26 1991, 11:12 am Subject: What would you like to see most in minix? To: comp.os.minix Hello everybody out there using minix Im doing a (free) operating system (just a hobby, wont be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. Id like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things).

Ive currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that Ill get something practical within a few months, and Id like to know what features most people would want. Any suggestions are welcome, but I wont promise Ill implement them :-) Linus (torva@kruuna.helsinki.fi) PS. Yes its free of any minix code, and it has a multi-threaded fs. It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as thats all I have :-(. He wrote the first Linux kernel in 1991. Two years after Linus post, there were 12,000 Linux users. Linux is an essentially full UNIX clone. And now the Linux kernel is over six million lines of code! The main reason why Linux gained its popularity is because it was released under the GNU GPL licence (arguably one of the strongest copyleft or quid pro quo licences). Later, Slackware, Red Hat, SuSE, Debian, et al, sprung up to provide packaged Linux distributions. This further spurred the growth of the GNU/Linux community. Some of the earlier large-scale users include Amazon, the US Post Office and the German Army. It is interesting to note that Linux machine clusters were used while making movies like Titanic and Shrek. Today, we have Linux on gadgets like PDAs, mobiles, embedded applications and even on wristwatches. The introduction of 3D acceleration support, support for USB devices, single-click updates of systems and packages made Linux more popular. Desktop users can now log in graphically and start all applications without typing a single character on a terminal. At the same time, you still have the ability to access the core of the system. Here are a few other utilities and programs (mostly from GNU) that added power to Linux:

Bash: The GNU shell GCC: The GNU C compiler coreutils: A set of basic UNIX-style utilities findutils: For searching and finding files GDB: The GNU debugger fontutils: For converting fonts from one format to another or making new fonts Emacs: A very powerful editor Ghostscript and Ghostview: An interpreter and graphical front-end for the PostScript files GNU Photo: A software to manipulate the inputs from digital cameras Octave: To perform numerical computations GNOME and KDE: GUI desktop environments GNU SQL: A relational database system Radius: A remote authentication and accounting server

An introduction to the tech side

The Linux kernel is monolithic with module support. Beginners may not fully appreciate the meaning of this statement. A monolithic kernel is essentially a single large complex do-ityourself kernel program, which has several different logical entities, and these run in kernel

mode. The Linux kernel has a modular design. When you boot the system, only a minimal resident kernel is loaded into memory. When you request a feature that is not in the kernel, a kernel module (or driver) is dynamically loaded into memory. The object code normally has a collection of functions that implements a file system, a device driver or any other feature at the kernels upper layer. This has the following advantages: minimal main memory usage, modularised structure and platform independence.

Our first module

Lets begin the technical side by writing a module:
#include <linux/module.h> MODULE_AUTHOR(Aasis Vinayak PG); MODULE_LICENSE(GPL GPLv3); static int new__init(void) { printk(Lets begin our new segment!n); return 0; } static void goodbye__old(void) { printk(Adieu to segment 2!n); } module_init(new__init); module_exit(goodbye__old);

We have written a module. Lets save this as module1.c in the home directory. Now we need a makefile:
obj-m += module.o all: make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) modules clean: make -C /lib/modules/$(shell uname -r)/build/ M=$(PWD) clean clean-files := Module.symvers

Before proceeding, check the kernel version you are using:

aasisvinayak@GNU-BOX:~$ uname -r 2.6.27-7-generic

Now we can make our module:

aasisvinayak@GNU-BOX:~$ make -C /lib/modules/`uname -r`/build/ M=`pwd` make: Entering directory `/usr/src/linux-headers-2.6.27-7-generic CC [M] /home/aasisvinayak/module1.o Building modules, stage 2. MODPOST 1 modules CC /home/aasisvinayak/module1.mod.o LD [M] /home/aasisvinayak/module1.ko make: Leaving directory `/usr/src/linux-headers-2.6.27-7-generic

You may change the directory by changing the pwd command to your directory. We can check our module by issuing the following commands:
aasisvinayak@GNU-BOX:~$ sudo dmesg -c > /dev/null aasisvinayak@GNU-BOX:~$ sudo insmod module1.ko aasisvinayak@GNU-BOX:~$ sudo dmesg -c [74091.826941] Lets begin our new segment! aasisvinayak@GNU-BOX:~$ sudo rmmod module1 aasisvinayak@GNU-BOX:~$ sudo dmesg -c [74119.021887] Adieu to segment 2!

You can see that a module1.mod.c file was also created in the process:
#include <linux/module.h> #include <linux/vermagic.h> #include <linux/compiler.h> MODULE_INFO(vermagic, VERMAGIC_STRING); struct module __this_module __attribute__((section(.gnu.linkonce.this_module))) = { .name = KBUILD_MODNAME, .init = init_module, #ifdef CONFIG_MODULE_UNLOAD .exit = cleanup_module, #endif .arch = MODULE_ARCH_INIT, }; static const struct modversion_info ____versions[] __used __attribute__((section(__versions))) = { { 0xa257c5a3, struct_module }, { 0xb72397d5, printk }, { 0xb4390f9a, mcount }, }; static const char __module_depends[] __used __attribute__((section(.modinfo))) = depends=; MODULE_INFO(srcversion, 4675BE4AD96DEF402B04BD1);

You will find that the code requires the following files as well:
#include #include #include #include #include #include #include #include #include <linux/list.h> <linux/stat.h> <linux/compiler.h> <linux/cache.h> <linux/kmod.h> <linux/elf.h> <linux/stringify.h> <linux/kobject.h> <linux/moduleparam.h>

#include <linux/marker.h> #include <asm/local.h> #include <asm/module.h>

If you are an experienced programmer, then by looking at the dependencies you will know how the new file has been created. Novice users need not worry, as this column will address everything from scratch. For the time being, you can just explore the directories /lib/modules/2.x.x-version and /usr/src/linux-headers- 2.x.x-version. As I mentioned before, you can load a module by issuing commands. lsmod will list the modules (with a small description about its current usage) in your terminal. Figure 1 shows the result of the execution.

Figure 1: A typical lsmod output You can also view the list of modules to load at boot time by checking the /etc/modules file. It will have entries similar to the one listed below:
# /etc/modules: kernel modules to load at boot time. # # This file contains the names of kernel modules that should be loaded # at boot time, one per line. fuse lp sbp2 # Generated by sensors-detect on Wed Jan 14 21:14:15 2009

# Chip drivers coretemp

You can issue modprobe for probing. Its usage is shown below:
Usage: modprobe [-v] [-V] [-C config-file] [-n] [-i] [-q] [-Q] [-b] [-o <modname>] [ --dump-modversions ] <modname> [parameters...] modprobe -r [-n] [-i] [-v] <modulename> ... modprobe -l -t <dirname> [ -a <modulename> ...]

As this is an introductory column, the kernel build process (which has five parts, viz. Makefile, .config, arch/$(ARCH)/Makefile, scripts/Makefile, Kbuild Makefiles) will be explained in detail in subsequent articles.

Compiling the kernel

Now we will look at how to compile a kernel:

Download the latest Linux source. If you have installed Linux with the kernel source option, the source will already be there. You may copy the kernel source to a safe location so that you can restore it later. It is a good idea to backup your /boot directory as well or at least the kernel image. On my machine, the source was located in /usr/src/linux-2.6.27-7. Create a soft link to your Linux folder, so that you have /usr/src/linux:
ln -s /usr/src/linux-2.6.27-7 /usr/src/linux

Make the necessary changes to the kernel

make clean make mrproper make clean

Configure the modules. You have to specify the modules to be loaded and those to be compiled in the kernel. It would be ideal to start with your existing configuration file.
make dep

Modify your Makefile EXTRAVERSION tag to give a unique name to your kernel Compile the kernel image
nohup make bzImage & tail -f nohup.out

Compile the modules

nohup make modules 1> modules.out 2> modules.err & tail -f modules.err make modules_install

Copy your kernel image and config file to the /boot directory:
cp arch/i386/boot/bzImage /boot/cpKernel cp .config /boot/cpKernelConfig

Create a new initrd image so that the modules can be loaded while booting. The module created in the make modules step will be located in /lib/modules:

/sbin/mkinitrd /boot/initrd-2.6.27-7cpKernel.img /lib/modules/2.6.27-7cpKernel

Now you can edit your /boot/grub/menu.lst file (if you are using the GRUB bootloader), so that while booting you can select which kernel to start. Here is a sample entry in /boot/grub/menu.lst:
timeout 10 splashimage=/boot/grub/splashimages/firework.xpm.gz title VINU, kernel 2.6.27-7 uuid c21385ba-7c68-4ac0-924f-8bfafdaddc5f kernel /boot/vmlinuz-2.6.27-7 root=UUID=c21385ba-7c68-4ac0-924f-8bfafdaddc5f ro quiet splash initrd /boot/initrd.img-2.6.27-7 quiet title VINU, kernel 2.6.27-7 uuid c21385ba-7c68-4ac0-924f-8bfafdaddc5f kernel /boot/vmlinuz-2.6.27-7 root=UUID=c21385ba-7c68-4ac0-924f-8bfafdaddc5f ro initrd /boot/initrd.img-2.6.27-7 title uuid kernel quiet VINU, memtest86+ c21385ba-7c68-4ac0-924f-8bfafdaddc5f /boot/memtest86+.bin

single

(Optional step) Delete the entry for the old kernel version in the config file Reboot. When the machine restarts, you can select the new kernel!

This is the standard procedure. There are simpler methods as well. (For example, in the case of Debian-based distros, you have a very easy method. Just Google it and find the resources.)

Kernel structure
Within the kernel layer, Linux is composed of five (or six, based on the classification style) major subsystems: the process scheduler (sched), the memory manager (mm), the virtual file system (vfs), the network interface (net), and the inter-process communication (ipc).

Organisation of code
We shall now discuss the way in which the source layout is organised. If you go to the root of the source tree and list the contents, you will see the following directories:

arch Contains architecture-specific files block Has the implementation of I/O scheduling algorithms

crypto Has the implementation for cipher operations and contains the cryptographic API for implementing encryption algorithms Documentation Contains descriptions about various kernel subsystems drivers Device drivers for various device classes and peripheral controllers can be found here fs Contains the implementation of file systems include Kernel header files init High-level initialisation and start-up code ipc Contains support for Inter-Process Communication (IPC) mechanisms kernel Architecture-independent portions of the base kernel lib Has the library routines mm Holds the memory management implementation net Networking protocols scripts Scripts used for kernel build security Holds the framework for security sound Linux audio subsystem usr Has the initramfs implementation

We will dedicate a few more sessions to reviewing the kernel structure before we meddle with programming. So look out for the forthcoming columns to hack the kernel!
Related Posts:

Device Drivers, Part 2: Writing Your First Linux Driver in the Classroom Device Drivers, Part 10: Kernel-Space Debuggers in Linux Kernel Debugging Using Kprobe and Jprobe Kernel Development & Debugging Using the Eclipse IDE Device Drivers, Part 17: Module Interactions