CS 635

http://cs.usfca.edu/~cruse/cs635f07/

CS 635: Advanced Systems Programming (Fall 2007)

Lecture: Tuesday-Thursday 7:30pm-9:15pm (235 Harney Science Center) Instructor: Allan Cruse
cruse@usfca.edu Phone: (415) 422-6562 Office: 212 Harney Science Center Office Hours: (see my homepage)

Synopsis:
Focuses on Open Source Linux operating system for Intel x86-family processors kernel version 2.6.22 Assumes familiarity with C/C++ programming language, x86 machine architecture, Unix operating system commands, and standard data-structures and algorithms (e.g., CS 112/210/245/326) Open to USF Graduate Students (and to qualified undergraduates or non-degree students with instructor permission)

List of planned course topics:

Loadable kernel modules Shared runtime libraries Thread and memory management Asynchronous input/output The '/proc' and 'ext2' filesystems Character and Block Device-Drivers System data-structures and algorithms Kernel threads, timers, and wait-queues Interrupts, Exceptions, and System-Calls IA32 Symmetric Multiprocessing architecture The course includes lectures, readings, discussions, demonstrations and programming projects.

Course Textbooks:
Daniel P. Bovet and Marco Cesati, Understanding the Linux Kernel (Third Edition) (O'Reilly Media, Incorporated (2006) ISBN 0-596-00565-2 Corbet, Rubini and Kroah-Hartman, Linux Device Drivers (3rd Edition) (O'Reilly Media, Incorporated, 2005) ISBN 0-596-00590-3

Learning Outcomes:

1 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

You will be able to read and write code-modules for an operating system You will be able to implement customized extensions to the Linux kernel You will be able to craft your own tools that let users control their PC You will be able to identify system features that impact performance

Resources
The classic C language tutorial textbook and reference guide by Kernighan and Ritchie The Linux Kernel Archives (where you can download the latest official kernel release) Here you can read the GNU General Purpose License (GPL) online Homepage for the Linux Cross-Reference Project for browsing kernel source files Official Data Sheet for the Dallas Semiconductor DS12887 Real-Time Clock and CMOS Memory A typical CMOS RAM Memory-map for PC/AT compatible machines (at www.bioscentral.com) Documentation for the Linux Executable and Linkable Format (ELF) files: elf.pdf Online summary of UNIX Standard Error Codes (for use in our device drivers) Official IA-32 Programmer Reference Manuals, Intel Corporation Official industry standard: Intel Multiprocessor Specification (version 1.4) (.pdf) Reference Manual for the I/O Controller Hub 6 (ICH6) Family Datasheet (.pdf), Intel Corporation Official datasheet for the 16550 Serial UART (.pdf format), National Semiconductor, Inc Online in-depth tutorial: The Serial UART by Lammert Bies Online article: API changes in the 2.6 kernel series posted by Jonathan Corbet Articles by Jonathan Corbet and David Howells specifically about the 'workqueue' API changes Intel PCI-express Gigabit Ethernet controllers Open Source Software Developer's Manual (.pdf format) Online guide to ANSI terminal escape-sequences

Readings
1. For Tue, Aug 28: Linux Device Drivers: Chapter 1 "An Introduction to Device Drivers" and Chapter 2: "Building and Running Modules" 2. For Thu, Aug 30: Understanding the Linux Kernel: Preface and Chapter 1 "Introduction" 3. For Tue, Sep 04: Linux Device Drivers: Chapter 3 "Char Drivers" 4. For Thu, Sep 06: Understanding the Linux Kernel: Chapter 2 "Memory Addressing" 5. For Tue, Sep 11: Linux Device Drivers: Chapter 4 "Debugging Techniques" 6. For Thu, Sep 13: Understanding the Linux Kernel: Chapter 3 "Processes" 7. For Tue, Sep 18: Linux Device Drivers: Chapter 5 "Concurency and Race Conditions" 8. For Thu, Sep 20: Understanding the Linux Kernel: Chapter 5 "Kernel Synchronization" 9. For Tue, Sep 25: No new reading assignment: submit completed programming project #1. 10. For Thu, Sep 27: Understanding the Linux Kernel: Chapter 4 "Interrupts and Exceptions" 11. For Tue, Oct 02: No new reading assignment: review for Midterm Exam I. 12. For Thu, Oct 04: Understanding the Linux Kernel: Chapter 8 "Memory Management" 13. For Tue, Oct 09: Linux Device Drivers: Chapter 8 "Allocating Memory" 14. For Thu, Oct 11: Read three sections in PC16550D Serial UART: "General Description",

2 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

15. 16. 17. 18. 19. 20. 21. 22.

23.

24. 25. 26. 27. 28. 29.

"Features", and "Registers" and browse The Serial UART online tutorial by Lammert Bies For Tue, Oct 16: Linux Device Drivers: Chapter 6 "Advanced Char Driver Operations" For Thu, Oct 18: Linux Device Drivers: Chapter 7 "Time, Delays, and Deferred Work" For Tue, Oct 23: No new reading assignment: submit completed programming project #2. For Thu, Oct 25: Understanding the Linux Kernel: Chapter 6 "Timing Measurements" For Tue, Oct 30: Linux Device Drivers: Chapter 9 "Communicating with Hardware" For Thu, Nov 01: Linux Device Drivers: Chapter 12 "PCI Drivers" For Tue, Nov 06: Understanding the Linux Kernel: Chapter 7 "Process Scheduling" For Thu, Nov 08: PCIe GbE Controllers Open Source Software Developer's Manual (.pdf format): Read Chapter 1: "Introduction" and Chapter 14: "General Initialization and Reset Operation" For Tue, Nov 13: PCIe GbE Controllers Open Source Software Developer's Manual (.pdf format): Read Chapter 3: "Receive and Transmit Descriptors" For Thu, Nov 15: No new reading assignment: review for Midterm Exam II. For Tue, Nov 20: Understanding the Linux Kernel: Chapter 9 "Process Address Space" For Thu, Nov 22: No new reading assignment: University holiday (Thanksgiving recess) For Tue, Nov 27: Understanding the Linux Kernel: Chapter 10 "System Calls" For Thu, Nov 29: Understanding the Linux Kernel: Chapter 10 "Interrupt Handling" For Tue, Dec 04: No new reading assignment: review for Final Examination.

Systems Software
Utility program: mmake.cpp (a tool you can use for compiling Linux 2.6 kernel modules) Development tool: newmod.cpp builds skeleton C code for a new Linux Kernel Module Development tool: newinfo.cpp builds the skeleton for an LKM that creates a '/proc' file -obsoleted by kernel 2.6.26 Development tool: newproc.cpp replacement for the obsolete 'newinfo.cpp' utility above -posted on 07/17/2008 Device-driver: dram.c needed to allow reading from your workstation's physical memory -revised on 02/06/2008 Utility program: fileview.cpp (a navigation tool for viewing the contents of binary files) Utility program: setconsole.cpp (a tool for redirecting your 'printk()' console messages) Utility program: dump.cpp (a tool that displays the contents of any file in hex and ascii) Utility program: ljpages.cpp (a tool that submits your program source to a laser printer) Utility program: iopl3.cpp (a tool by Alex Fedosov that lets user-programs do direct I/O) -revised on 10/13/2007 Utility program: liveview.cpp (enhancement of 'fileview' for viewing volatile device-files)

Handouts
Quick course-preview: intro635.ppt (PowerPoint Slides) 0203-635-01: Course syllabus(.PDF) lesson1.ppt (PowerPoint Slides) Demo program: hello.c illustrates a minimal application-program, written in the C language

3 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

Kernel module: kello.c modifies the 'hello.c' program, so it becomes a Linux kernel module lesson2.ppt (PowerPoint Slides) Demo program: mycat.c shows how you could implement your own emulation for the 'cat' command Kernel module: cmos.c shows how to create a pseudo-file that will display some device information -- revised on 11/30/2007 Kernel module: moreinfo.c shows how '/proc' files can be designed to display a lot more information lesson3.ppt (PowerPoint Slides) Kernel module: tempcdev.c shows how a kernel module can create a temporary device special file -- revised on 9/03/2007 Kernel module: seesetfs.c to clarify the effect of the misleadingly-named 'set_fs()' macrostatement Demo program: devtfix.cpp it clarifies what 'dev_t' parameter-value to use in a 'mknod()' system-call lesson4.ppt (PowerPoint Slides) Demo program: elfcheck.cpp illustrating use of standard file-I/O library functions by a user program Kernel module: cmosram.c showing the implementation for a simple character-mode devicedriver lesson5.ppt (PowerPoint Slides) Kernel module: tasklist.c creates a pseudo-file that will show a list of all the system's active tasks lesson6.ppt (PowerPoint Slides) Demo module: stash.c illustrates the use of 'sleeping' and 'waking' in a character devicedriver lesson7.ppt (PowerPoint Slides) Demo program: concur1.cpp showing why thread-synchronization is needed in multiprogramming Demo program: concur2.cpp revises 'concur1.cpp' so it incorporate the use of 'mutual exclusion' Demo program: yielding.cpp demonstrates the 'nanosleep' system-call in Linux multiprogramming Demo program: concur3.cpp revises 'concur2.cpp' so it uses 'nanosleep' to reduce 'busywaiting' Demo module: newstash.c uses kernel semaphores to remove 'race conditions' observed in 'stash.c' lesson8.ppt (PowerPoint Slides) -- corrected 'cmpxchg' (slide 20) on 9/19/2007 Demo program: showgdt.cpp shows the CPU's Global Descriptor Table (if our 'dram.ko' is installed) Code-fragment: cmosread.asm a commented disassembly of the kernel's 'rtc_cmos_read()' function Student-contributed tool: findFunction.sh (automates the dissassembly of a specified kernelfunction) lesson9.ppt (PowerPoint Slides) Exercise solution: cmpxchg.cpp (to illustrate the effect of the 'cmpxchg' instruction upon the ZF-flag) lesson10.ppt (PowerPoint Slides) Demo program: showidt.cpp shows the CPU's Interrupt Descriptor Table (if our 'dram.ko' is

4 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

installed) Demo module: smpinfo.c shows the MP Configuration Table and Base Configuration Table's entries Demo module: ioapic.c lets users view the current register-values in the memory-mapped I/O APIC -- revised on 9/26/2007 lesson11.ppt (PowerPoint Slides) Demo module: ioremap.c for illustrating effects of the kernel's 'ioremap()' and 'iounmap()' functions Exercise solution: delaysix.cpp showing one possible way to perform the desired six-second delay lesson12.ppt (PowerPoint Slides) Demo module: pgdir.c for providing a visualization of the current task's 'mapped' memory regions Demo module: cr3.c creates a pseudo-file showing current values held in registers CR3 and CR4 Demo module: mm.c creates a pseudo-file showing information from a task's 'mm_struct' object Demo module: vma.c creates a pseudo-file that shows a task's list of 'vm_area_struct' objects Demo program: domalloc.cpp examines the effect of 'malloc()' on a task's kernel data-structures Demo program: dofork.cpp examines the effect of 'fork()' on the list of 'vm_area_struct' objects lesson13.ppt (PowerPoint Slides) Demo program: testuart.cpp uses 'loopback' mode to illustrate programming of the serial UART Demo program: loopback.cpp showing how a UART could support interprocess communication Demo programs: uartecho.cpp and trycable.cpp for checking that your null-modem cable works lesson14.ppt (PowerPoint Slides) Demo module: uartintr.c provides an example of an interrupt-handler written for the serial-UART lesson15.ppt (PowerPoint Slides) Demo module: vram.c this device-driver provides user-mode access to the video display memory Demo program: rotation.cpp uses 'vram.c' to perform rotation of (red,green,blue) colorcomponents Demo program: inherit.cpp demonstrates inheritance of a parent's memory-map by a childprocess lesson16.ppt (PowerPoint Slides) Demo module: baudrate.c shows the implementation of an 'ioctl()' method in a Linux devicedriver Demo program: setbaud.cpp allows users to query or modify the serial-UART's current baud-rate -- revised on 10/21/2007 lesson17.ppt (PowerPoint Slides) Demo module: tasklet.c illustrates use of a Linux 'tasklet' for doing bottom-half interrupt handling Working example: myserial.c shows a possible solution to the assigned programming project #2

5 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

lesson18.ppt (PowerPoint Slides) Demo module: foo.c a very simple device-driver that lets application programs read 'jiffies_64' Demo program: watchfoo.cpp a program that continually displays the current 'jiffies_64' value Demo module: timedfoo.c this driver uses a kernel timer to curtail the frequency of 'read()' calls Demo module: trytimer.c illustrating how a kernel module might utilize a dynamic kernel timer Demo module: trymacro.c illustrating use of 'sizeof' and 'offsetof', plus the 'container_of()' macro Demo module: tryworkq.c illustrating the use of a 'workqueue' for scheduling some future action Demo module: announce.c shows how a kernel module can display its messages in a window Demo module: defermsg.c uses a workqueue to delay a message's appearance for ten seconds lesson19.ppt (PowerPoint Slides) Demo program: pciprobe.cpp shows how software can enumerate a system's PCI devicefunctions Demo module: utsname.c shows how a module can find out the name of the node it is running on Demo module: tigon3.c shows our network interface's MAC-address and PCI configuration space Exercise solution: anchor.c modifies 'tigon3.c' so it performs the same function on Intel 82573L nic lesson20.ppt (PowerPoint Slides) Demo program: noncanon.cpp shows how applications can enable 'noncanonical' keyboard input Demo program: mapwatch.cpp gives a dynamic visualization of current 'permanent' kernel-maps Demo module: pkmaps.c creates the two pseudo-files required by our 'mapwatch' program above Demo program: smpwatch.cpp shows a real-time display of dual-CPU interrupt-activity under Linux Kernel module: smpwatch.c this driver is required when running our Linux 'smpwatch.cpp' application lesson21.ppt (PowerPoint Slides) Demo module: gbstatus.c it allows users to see the Intel 82573L controller's Device Status register Developer tool: igbe.c to aid with debugging of a device-driver for Intel's GigaBit Ethernet controller lesson22.ppt (PowerPoint Slides) Demo module: tryreset.c performs an experiment with the 'reset' capabilities of the 82573L controller lesson23.ppt (PowerPoint Slides) Demo module: seereset.c shows the sequence of register state-changes during 'reset' of the 82573L Kernel module: setupnic.c performs a system administration chore needed by a 82573L device-driver Shell script: setupnic it will accelerate our attempts at loading the foregoing 'setupnic.ko' kernel-object Database template: ethers for translating from Ethernet-address to Hostname on our 'anchor'

6 of 8

9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

cluster Kernel module: audio.c showing one possible way of answering Question V on our Midterm Exam II lesson24.ppt (PowerPoint Slides) Kernel module: xmit1000.c supports writing a message to our '/dev/nic' device-file via Intel's 82573L -- revised on 11/24/2007 lesson25.ppt (PowerPoint Slides) Kernel module: recv1000.c receives packets sent by our 'xmit1000.c' driver via Intel's 82573L nic lesson26.ppt (PowerPoint Slides) Kernel module: nic.c combines 'write()' and 'read()' from previous 82573L drivers, and adds 'ioctl()' Demo program: sendto.cpp uses 'ethers' lookup and 'ioct()' call to send packets to a specific node lesson27.ppt (PowerPoint Slides) Demo program: syswrite.cpp shows how applications can issue a system-call directly to the kernel Demo module: myexport.c this module can 'export' the address of the kernel's 'sys_call_table[]' array Shell script: myscript automates the address-lookup and module-installation steps for 'myexport.ko' Demo module: newcall.c replaces the code for an obsolete system-call with our own kernel function Demo program: try17.cpp this program can be used for testing our new system-call implementation Exercise solution: unused.c creates a pseudo-file that lists all the obsolete system-call ID-numbers

Announcements
Enrolled students may join the class mailing-list here project 1: Due 7:30pm Tuesday, 25 September 2007 Midterm Exam I: Tuesday, October 2 project 2: Due 7:30pm Tuesday, 23 October 2007 Midterm Exam II: Thursday, November 15 FINAL EXAMINATION: Tuesday, 11 December 2007, 7:30pm ADDENDUM POSTED 17 JULY 2008: My thanks to Parag Nemade for alerting me to the fact that the Linux kernel developers have now removed support for the older 'create_proc_info_entry()' inferface, used in about two dozen of our demo-modules above, beginning with kernel version 2.6.26 (just released). Simplest 'fix' is just to substitute the 'create_proc_read_entry()' interface in place of 'create_proc_info_entry()'; this involves editing about four lines of source-code in each such module-example. See the LDD3 course-textbook, pages 84-86, for complete details [as well as our newly-posted 'newproc.cpp' utility under Systems Software above].
7 of 8 9/11/2013 9:11 PM

CS 635

http://cs.usfca.edu/~cruse/cs635f07/

ADDENDUM POSTED 18 JUN 2013: My thanks to David Robinson for providing a revised and improved version of my tempcdev.c demo, designed to be used with a 64-bit version of the Linux kernel (such as Linux 3.0.4).

Last updated on 06/18/2013

8 of 8

9/11/2013 9:11 PM