High performance computing for non-programmers

Glib Ivashkevych
junior researcher, NSC KIPT

Lecture #8: Linux internals

Linux boot birds-eye view

1 BIOS (UEFI) 2 GRUB1 & GRUB2 3 Kernel image 4 init/systemd 5 Functional system

Linux boot BIOS stage

1 power up, CPU reset
one CPU becomes bootstrap, other - application processors

2 CPU in real mode, POST

CPU behaves like Intel 8086, HW is tested and initialized (kinda)

3 Search for boot device, MBR

MBR - first sector (512 bytes) of boot drive, stage-1 BL + partition table. To get MBR: # dd if=/dev/sda of=/tmp/mbr.bkp bs=512 count=1 bootstrap code 446 bytes partition table 72 bytes 55AA

Linux boot GRUB

1 GRUB stage 1 BL from MBR
can do little (since its about 446 bytes)

2 GRUB stage 1.5

knows filesystems, can access by normal path

3 GRUB stage 2
presents menu, loads kernel image, maneuvers in unreal mode, since kernel image can be larger than 640K Config files: /boot - they are here. /boot/grub2/grub.cfg is a main config file.

Linux boot Kernel

1 Kernel image is loaded
from /boot/vmlinuz-*. bootstrap kernel code is <1Mb, CPU in real mode

2 Kernel bootstrap code

initializations and preparations to transition to protected mode GDT & IDT are set here, A20 line is opened by kernel image is decompressed, paging is set up and

3 Kernel
kernel is now PID 0, setups remaining CPUs, hardware is fully functional kernel launches init (tries /sbin/init, /etc/init, /bin/init, /bin/sh) kernel thread waits for requests

Linux boot init (systemd)

1 systemd reads config
/usr/lib/systemd/system/default.target

2 systemd starts launching units

unit can be: target, service, socket, etc. unit config file specifies requirements and dependencies if no unit config file - use file in /etc/init.d instead

systemd becomes a parent of any process which lost its direct parent (and what the hell does that
mean?! believe it or not, any process in Linux has a parent. except PID 0 and PID 1).

Linux boot process

Instance of running program Has ID called PID and parent Can have command line args and has access to environment Has current working directory

Linux birth of a process

fork/exec pair of system calls fork
is called from parent process returns twice (0 in child, PID of created process in parent)

exec
loads new program to memory typically, never returns

Linux process: CL args and env

argv[0] is a name of a program by convention Depending on actual form of exec environment can be passed from parent or not. If so, it is passed as a copy Process can change its environment variables

Linux process: memory layout

Process has its own address space, which is linear and flat Memory is virtual. Logical addresses in program do not correspond directly to physical addresses Memory is allocated in pages (4KB)

Linux process: memory layout

text: program code, literals, constants
args, env

stack

initialized: explicitly initialized data uninitialized (bss): obvious, mainly

heap virtual address uninitialized data initialized data text

stack: per function call, contains local variables, return values and where to return. One stack per thread heap: dynamically allocated memory
0x00000000

Linux process: memory paging

page 6 page 5 page table page 4 page 3 page 2 page 1 page 0
page frame 3 page frame 1 page frame 2 page frame 0 page frame 6 page frame 4 page frame 5

frame 6 frame 5 frame 4 frame 3 frame 2 frame 1 frame 0 RAM

virtual address

Linux process info

Virtual filesystem /proc /proc/<PID>/ - directory per process
/proc/<PID>/status - various info about process /proc/<PID>/cmdline /proc/<PID>/cwd /proc/<PID>/environ /proc/<PID>/maps, /proc/<PID>/smaps

Linux process execution

Cooperative vs. preemptive multitasking scheduler interrupts system calls signals

mechanisms of control

Linux process scheduling

CPU-bound processes
spend most of the time executing code high throughput

vs. I/O bound processes

conflict

spend most the time waiting for I/O low latency

Linux process scheduling

Linux Completely Fair Scheduler
Tries to emulate ideal multiprocessing CPU time is divided into epochs Process has nice value, which weight their timeslice Processes with lowest CPU time are scheduled

Linux interrupts
Interrupt: signal, that breaks current CPU activity Async: generated by hardware Sync: generated by CPU itself (exceptionally conditions, etc.)

Linux interrupts
Kernel should handle interrupts
peripheral device kernel decides what to do: schedule process, return to previously running critical actions (other interrupts disabled) non-critical deferred

interrupt controller

CPU

stop execution load kernel routine called ISR

(interrupt service routine)

Linux system calls

A way to utilize kernel services from user-space (say, open file or fork) Used via C standard library Kernel enters the scene and performs task in kernel space

Linux process death

Process died Zombie Zombie = entry in process table with exit code Zombies retain little memory, but consume PIDs Parent should use wait system call, or exit, so that zombie child can be inherited by init process

Linux processes: tools to manage

top
useful keys: -p <PID> -n -b -u <UID> shortcuts: P (sort by CPU%) M (sort by memory%) k (send signal to process) c, z

iotop to check disk usage

useful keys: -o -k -b -t shortcuts: r (choose sorting, reverse)

Linux processes: tools to manage

pstree
display processes tree useful keys: -p -s

ps
takes info from /proc many different options

Linux virtual filesystem

Skipped: learn it yourself
Consider this: Ch. 13 in Linux Kernel Development by Robert Love or Ch. 2 in How Linux Works by Brian Ward

Linux
The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquility it's right. If it disturbs you it's wrong until either the machine or your mind is changed. Robert Pirsig, in Zen and the Art of Motorcycle Maintenance

Questions?