FREE-ELECTRONS-embedded Linux Kernel and Drivers

EmbeddedLinuxdriverdevelopment
EmbeddedLinux
kernelanddriver
development
MichaelOpdenacker
ThomasPetazzoni
FreeElectrons
Copyright20042009,FreeElectrons.
CreativeCommonsBYSA3.0license
Latestupdate:May26,2010,
Documentsources,updatesandtranslations:
http://freeelectrons.com/docs/kernel
Corrections,suggestions,contributionsandtranslationsarewelcome!
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents
Driverdevelopment
Loadablekernelmodules
Sleeping,Interruptmanagement
Memorymanagement
Handlingconcurrency
I/Omemoryandports
Debugging
Characterdrivers
mmap
Processesandscheduling
Devicemodel,sysfs
2
Driverdevelopment
Loadablekernelmodules
3
hellomodule
/*hello.c*/
#include<linux/init.h>
#include<linux/module.h>
#include<linux/kernel.h>
staticint__inithello_init(void)
{
printk(KERN_ALERT"Goodmorrow");
printk(KERN_ALERT"tothisfairassembly.\n");
return0;
}
staticvoid__exithello_exit(void)
{
printk(KERN_ALERT"Alas,poorworld,whattreasure");
printk(KERN_ALERT"hastthoulost!\n");
}
__init:
removedafterinitialization
(statickernelormodule).
__exit:discardedwhen
modulecompiledstatically
intothekernel.
module_init(hello_init);
module_exit(hello_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Greetingmodule");
MODULE_AUTHOR("WilliamShakespeare");
Exampleavailableonhttp://freeelectrons.com/doc/c/hello.c
Modulelicense
Severalusages
Usedtorestrictthekernelfunctionsthatthemodulecanuseifit
isn'taGPLlicensedmodule
DifferencebetweenEXPORT_SYMBOL()and
EXPORT_SYMBOL_GPL()
Usedbykerneldeveloperstoidentifyissuescomingfrom
proprietarydrivers,whichtheycan'tdoanythingabout
(Taintedkernelnoticeinkernelcrashesandoopses).
Usefulforuserstocheckthattheirsystemis100%free
(check/proc/sys/kernel/tainted)
Values
GPL,GPLv2,GPLandadditionalrights,DualMIT/GPL,Dual
BSD/GPL,DualMPL/GPL,Proprietary
5
Compilingaoutoftreemodule
ThebelowMakefileshouldbereusableforanyoutoftree
Linux2.6module
Intreemodulesarecoveredlater
Justrunmaketobuildthehello.kofile
Caution:makesurethereisa[Tab]characteratthe
beginningofthe$(MAKE)line(makesyntax)
Either
#Makefileforthehellomodule
[Tab]!
(nospaces)
objm:=hello.o
KDIR:=/lib/modules/$(shellunamer)/build
PWD:=$(shellpwd)
default:
$(MAKE)C$(KDIR)SUBDIRS=$(PWD)modules
fullkernel
sourcedirectory
(configuredand
compiled)
orjustkernel
headersdirectory
(minimum
needed)
Exampleavailableonhttp://freeelectrons.com/doc/c/Makefile
6
Modulesandkernelversion
Tobecompiled,akernelmoduleneedsaccesstothekernel
headers,containingthefunctions,typesandconstantsdefinitions
Twosolutions
Fullkernelsources
Onlykernelheaders(linuxheaders*packagesinDebian/Ubuntu
distributions)
Thesourcesorheadersmustbeconfigured
Manymacrosorfunctionsdependontheconfiguration
AkernelmodulecompiledagainstversionXofkernelheaders
willnotloadinkernelversionY
modprobe/insmodwillsayInvalidmoduleformat
7
Symbolsexportedtomodules
Fromakernelmodule,
onlyalimitednumberofkernelfunctionscanbecalled
Functionsandvariableshavetobeexplicitlyexported
bythekerneltobevisiblefromakernelmodule
Twomacrosareusedinthekernel
toexportfunctionsandvariables:
EXPORT_SYMBOL(symbolname),whichexportsa
functionorvariabletoallmodules
EXPORT_SYMBOL_GPL(symbolname),whichexportsa
functionorvariableonlytoGPLmodules
Anormaldrivershouldnotneedanynonexportedfunction.
8
hellomodulewithparameters
/*hello_param.c*/
#include<linux/init.h>
#include<linux/module.h>
#include<linux/moduleparam.h>
MODULE_LICENSE("GPL");
Thanksto
JonathanCorbet
fortheexample!
/*Acoupleofparametersthatcanbepassedin:howmanytimeswesay
hello,andtowhom*/
staticchar*whom="world";
module_param(whom,charp,0);
staticinthowmany=1;
module_param(howmany,int,0);
staticint__inithello_init(void)
{
inti;
for(i=0;i<howmany;i++)
printk(KERN_ALERT"(%d)Hello,%s\n",i,whom);
return0;
}
staticvoid__exithello_exit(void)
{
printk(KERN_ALERT"Goodbye,cruel%s\n",whom);
}
module_init(hello_init);
module_exit(hello_exit);
Exampleavailableonhttp://freeelectrons.com/doc/c/hello_param.c
Declaringamoduleparameter
module_param(
name,
/*nameofanalreadydefinedvariable*/
type,
/*eitherbyte,short,ushort,int,uint,long,
ulong,charp,orbool.
(checkedatcompiletime!)*/
perm
/*for/sys/module/<module_name>/parameters/<param>
0:nosuchmoduleparametervaluefile*/
);
Example
intirq=5;
module_param(irq,int,S_IRUGO);
10
Declaringamoduleparameterarray
module_param_array(
name,
/*nameofanalreadydefinedarray*/
type,
/*sameasinmodule_param*/
num,
/*numberofelementsinthearray,orNULL(nocheck?)*/
perm
/*sameasinmodule_param*/
);
Example
staticintbase[MAX_DEVICES]={0x820,0x840};
module_param_array(base,int,NULL,0);
11
Driverdevelopment
Addingsourcestothekerneltree
12
Newdriverinkernelsources(1)
Toaddanewdrivertothekernelsources:
Addyournewsourcefiletotheappropriatesourcedirectory.
Example:drivers/usb/serial/navman.c
Describetheconfigurationinterfaceforyournewdriver
byaddingthefollowinglinestotheKconfigfileinthisdirectory:
configUSB_SERIAL_NAVMAN
tristate"USBNavmanGPSdevice"
dependsonUSB_SERIAL
help
Tocompilethisdriverasamodule,chooseMhere:the
modulewillbecallednavman.
13
Newdriverinkernelsources(2)
AddalineintheMakefilefilebasedontheKconfigsetting:
obj$(CONFIG_USB_SERIAL_NAVMAN)+=navman.o
Runmakexconfigandseeyournewoptions!
Runmakeandyournewfilesarecompiled!
SeeDocumentation/kbuild/fordetails
14
HowtocreateLinuxpatches
Downloadthelatestkernelsources
Makeacopyofthesesources:
rsyncalinux2.6.9rc2/linux2.6.9rc2patch/
Applyyourchangestothecopiedsources,andtestthem.
Runmakedistcleantokeeponlysourcefiles.
Createapatchfile:
diffNurlinux2.6.9rc2/\
linux2.6.9rc2patch/>patchfile
Alwayscomparethewholesourcestructures
(suitableforpatchp1)
Patchfilename:shouldrecallwhatthepatchisabout.
Ifyouneedtomanagealotofpatches,usegitorquiltinstead
ThankstoNicolasRougier(Copyright2003,
http://webloria.loria.fr/~rougier/)fortheTuximage
15
PracticallabWritingmodules
Writeakernelmodulewithseveral
capabilities,includingmodule
parameters.
Accesskernelinternalsfromyour
module.
Setuptheenvironmenttocompileit
16
Driverdevelopment
Memorymanagement
17
Physicalandvirtualmemory
Physicaladdressspace
Virtualaddressspaces
0xFFFFFFFF
0xFFFFFFFFF
I/Omemory3
I/Omemory2
I/Omemory1
Flash
0xFFFFFFFF
0xC0000000
Memory
Management
Unit
MMU
Process1
0x00000000
0x00000000
CPU
0xFFFFFFFF
RAM1
RAM0
Kernel
Kernel
0xC0000000
Alltheprocesseshavetheir
ownvirtualaddressspace,and
runasiftheyhadaccesstothe
wholeaddressspace.
Process2
0x00000000
0x00000000
18
kmallocandkfree
Basicallocators,kernelequivalentsofglibc'smallocand
free.
#include<linux/slab.h>
staticinlinevoid*kmalloc(size_tsize,intflags);
size:numberofbytestoallocate
flags:priority(explainedinafewpages)
voidkfree(constvoid*objp);
Example:(drivers/infiniband/core/cache.c)
structib_update_work*work;
work=kmalloc(sizeof*work,GFP_ATOMIC);
...
kfree(work);
19
kmallocfeatures
Quick(unlessit'sblockedwaitingformemorytobefreed).
Doesn'tinitializetheallocatedarea.
TheallocatedareaiscontiguousinphysicalRAM.
Allocatesby2nsizes,andusesafewmanagementbytes.
So,don'taskfor1024whenyouneed1000!You'dget2048!
Caution:driversshouldn'ttrytokmalloc
morethan128KB(upperlimitinsomearchitectures).
Minimummemoryconsumption:
32or64bytes(pagesizedependent).
20
Mainkmallocflags(1)
Definedininclude/linux/gfp.h(GFP:__get_free_pages)
GFP_KERNEL
Standardkernelmemoryallocation.Mayblock.Fineformost
needs.
GFP_ATOMIC
RAMallocatedfromcodewhichisnotallowedtoblock(interrupt
handlers)orwhichdoesn'twanttoblock(criticalsections).Never
blocks.
GFP_USER
Allocatesmemoryforuserprocesses.Mayblock.Lowestpriority.
21
Mainkmallocflags(2)
Extraflags(canbeaddedwith|)
__GFP_DMAorGFP_DMA
AllocateinDMAzone
__GFP_ZERO
Returnsazeroedpage.
__GFP_NOFAIL
Mustnotfail.Nevergives
up.Caution:useonlywhen
mandatory!
__GFP_NORETRY
Ifallocationfails,doesn'ttry
togetfreepages.
Example:
GFP_KERNEL|__GFP_DMA
Note:almostonly
__GFP_DMAorGFP_DMA
usedindevicedrivers.
22
Relatedallocationfunctions
Again,namessimilartothoseofClibraryfunctions
staticinlinevoid*kzalloc(
size_tsize,gfp_tflags);
Zeroestheallocatedbuffer.
staticinlinevoid*kcalloc(
size_tn,size_tsize,gfp_tflags);
Allocatesmemoryforanarrayofnelementsofsizesize,
andzeroesitscontents.
void*__must_checkkrealloc(
constvoid*,size_t,gfp_t);
Changesthesizeofthegivenbuffer.
23
Availableallocators
Memoryisallocatedusingslabs(groupsofoneormorecontinuouspages
fromwhichobjectsareallocated).Severalcompatibleslaballocatorsare
available:
SLAB:original,wellprovenallocatorinLinux2.6.
SLOB:muchsimpler.Morespaceefficientbutdoesn'tscalewell.Saves
afewhundredsofKBinsmallsystems(dependson
CONFIG_EMBEDDED)
SLUB:thenewdefaultallocatorsince2.6.23,simplerthanSLAB,
scalingmuchbetter(inparticularforhugesystems)andcreatingless
fragmentation.
24
Slabcachesandmemorypools
Slabcaches:makeitpossibletoallocatemultiple
objectsofthesamesize,withoutwastingRAM.
Sofar,mainlyusedincoresubsystems,
butnotmuchindevicedrivers
(exceptUSBandSCSIdrivers)
Memorypools:poolsofpreallocatedobjects,
toincreasethechancesofallocationstosucceed.
Oftenusedwithfilecaches.
25
Allocatingbypages
MoreappropriatewhenyouneedbigslicesofRAM:
Apageisusually4K,butcanbemadegreaterinsomearchitectures
(sh,mips:4,8,16or64K,butnotconfigurableini386orarm).
unsignedlongget_zeroed_page(intflags);
Returnsapointertoafreepageandfillsitupwithzeros
unsignedlong__get_free_page(intflags);
Same,butdoesn'tinitializethecontents
unsignedlong__get_free_pages(intflags,
unsignedintorder);
ReturnsapointeronanareaofseveralcontiguouspagesinphysicalRAM.
order:log2(<number_of_pages>)
Ifvariable,canbecomputedfromthesizewiththeget_orderfunction.
Maximum:8192KB(MAX_ORDER=11ininclude/linux/mmzone.h),
exceptinafewarchitectureswhenoverwrittenwithCONFIG_FORCE_MAX_ZONEORDER.
26
Freeingpages
voidfree_page(unsignedlongaddr);
voidfree_pages(unsignedlongaddr,
unsignedintorder);
Needtousethesameorderasinallocation.
27
vmalloc
vmalloccanbeusedtoobtaincontiguousmemoryzones
invirtualaddressspace(evenifpagesmaynotbe
contiguousinphysicalmemory).
void*vmalloc(unsignedlongsize);
voidvfree(void*addr);
28
Memoryutilities
void*memset(void*s,intc,size_tcount);
Fillsaregionofmemorywiththegivenvalue.
void*memcpy(void*dest,
constvoid*src,
size_tcount);
Copiesoneareaofmemorytoanother.
Usememmovewithoverlappingareas.
LotsoffunctionsequivalenttostandardClibraryonesdefined
ininclude/linux/string.h
andininclude/linux/kernel.h(sprintf,etc.)
29
Kernelmemorydebugging
Debuggingfeaturesavailablesince2.6.31
Kmemcheck
Dynamiccheckerforaccesstouninitializedmemory.
Onlyavailableonx86sofar,butwillhelptoimprovearchitecture
independentcodeanyway.
SeeDocumentation/kmemcheck.txtfordetails.
Kmemleak
Dynamiccheckerformemoryleaks
Thisfeatureisavailableforallarchitectures.
SeeDocumentation/kmemleak.txtfordetails.
Bothhaveasignificantoverhead.Onlyusethemindevelopment!
30
Driverdevelopment
I/Omemoryandports
31
PortI/Ovs.MemoryMappedI/O
MMIO
PIO
Sameaddressbustoaddress
memoryandI/Odevices
Differentaddressspacesfor
memoryandI/Odevices
AccesstotheI/Odevices
usingregularinstructions
UsesaspecialclassofCPU
instructionstoaccessI/O
devices
MostwidelyusedI/Omethod
acrossthedifferent
architecturessupportedby
Linux
Exampleonx86:INandOUT
instructions
32
RequestingI/Oports
/proc/ioportsexample(x86)
0000001f:dma1
00200021:pic1
00400043:timer0
00500053:timer1
0060006f:keyboard
00700077:rtc
0080008f:dmapagereg
00a000a1:pic2
00c000df:dma2
00f000ff:fpu
0100013f:pcmcia_socket0
01700177:ide1
01f001f7:ide0
03760376:ide1
0378037a:parport0
03c003df:vga+
03f603f6:ide0
03f803ff:serial
0800087f:0000:00:1f.0
08000803:PM1a_EVT_BLK
08040805:PM1a_CNT_BLK
0808080b:PM_TMR
08200820:PM2_CNT_BLK
0828082f:GPE0_BLK
...
structresource*request_region(
unsignedlongstart,
unsignedlonglen,
char*name);
Triestoreservethegivenregionandreturns
NULLifunsuccessful.Example:
request_region(0x0170,8,"ide1");
voidrelease_region(
unsignedlongstart,
unsignedlonglen);
Seeinclude/linux/ioport.hand
kernel/resource.c
33
MappingI/Oportsinvirtualmemory
SinceLinux2.6.9,itispossibletogetavirtualaddress
correspondingtoanI/Oportsrange.
AllowsfortransparentuseofPIOandMMIOatthesame
time,asthismemorycanbeaccessedinthesamewayas
MMIOmemory.
Themappingisdoneusingtheioport_mapfunction:
#include<asm/io.h>;
void*ioport_map(unsignedlongport,
unsignedlongnr);
voidioport_unmap(void*address);
34
RequestingI/Omemory
/proc/iomemexample
Equivalentfunctionswiththesame
000000000009efff:SystemRAM
0009f0000009ffff:reserved
000a0000000bffff:VideoRAMarea
000c0000000cffff:VideoROM
000f0000000fffff:SystemROM
001000003ffadfff:SystemRAM
001000000030afff:Kernelcode
0030b000003b4bff:Kerneldata
3ffae0003fffffff:reserved
40000000400003ff:0000:00:1f.1
4000100040001fff:0000:02:01.0
4000100040001fff:yenta_socket
4000200040002fff:0000:02:01.1
4000200040002fff:yenta_socket
40400000407fffff:PCICardBus#03
4080000040bfffff:PCICardBus#03
40c0000040ffffff:PCICardBus#07
41000000413fffff:PCICardBus#07
a0000000a0000fff:pcmcia_socket0
a0001000a0001fff:pcmcia_socket1
e0000000e7ffffff:0000:00:00.0
e8000000efffffff:PCIBus#01
e8000000efffffff:0000:01:00.0
...
interface
structresource*request_mem_region(
unsignedlongstart,
unsignedlonglen,
char*name);
voidrelease_mem_region(
unsignedlongstart,
unsignedlonglen);
35
MappingI/Omemoryinvirtualmemory
ToaccessI/Omemory,driversneedtohaveavirtual
addressthattheprocessorcanhandle.
Theioremapfunctionssatisfythisneed:
#include<asm/io.h>;
void*ioremap(unsignedlongphys_addr,
unsignedlongsize);
voidiounmap(void*address);
Caution:checkthatioremapdoesn'treturnaNULL
address!
Notethatanioremap_nocachefunctionexists.
ThisdisablestheCPUcacheatthegivenaddressrange.
36
Differenceswithstandardmemory
Readsandwritesonmemorycanbecached
Thecompilermaychoosetowritethevalueinacpu
register,andmayneverwriteitinmainmemory.
Thecompilermaydecidetooptimizeorreorderreadand
writeinstructions.
37
AvoidingI/Oaccessissues
CachingonI/Oportsormemoryalreadydisabled,eitherbythe
hardwareorbyLinuxinitcode.
UsethevolatilestatementinyourCcodetopreventthe
compilerfromusingregistersinsteadofwritingtomemory.
Memorybarriersaresuppliedtoavoidreordering
Hardwareindependent
#include<asm/kernel.h>
voidbarrier(void);
Hardwaredependent
#include<asm/system.h>
voidrmb(void);
Onlyimpactsthebehaviorofthe
voidwmb(void);
compiler.Doesn'tpreventreordering voidmb(void);
intheprocessor:maynotworkon
Safeonallarchitectures!
SMPoroutoforderarchitectures!
38
AccessingI/OportsTheoldstyle
Functionstoread/writebytes,wordandlongstoI/Oports:
unsignedin[bwl](unsignedlong*addr);
voidout[bwl](unsignedport,unsignedlong*addr);
Andthestringsvariants:oftenmoreefficientthanthecorresponding
Cloop,iftheprocessorsupportssuchoperations!
voidins[bwl](unsignedport,void*addr,unsignedlong
count);
voidouts[bwl](unsignedport,void*addr,unsignedlong
count);
Notusablewithioport_map():
thosefunctionsneedanI/Oportnumber,notapointertomemory.
PerfectlyfineifdoingonlyPIO
39
AccessingMMIOdevicesTheoldstyle
Directlyreadingfromorwritingtoaddressesreturnedby
ioremap(pointerdereferencing)maynotworkonsome
architectures.
TodoPCIstyle,littleendianaccesses:
unsignedread[bwl](void*addr);
voidwrite[bwl](unsigneda,void*addr);
PerfectlyfineifdoingonlyMMIO.WhendoingmixedPIOand
MMIO,theioread/iowritefamilyoffunctionscanbeused
everywhereinsteadofread/write
40
AccessingI/OmemoryThenewstyle
Thankstoioport_map(),itispossibletomixPIOandMMIOin
atransparentway.Thefollowingfunctionscanbeusedtoaccess
memoryareasreturnedbyioport_map()orioremap():
unsignedintioread8(void*addr);(samefor16and32)
voidiowrite8(u8value,void*addr);(samefor16and32)
Toreadorwriteaseriesofvalues:
voidioread8_rep(void*addr,void*buf,unsignedlongcount);
voidiowrite8_rep(void*addr,constvoid*buf,unsignedlongcount);
Otherusefulfunctions:
voidmemset_io(void*addr,u8value,unsignedintcount);
voidmemcpy_fromio(void*dest,void*source,unsignedintcount);
voidmemcpy_toio(void*dest,void*source,unsignedintcount);
Note:manydriversstilluseoldfunctionsinstead:
readb,readl,readw,writeb,writel,writew,outb,inb,...
41
/dev/mem
Usedtoprovideuserspaceapplications
withdirectaccesstophysicaladdresses.
Usage:open/dev/memandreadorwriteatgivenoffset.
Whatyoureadorwriteisthevalue
atthecorrespondingphysicaladdress.
UsedbyapplicationssuchastheXserver
towritedirectlytodevicememory.
Since2.6.26(x86only,2.6.32status):onlynonRAMcan
beaccessedforsecurityreasons,unlessexplicitly
configuredotherwise(CONFIG_STRICT_DEVMEM).
42
PracticallabI/Omemoryandports
Makearemoteconnectiontoyour
boardthroughssh.
Accessthesystemconsolethrough
thenetwork.
ReservetheI/Omemoryaddresses
usedbytheserialport.
Readdeviceregistersandwritedata
tothem,tosendcharactersonthe
serialport.
43
Driverdevelopment
Characterdrivers
44
Usefulnessofcharacterdrivers
Exceptforstoragedevicedrivers,mostdriversfordeviceswith
inputandoutputflowsareimplementedascharacterdrivers.
So,mostdriversyouwillfacewillbecharacterdrivers
Youwillregretifyousleepduringthispart!
45
Creatingacharacterdriver
Userspace
Userspaceneeds
Thenameofadevicefilein/devto
interactwiththedevicedriverthrough
regularfileoperations(open,read,write,
close...)
Read
buffer
read
write
Thekernelneeds
Copytouser
major/minor
Read
handler
Write
handler
Copyfromuser
/dev/foo
Toknowwhichdriverisinchargeofdevice
fileswithagivenmajor/minornumberpair
Foragivendriver,tohavehandlers(file
operations)toexecutewhenuserspace
opens,reads,writesorclosesthedevice
file.
Write
string
Devicedriver
Kernelspace
46
Declaringacharacterdriver
Devicenumberregistration
Needtoregisteroneormoredevicenumbers(major/minor
pairs),dependingonthenumberofdevicesmanagedbythe
driver.
Needtofindfreeones!
Fileoperationsregistration
Needtoregisterhandlerfunctionscalledwhenuserspace
programsaccessthedevicefiles:open,read,write,
ioctl,close...
47
Informationonregistereddevices
Registereddevicesarevisiblein/proc/devices:
Characterdevices:
1mem
4/dev/vc/0
4tty
4ttyS
5/dev/tty
5/dev/console
5/dev/ptmx
6lp
10misc
13input
14sound
...
Blockdevices:
1ramdisk
3ide0
8sd
9md
22ide1
65sd
66sd
67sd
68sd
Major
number
Registered
name
48
dev_tdatatype
Kerneldatatypetorepresentamajor/minornumberpair
Alsocalledadevicenumber.
Definedin<linux/kdev_t.h>
Linux2.6:32bitsize(major:12bits,minor:20bits)
Macrotocomposethedevicenumber:
MKDEV(intmajor,intminor);
Macrotoextracttheminorandmajornumbers:
MAJOR(dev_tdev);
MINOR(dev_tdev);
49
Registeringdevicenumbers(1)
#include<linux/fs.h>
intregister_chrdev_region(
dev_tfrom,
/*Startingdevicenumber*/
unsignedcount,
/*Numberofdevicenumbers*/
constchar*name); /*Registeredname*/
Returns0iftheallocationwassuccessful.
Example
staticdev_tacme_dev=MKDEV(202,128);
if(register_chrdev_region(acme_dev,acme_count,acme)){
printk(KERN_ERRFailedtoallocatedevicenumber\n);
...
50
Registeringdevicenumbers(2)
Ifyoudon'thavefixeddevicenumbersassignedtoyourdriver
Betternottochoosearbitraryones.
Therecouldbeconflictswithotherdrivers.
ThekernelAPIoffersaalloc_chrdev_regionfunction
tohavethekernelallocatefreeonesforyou.Youcanfindthe
allocatedmajornumberin/proc/devices.txt.
51
Fileoperations(1)
Beforeregisteringcharacterdevices,youhavetodefine
file_operations(calledfops)forthedevicefiles.
Herearethemainones:
int(*open)(
structinode*,/*Correspondstothedevicefile*/
structfile*);/*Correspondstotheopenfiledescriptor*/
Calledwhenuserspaceopensthedevicefile.
int(*release)(
structinode*,
structfile*);
Calledwhenuserspaceclosesthefile.
52
Thefilestructure
Iscreatedbythekernelduringtheopencall.
Representsopenfiles.
mode_tf_mode;
Thefileopeningmode(FMODE_READand/orFMODE_WRITE)
loff_tf_pos;
Currentoffsetinthefile.
structfile_operations*f_op;
Allowstochangefileoperationsfordifferentopenfiles!
structdentry*f_dentry
Usefultogetaccesstotheinode:f_dentry>d_inode.
53
Fileoperations(2)
ssize_t(*read)(
structfile*,
/*Openfiledescriptor*/
__userchar*,
/*Userspacebuffertofillup*/
size_t,
/*Sizeoftheuserspacebuffer*/
loff_t*);
/*Offsetintheopenfile*/
Calledwhenuserspacereadsfromthedevicefile.
ssize_t(*write)(
structfile*,
__userconstchar*,
/*Userspacebuffertowrite
tothedevice*/
size_t,
/*Sizeoftheuserspacebuffer*/
loff_t*);
/*Offsetintheopenfile*/
Calledwhenuserspacewritestothedevicefile.
54
Exchangingdatawithuserspace(1)
Indrivercode,youcan'tjustmemcpybetween
anaddresssuppliedbyuserspaceand
theaddressofabufferinkernelspace!
Correspondtocompletelydifferent
addressspaces(thankstovirtualmemory)
Theuserspaceaddressmaybeswappedouttodisk
Theuserspaceaddressmaybeinvalid
(userspaceprocesstryingtoaccessunauthorizeddata)
55
Exchangingdatawithuserspace(2)
Youmustusededicatedfunctionssuchasthefollowingones
inyourreadandwritefileoperationscode:
include<asm/uaccess.h>
unsignedlongcopy_to_user(void__user*to,
constvoid*from,
unsignedlongn);
unsignedlongcopy_from_user(void*to,
constvoid__user*from,
unsignedlongn);
Makesurethatthesefunctionsreturn0!
Anotherreturnvaluewouldmeanthattheyfailed.
56
Fileoperations(3)
int(*ioctl)(structinode*,structfile*,
unsignedint,unsignedlong);
Canbeusedtosendspecificcommandstothedevice,whichare
neitherreadingnorwriting(e.g.changingthespeedofaserial
port,settingvideooutputformat,queryingadeviceserial
number...).
int(*mmap)(structfile*,
structvm_area_struct*);
Askingfordevicememorytobemapped
intotheaddressspaceofauserprocess.
Moreinourmmapsection.
Thesewerejustthemainones:
about25fileoperationscanbeset,correspondingtoall
thesystemcallsthatcanbeperformedonopenfiles.
57
readoperationexample
staticssize_t
acme_read(structfile*file,char__user*buf,size_tcount,loff_t*ppos)
{
/*Theacme_bufaddresscorrespondstoadeviceI/Omemoryarea*/
/*ofsizeacme_bufsize,obtainedwithioremap()*/
intremaining_size,transfer_size;
remaining_size=acme_bufsize(int)(*ppos);//byteslefttotransfer
if(remaining_size==0){/*Allread,returning0(EndOfFile)*/
return0;
}
/*Sizeofthistransfer*/
transfer_size=min(remaining_size,(int)count);
if(copy_to_user(buf/*to*/,acme_buf+*ppos/*from*/,transfer_size)){
returnEFAULT;
}else{/*Increasethepositionintheopenfile*/
*ppos+=transfer_size;
returntransfer_size;
}
}
Readmethod
Pieceofcodeavailablein
http://freeelectrons.com/doc/c/acme.c
58
writeoperationexample
staticssize_t
acme_write(structfile*file,constchar__user*buf,size_tcount,loff_t*ppos)
{
intremaining_bytes;
/*Numberofbytesnotwrittenyetinthedevice*/
remaining_bytes=acme_bufsize(*ppos);
if(count>remaining_bytes){
/*Can'twritebeyondtheendofthedevice*/
returnEIO;
}
if(copy_from_user(acme_buf+*ppos/*to*/,buf/*from*/,count)){
returnEFAULT;
}else{
/*Increasethepositionintheopenfile*/
*ppos+=count;
returncount;
}
}
Writemethod
Pieceofcodeavailablein
http://freeelectrons.com/doc/c/acme.c
59
fileoperationsdefinitionexample(3)
Definingafile_operationsstructure:
#include<linux/fs.h>
staticstructfile_operationsacme_fops=
{
.owner=THIS_MODULE,
.read=acme_read,
.write=acme_write,
};
Youjustneedtosupplythefunctionsyouimplemented!Defaultsfor
otherfunctions(suchasopen,release...)arefineifyoudonot
implementanythingspecial.
60
Characterdeviceregistration(1)
Thekernelrepresentscharacterdriverswithacdevstructure
Declarethisstructureglobally(withinyourmodule):
#include<linux/cdev.h>
staticstructcdevacme_cdev;
Intheinitfunction,initializethestructure:
cdev_init(&acme_cdev,&acme_fops);
61
Characterdeviceregistration(2)
Then,nowthatyourstructureisready,addittothesystem:
intcdev_add(
structcdev*p,
/*Characterdevicestructure*/
dev_tdev,
/*Startingdevicemajor/minornumber
*/
unsignedcount); /*Numberofdevices*/
Example(continued):
if(cdev_add(&acme_cdev,acme_dev,acme_count)){
printk(KERN_ERRChardriverregistrationfailed\n);
...
62
Characterdeviceunregistration
Firstdeleteyourcharacterdevice:
voidcdev_del(structcdev*p);
Then,andonlythen,freethedevicenumber:
voidunregister_chrdev_region(dev_tfrom,
unsignedcount);
Example(continued):
cdev_del(&acme_cdev);
unregister_chrdev_region(acme_dev,acme_count);
63
Linuxerrorcodes
Trytoreporterrorswitherrornumbersasaccurateas
possible!Fortunately,macronamesareexplicitandyou
canrememberthemquickly.
Genericerrorcodes:
include/asmgeneric/errnobase.h
Platformspecificerrorcodes:
include/asm/errno.h
64
Chardriverexamplesummary(1)
staticvoid*acme_buf;
staticintacme_bufsize=8192;
staticintacme_count=1;
staticdev_tacme_dev=MKDEV(202,128);
staticstructcdevacme_cdev;
staticssize_tacme_write(...){...}
staticssize_tacme_read(...){...}
staticstructfile_operationsacme_fops=
{
.owner=THIS_MODULE,
.read=acme_read,
.write=acme_write
};
65
Chardriverexamplesummary(2)
Showshowtohandleerrorsanddeallocateresourcesintherightorder!
staticint__initacme_init(void)
{
interr;
acme_buf=ioremap(ACME_PHYS,
acme_bufsize);
if(!acme_buf){
err=ENOMEM;
gotoerr_exit;
}
if(register_chrdev_region(acme_dev,
acme_count,acme)){
err=ENODEV;
gotoerr_free_buf;
}
cdev_init(&acme_cdev,&acme_fops);
if(cdev_add(&acme_cdev,acme_dev,
acme_count)){
err=ENODEV;
gotoerr_dev_unregister;
}
return0;
err_dev_unregister:
unregister_chrdev_region(
acme_dev,acme_count);
err_free_buf:
iounmap(acme_buf);
err_exit:
returnerr;
}
staticvoid__exitacme_exit(void)
{
cdev_del(&acme_cdev);
unregister_chrdev_region(acme_dev,
acme_count);
iounmap(acme_buf);
}
Completeexamplecodeavailableonhttp://freeelectrons.com/doc/c/acme.c
66
Definethefileoperationscallbacksforthedevicefile:read,write,ioctl...
Inthemoduleinitfunction,reservemajorandminornumberswith
register_chrdev_region(),initacdevstructurewithyourfileoperationsandadditto
thesystemwithcdev_add().
Inthemoduleexitfunction,callcdev_del()andunregister_chrdev_region()
Systemadministration
Loadthecharacterdrivermodule
Createdevicefileswithmatchingmajorandminornumbersifneeded
Thedevicefileisreadytouse!
Systemuser
Userspace
Characterdriverwriter
Kernel
Characterdriversummary
Kernel
Executesthecorrespondingfileoperations
Kernel
Openthedevicefile,read,write,orsendioctl'stoit.
67
PracticallabCharacterdrivers
Writingasimplecharacterdriver,to
writedatatotheserialport.
Onyourworkstation,checkingthat
transmitteddataisreceivedcorrectly.
Exchangingdatabetweenuserspace
andkernelspace.
Practicingwiththecharacterdevice
driverAPI.
Usingkernelstandarderrorcodes.
68
EmbeddedLinuxDriverDevelopment
Driverdevelopment
Processesandscheduling
69
Processes
Aprocessisaninstanceofarunningprogram
Multipleinstancesofthesameprogramcanberunning.
Programcode(textsection)memoryisshared.
Eachprocesshasitsowndatasection,addressspace,processor
state,openfilesandpendingsignals.
Thekernelhasaseparatedatastructureforeachprocess.
70
Threads
InLinux,threadsarejustimplementedasprocesses!
Newthreadsareimplementedasregularprocesses,
withtheparticularitythattheyarecreatedwiththesameaddress
space,filesystemresources,filedescriptorsandsignalhandlers
astheirparentprocess.
71
Aprocesslife
EXIT_ZOMBIE
Parentprocess
Callsfork()
andcreates
anewprocess
Theprocessiselected
bythescheduler
TASK_RUNNING
Readybut
notrunning
Theeventoccurs
ortheprocessreceives
asignal.Processbecomes
runnableagain
Theprocessispreempted
bytheschedulertorun
ahigherprioritytask
TASK_INTERRUPTIBLE
TASK_UNINTERRUPTIBLE
orTASK_KILLABLE
Taskterminatedbutits
resourcesarenotfreedyet.
Waitingforitsparent
toacknowledgeitsdeath.
TASK_RUNNING
Actuallyrunning
Decidestosleep
onawaitqueue
foraspecificevent
Waiting
72
Processcontext
Userspaceprogramsandsystemcallsarescheduledtogether
Processcontinuinginuserspace...
(orreplacedbyahigherpriorityprocess)
(canbepreempted)
Processexecutinginuserspace...
(canbepreempted)
Systemcall
orexception
Kernelcodeexecuted
onbehalfofuserspace
(canbepreemptedtoo!)
Stillhasaccesstoprocess
data(openfiles...)
73
Kernelthreads
Thekerneldoesnotonlyreactfromuserspace(systemcalls,
exceptions)orhardwareevents(interrupts).Italsorunsitsown
processes.
Kernelthreadsarestandardprocessesscheduledandpreemptedin
thesameway(youcanviewthemwithtoporps!)Theyjusthave
nospecialaddressspaceandusuallyrunforever.
Kernelthreadexamples:
pdflush:regularlyflushesdirtymemorypagestodisk
(filechangesnotcommittedtodiskyet).
migration/<n>:PerCPUthreadstomigrateprocesses
betweenprocessors,tobalanceCPUloadbetweenprocessors.
74
Processpriorities
Regularprocesses
Prioritiesfrom20(maximum)to19(minimum)
Onlyrootcansetnegativepriorities
(rootcangiveanegativeprioritytoaregularuserprocess)
Usethenicecommandtorunajobwithagivenpriority:
nicen<priority><command>
Usetherenicecommandtochangeaprocesspriority:
renice<priority>p<pid>
75
Timeslices
Theschedulerprioritizeshighpriorityprocesses
bygivingthemabiggertimeslice.
Initialprocesstimeslice:parent'stimeslicesplitin2
(otherwiseprocesswouldcheatbyforking).
Minimumpriority:5msor1jiffy(whicheverislarger)
Defaultpriorityinjiffies:100ms
Maximumpriority:800ms
Note:actuallydependsonHZ.
Seekernel/sched.cfordetails.
76
Realtimepriorities
Processeswithrealtimeprioritycanbestartedbyrootusingthe
POSIXAPI
Availablethrough<sched.h>(seemansched.hfordetails)
100realtimeprioritiesavailable
SCHED_FIFOschedulingclass:
TheprocessrunsuntilcompletionunlessitisblockedbyanI/O,
voluntarilyrelinquishestheCPU,orispreemptedbyahigherpriority
process.
SCHED_RRschedulingclass:
Difference:theprocessesarescheduledinaRoundRobinway.
Eachprocessisrununtilitexhaustsamaxtimequantum.Thenother
processeswiththesamepriorityarerun,andsoandso...
77
Whenisschedulingrun?
Eachprocesshasaneed_reschedflagwhichisset:
Afteraprocessexhausteditstimeslice.
Afteraprocesswithahigherpriorityisawakened.
Thisflagischecked(possiblycausingtheexecutionofthe
scheduler)
Whenreturningtouserspacefromasystemcall
Whenreturningfrominterrupts(includingthecputimer),
whenkernelpreemptionisenabled.
Schedulingalsohappenswhenkernelcodeexplicitlyruns
schedule()orexecutesanactionthatsleeps.
78
Driverdevelopment
Sleeping
79
Sleeping
Sleepingisneededwhenaprocess(userspaceorkernelspace)
iswaitingfordata.
Userspaceprocess...
...Userspace
Other
processes
are
scheduled
readdevicefile
return
Systemcall...
askfor
data
sleep
...Systemcall
wakeup
Interrupt
handler
datareadynotification
80
Howtosleep(1)
Mustdeclareawaitqueue
Staticqueuedeclaration
DECLARE_WAIT_QUEUE_HEAD(module_queue);
Ordynamicqueuedeclaration
wait_queue_head_tqueue;
init_waitqueue_head(&queue);
81
Howtosleep(2)
Severalwaystomakeakernelprocesssleep
wait_event(queue,condition);
SleepsuntilthetaskiswokenupandthegivenCexpressionistrue.
Caution:can'tbeinterrupted(can'tkilltheuserspaceprocess!)
wait_event_killable(queue,condition);(SinceLinux2.6.25)
Canbeinterrupted,butonlybyafatalsignal(SIGKILL)
wait_event_interruptible(queue,condition);
Canbeinterruptedbyanysignal
wait_event_timeout(queue,condition,timeout);
Alsostopssleepingwhenthetaskiswokenupandthetimeoutexpired.
wait_event_interruptible_timeout(queue,condition,timeout);
Sameasabove,interruptible.
82
HowtosleepExample
Fromdrivers/ieee1394/video1394.c
wait_event_interruptible(
d>waitq,
(d>buffer_status[v.buffer]
==VIDEO1394_BUFFER_READY)
);
if(signal_pending(current))
returnEINTR;
Currentlyrunningprocess
83
Wakingup!
Typicallydonebyinterrupthandlerswhendatasleeping
processesarewaitingforareavailable.
wake_up(&queue);
Wakesupallthewaitingprocessesonthegivenqueue
wake_up_interruptible(&queue);
Wakesuponlytheprocesseswaitinginaninterruptible
sleeponthegivenqueue
84
Sleepingandwakingupimplementation
Theschedulerdoesn'tkeepevaluatingthesleepingcondition!
wait_event_interruptible(&queue,condition);
TheprocessisputintheTASK_INTERRUPTIBLEstate.
wake_up_interruptible(&queue);
Forallprocesseswaitinginqueue,conditionisevaluated.
Whenitevaluatestotrue,theprocessisputback
totheTASK_RUNNINGstate,andtheneed_reschedflagforthe
currentprocessisset.
Thisway,severalprocessescanbewokenupatthesametime.
85
Driverdevelopment
Interruptmanagement
86
Interrupthandlerconstraints
Notrunfromausercontext:
Can'ttransferdatatoandfromuserspace
(needtobedonebysystemcallhandlers)
InterrupthandlerexecutionismanagedbytheCPU,not
bythescheduler.Handlerscan'trunactionsthatmay
sleep,becausethereisnothingtoresumetheir
execution.Inparticular,needtoallocatememorywith
GFP_ATOMIC.
Havetocompletetheirjobquicklyenough:
theyshouldn'tblocktheirinterruptlinefortoolong.
87
Registeringaninterrupthandler(1)
Definedininclude/linux/interrupt.h
intrequest_irq(
Returns0ifsuccessful
unsignedintirq,
Requestedirqchannel
irq_handler_thandler,
Interrupthandler
unsignedlongirq_flags,
Optionmask(seenextpage)
constchar*devname,
Registeredname
void*dev_id);
Pointertosomehandlerdata
CannotbeNULLandmustbeuniqueforsharedirqs!
voidfree_irq(unsignedintirq,void*dev_id);
dev_idcannotbeNULLandmustbeuniqueforsharedirqs.
Otherwise,onasharedinterruptline,
free_irqwouldn'tknowwhichhandlertofree.
88
Registeringaninterrupthandler(2)
irq_flagsbitvalues(canbecombined,noneisfinetoo)
IRQF_DISABLED
"Quick"interrupthandler.Runwithallinterruptsdisabledonthecurrentcpu
(insteadofjustthecurrentline).Forlatencyreasons,shouldonlybeused
whenneeded!
IRQF_SHARED
Runwithinterruptsdisabledonlyonthecurrentirqlineandonthelocalcpu.
Theinterruptchannelcanbesharedbyseveraldevices.Requiresa
hardwarestatusregistertellingwhetheranIRQwasraisedornot.
89
Whentoregisterthehandler
Eitheratdriverinitializationtime:
consumeslotsofIRQchannels!
Oratdeviceopentime(firstcalltotheopenfileoperation):
betterforsavingfreeIRQchannels.
Needtocountthenumberoftimesthedeviceisopened,to
beabletofreetheIRQchannelwhenthedeviceisno
longerinuse.
90
Informationoninstalledhandlers
/proc/interrupts
CPU0
0:5616905XTPICtimer#Registeredname
1:9828XTPICi8042
2:0XTPICcascade
3:1014243XTPICorinoco_cs
7:184XTPICIntel82801DBICH4
8:1XTPICrtc
9:2XTPICacpi
11:566583XTPICehci_hcd,uhci_hcd,
uhci_hcd,uhci_hcd,yenta,yenta,radeon@PCI:1:0:0
12:5466XTPICi8042
14:121043XTPICide0
15:200888XTPICide1
NMI:0
NonMaskableInterrupts
ERR:0
Spuriousinterruptcount
91
Totalnumberofinterrupts
cat/proc/stat|grepintr
intr819076760929671037701102775520196...
Totalnumber
ofinterrupts
IRQ1
total
IRQ2 IRQ3
total
...
92
Theinterrupthandler'sjob
Acknowledgetheinterrupttothedevice
(otherwisenomoreinterruptswillbegenerated)
Read/writedatafrom/tothedevice
Wakeupanywaitingprocesswaitingforthecompletion
ofthisread/writeoperation:
wake_up_interruptible(&module_queue);
93
Interrupthandlerprototype
irqreturn_t(*handler)(
int, //irqnumberofthecurrentinterrupt
void*dev_id
//Pointerusedtokeeptrack
//ofthecorrespondingdevice.
//Usefulwhenseveraldevices
//aremanagedbythesamemodule
);
Returnvalue:
IRQ_HANDLED:recognizedandhandledinterrupt
IRQ_NONE:notonadevicemanagedbythemodule.Usefulto
shareinterruptchannelsand/orreportspuriousinterruptsto
thekernel.
94
Tophalfandbottomhalfprocessing(1)
Splittingtheexecutionofinterrupthandlersin2parts
Tophalf:theinterrupthandlermustcompleteasquickly
aspossible.Onceitacknowledgedtheinterrupt,itjust
schedulesthelengthyrestofthejobtakingcareofthe
data,foralaterexecution.
Bottomhalf:completingtherestoftheinterrupthandler
job.Handlesdata,andthenwakesupanywaitinguser
process.
Bestimplementedbytasklets(alsocalledsoftirqs).
95
Tophalfandbottomhalfprocessing(2)
Declarethetaskletinthemodulesourcefile:
DECLARE_TASKLET(module_tasklet,/*name*/
module_do_tasklet,/*function*/
data/*params*/
);
Schedulethetaskletinthetophalfpart(interrupthandler):
tasklet_schedule(&module_tasklet);
Notethatatasklet_hi_schedulefunctionisavailableto
definehighprioritytaskletstorunbeforeordinaryones.
Bydefault,taskletsareexecutedrightafteralltophalves
(hardirqs)
96
Interruptmanagementfun
Inatraininglab,somebodyforgottounregisterahandleron
asharedinterruptlineinthemoduleexitfunction.
Whydidhiskerneloopswithasegmentationfault
atmoduleunload?
Answer...
Inatraininglab,somebodyfreedthetimerinterrupthandler
bymistake(usingthewrongirqnumber).Thesystemfroze.
Rememberthekernelisnotprotectedagainstitself!
97
Interruptmanagementsummary
Devicedriver
Whenthedevicefileisfirst
open,registeraninterrupt
handlerforthedevice's
interruptchannel.
Interrupthandler
Calledwhenaninterruptis
raised.
Acknowledgetheinterrupt
Tasklet
Processthedata
Wakeupprocesseswaiting
forthedata
Devicedriver
Whenthedeviceisnolonger
openedbyanyprocess,
unregistertheinterrupt
handler.
Ifneeded,scheduleatasklet
takingcareofhandlingdata.
Otherwise,wakeupprocesses
waitingforthedata.
98
PracticallabInterrupts
Addingreadcapabilitytothe
characterdriverdevelopedearlier.
Registeraninterrupthandler.
Waitingfordatatobeavailableinthe
readfileoperation.
Wakingupthecodewhendatais
availablefromthedevice.
99
Driverdevelopment
Concurrentaccesstoresources
100
Sourcesofconcurrencyissues
Thesameresourcescanbeaccessedbyseveralkernel
processesinparallel,causingpotentialconcurrencyissues
Severaluserspaceprogramsaccessingthesamedevicedata
orhardware.Severalkernelprocessescouldexecutethesame
codeonbehalfofuserprocessesrunninginparallel.
Multiprocessing:thesamedrivercodecanberunningon
anotherprocessor.ThiscanalsohappenwithsingleCPUswith
hyperthreading.
Kernelpreemption,interrupts:kernelcodecanbeinterruptedat
anytime(justafewexceptions),andthesamedatamaybe
accessbyanotherprocessbeforetheexecutioncontinues.
101
Avoidingconcurrencyissues
Avoidusingglobalvariablesandshareddatawhenever
possible
(cannotbedonewithhardwareresources).
Usetechniquestomanageconcurrentaccessto
resources.
SeeRustyRussell'sUnreliableGuideToLocking
Documentation/DocBook/kernellocking/
inthekernelsources.
102
Concurrencyprotectionwithlocks
Process1
Failed
Acquirelock
Success
Criticalcodesection
Tryagain
Process2
Waitlockrelease
Success
Sharedresource
Releaselock
103
Linuxmutexes
ThemainlockingprimitivesinceLinux2.6.16.
Betterthancountingsemaphoreswhenbinaryonesare
enough.
Mutexdefinition:
#include<linux/mutex.h>
Initializingamutexstatically:
DEFINE_MUTEX(name);
Orinitializingamutexdynamically:
voidmutex_init(structmutex*lock);
104
lockingandunlockingmutexes
voidmutex_lock(structmutex*lock);
Triestolockthemutex,sleepsotherwise.
Caution:can'tbeinterrupted,resultinginprocessesyoucannotkill!
intmutex_lock_killable(structmutex*lock);
Same,butcanbeinterruptedbyafatal(SIGKILL)signal.Ifinterrupted,returns
anonzerovalueanddoesn'tholdthelock.Testthereturnvalue!!!
intmutex_lock_interruptible(structmutex*lock);
Same,butcanbeinterruptedbyanysignal.
intmutex_trylock(structmutex*lock);
Neverwaits.Returnsanonzerovalueifthemutexisnotavailable.
intmutex_is_locked(structmutex*lock);
Justtellswhetherthemutexislockedornot.
voidmutex_unlock(structmutex*lock);
Releasesthelock.Doitassoonasyouleavethecriticalsection.
105
Reader/writersemaphores
Allowsharedaccessbyunlimitedreaders,orbyonly1writer.Writersget
priority.
voidinit_rwsem(structrw_semaphore*sem);
voiddown_read(structrw_semaphore*sem);
intdown_read_trylock(structrw_semaphore*sem);
intup_read(structrw_semaphore*sem);
voiddown_write(structrw_semaphore*sem);
intdown_write_trylock(structrw_semaphore*sem);
intup_write(structrw_semaphore*sem);
Wellsuitedforrarewrites,holdingthesemaphorebriefly.Otherwise,
readersgetstarved,waitingtoolongforthesemaphoretobereleased.
106
Spinlocks
Lockstobeusedforcodethatisnotallowedtosleep
(interrupthandlers),orthatdoesn'twanttosleep(critical
sections).Beverycarefulnottocallfunctionswhichcan
sleep!
Originallyintendedformultiprocessorsystems
Spinlocksneversleepandkeepspinning
inaloopuntilthelockisavailable.
Stilllocked?
Spinlock
Spinlockscausekernelpreemptiontobedisabled
ontheCPUexecutingthem.
107
Initializingspinlocks
Static
spinlock_tmy_lock=SPIN_LOCK_UNLOCKED;
Dynamic
voidspin_lock_init(spinlock_t*lock);
108
Usingspinlocks
Severalvariants,dependingonwherethespinlockiscalled:
voidspin_[un]lock(spinlock_t*lock);
Doesn'tdisableinterrupts.Usedforlockinginprocesscontext
(criticalsectionsinwhichyoudonotwanttosleep).
voidspin_lock_irqsave/spin_unlock_irqrestore
(spinlock_t*lock,unsignedlongflags);
Disables/restoresIRQsonthelocalCPU.
Typicallyusedwhenthelockcanbeaccessedinbothprocess
andinterruptcontext,topreventpreemptionbyinterrupts.
voidspin_[un]lock_bh(spinlock_t*lock);
Disablessoftwareinterrupts,butnothardwareones.
Usefultoprotectshareddataaccessedinprocesscontext
andinasoftinterrupt(bottomhalf).Noneedtodisable
hardwareinterruptsinthiscase.
Notethatreader/writerspinlocksalsoexist.
109
Deadlocksituations
Theycanlockupyoursystem.Makesuretheyneverhappen!
Don'tcallafunctionthatcan
trytogetaccesstothesame
lock
Holdingmultiplelocksisrisky!
Getlock1
Getlock1
Getlock2
call
Getlock2
Waitforlock1
Dead
Lock!
Getlock1
Dead
Lock!
110
Kernellockvalidator
FromIngoMolnarandArjanvandeVen
Addsinstrumentationtokernellockingcode
Detectviolationsoflockingrulesduringsystemlife,suchas:
Locksacquiredindifferentorder
(keepstrackoflockingsequencesandcomparesthem).
Spinlocksacquiredininterrupthandlersandalsoinprocess
contextwheninterruptsareenabled.
Notsuitableforproductionsystemsbutacceptableoverheadin
development.
SeeDocumentation/lockdepdesign.txtfordetails
111
Alternativestolocking
Aswehavejustseen,lockingcanhaveastrongnegative
impactonsystemperformance.Insomesituations,youcould
dowithoutit.
ByusinglockfreealgorithmslikeReadCopyUpdate(RCU).
RCUAPIavailableinthekernel
(Seehttp://en.wikipedia.org/wiki/RCU).
Whenavailable,useatomicoperations.
112
Atomicvariables
Usefulwhenthesharedresourceisan
integervalue
Evenaninstructionliken++isnot
guaranteedtobeatomiconall
processors!
Header
#include<asm/atomic.h>
Type
atomic_t
containsasignedinteger(atleast24
bits)
Operationswithoutreturnvalue:
voidatomic_inc(atomic_t*v);
voidatomic_dec(atomic_t*v);
voidatomic_add(inti,atomic_t*v);
voidatomic_sub(inti,atomic_t*v);
Simularfunctionstestingtheresult:
intatomic_inc_and_test(...);
intatomic_dec_and_test(...);
intatomic_sub_and_test(...);
Functionsreturningthenewvalue:
intatomic_inc_and_return(...);
intatomic_dec_and_return(...);
intatomic_add_and_return(...);
intatomic_sub_and_return(...);
Atomicoperations(mainones)
Setorreadthecounter:
atomic_set(atomic_t*v,inti);
intatomic_read(atomic_t*v);
113
Atomicbitoperations
Supplyveryfast,atomicoperations
Onmostplatforms,applytoanunsignedlongtype.
Applytoavoidtypeonafewothers.
Set,clear,toggleagivenbit:
voidset_bit(intnr,unsignedlong*addr);
voidclear_bit(intnr,unsignedlong*addr);
voidchange_bit(intnr,unsignedlong*addr);
Testbitvalue:
inttest_bit(intnr,unsignedlong*addr);
Testandmodify(returnthepreviousvalue):
inttest_and_set_bit(...);
inttest_and_clear_bit(...);
inttest_and_change_bit(...);
114
PracticallabLocking
Addlockingtothedrivertoprevent
concurrentaccessestoshared
ressources
115
Driverdevelopment
Debuggingandtracing
116
Debuggingwithprintk
Universaldebuggingtechniqueusedsincethebeginningof
programming(firstfoundincavemendrawings)
Printedornotintheconsoleor/var/log/messages
accordingtothepriority.Thisiscontrolledbytheloglevel
kernelparameter,orthrough/proc/sys/kernel/printk
(seeDocumentation/sysctl/kernel.txt)
Availablepriorities(include/linux/kernel.h):
#defineKERN_EMERG"<0>"/*systemisunusable*/
#defineKERN_ALERT"<1>"/*actionmustbetakenimmediately*/
#defineKERN_CRIT"<2>"/*criticalconditions*/
#defineKERN_ERR"<3>"/*errorconditions*/
#defineKERN_WARNING"<4>"/*warningconditions*/
#defineKERN_NOTICE"<5>"/*normalbutsignificantcondition*/
#defineKERN_INFO"<6>"/*informational*/
#defineKERN_DEBUG"<7>"/*debuglevelmessages*/
117
Debuggingwith/procor/sys
Insteadofdumpingmessagesinthekernellog,youcanhaveyour
driversmakeinformationavailabletouserspace
Throughafilein/procor/sys,whichcontentsarehandledby
callbacksdefinedandregisteredbyyourdriver.
Canbeusedtoshowanypieceofinformation
aboutyourdeviceordriver.
Canalsobeusedtosenddatatothedriverortocontrolit.
Caution:anybodycanusethesefiles.
Youshouldremoveyourdebugginginterfaceinproduction!
Sincethearrivalofdebugfs,nolongerthepreferreddebugging
mechanism
118
Debugfs
Avirtualfilesystemtoexportdebugginginformationtouserspace.
Kernelconfiguration:DEBUG_FS
Kernelhacking>DebugFilesystem
Muchsimplertocodethananinterfacein/procor/sys.
ThedebugginginterfacedisappearswhenDebugfsisconfigured
out.
Youcanmountitasfollows:
sudomounttdebugfsnone/mnt/debugfs
Firstdescribedonhttp://lwn.net/Articles/115405/
APIdocumentedintheLinuxKernelFilesystemAPI:
http://freeelectrons.com/kerneldoc/latest/DocBook/filesystems/index.html
119
Simpledebugfsexample
#include<linux/debugfs.h>
staticchar*acme_buf;
staticunsignedlongacme_bufsize;
staticstructdebugfs_blob_wrapperacme_blob;
staticstructdentry*acme_buf_dentry;
//modulebuffer
staticu32acme_state;
staticstructdentry*acme_state_dentry;
//modulevariable
/*Moduleinit*/
acme_blob.data=acme_buf;
acme_blob.size=acme_bufsize;
acme_buf_dentry=debugfs_create_blob("acme_buf",S_IRUGO,
NULL,&acme_blob);
acme_state_dentry=debugfs_create_bool("acme_state",S_IRUGO,
NULL,&acme_state);
/*Moduleexit*/
debugfs_remove(acme_buf_dentry);
debugfs_remove(acme_state_dentry);
//Create
//newfiles
//indebugfs
//removingthefilesfromdebugfs
120
Debuggingwithioctl
Canusetheioctl()systemcalltoqueryinformation
aboutyourdriver(ordevice)orsendcommandstoit.
Thiscallstheioctlfileoperationthatyoucanregisterin
yourdriver.
Advantage:yourdebugginginterfaceisnotpublic.
Youcouldevenleaveitwhenyoursystem(oritsdriver)isin
thehandsofitsusers.
121
UsingMagicSysRq
Linuxalsohas3fingerkeystosaveyourwork;)
Allowstorunmultipledebug/rescuecommandsevenwhenthe
kernelseemstobeindeeptrouble.Examplecommands:
[ALT][SysRq][d]:killsallprocesses,exceptinit.
[ALT][SysRq][n]:makesRTprocessesniceable.
[ALT][SysRq][s]:attemptstosyncallmountedfilesystems.
[ALT][SysRq][b]:immediatelyrebootwithoutsyncingand
unmounting.
Typicalcombination:[ALT][SysRq][s]
andthen[ALT][SysRq][b]
DetailedinDocumentation/sysrq.txt
122
Debuggingwithgdb
Ifyouexecutethekernelfromadebuggeronthesame
machine,thiswillinterferewiththekernelbehavior.
However,youcanaccessthecurrentkernelstatewithgdb:
gdb/usr/src/linux/vmlinux/proc/kcore
uncompressedkernelkerneladdressspace
Youcanaccesskernelstructures,followpointers...
(readonly!)
Requiresthekerneltobecompiledwith
CONFIG_DEBUG_INFO(Kernelhackingsection)
123
kgdbAkerneldebugger
Theexecutionofthekernelisfullycontrolledbygdbfrom
anothermachine,connectedthroughaserialline.
Candoalmosteverything,includinginsertingbreakpointsin
interrupthandlers.
FeatureincludedinstandardLinuxsince2.6.26(x86and
sparc).arm,mipsandppcsupportmergedin2.6.27.
124
Usingkgdb
Detailsavailableinthekerneldocumentation:
http://freeelectrons.com/kerneldoc/latest/DocBook/kgdb/
RecommendedtoturnonCONFIG_FRAME_POINTERtoaid
inproducingmorereliablestackbacktracesingdb.
YoumustincludeakgdbI/Odriver.Oneofthemiskgdb
overserialconsole(kgdboc:kgdboverconsole,enabled
byCONFIG_KGDB_SERIAL_CONSOLE)
Configurekgdbocatboottimebypassingtothekernel:
kgdboc=<ttydevice>,[baud].Forexample:
kgdboc=ttyS0,115200
125
Usingkgdb(2)
Thenalsopasskgdbwaittothekernel:
itmakeskgdbwaitforadebuggerconnection.
Bootyourkernel,andwhentheconsoleisinitialized,interruptthe
kernelwith[Alt][SyrRq][g].
Onyourworkstation,startgdbasfollows:
%gdb./vmlinux
(gdb)setremotebaud115200
(gdb)targetremote/dev/ttyS0
Onceconnected,youcandebugakernelthewayyouwould
debuganapplicationprogram.
126
DebuggingwithaJTAGinterface
TwotypesofJTAGdongles
Thoseofferingagdbcompatibleinterface,overaserialportoranEthernet
connexion.Gdbcandirectlyconnecttothem.
Thosenotofferingagdbcompatibleinterfacearegenerallysupportedby
OpenOCD(OpenOnChipDebugger)
OpenOCDisthebridgebetweenthegdbdebugginglanguageandthe
JTAGdonglespecificlanguage
http://openocd.berlios.de/web/
Seetheverycompletedocumentation:http://openocd.berlios.de/doc/
Foreachboard,you'llneedanOpenOCDconfigurationfile(askyour
supplier)
SeeveryusefuldetailsonusingEclipse/gcc/gdb/OpenOCDonWindows:
http://www2.amontec.com/sdk4arm/ext/jlynchtutorial20061124.pdfand
http://www.yagarto.de/howto/yagarto2/
127
Morekerneldebuggingtips
EnableCONFIG_KALLSYMS_ALL
(GeneralSetup>Configurestandardkernelfeatures)
togetoopsmessageswithsymbolnamesinsteadofrawaddresses
(thisobsoletestheksymoopstool).
Ifyourkerneldoesn'tbootyetorhangswithoutanymessage,youcan
activateLowLeveldebugging(KernelHackingsection,onlyavailableon
arm):
CONFIG_DEBUG_LL=y
TechniquestolocatetheCinstructionwhichcausedanoops:
http://kerneltrap.org/node/3648
MoreaboutkerneldebugginginthefreeLinuxDeviceDriversbook:
http://lwn.net/images/pdf/LDD3/ch04.pdf
128
TracingwithSystemTap
http://sourceware.org/systemtap/
Infrastructuretoaddinstrumentationtoarunningkernel:
tracefunctions,readandwritevariables,followpointers,gatherstatistics...
Eliminatestheneedtomodifythekernelsourcestoaddone'sown
instrumentationtoinvestigatedafunctionalorperformanceproblem.
Usesasimplescriptinglanguage.
Severalexamplescriptsandprobepointsareavailable.
BasedontheKprobesinstrumentationinfrastructure.
SeeDocumentation/kprobes.txtinkernelsources.
Linux2.6.26:supportedonmostpopularCPUs(armincludedin2.6.25).
However,lackofrecentsupportformips(2.6.16only!).
129
SystemTapscriptexample(1)
#!/usr/bin/envstap
#Usingstatisticsandmapstoexaminekernelmemoryallocations
globalkmalloc
probekernel.function("__kmalloc"){
kmalloc[execname()]<<<$size
}
#Exitafter10seconds
probetimer.ms(10000){exit()}
probeend{
foreach([name]inkmalloc){
printf("Allocationsfor%s\n",name)
printf("Count:%dallocations\n",@count(kmalloc[name]))
printf("Sum:%dKbytes\n",@sum(kmalloc[name])/1000)
printf("Average:%dbytes\n",@avg(kmalloc[name]))
printf("Min:%dbytes\n",@min(kmalloc[name]))
printf("Max:%dbytes\n",@max(kmalloc[name]))
print("\nAllocationsbysizeinbytes\n")
print(@hist_log(kmalloc[name]))
printf("\n\n");
}
}
130
SystemTapscriptexample(2)
#!/usr/bin/envstap
#Logseachfilereadperformedbyeachprocess
probekernel.function("vfs_read")
{
dev_nr=$file>f_dentry>d_inode>i_sb>s_dev
inode_nr=$file>f_dentry>d_inode>i_ino
printf("%s(%d)%s0x%x/%d\n",
execname(),pid(),probefunc(),dev_nr,inode_nr)
}
Nicetutorialonhttp://sources.redhat.com/systemtap/tutorial.pdf
131
Kernelcrashanalysiswithkexec/kdump
kexecsystemcall:makesitpossibleto
callanewkernel,withoutrebootingand
goingthroughtheBIOS/firmware.
Idea:afterakernelpanic,makethe
kernelautomaticallyexecuteanew,
cleankernelfromareservedlocationin
RAM,toperformpostmortemanalysis
ofthememoryofthecrashedkernel.
1.Copydebug
kernelto
reserved
RAM
3.Analyze
crashed
kernelRAM
Standardkernel
2.kernel
panic,kexec
debugkernel
Debugkernel
SeeDocumentation/kdump/kdump.txt
inthekernelsourcesfordetails.
RegularRAM
132
Kernelmarkers
Capabilitytoaddstaticmarkerstokernelcode,
mergedinLinux2.6.24byMatthieuDesnoyers.
Almostnoimpactonperformance,untilthemarkerisdynamically
enabled,byinsertingaprobekernelmodule.
Usefultoinserttracepointsthatwon'tbeimpactedbychangesin
theLinuxkernelsources.
Seemarkerandprobeexample
insamples/markersinthekernelsources.
Seehttp://en.wikipedia.org/wiki/Kernel_marker
133
LTTng
http://lttng.org
ThesuccessoroftheLinuxTraceToolkit(LTT)
Toolkitallowingtocollectandanalyzetracinginformationfrom
thekernel,basedonkernelmarkersandkerneltracepoints.
Sofar,basedonkernelpatches,butdoingitsbesttouseintree
solutions,andtobemergedinthefuture.
Veryprecisetimestamps,verylittleoverhead.
Usefuldocumentationonhttp://lttng.org/?q=node/2#manuals
134
LTTV
ViewerforLTTngtraces
Supportforhugetraces(testedwith15GBones)
Cancombinemultipletracefilesinasingleview.
Graphicalortextinterface
Seehttp://lttng.org/files/lttvdoc/user_guide/
135
PracticallabKerneldebugging
Loadabrokendriverandseeitcrash
Analyzetheerrorinformation
dumpedbythekernel.
Disassemblethecodeandlocate
theexactCinstructionwhichcaused
thefailure.
UsetheJTAGandOpenOCDto
remotelycontrolthekernelexecution
136
Driverdevelopment
mmap
137
mmap(1)
Possibilitytohavepartsofthevirtualaddressspaceofaprogram
mappedtothecontentsofafile!
>cat/proc/1/maps(initprocess)
startend
permoffsetmajor:minorinodemappedfilename
007710000077f000rxp0000000003:051165839/lib/libselinux.so.1
0077f00000781000rwp0000d00003:051165839/lib/libselinux.so.1
0097d00000992000rxp0000000003:051158767/lib/ld2.3.3.so
0099200000993000rp0001400003:051158767/lib/ld2.3.3.so
0099300000994000rwp0001500003:051158767/lib/ld2.3.3.so
0099600000aac000rxp0000000003:051158770/lib/tls/libc2.3.3.so
00aac00000aad000rp0011600003:051158770/lib/tls/libc2.3.3.so
00aad00000ab0000rwp0011700003:051158770/lib/tls/libc2.3.3.so
00ab000000ab2000rwp00ab000000:000
0804800008050000rxp0000000003:05571452/sbin/init(text)
0805000008051000rwp0000800003:05571452/sbin/init(data,stack)
08b4300008b64000rwp08b4300000:000
f6fdf000f6fe0000rwpf6fdf00000:000
fefd4000ff000000rwpfefd400000:000
ffffe000fffff000p0000000000:000
138
mmap(2)
Particularlyusefulwhenthefileisadevicefile!
AllowstoaccessdeviceI/Omemoryandportswithouthavingto
gothrough(expensive)read,writeorioctlcalls!
Xserverexample(mapsexcerpt)
startend
permoffsetmajor:minorinodemappedfilename
08047000081be000rxp0000000003:05310295/usr/X11R6/bin/Xorg
081be000081f0000rwp0017600003:05310295/usr/X11R6/bin/Xorg
...
f4e08000f4f09000rwse000000003:05655295/dev/dri/card0
f4f09000f4f0b000rws4281a00003:05655295/dev/dri/card0
f4f0b000f6f0b000rwse800000003:05652822/dev/mem
f6f0b000f6f8b000rwsfcff000003:05652822/dev/mem
Amoreuserfriendlywaytogetsuchinformation:pmap<pid>
139
mmapoverview
Process
mmap
system
call(once)
access
virtual
address
Processvirtualaddressspace
MMU
Devicedriver
mmapfopcalled
initializesthemapping
access
physical
address
Physicaladdressspace
140
HowtoimplementmmapUserspace
Openthedevicefile
Callthemmapsystemcall(seemanmmapfordetails):
void*mmap(
void*start,
/*Often0,preferredstartingaddress*/
size_tlength, /*Lengthofthemappedarea*/
intprot,
/*Permissions:read,write,execute*/
intflags,
/*Options:sharedmapping,privatecopy...
*/
intfd,
off_toffset
/*Offsetinthefile*/
);
Yougetavirtualaddressyoucanwritetoorreadfrom.
141
HowtoimplementmmapKernelspace
Characterdriver:implementammapfileoperation
andaddittothedriverfileoperations:
int(*mmap)(
structfile*,
/*Openfilestructure*/
structvm_area_struct* /*KernelVMAstructure*/
);
Initializethemapping.
Canbedoneinmostcaseswiththeremap_pfn_range()
function,whichtakescareofmostofthejob.
142
remap_pfn_range()
pfn:pageframenumber
Themostsignificantbitsofthepageaddress
(withoutthebitscorrespondingtothepagesize).
#include<linux/mm.h>
intremap_pfn_range(
structvm_area_struct*,
/*VMAstruct*/
unsignedlongvirt_addr, /*Startinguservirtualaddress*/
unsignedlongpfn, /*pfnofthestartingphysicaladdress*/
unsignedlongsize,
/*Mappingsize*/
pgprot_t
/*Pagepermissions*/
);
143
Simplemmapimplementation
staticintacme_mmap(
structfile*file,structvm_area_struct*vma)
{
size=vma>vm_endvma>vm_start;
if(size>ACME_SIZE)
returnEINVAL;
if(remap_pfn_range(vma,
vma>vm_start,
ACME_PHYS>>PAGE_SHIFT,
size,
vma>vm_page_prot))
returnEAGAIN;
return0;
}
144
devmem2
http://freeelectrons.com/pub/mirror/devmem2.c,byJanDerkBakker
Veryusefultooltodirectlypeek(read)orpoke(write)I/Oaddresses
mappedinphysicaladdressspacefromashellcommandline!
Veryusefulforearlyinteractionexperimentswithadevice,without
havingtocodeandcompileadriver.
Usesmmapto/dev/mem.
Examples(b:byte,h:half,w:word)
devmem20x000c0004h(reading)
devmem20x000c0008w0xffffffff(writing)
devmemisnowavailableinBusyBox,makingiteveneasiertouse.
145
mmapsummary
Thedevicedriverisloaded.
Itdefinesanmmapfileoperation.
Auserspaceprocesscallsthemmapsystemcall.
Themmapfileoperationiscalled.
Itinitializesthemappingusingthedevicephysicaladdress.
Theprocessgetsastartingaddresstoreadfromandwriteto
(dependingonpermissions).
TheMMUautomaticallytakescareofconvertingtheprocess
virtualaddressesintophysicalones.
Directaccesstothehardware!
Noexpensivereadorwritesystemcalls!
146
Driverdevelopment
Kernelarchitecturefordevicedrivers
147
Kernelanddevicedrivers
Userspace
Application
Systemcallinterface
Framework
Kernel
Driver
Businfrastructure
Hardware
148
Kernelanddevicedrivers
Manydevicedriversarenotimplementeddirectlyascharacter
drivers
Theyareimplementedunderaframework,specifictoagiven
devicetype(framebuffer,V4L,serial,etc.)
Theframeworkallowstofactorizethecommonpartsofdriversfor
thesametypeofdevices
Fromuserspace,theyarestillseenascharacterdevicesbythe
applications
Theframeworkallowstoprovideacoherentuserspaceinterface
(ioctl,etc.)foreverytypeofdevice,regardlessofthedriver
Thedevicedriversrelyonthebusinfrastructuretoenumerate
thedevicesandcommunicatewiththem.
149
Kernelframeworks
150
Frameworks
Application
Application
Application
Systemcallinterface
Character
driver
Framebuffer
core
V4L
core
TTY
core
Framebuffer
driver
V4L
driver
Serial
core
IDE
core
SCSI
core
Serial
driver
IDE
driver
USBstorage
driver
Businfrastructure
Block
core
151
Example:framebufferframework
KerneloptionCONFIG_FB
menuconfigFB
tristate"Supportforframebufferdevices"
Implementedindrivers/video/
fb.c,fbmem.c,fbmon.c,fbcmap.c,fbsysfs.c,
modedb.c,fbcvt.c
Implementsasinglecharacterdriveranddefinestheuser/kernel
API
Firstpartofinclude/linux/fb.h
Definesthesetofoperationsaframebufferdrivermustimplement
andhelperfunctionsforthedrivers
structfb_ops
Secondpartofinclude/linux/fb.h
(in#ifdef__KERNEL__)
152
Framebufferdriverskeleton
Skeletondriverindrivers/video/skeletonfb.c
Implementsthesetofframebufferspecificoperationsdefined
bythestructfb_opsstructure
xxxfb_open()
xxxfb_fillrect()
xxxfb_read()
xxxfb_copyarea()
xxxfb_write()
xxxfb_imageblit()
xxxfb_release()
xxxfb_cursor()
xxxfb_checkvar()
xxxfb_rotate()
xxxfb_setpar()
xxxfb_sync()
xxxfb_setcolreg()
xxxfb_ioctl()
xxxfb_blank()
xxxfb_mmap()
xxxfb_pan_display()
153
Aftertheimplementationoftheoperations,definitionofastruct
fb_opsstructure
staticstructfb_opsxxxfb_ops={
.owner=THIS_MODULE,
.fb_open=xxxfb_open,
.fb_read=xxxfb_read,
.fb_write=xxxfb_write,
.fb_release=xxxfb_release,
.fb_check_var=xxxfb_check_var,
.fb_set_par=xxxfb_set_par,
.fb_setcolreg=xxxfb_setcolreg,
.fb_blank=xxxfb_blank,
.fb_pan_display=xxxfb_pan_display,
.fb_fillrect=xxxfb_fillrect,/*Needed!!!*/
.fb_copyarea=xxxfb_copyarea,/*Needed!!!*/
.fb_imageblit=xxxfb_imageblit,/*Needed!!!*/
.fb_cursor=xxxfb_cursor,/*Optional!!!*/
.fb_rotate=xxxfb_rotate,
.fb_sync=xxxfb_sync,
.fb_ioctl=xxxfb_ioctl,
.fb_mmap=xxxfb_mmap,
};
154
Intheprobe()function,registrationoftheframebufferdevice
andoperations
staticint__devinitxxxfb_probe
(structpci_dev*dev,
conststructpci_device_id*ent)
{
structfb_info*info;
[...]
info=framebuffer_alloc(sizeof(structxxx_par),device);
[...]
info>fbops=&xxxfb_ops;
[...]
if(register_framebuffer(info)<0)
returnEINVAL;
[...]
}
register_framebuffer()willcreatethecharacterdevice
thatcanbeusedbyuserspaceapplicationwiththegeneric
framebufferAPI
155
DeviceModelandBusInfrastructure
156
Unifieddevicemodel
The2.6kernelincludedasignificantnewfeature:
aunifieddevicemodel
Insteadofhavingdifferentadhocmechanismsinthevarious
subsystems,thedevicemodelunifiesthedescriptionofthedevices
andtheirtopology
Minimizingcodeduplication
Commonfacilities(referencecounting,eventnotification,power
management,etc.)
Enumeratethedevices,viewtheirinterconnections,linkthedevicesto
theirbusesanddrivers,categorizethembyclasses.
157
Busdrivers
Thefirstcomponentofthedevicemodelisthebusdriver
Onebusdriverforeachtypeofbus:USB,PCI,SPI,MMC,ISA,etc.
Itisresponsiblefor
Registeringthebustype
Allowingtheregistrationofadapterdrivers(USBcontrollers,I2C
adapters,etc.),ableofdetectingtheconnecteddevices
Allowingtheregistrationofdevicedrivers(USBdevices,I2C
devices,PCIdevices,etc.),managingthedevices
Matchingthedevicedriversagainstthedevicesdetectedbythe
adapterdrivers.
158
Listofdeviceidentifiers
Dependingonthebustype,themethodforbindingadevicetoa
driverisdifferent.Formanybuses,itisbasedonuniqueidentifiers
Thedevicedriverdefinesatablewiththelistofdeviceidentifiersit
isabletomanage:
staticconststructpci_device_idrhine_pci_tbl[]={
{0x1106,0x3043,PCI_ANY_ID,PCI_ANY_ID,},/*VT86C100A*/
{0x1106,0x3053,PCI_ANY_ID,PCI_ANY_ID,},/*VT6105M*/
{}/*terminatelist*/
};
MODULE_DEVICE_TABLE(pci,rhine_pci_tbl);
Codeonthisslideandonthenextslidesaretaken
fromtheviarhinedriverindrivers/net/viarhine.c
159
Definingthedriver
Thedevicedriverdefinesadriverstructure,usuallyspecializedby
thebusinfrastructure(pci_driver,usb_driver,etc.)
Thestructurepointsto:thedevicetable,aprobefunction,called
whenadeviceisdetectedandvariousothercallbacks
staticstructpci_driverrhine_driver={
.name=DRV_NAME,
.id_table=rhine_pci_tbl,
.probe=rhine_init_one,
.remove=__devexit_p(rhine_remove_one),
#ifdefCONFIG_PM
.suspend=rhine_suspend,
.resume=rhine_resume,
#endif/*CONFIG_PM*/
.shutdown=rhine_shutdown,
};
160
Registeringthedriver
Inthemoduleinitializationfunction,thedriverisregisteredtothe
businfrastructure,inordertoletthebusknowthatthedriveris
availabletohandledevices.
staticint__initrhine_init(void)
{
[...]
returnpci_register_driver(&rhine_driver);
}
staticvoid__exitrhine_cleanup(void)
{
pci_unregister_driver(&rhine_driver);
}
IfanewPCIdevicematchesoneoftheidentifiersofthetable,the
probe()methodofthePCIdriverwillgetcalled.
161
Probemethod
Theprobe()methodreceivesasargumentastructure
describingthedevice,usuallyspecializedbythebus
infrastructure(pci_dev,usb_device,etc.)
Thisfunctionisresponsiblefor
Initializingthedevice,mappingI/Omemory,registeringthe
interrupthandlers.Thebusinfrastructureprovidesmethodsto
gettheaddresses,interruptsnumbersandotherdevice
specificinformation.
Registeringthedevicetotheproperkernelframework,for
examplethenetworkinfrastructure.
162
Devicedriver(5)
staticint__devinitrhine_init_one(structpci_dev*pdev,
conststructpci_device_id*ent)
{
structnet_device*dev;
[...]
rc=pci_enable_device(pdev);
[...]
pioaddr=pci_resource_start(pdev,0);
memaddr=pci_resource_start(pdev,1);
[...]
dev=alloc_etherdev(sizeof(structrhine_private));
[...]
SET_NETDEV_DEV(dev,&pdev>dev);
[...]
rc=pci_request_regions(pdev,DRV_NAME);
[...]
ioaddr=pci_iomap(pdev,bar,io_size);
[...]
rc=register_netdev(dev);
[...]
}
163
Globalarchitecture
PCIbusdriver
Thedevice
driver
registers
itselfandthe
device
identifiersto
thebus
driver
Thebusdriver
detectsa
matchingdevice,
andcallsthe
probe()method
ofthedevice
driver.
viarhine
devicedriver
Network
device
framework
3
Theprobe()
methodofthe
devicedriver
initializesthe
deviceand
registersanew
networkinterface
164
sysfs
Thebus,device,drivers,etc.structuresareinternaltothe
kernel
Thesysfsvirtualfilesystemoffersamechanismtoexportsuch
informationtouserspace
Usedforexamplebyudevtoprovideautomaticmoduleloading,
firmwareloading,devicefilecreation,etc.
sysfsisusuallymountedin/sys
/sys/bus/containsthelistofbuses
/sys/devices/containsthelistofdevices
/sys/classenumeratesdevicesbyclass(net,input,
block...),whateverthebustheyareconnectedto.Veryuseful!
Takeyourtimetoexplore/sysonyourworkstation.
165
Platformdevices
Onembeddedsystems,devicesareoftennotconnected
throughabusallowingenumeration,hotplugging,andproviding
uniqueidentifiersfordevices.
However,westillwantthedevicestobepartofthedevice
model.
Thesolutiontothisistheplatformdriver/platformdevice
infrastructure.
Theplatformdevicesarethedevicesthataredirectlyconnected
totheCPU,withoutanykindofbus.
166
Implementationoftheplatformdriver
Thedriverimplementsaplatform_driverstructure
(exampletakenfromdrivers/serial/imx.c)
staticstructplatform_driverserial_imx_driver={
.probe=serial_imx_probe,
.remove=serial_imx_remove,
.driver={
.name="imxuart",
.owner=THIS_MODULE,
},
};
Andregistersitsdrivertotheplatformdriverinfrastructure
staticint__initimx_serial_init(void)
{
[...]
ret=platform_driver_register(&serial_imx_driver);
[...]
}
167
Platformdeviceinstantiation(1)
Intheboardspecificcode,theplatformdevicesare
instantiated(arch/arm/machimx/mx1ads.c):
staticstructplatform_deviceimx_uart1_device={
.name="imxuart",
.id=0,
.num_resources=ARRAY_SIZE(imx_uart1_resources),
.resource=imx_uart1_resources,
.dev={
.platform_data=&uart_pdata,
}
};
Thematchbetweenthedeviceandthedriverismadeusing
thename.Itmustbeuniqueamongstdrivers!
168
Platformdeviceinstantiation(2)
Thedeviceispartofalist
staticstructplatform_device*devices[]__initdata={
&cs89x0_device,
&imx_uart1_device,
&imx_uart2_device,
};
Andthelistofdevicesisaddedtothesystem
duringboardinitialization
staticvoid__initmx1ads_init(void)
{
[...]
platform_add_devices(devices,ARRAY_SIZE(devices));
}
169
I/Oresources
EachplatformdeviceisassociatedwithasetofI/Oresources,
referencedintheplatform_devicestructure
staticstructresourceimx_uart1_resources[]={
[0]={
.start=0x00206000,
.end=0x002060FF,
.flags=IORESOURCE_MEM,
},
[1]={
.start=(UART1_MINT_RX),
.end=(UART1_MINT_RX),
.flags=IORESOURCE_IRQ,
},
};
ItallowsthedrivertobeindependentoftheI/Oaddresses,IRQ
numbers!Seeimx_uart2_deviceforanotherdeviceusingthe
sameplatformdriver.
170
Insidetheplatformdriver
Whenaplatform_deviceisaddedtothesystemusing
platform_add_device(),theprobe()methodoftheplatform
drivergetscalled
Thismethodisresponsibleforinitializingthehardware,
registeringthedevicetotheproperframework(inourcase,the
serialdriverframework)
TheplatformdriverhasaccesstotheI/Oresources:
res=platform_get_resource(pdev,IORESOURCE_MEM,0);
base=ioremap(res>start,PAGE_SIZE);
sport>rxirq=platform_get_irq(pdev,0);
171
Frameworkandbusinfrastructure
Atypicaldriverwill
Beregisteredinsideaframework
Relyonabusinfrastructureandthedevicemodel
ExamplewiththeiMXserialdriver,drivers/serial/imx.c
Atmoduleinitializationtime,thedriverregistersitselfbothtothe
UARTframeworkandtotheplatformbusinfrastructure
staticint__initimx_serial_init(void)
{
ret=uart_register_driver(&imx_reg);
[...]
ret=platform_driver_register(&serial_imx_driver);
[...]
return0;
}
172
iMXserialdriver
DefinitionoftheiMXUARTdriver
staticstructuart_driverimx_reg={
.owner=THIS_MODULE,
.driver_name=DRIVER_NAME,
.dev_name=DEV_NAME,
.major=SERIAL_IMX_MAJOR,
.minor=MINOR_START,
.nr=ARRAY_SIZE(imx_ports),
.cons=IMX_CONSOLE,
};
DefinitionoftheiMXplatformdriver
staticstructplatform_driverserial_imx_driver={
.probe=serial_imx_probe,
.remove=serial_imx_remove,
[...]
.driver={
.name="imxuart",
.owner=THIS_MODULE,
},
};
173
iMXserialdriver
Whentheplatformdeviceisinstantiated,theprobe()
methodiscalled
staticintserial_imx_probe(structplatform_device*pdev)
{
structimx_port*sport;
sport=kzalloc(sizeof(*sport),GFP_KERNEL);
res=platform_get_resource(pdev,IORESOURCE_MEM,0);
base=ioremap(res>start,PAGE_SIZE);
/*sportinitialization*/
sport>port.irq=platform_get_irq(pdev,0);
sport>port.ops=&imx_pops;
sport>clk=clk_get(&pdev>dev,"uart_clk");
clk_enable(sport>clk);
uart_add_one_port(&imx_reg,&sport>port);
}
174
iMXserialdriver
Theoperationstructureuart_opsisassociatedtoeach
port.TheoperationsareimplementedintheiMXdriver
staticstructuart_opsimx_pops={
.tx_empty=imx_tx_empty,
.set_mctrl=imx_set_mctrl,
.get_mctrl=imx_get_mctrl,
.stop_tx=imx_stop_tx,
.start_tx=imx_start_tx,
.stop_rx=imx_stop_rx,
.enable_ms=imx_enable_ms,
.break_ctl=imx_break_ctl,
.startup=imx_startup,
.shutdown=imx_shutdown,
.set_termios=imx_set_termios,
.type=imx_type,
.release_port=imx_release_port,
.request_port=imx_request_port,
.config_port=imx_config_port,
.verify_port=imx_verify_port,
};
175
References
Kerneldocumentation
Documentation/drivermodel/
Documentation/filesystems/sysfs.txt
Linux2.6DeviceModel
http://www.bravegnu.org/devicemodel/devicemodel.html
LinuxDeviceDrivers,chapter14TheLinuxDeviceModel
http://lwn.net/images/pdf/LDD3/ch14.pdf
Thekernelsourcecode
Fullofexamplesofotherdrivers!
176
Annexes
Quizanswers
177
Quizanswers
Interrupthandling
Q:Whydidthekernelsegfaultatmoduleunload(forgettingto
unregisterahandlerinasharedinterruptline)?
A:Kernelmemoryisallocatedatmoduleloadtime,tohostmodule
code.Thismemoryisfreedatmoduleunloadtime.Ifyouforgetto
unregisterahandlerandaninterruptcomes,thecpuwilltrytojump
totheaddressofthehandler,whichisinafreedmemoryarea.
Crash!
178
Relateddocuments
Allourtechnicalpresentations
onhttp://freeelectrons.com/docs
Linuxkernel
Devicedrivers
Architecturespecifics
EmbeddedLinuxsystemdevelopment
Howtohelp
Youcanhelpustoimproveandmaintainthisdocument...
Bysendingcorrections,suggestions,contributionsand
translations
Byaskingyourorganizationtoorderdevelopment,consulting
andtrainingservicesperformedbytheauthorsofthese
documents(seehttp://freeelectrons.com/).
Bysharingthisdocumentwithyourfriends,colleagues
andwiththelocalFreeSoftwarecommunity.
Byaddinglinksonyourwebsitetoouronlinematerials,
toincreasetheirvisibilityinsearchengineresults.
Linuxkernel
Linuxdevicedrivers
Boardsupportcode
Mainstreamingkernelcode
Kerneldebugging
EmbeddedLinuxTraining
Allmaterialsreleasedwithafreelicense!
UnixandGNU/Linuxbasics
Linuxkernelanddriversdevelopment
RealtimeLinux,uClinux
Developmentandprofilingtools
Lightweighttoolsforembeddedsystems
Rootfilesystemcreation
Audioandmultimedia
Systemoptimization
FreeElectrons
Ourservices
CustomDevelopment
Systemintegration
EmbeddedLinuxdemosandprototypes
Systemoptimization
Applicationandinterfacedevelopment
Consultingandtechnicalsupport
Helpindecisionmaking
Systemarchitecture
Systemdesignandperformancereview
Developmenttoolandapplicationsupport
Investigatingissuesandfixingtoolbugs

FREE-ELECTRONS-embedded Linux Kernel and Drivers

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

FREE-ELECTRONS-embedded Linux Kernel and Drivers

Încărcat de

Drepturi de autor:

Formate disponibile

EmbeddedLinuxdriverdevelopment

S-ar putea să vă placă și