Beagleboard

FreeElectrons
EmbeddedLinux training
GregoryClement ThomasPetazzoni MichaelOpdenacker MaximeRipard
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Rightstocopy
Copyright20042011,FreeElectrons feedback@freeelectrons.com Electronicversionofthisdocumentavailableon http://freeelectrons.com/doc/training/embeddedlinux Updateswillbeavailableon http://freeelectrons.com/doc/training/embeddedlinux/
AttributionShareAlike3.0 Youarefree tocopy,distribute,display,andperformthework tomakederivativeworks tomakecommercialuseofthework Underthefollowingconditions Attribution.Youmustgivetheoriginalauthorcredit. ShareAlike.Ifyoualter,transform,orbuilduponthiswork,you maydistributetheresultingworkonlyunderalicenseidenticalto thisone. Foranyreuseordistribution,youmustmakecleartoothersthelicense termsofthiswork. Anyoftheseconditionscanbewaivedifyougetpermissionfromthe copyrightholder. Yourfairuseandotherrightsareinnowayaffectedbytheabove. Licensetext:http://creativecommons.org/licenses/bysa/3.0/legalcode
Corrections,suggestions, contributionsandtranslationsarewelcome! Latestupdate:May31,2011
Linuxkernel Linuxdevicedrivers Boardsupportcode Mainstreamingkernelcode Kerneldebugging EmbeddedLinuxTraining Allmaterialsreleasedwithafreelicense! UnixandGNU/Linuxbasics Linuxkernelanddriversdevelopment RealtimeLinux,uClinux Developmentandprofilingtools Lightweighttoolsforembeddedsystems Rootfilesystemcreation Audioandmultimedia Systemoptimization
FreeElectrons
Ourservices
CustomDevelopment Systemintegration EmbeddedLinuxdemosandprototypes Systemoptimization Applicationandinterfacedevelopment Consultingandtechnicalsupport Helpindecisionmaking Systemarchitecture Systemdesignandperformancereview Developmenttoolandapplicationsupport Investigatingissuesandfixingtoolbugs
Hardwareusedinthistrainingsession
TIDM3730
OMAP3CPU@1Ghz DSP,OpenGL
ISEEIGEPv2
512MBRAM,512MBflash Ethernet,Wifi,Bluetooth RS232serial USBHost,USBOTG DVID,AudioInandOut microSD Expansionconnectors:UART,SPI,I2C,GPIO,keyboard,LCD Approximately130EUR http://www.igep.es

1
Participate!
Duringthelectures... Don'thesitatetoaskquestions.Otherpeopleintheaudience mayhavesimilarquestionstoo. Thishelpsthetrainertodetectanyexplanationthatwasn'tclear ordetailedenough. Don'thesitatetoshareyourexperience,forexampletocompare Linuxwithotheroperatingsystemsusedinyourcompany. Yourpointofviewismostvaluable,becauseitcanbesimilarto yourcolleagues'anddifferentfromthetrainer's. Yourparticipationcanmakeoursessionmoreinteractiveand makethetopicseasiertolearn.
Practicallabguidelines
Duringpracticallabs... Wecannotsupportmorethan8workstationsatonce(eachwith itsboardandequipment).Havingmorewouldmakethewhole classprogressslower,compromisingthecoverageofthewhole trainingagenda(exceptionforpublicsessions:upto10people). So,ifyouaremorethan8participants,pleaseformupto8 workinggroups. Opentheelectroniccopyofyourlecturematerials,anduseit throughoutthepracticallabstofindtheslidesyouneedagain. Don'tcopyandpastefromthePDFslides. TheslidescontainUTF8charactersthatlookthesameasASCII ones,butwon'tbeunderstoodbyshellsorcompilers.
Cooperate!
AsintheFreeSoftwareandOpenSourcecommunity, cooperationduringpracticallabsisvaluableinthistrainingsession: Ifyoucompleteyourlabsbeforeotherpeople,don'thesitateto helpotherpeopleandinvestigatetheissuestheyface.Thefaster weprogressasagroup,themoretimewehavetoexploreextra topics. Explainwhatyouunderstoodtootherparticipantswhenneeded. Italsohelpstoconsolidateyourknowledge. Don'thesitatetoreportpotentialbugstoyourinstructor. Don'thesitatetolookforsolutionsontheInternetaswell.
Commandmementosheet
Thismementosheetgives commandexamplesforthemost typicalneeds(lookingforfiles, extractingatararchive...) Itsavesus1dayofUNIX/Linux commandlinetraining. Ourbesttip:inthecommandline shell,alwayshitthe[Tab]keyto completecommandnamesandfile paths.Thisavoids95%oftyping mistakes. Getanelectroniccopyon http://freeelectrons.com/docs/commandline
vibasiccommands
Thevieditorisveryuseful tomakequickchangesto filesinaembeddedtarget. Thoughnotveryuser friendlyatfirst,viisvery powerfulanditsmain15 commandsareeasyto learnandaresufficientfor 99%ofeveryone'sneeds! Youcanalsotakethequicktutorialbyrunningvimtutor. Thisisaworthyinvestment! Getanelectroniccopyon http://freeelectrons.com/docs/commandline
FreeElectrons
Introduction toembedded Linux

MichaelOpdenacker ThomasPetazzoni FreeElectrons
Copyright20092011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!
Birthoffreesoftware
1983,RichardStallman,GNUprojectandthefreesoftware concept.Beginningofthedevelopmentofgcc,gdb,glibcand otherimportanttools 1991,LinusTorvalds,Linuxkernelproject,aUnixlikeoperating systemkernel.TogetherwithGNUsoftwareandmanyother opensourcecomponents:acompletelyfreeoperatingsystem, GNU/Linux ~1995,Linuxismoreandmorepopularonserversystems ~2000,Linuxismoreandmorepopularonembeddedsystems ~2005,Linuxismoreandmorepopularondesktopsystems
2
Freesoftware?
Aprogramisconsideredfreewhenitslicenseofferstoallits usersthefollowingfourfreedoms
Freedomtorunthesoftwareforanypurpose Freedomtostudythesoftwareandtochangeit Freedomtoredistributecopies Freedomtodistributecopiesofmodifiedversions
Thosefreedomsaregrantedforbothcommercialandnon commercialuse Theyimplytheavailabilityofsourcecode,softwarecanbe modifiedanddistributedtocustomers Goodmatchforembeddedsystems!

3
WhatisembeddedLinux?
EmbeddedLinuxistheusage oftheLinuxkernelandvarious opensourcecomponentsin embeddedsystems
4
WhyembeddedLinux?
AdvantagesofLinuxandopensourcefor embeddedsystems
5
Reusingcomponents
ThekeyadvantageofLinuxandopensourceinembedded systemsistheabilitytoreusecomponents Theopensourceecosystemalreadyprovidesmanycomponents forstandardfeatures,fromhardwaresupporttonetwork protocols,goingthroughmultimedia,graphic,cryptographic libraries,etc. Assoonasahardwaredevice,oraprotocol,orafeatureiswide spreadenough,highchanceofhavingopensourcecomponents thatsupportit. Allowstoquicklydesignanddevelopcomplicatedproducts, basedonexistingcomponents. Nooneshouldredevelopyetanotheroperatingsystemkernel, TCP/IPstack,USBstackoranothergraphicaltoolkitlibrary. Allowstofocusontheaddedvalueofyourproduct.
6
Lowcost
Freesoftwarecanbeduplicatedonasmanydevicesasyou want,freeofcharge. Ifyourembeddedsystemusesonlyfreesoftware,youcan reducethecostofsoftwaretozero.Eventhedevelopmenttools arefree,unlessyouchooseacommercialembeddedLinux edition. Allowstohaveanhigherbudgetforthehardwareorto increasethecompanysskillsandknowledge
7
Fullcontrol
Withopensource,youhavethesourcecodeforallcomponents inyoursystem Allowsunlimitedmodifications,changes,tuning,debugging, optimization,foranunlimitedperiodoftime Withoutlockingordependencyfromathirdpartyvendor
Tobetrue,nonopensourcecomponentsmustbeavoidedwhen thesystemisdesignedanddeveloped
Allowstohavefullcontroloverthesoftwarepartofyour system
8
Quality
Manyopensourcecomponentsarewidelyused,onmillionsof systems Usuallyhigherqualitythanwhataninhousedevelopmentcan produce,orevenproprietaryvendors Ofcourse,notallopensourcecomponentsareofgoodquality, butmostofthewidelyusedonesare. Allowstodesignyoursystemwithhighqualitycomponents atthefoundations
9
Easestestingofnewfeatures
Opensourcebeingfreelyavailable,itiseasytogetoneand evaluateit Allowstoeasilystudyseveraloptionswhilemakingachoice Mucheasierthanpurchasinganddemonstrationprocedures neededwithmostproprietaryproducts Allowstoeasilyexplorenewpossibilitiesandsolutions
10
Communitysupport
Opensourcesoftwarecomponentsaredevelopedby communitiesofdevelopersandusers Thiscommunitycanprovideahighqualitysupport:youcan directlycontactthemaindevelopersofthecomponentyouare using Oftenbetterthantraditionalsupport,butoneneedstounderstand howthecommunityworkstoproperlyusethecommunitysupport possibilities Allowstospeeduptheresolutionofproblemswhen developingyoursystem
11
Takingpartintothecommunity
Possibilityoftakingpartintothedevelopmentcommunityofsome ofthecomponentsusedintheembeddedsystems:bug reporting,testofnewversionsorfeatures,patchesthatfixbugs oraddnewfeatures,etc. Mostofthetimetheopensourcecomponentsarenotthecore valueoftheproduct:itstheinterestofeverybodytocontribute back. Fortheengineers:averymotivatingwayofbeingrecognized outsidethecompany,communicationwithothersinthesame field,openingofnewpossibilities,etc. Forthemanagers:motivationfactorforengineers,allowsthe companytoberecognizedintheopensourcecommunityand thereforegetsupportmoreeasilyandbemoreattractiveto opensourcedevelopers
12
SystemsrunningLinux
Afewexamplesofembeddedsystems runningLinux
13
Personalrouters
14
Television
15
Pointofsaleterminal
16
Lasercuttingmachine
17
Viticulturemachine
18
Hardware
Embeddedhardwarefor Linuxsystems
19
Processorandarchitecture(1)
TheLinuxkernelandmostotherarchitecturedependent componentsupportawiderangeof32and64bitsarchitectures
x86andx8664,asfoundonPCplatforms,butalsoembedded systems(multimedia,industrial) ARM,withhundredsofdifferentSoC(multimedia,industrial) PowerPC(mainlyrealtime,industrialapplications) MIPS(mainlynetworkingapplications) SuperH(mainlysettopboxandmultimediaapplications) Blackfin(DSParchitecture) Microblaze(softcoreforXilinxFPGA) Coldfire,SCore,Tile,Xtensa,Cris,FRV,AVR32,M32R
20
Processorandarchitecture(2)
BothMMUandnoMMUarchitecturesaresupported,even thoughnoMMUarchitectureshaveafewlimitations. Linuxisnotdesignedforsmallmicrocontrollers. Besidesthetoolchain,thebootloaderandthekernel,allother componentsaregenerallyarchitectureindependent
21
RAMandstorage
RAM:averybasicLinuxsystemcanworkwithin8MBofRAM, butamorerealisticsystemwillusuallyrequireatleast32MBof RAM.Dependsonthetypeandsizeofapplications. Storage:averybasicLinuxsystemcanworkwithin4MBof storage,butusuallymoreisneeded.
Flashstorageissupported,bothNANDandNORflash,with specificfilesystems BlockstorageincludingSD/MMCcardsandeMMCissupported
Notnecessarilyinterestingtobetoorestrictiveontheamountof RAM/storage:havingflexibilityatthislevelallowstoreuseas manyexistingcomponentsaspossible.
22
Communication
TheLinuxkernelhassupportformanycommoncommunication busses
I2C SPI CAN 1wire SDIO USB
Andalsoextensivenetworkingsupport
Ethernet,Wifi,Bluetooth,CAN,etc. IPv4,IPv6,TCP,UDP,SCTP,DCCP,etc. Firewalling,advancedrouting,multicast
23
Typesofhardwareplatforms
EvaluationplatformsfromtheSoCvendor.Usuallyexpensive, butmanyperipheralsarebuiltin.Generallyunsuitableforreal products. ComponentonModule,asmallboardwithonlyCPU/RAM/flash andafewothercorecomponents,withconnectorstoaccessall otherperipherals.Canbeusedtobuildendproductsforsmallto mediumquantities. Communitydevelopmentplatforms,anewtrendtomakea particularSoCpopularandeasilyavailable.Thosearereadyto useandlowcost,butusuallyhavelessperipheralsthan evaluationplatforms.Tosomeextent,canalsobeusedforreal products. Customplatform.Schematicsforevaluationboardsor developmentplatformsaremoreandmorecommonlyfreely available,makingiteasiertodevelopcustomplatforms.
24
Criteriasforchoosingthehardware
Makesurethehardwareyouplantouseisalreadysupportedby theLinuxkernel,andhasanopensourcebootloader,especially theSoCyouretargeting. Havingsupportintheofficialversionsoftheprojects(kernel, bootloader)isalotbetter:qualityisbetter,andnewversionsare available. SomeSoCvendorsand/orboardvendorsdonotcontributetheir changesbacktothemainlineLinuxkernel.Askthemtodoso,or useanotherproductifyoucan.Agoodmeasurementistosee thedeltabetweentheirkernelandtheofficialone. BetweenproperlysupportedhardwareintheofficialLinux kernelandpoorlysupportedhardware,therewillbehuge differencesindevelopmenttimeandcost.
25
EmbeddedLinuxsystemarchitecture
EmbeddedLinuxsystemarchitecture
26
Globalarchitecture
Application Application
Tools
Library
Library
Library
StandardClibrary Linuxkernel Bootloader Hardware

27
Softwarecomponents
Crosscompilationtoolchain
Compilerthatrunsonthedevelopmentmachine,butgenerates codeforthetarget
Bootloader
Startedbythehardware,responsibleforbasicinitialization,loading andexecutingthekernel
LinuxKernel
Containstheprocessandmemorymanagement,networkstack, devicedriversandprovidesservicestouserspaceapplications
Clibrary
Theinterfacebetweenthekernelandtheuserspaceapplications
Librariesandapplications
Thirdpartyorinhouse
28
EmbeddedLinuxwork
Severaldistincttasksareneededwhendeploying embeddedLinuxinaproduct BoardSupportPackagedevelopment
ABSPcontainsabootloaderandkernelwiththesuitabledevice driversforthetargetedhardware PurposeofourKernelDevelopmenttraining
Systemintegration
Integrateallthecomponents,bootloader,kernel,thirdpartylibraries andapplicationsandinhouseapplicationsintoaworkingsystem Purposeofthistraining
Developmentofapplications
NormalLinuxapplications,butusingspecificallychosenlibraries
29
FreeElectrons
EmbeddedLinux development environment

Copyright20092011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!
EmbeddedLinuxsolutions
TwowaystoswitchtoembeddedLinux
Usesolutionsprovidedandsupportedbyvendorslike MontaVista,WindRiverorTimeSys.Thesesolutionscomewith theirowndevelopmenttoolsandenvironment.Theyuseamixof opensourcecomponentsandproprietarytools. Usecommunitysolutions.Theyarecompletelyopen,supported bythecommunity.
InFreeElectronstrainings,wedonotpromoteaparticular vendor,andthereforeusecommunitysolutions
However,knowingtheconcepts,switchingtovendorsolutionswill beeasy
2
OSforLinuxdevelopment
WestronglyrecommendtouseLinuxasthedesktopoperating systemtoembeddedLinuxdevelopers,formultiplereasons. AllcommunitytoolsaredevelopedanddesignedtorunonLinux. Tryingtousethemonotheroperatingsystems(Windows,Mac OSX)willleadtotroubles,andtheirusageonthosesystemsis generallynotsupportedbycommunitydevelopers. AsLinuxalsorunsontheembeddeddevice,alltheknowledge gainedfromusingLinuxonthedesktopwillapplysimilarlytothe embeddeddevice.
:)
3
DesktopLinuxdistribution
AnygoodandsufficientlyrecentLinux desktopdistributioncanbeusedforthe developmentworkstation
Ubuntu,Debian,Fedora,OpenSuse, Redhat,etc.
WehavechosenUbuntu,asitisa widelyusedandeasytousedesktop Linuxdistribution TheUbuntusetuponthetraininglaptops hasintentionallybeenleftuntouched afterthenormalinstallationprocess. LearningembeddedLinuxisalsoabout learningthetoolsneededonthe developmentworkstation!

4
Linuxrootandnonrootusers
Linuxisamultiuseroperatingsystem
Therootuseristheadministrator,anditcandopriviledged operationssuchas:mountingfilesystems,configuringthenetwork, creatingdevicefiles,changingthesystemconfiguration,installing orremovingsoftware Allotherusersareunpriviledged,andcannotperformthose administratorleveloperations
OnanUbuntusystem,itisnotpossibletologinasroot,onlyas anormaluser. Thesystemhasbeenconfiguredsothattheuseraccount createdfirstisallowedtorunpriviledgedoperationsthrougha programcalledsudo.

Example:sudomount/dev/sda2/mnt/disk
5
Softwarepackages
ThedistributionmechanismforsoftwareinGNU/Linuxisdifferent fromtheoneinWindows Linuxdistributionsprovidesacentralandcoherentwayof installing,updatingandremovingapplicationsandlibraries: packages Packagescontainstheapplicationorlibraryfiles,andassociated metainformation,suchastheversionandthedependencies
.debonDebianandUbuntu,.rpmonMandriva,Fedora,OpenSUSE
Packagesarestoredinrepositories,usuallyonHTTPorFTP servers Oneshouldonlyusepackagesfromofficialrepositoriesofits distribution,unlessstrictlyrequired.

6
Managingsoftwarepackages(1)
InstructionsforDebianbasedGNU/Linuxsystems (Debian,Ubuntu...) Packagerepositoriesarespecifiedin /etc/apt/sources.list Toupdatepackagerepositorylists: sudoaptgetupdate Tofindthenameofapackagetoinstall,thebestistousethe searchengineonhttp://packages.debian.orgoron http://packages.ubuntu.com.Youmayalsouse: aptcachesearch<keyword>
7
Managingsoftwarepackages(2)
Toinstallagivenpackage: sudoaptgetinstall<package> Toremoveagivenpackage: sudoaptgetremove<package> Toinstallallavailablepackageupdates: sudoaptgetdistupgrade Getinformationaboutapackage: sudoaptcacheshow<package> Graphicalinterfaces
SynapticforGNOME KPackageKitforKDE
Furtherdetailsonpackagemanagement: http://www.debian.org/doc/manuals/apthowto/
8
Hostvs.target
Whendoingembeddeddevelopment,thereisalwaysasplit between
Thehost,thedevelopmentworkstation,whichistypicallya powerfulPC Thetarget,whichisthetargetitself
Theyareconnectedbyvariousmeans:almostalwaysaserial linefordebuggingpurposes,frequentlyanEthernetconnection, sometimesaJTAGforlowleveldebugging Host

x86orx86_64PC FullfeaturedLinux desktopsystem
Serial Ethernet,etc.
Target
ARM,PowerPC,MIPS, x86..platform Moreminimalistic Linuxsystem 9
Seriallinecommunicationprogram
Anessentialtoolforembeddeddevelopmentisaserialline communicationprogram TherearemultipleoptionsavailableinLinux:Minicom,Picocom, Gtkterm,Putty,etc. Inthistrainingsession,werecommendusingthesimplestof them:picocom
Installationwithaptgetinstallpicocom RunwithpicocombBAUD_RATE/dev/SERIAL_DEVICE ExitwithControlAControlX
SERIAL_DEVICEistypically
ttyUSBxforUSBtoserialconverters ttySxforrealserialports
10
Commandlinetips
Usingthecommandlineismandatoryformanyoperations neededforembeddedLinuxdevelopment Itisaverypowerfulwayofinteractingwiththesystem,withwhich youcansavealotoftime. Someusefultips
YoucanuseseveraltabsintheGnomeTerminal Rememberthatyoucanuserelativepaths(forexample: ../../linux)inadditiontoabsolutepaths(forexample: /home/user) Inashell,hitControlr,thenakeyword,willsearchthroughthe commandhistory.HitControlragaintosearchbackwardinthe history Youcancopy/pastepathsdirectlyfromthefilemanagertothe terminalbydraganddrop.
11
EmbeddedLinuxsystemdevelopment
Crosscompiling toolchains
ThomasPetazzoni MichaelOpdenacker FreeElectrons
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/toolchains Corrections,suggestions,contributionsandtranslationsarewelcome!
Definition(1)
TheusualdevelopmenttoolsavailableonaGNU/Linuxworkstationisa nativetoolchain Thistoolchainrunsonyourworkstationandgeneratescodeforyour workstation,usuallyx86 Forembeddedsystemdevelopment,itisusuallyimpossibleornot interestingtouseanativetoolchain Thetargetistoorestrictedintermsofstorageand/ormemory Thetargetisveryslowcomparedtoyourworkstation Youmaynotwanttoinstallalldevelopmenttoolsonyourtarget. Therefore,crosscompilingtoolchainsaregenerallyused.Theyrun onyourworkstationbutgeneratecodeforyourtarget.
2
Definition(2)
Sourcecode Crosscompiling toolchain x86 Execution machine ARM Compilation machine
Nativetoolchain
x86binary x86
ARMbinary
3
Components
Binutils
Kernelheaders
C/C++libraries
GCCcompiler
GDBdebugger (optional)
4
Binutils
Binutilsisasetoftoolstogenerateandmanipulatebinariesfora givenCPUarchitecture
as,theassembler,thatgeneratesbinarycodefromassembler sourcecode ld,thelinker ar,ranlib,togenerate.aarchives,usedforlibraries objdump,readelf,size,nm,strings,toinspectbinaries. Veryusefulanalysistools! strip,tostripuselesspartsofbinariesinordertoreducetheirsize
http://www.gnu.org/software/binutils/ GPLlicense
5
Kernelheaders(1)
TheClibraryandcompiledprograms needstointeractwiththekernel
Availablesystemcallsandtheir numbers Constantdefinitions Datastructures,etc.
Kernel Kernelheaders CLibrary Applications
Therefore,compilingtheClibrary requireskernelheaders,andmany applicationsalsorequirethem.
Availablein<linux/...>and<asm/...>andafewother directoriescorrespondingtotheonesvisibleininclude/inthe kernelsources

6
Kernelheaders(2)
Systemcallnumbers,in<asm/unistd.h>
#define__NR_exit1 #define__NR_fork2 #define__NR_read3
Constantdefinitions,herein<asmgeneric/fcntl.h>,included from<asm/fcntl.h>,includedfrom<linux/fcntl.h>
#defineO_RDWR00000002
Datastructures,herein<asm/stat.h>
structstat{ unsignedlongst_dev; unsignedlongst_ino; [...] };
7
Kernelheaders(3)
ThekerneltouserspaceABIisbackwardcompatible
Binariesgeneratedwithatoolchainusingkernelheadersolderthan therunningkernelwillworkwithoutproblem,butwon'tbeableto usethenewsystemcalls,datastructures,etc. Binariesgeneratedwithatoolchainusingkernelheadersnewer thantherunningkernelmightworkoniftheydon'tusetherecent features,otherwisetheywillbreak Usingthelatestkernelheadersisnotnecessary,unlessaccessto thenewkernelfeaturesisneeded
Thekernelheadersareextractedfromthekernelsourcesusing theheaders_installkernelMakefiletarget.
8
GCCcompiler
GNUCCompiler,thefamousfreesoftwarecompiler CancompileC,C++,Ada,Fortran,Java,Objective C,ObjectiveC++,andgeneratecodeforalarge numberofCPUarchitectures,includingARM,AVR, Blackfin,CRIS,FRV,M32,MIPS,MN10300, PowerPC,SH,v850,i386,x86_64,IA64,Xtensa,etc. http://gcc.gnu.org/ AvailableundertheGPLlicense, librariesundertheLGPL.
9
Clibrary
TheClibraryisanessentialcomponentofa Linuxsystem
Interfacebetweentheapplicationsandthe kernel ProvidesthewellknownstandardCAPIto easeapplicationdevelopment
Kernel
SeveralClibrariesareavailable: glibc,uClibc,eglibc,dietlibc,newlib,etc. ThechoiceoftheClibrarymustbemadeat thetimeofthecrosscompilingtoolchain generation,astheGCCcompileriscompiled againstaspecificClibrary.
CLibrary Applications
10
glibc
http://www.gnu.org/software/libc/ License:LGPL ClibraryfromtheGNUproject Designedforperformance,standardscomplianceand portability FoundonallGNU/Linuxhostsystems Ofcourse,activelymaintained Quitebigforsmallembeddedsystems:approx2.5MB onarm(version2.9libc:1.5MB,libm:750KB)
11
uClibc
http://www.uclibc.org/fromCodePoetConsulting License:LGPL LightweightClibraryforsmallembeddedsystems Highconfigurability:manyfeaturescanbeenabledor disabledthroughamenuconfiginterface WorksonlywithLinux/uClinux,worksonmostembedded architectures NostableABI,differentABIdependingonthelibrary configuration Focusonsizeratherthanperformance Smallcompiletime
12
uClibc(2)
MostoftheapplicationscompilewithuClibc.Thisappliestoall applicationsusedinembeddedsystems. Size(arm):4timessmallerthanglibc! uClibc0.9.30.1:approx.600KB(libuClibc:460KB,libm: 96KB) glibc2.9:approx2.5MB Somefeaturesnotavailableorlimited:priorityinheritancemutexes, NPTLsupportisverynew,fixedNameServiceSwitchfunctionality, etc. Usedonalargenumberofproductionembeddedproducts,including consumerelectronicdevices Activelymaintained,largedeveloperanduserbase SupportedandusedbyMontaVista,TimeSysandWindRiver.
13
Honey,Ishrunktheprograms!
Cprogram Plainhelloworld (stripped) Busybox (stripped)
Compiledwithsharedlibraries glibc uClibc 5.6K 5.4K (glibc2.9) (uClibc0.9.30.1) 245K (olderglibc) 231K (olderuClibc)
Compiledstatically glibc 472K (glibc2.9) 843K (olderglibc) uClibc 18K (uClibc 0.9.30.1) 311K (olderuClibc)
ExecutablesizecomparisononARM
14
eglibc
Embeddedglibc,undertheLGPL VariantoftheGNUCLibrary(GLIBC)designedtoworkwellon embeddedsystems StrivestobesourceandbinarycompatiblewithGLIBC eglibc'sgoalsincludereducedfootprint,configurablecomponents, bettersupportforcrosscompilationandcrosstesting. CanbebuiltwithoutsupportforNIS,locales,IPv6,andmanyother features. Supportedbyaconsortium,withFreescale,MIPS,MontaVistaand WindRiverasmembers. TheDebiandistributionisswitchingtoeglibctoo: http://blog.aurel32.net/?p=47 http://www.eglibc.org
15
OthersmallerClibraries
SeveralothersmallerClibrarieshavebeendeveloped,but noneofthemhavethegoalofallowingthecompilationof largeexistingapplications Theyneedspeciallywrittenprogramsandapplications Choices:
Dietlibc,http://www.fefe.de/dietlibc/.Approximately70KB. Newlib,http://sourceware.org/newlib/
Klibc,http://www.kernel.org/pub/linux/libs/klibc/,designed foruseinaninitramfsorinitrdatboottime.
16
Machinesinbuildprocedures
Threemachinesmustbedistinguishedwhendiscussing toolchaincreation
Thebuildmachine,wherethetoolchainisbuilt. Thehostmachine,wherethetoolchainwillbeexecuted. Thetargetmachine,wherethebinariescreatedbythe toolchainwillbeexecuted.
Fourcommonbuildtypesarepossiblefortoolchains
17
Differenttoolchainbuildprocedures
build host Nativebuild
usedtobuildthenormalgccofa workstation
target
build
host Crossbuild
target
usedtobuildatoolchainthatrunsonyour workstationbutgeneratesbinariesforthe target Themostcommonsolutioninembedded
build
host
target
build
host
target
Crossnativebuild
usedtobuildatoolchainthatrunsonyour targetandgeneratesbinariesforthe target
Canadianbuild
usedtobuildonarchitectureAatoolchain thatrunsonarchitectureBandgenerates binariesforarchitectureC 18
ABI
Whenbuildingatoolchain,theABIusedtogeneratebinaries needstobedefined ABI,forApplicationBinaryInterface,definesthecalling conventions(howfunctionargumentsarepassed,howthereturn valueispassed,howsystemcallsaremade)andthe organizationofstructures(alignement,etc.) AllbinariesinasystemmustbecompiledwiththesameABI,and thekernelmustunderstandthisABI. OnARM,twomainABIs:OABIandEABI
NowadayseverybodyusesEABI
OnMIPS,severalABI:o32,o64,n32,n64 http://en.wikipedia.org/wiki/Application_Binary_Interface
19
Floatingpointsupport
Someprocessorshaveafloatingpointunit,someothersdonot.
Forexample,manyARMv4andARMv5CPUsdonothavea floatingpointunit.SinceARMv7,aVFPunitismandatory.
Forprocessorshavingafloatingpointunit,thetoolchainshould generatehardfloatcode,inordertousethefloatingpoint instructionsdirectly Forprocessorswithoutafloatingpointunit,twosolutions

Generatehardfloatcodeandrelyonthekerneltoemulatethe floatingpointinstructions.Thisisveryslow. Generatesoftfloatcode,sothatinsteadofgeneratingfloating pointinstructions,callstoanuserspacelibraryaregenerated
Decisiontakenattoolchainconfigurationtime Alsopossibletoconfigurewhichfloatingpointunitshouldbe used
20
CPUoptimizationflags
AsetofcrosscompilingtoolsisspecifictoaCPUarchitecture (ARM,x86,MIPS,PowerPC) However,withthemarch=,mcpu=,mtune=options,onecan selectmorepreciselythetargettedCPUtype
Forexample,march=armv7mcpu=cortexa8
Atthetoolchaincompilationtime,valuescanbechosen.They areused:
Asthedefaultvaluesforthecrosscompilingtools,whennoother march,mcpu,mtuneoptionsarepassed TocompiletheClibrary
EveniftheClibraryhasbeencompiledforarmv5t,itdoesn't preventfromcompilingotherprogramsforarmv7
21
Buildingatoolchainmanually
Buildingacrosscompilingtoolchainbyyourselfisadifficult andpainfultask!Cantakedaysorweeks!
Lotsofdetailstolearn:manycomponentstobuild,complicated configuration Lotsofdecisionstomake(suchasClibraryversion,ABI,floating pointmechanisms,componentversions) NeedkernelheadersandClibrarysources Needtobefamiliarwithcurrentgccissuesandpatchesonyour platform Usefultobefamiliarwithbuildingandconfiguringtools SeetheCrosstoolNGdocs/directoryfordetailsonhow toolchainsarebuilt.
22
Getaprecompiledtoolchain
Solutionthatmanypeoplechoose Advantage:itisthesimplestandmostconvenientsolution Drawback:youcan'tfinetunethetoolchaintoyourneeds Determinewhattoolchainyouneed:CPU,endianism,C library,componentversions,ABI,softfloatorhardfloat,etc. Checkwhethertheavailabletoolchainsmatchyour requirements. Possiblechoices CodeSourcerytoolchains Linarotoolchains Morereferencesathttp://elinux.org/Toolchains
23
CodeSourcery
Companythathasanextendedexpertise onfreesoftwaretoolchains:gcc,gdb, binutilsandglibc Theyselltoolchainswithsupport,buttheyalsoprovideaLite version,whichisfreeandusableforcommercialproducts Theyhavetoolchainsavailablefor
ARM MIPS PowerPC SuperH
BesuretousetheGNU/Linuxversions.TheEABIversionsare forbaremetaldevelopment(nooperatingsystem)
24
Linarotoolchains
https://wiki.linaro.org/WorkingGroups/ToolChain
Linarocontributestoimprovingmainstream gcconARM,inparticularbyhiring CodeSourcerydevelopers. Forpeoplewhocan'twaitforthenextreleasesofgcc,Linaro releasesmodifiedsourcesofstablereleasesofgcc,withthese optimizationsforARM(mainlyforrecentCortexACPUs). Asanygccrelease,thesesourcescanbeusedbybuildtoolsto buildtheirownbinarytoolchains(Buildroot,OpenEmbedded...) Thisallowstosupportglibc,uClibcandeglibc. BinarypackagesareavailableforUbuntuusers: Seehttps://launchpad.net/~linaromaintainers/+archive/toolchain
25
Installingandusingaprecompiledtoolchain
Followtheinstallationprocedureproposedbythevendor Usually,itissimplyamatterofextractingatarball Oldtoolchains(gcc3.xbased)werenotrelocatable,they hadtobeinstalled.Youmustinstalltheminthelocation theywerebuiltfor. Nowyoucaninstallthetoolchainswhereveryouwant. Then,addthepathtotoolchainbinariesinyourPATH: exportPATH=/path/to/toolchain/bin/:$PATH Finally,compileyourapplications PREFIXgccofoobarfoobar.c ThePREFIXdependsonthetoolchainconfiguration,and allowstodistinguishcrosscompilationtoolsfromnative compilationutilities
26
Toolchainbuildingutilities
Anothersolutionistouseutilitiesthatautomatetheprocessof buildingthetoolchain Sameadvantageastheprecompiledtoolchains:youdon'tneed tomessupwithallthedetailsofthebuildprocess Butalsooffersmoreflexibilityintermsoftoolchainconfiguration, componentversionselection,etc. Theyalsousuallycontainseveralpatchesthatfixknownissues withthedifferentcomponentsonsomearchitectures Identicalprinciple:shellscriptsorMakefilethatautomatically fetch,extract,configure,compileandinstallthedifferent components
27
Toolchainbuildingutilities(2)
Crosstool
Theprecursor,writtenbyDanKegel Setofscriptsandpatches,glibconly Notmaintainedanymore,don'tuseit! http://www.kegel.com/crosstool
Crosstoolng
RewriteofCrosstool,withamenuconfiglikeconfigurationsystem Featurefull:supportsuClibc,glibc,eglibc,hardandsoftfloat,many architectures Activelymaintained http://crosstoolng.org/
28
Toolchainbuildingutilities(3)
Manyrootfilesystembuildingsystemsalsoallowtheconstructionof crosscompilingtoolchain Buildroot Makefilebased,hasaCrosstoolNGbackend,maintainedby thecommunity http://www.buildroot.net PTXdist Makefilebased,uClibcorglibc,maintainedmainlyby Pengutronix http://www.pengutronix.de/software/ptxdist/index_en.html OpenEmbedded Thefeaturefull,butcomplexbuildingsystem http://www.openembedded.org/
29
CrosstoolNG:installationandusage
InstallationofCrosstoolNGcanbedonesystemwide,orjust locallyinthesourcedirectory.Forlocalinstallation: ./configurelocal make makeinstall Somesampleconfigurationsforvariousarchitecturesare availableinsamples,theycanbelistedusing ./ctnglistsamples Toloadasampleconfiguration ./ctng<samplename> Toadjusttheconfiguration ./ctngmenuconfig Tobuildthetoolchain ./ctngbuild
30
Toolchaincontents
Thecrosscompilationtoolbinaries,inbin/
ThisdirectorycanbeaddedtoyourPATHtoeaseusageofthe toolchain
Oneorseveralsysroot,eachcontaining
TheClibraryandrelatedlibraries,compiledforthetarget TheClibraryheadersandkernelheaders
Thereisonesysrootforeachvariant:toolchainscanbemultilibif theyhaveseveralcopiesoftheClibraryfordifferent configurations(forexample:ARMv4T,ARMv5T,etc.)

CodeSourceryARMtoolchainaremultilib,thesysrootsareinarm nonelinuxgnueabi/libc/,armnonelinux gnueabi/libc/armv4t/,armnonelinux gnueabi/libc/thumb2 CrosstoolNGtoolchainsarenevermultilib,thesysrootisinarm unknownlinuxuclibcgnueabi/sysroot
31
PracticallabUsingCrosstoolNG
Timetobuildyourtoolchain ConfigureCrosstoolNG Runittobuildyourowncross compilingtoolchain
32
Bootloaders
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/bootloaders Corrections,suggestions,contributionsandtranslationsarewelcome!
Bootloaders
Thebootloaderisapieceofcoderesponsiblefor
Basichardwareinitialization Loadingofanapplicationbinary,usuallyanoperatingsystem kernel,fromflashstorage,fromthenetwork,orfromanothertypeof nonvolatilestorage. Possiblyuncompressionoftheapplicationbinary Executionoftheapplication
Besidesthesebasicfunctions,mostbootloadersprovideashell withvariouscommandsimplementingdifferentoperations.
Loadingofdatafromstorageornetwork,memoryinspection, hardwarediagnosticsandtesting,etc.
2
Bootloadersonx86(1)
Thex86processorsaretypicallybundledonaboard withanonvolatilememorycontainingaprogram,the BIOS. ThisprogramgetsexecutedbytheCPUafterreset, andisresponsibleforbasichardwareinitializationand loadingofasmallpieceofcodefromnonvolatile storage.
Thispieceofcodeisusuallythefirst512bytesofa storagedevice
BIOS
fromROM
Stage1
512bytes fromraw storage
Stage2
from rawstorage
Thispieceofcodeisusuallya1ststagebootloader, whichwillloadthefullbootloaderitself.
Thebootloadercanthenofferallitsfeatures.Ittypically Kernel understandsfilesystemformatssothatthekernelfile from canbeloadeddirectlyfromanormalfilesystem. filesystem

3
Bootloadersonx86(2)
GRUB,GrandUnifiedBootloader,themostpowerfulone. http://www.gnu.org/software/grub/ Canreadmanyfilesystemformatstoloadthekernelimage andtheconfiguration,providesapowerfulshellwithvarious commands,canloadkernelimagesoverthenetwork,etc. Seeourdedicatedpresentationfordetails: http://freeelectrons.com/docs/grub/ Syslinux,fornetworkandremovablemediabooting http://syslinux.zytor.com
4
BootingonembeddedCPUs:case1
Whenpowered,theCPUstartsexecutingcode atafixedaddress Thereisnootherbootingmechanismprovided bytheCPU ThehardwaredesignmustensurethataNOR flashiswiredsothatitisaccessibleatthe addressatwhichtheCPUstartsexecuting instructions Thefirststagebootloadermustbeprogrammed atthisaddressintheNOR NORismandatory,becauseitallowsrandom access,whichNANDdoesn'tallow Notverycommonanymore(unpractical,and requiresNORflash)
Physical memory
Execution starts here
NOR flash
RAM
BootingonembeddedCPUs:case2
TheCPUhasanintegratedbootcodeinROM
BootROMonAT91CPUs,ROMcodeonOMAP,etc. ExactdetailsareCPUdependent
Thisbootcodeisabletoloadafirststagebootloaderfroma storagedeviceintoaninternalSRAM(noDRAMaccesspossible)
Storagedevicecantypicallybe:MMC,NAND,SPIFlash,UART, etc.
Thefirststagebootloaderis
Limitedinsizeduetohardwareconstraints ProvidedeitherbytheCPUvendorofthroughcommunityprojects
ThisfirststagebootloadermustinitializetheDRAMandother hardwaredevicesandloadsasecondstagebootloaderintoRAM
6
BootingonARMAtmelAT91
RomBoot
StoredinROM
Triestofindavalidbootstrapimagefromvarious storagesources,andloaditintoSRAM(DRAMnot initializedyet).Sizelimitedto4KB.Nouserinteraction possibleinstandardbootmode. RunsfromSRAM.InitializestheDRAM,theNANDor SPIcontroller,andloadsthesecondarybootloaderinto RAMandstartsit.Nouserinteractionpossible.
AT91Bootstrap
StoredinNANDorSPIFlash RunsinSRAM
UBoot
StoredinNANDorSPIFlash RunsinRAM
RunsfromRAM.Initializessomeotherhardware devices(network,USB,etc.).Loadsthekernelimage fromstorageornetworktoRAMandstartsit.Shellwith commandsprovided. RunsfromRAM.Takesoverthesystemcompletely (bootloadersnolongerexists).

7
LinuxKernel
StoredinNAND RunsinRAM
BootingonARMOMAP3
ROMCode
StoredinROM
Triestofindavalidbootstrapimagefromvariousstorage sources,andloaditintoSRAMorRAM(RAMcanbe initializedbyROMcodethroughaconfigurationheader). Sizelimitedto<64KB.Nouserinteractionpossible. RunsfromSRAM.InitializestheDRAM,theNANDor MMCcontroller,andloadsthesecondarybootloader intoRAMandstartsit.Nouserinteractionpossible.File calledMLO. RunsfromRAM.Initializessomeotherhardware devices(network,USB,etc.).Loadsthekernelimage fromstorageornetworktoRAMandstartsit.Shellwith commandsprovided.Filecalleduboot.bin. RunsfromRAM.Takesoverthesystemcompletely (bootloadersnolongerexists).
8
XLoader
StoredinMMCorNAND RunsinSRAM
UBoot
StoredinMMCorNAND RunsinRAM
LinuxKernel
StoredinMMCorNAND RunsinRAM
GenericbootloadersforembeddedCPUs
Wewillfocusonthegenericpart,themainbootloader,offering themostimportantfeatures. Thereareseveralopensourcegenericbootloaders. Herearethemostpopularones:
UBoot,theuniversalbootloaderbyDenx ThemostusedonARM,alsousedonPPC,MIPS,x86,m68k,NIOS,etc. Thedefactostandardnowadays.Wewillstudyitindetail. http://www.denx.de/wiki/UBoot Barebox,anewarchitectureneutralbootloader,writtenasasuccessorof UBoot.Betterdesign,bettercode,activedevelopment,butdoesn'tyet haveasmuchhardwaresupportasUBoot. http://www.barebox.org
Therearealsoalotofotheropensourceorproprietary bootloaders,oftenarchitecturespecific
RedBoot,Yaboot,PMON,etc.
9
TheUbootbootloader
TheUboot bootloader
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/uboot Corrections,suggestions,contributionsandtranslationsarewelcome!
UBoot
UBootisatypicalfreesoftwareproject Freelyavailableathttp://www.denx.de/wiki/UBoot Documentationavailableat http://www.denx.de/wiki/UBoot/Documentation ThelatestdevelopmentsourcecodeisavailableinaGit repository: http://git.denx.de/cgibin/gitweb.cgi?p=uboot.git;a=summary Developmentanddiscussionshappenaroundanopenmailing listhttp://lists.denx.de/pipermail/uboot/ Sincetheendof2008,itfollowsafixedintervalreleaseschedule. Everytwomonths,anewversionisreleased.Versionsare namedYYYY.MM.
2
UBootconfiguration
Getthesourcecodefromthewebsite,anduncompressit Theinclude/configs/directorycontainsoneconfiguration fileforeachsupportedboard
ItdefinestheCPUtype,theperipheralsandtheirconfiguration,the memorymapping,theUBootfeaturesthatshouldbecompiledin, etc. Itisasimple.hfilethatsetspreprocessorconstants.Seethe READMEfileforthedocumentationoftheseconstants.
AssumingthatyourboardisalreadysupportedbyUBoot,there shouldbeonefilecorrespondingtoyourboard,forexample include/configs/igep0020.h ThisfilecanalsobeadjustedforaddorremovefeaturesfromU Boot

3
UBootconfigurationfileexcerpt
#defineCONFIG_ARMV71 #defineCONFIG_OMAP1 #defineCONFIG_OMAP34XX1 #defineCONFIG_OMAP34301 #defineCONFIG_OMAP3_IGEP00201 [...] #defineCONFIG_NR_DRAM_BANKS2 #definePHYS_SDRAM_1OMAP34XX_SDRC_CS0 #definePHYS_SDRAM_1_SIZE(32<<20) #definePHYS_SDRAM_2OMAP34XX_SDRC_CS1 [...] /*USB*/ #defineCONFIG_MUSB_UDC1 #defineCONFIG_USB_OMAP31 #defineCONFIG_TWL4030_USB1 [...] #defineCONFIG_CMD_CACHE #defineCONFIG_CMD_EXT2 #defineCONFIG_CMD_FAT #defineCONFIG_CMD_I2C #defineCONFIG_CMD_MMC #defineCONFIG_CMD_ONENAND #defineCONFIG_CMD_NET #defineCONFIG_CMD_DHCP #defineCONFIG_CMD_PING #defineCONFIG_CMD_NFS #defineCONFIG_CMD_MTDPARTS
CPU configuration Memory configuration USB configuration Available commands andfeatures
4
ConfiguringandcompilingUBoot
UBootmustbeconfiguredbeforebeingcompiled
makeBOARDNAME_config WhereBOARDNAMEisusuallythenameoftheconfigurationfilein include/configs/,withoutthe.h
MakesurethatthecrosscompilerisavailableinPATH CompileUBoot,byspecifyingthecrosscompilerprefix. Example,ifyourcrosscompilerexecutableisarmlinuxgcc: makeCROSS_COMPILE=armlinux Theresultisauboot.binfile,whichistheUBootimage
5
InstallingUBoot
UBootmustusuallybeinstalledinflashmemorytobeexecuted bythehardware.Dependingonthehardware,theinstallationof UBootisdoneinadifferentway:
TheCPUprovidessomekindofspecificbootmonitorwithwhich youcancommunicatethroughserialportorUSBusingaspecific protocol TheCPUbootsfirstonremovablemedia(MMC)beforebooting fromfixedmedia(NAND).Inthiscase,bootfromMMCtoreflasha newversion UBootisalreadyinstalled,andcanbeusedtoflashanewversion ofUBoot.However,becareful:ifthenewversionofUBootdoesn't work,theboardisunusable TheboardprovidesaJTAGinterface,whichallowstowritetothe flashmemoryremotely,withoutanysystemrunningontheboard.It alsoallowstorescueaboardifthebootloaderdoesn'twork.
6
Ubootprompt
Connectthetargettothehostthroughaserialconsole Poweruptheboard.Ontheserialconsole,youwillsee somethinglike:
UBoot2010.062(May13201112:13:22) OMAP3630/3730GPES2.0,CPUOPP2,L3165MHz IGEPv2board+LPDDR/ONENAND I2C:ready DRAM:512MiB MuxedOneNAND(DDP)512MB1.8V16bit(0x58) OneNANDversion=0x0031 Chipsupportallblockunlock OneNAND:512MiB OneNAND:Readenvironmentfrom0x00200000 In:serial Out:serial Err:serial DieID#4d5400011ff0000001592f350202c01d Net:smc911x0
UBoot#
TheUBootshelloffersasetofcommands.Wewillstudy themostimportantones,seethedocumentationfora completereferenceorthehelpcommand.

7
Informationcommands
UBoot>flinfo DataFlash:AT45DB021 Nbpages:1024 PageSize:264 Size=270336bytes Logicaladdress:0xC0000000 Area0:C0000000toC0001FFF(RO)Bootstrap Area1:C0002000toC0003FFFEnvironment Area2:C0004000toC0041FFF(RO)UBoot UBoot>nandinfo Device0:NAND256MiB3,3V8bit,sectorsize128KiB UBoot>version UBoot2009.08(Nov15200914:48:35)
Flash information
NANDflash information UBoot information
Canvaryfromoneboardtotheother (accordingtotheUBootcompileconfiguration)
8
Importantcommands(1)
TheexactsetofcommandsdependsontheUBootconfiguration helpandhelpcommand boot,runsthedefaultbootcommand,storedinbootcmd bootm<address>,startsakernelimageloadedatthegiven addressinRAM ext2load,loadafilefromanext2filesystemtoRAM
Andalsoext2lstolistfiles,ext2infoforinformations
fatload,loadafilefromaFATfilesystemtoRAM
Andalsofatlsandfatinfo
tftp,loadafilefromthenetworktoRAM ping,totestthenetwork
9
Importantcommands(2)
loadb,loads,loady,loadafilefromtheseriallinetoRAM usb,toinitializeandcontroltheUSBsubsystem,mainlyusedfor USBstoragedevicessuchasUSBkeys mmc,toinitializeandcontroltheMMCsubsystem,usedforSD andmicroSDcards nand,toerase,readandwritecontentstotheNAND erase,protect,cp,toerase,modifyprotectionandwriteto aNORflash md,displaymemorycontents.Canbeusefultocheckthe contentsloadedinmemory,ortolookathardwareregisters. mm,modifymemorycontents.Canbeusefultomodifydirectly hardwareregisters,fortestingpurposes.
10
Environmentvariablescommands
UBootcanbeconfiguredthroughenvironmentvariables,which affectthebehaviorofthedifferentcommands. EnvironmentvariablesareloadedfromFlashtoRAMatUBoot startup,canbemodifiedandsavedbacktoFlashfor persistence ThereisadedicatedlocationinFlashtostoreUBoot environment,definedintheboardconfigurationfile Commandstomanipulateenvironmentvariables:
printenv,showallvariables printenv<variablename>,showvalueofonevariable setenv<variablename><variablevalue>,changethe valueofavariable,onlyinRAM saveenv,savetoFlashthecurrentstateoftheenvironment
11
Environmentvariablescommands(2)
uboot#printenv baudrate=19200 ethaddr=00:40:95:36:35:33 netmask=255.255.255.0 ipaddr=10.0.0.11 serverip=10.0.0.1 stdin=serial stdout=serial stderr=serial uboot#printenvserverip serverip=10.0.0.2 uboot#setenvserverip10.0.0.2 uboot#saveenv
12
ImportantUBootenvvariables
bootcmd,containsthecommandthatUBootwillautomatically executeatboottimeafteraconfigurabledelay,iftheprocessisnot interrupted bootargs,containstheargumentspassedtotheLinuxkernel, coveredlater serverip,theIPaddressoftheserverthatUBootwillcontactfor networkrelatedcommands ipaddr,theIPaddressthatUBootwilluse netmask,thenetworkmasktocontacttheserver ethaddr,theMACaddress,canonlybesetonce bootdelay,thedelayinsecondsbeforewhichUBootrunsbootcmd autostart,ifyes,UBootstartsautomaticallyanimagethathasbeen loadedintomemory
13
Scriptsinenvironmentvariables
Environmentvariablescancontainsmallscripts,toexecute severalcommandsandtesttheresultsofcommands.
Usefultoautomatebootingorupgradeprocesses Severalcommandcanbechainedusingthe;operator Testscanbedoneusingifcommand;then;else; fi Scriptsareexecutedusingrun<variablename> Youcanreferenceothervariablesusing${variablename}
Example setenvmmcboot'mmcinit0;iffatloadmmc0 80000000boot.ini;thensource;elseif fatloadmmc080000000uImage;thenrunmmc bootargs;bootm;fi;fi'

14
Transferringfilestothetarget
UBootismostlyusedtoloadandbootakernelimage,butitalso allowstochangethekernelimageandtherootfilesystemstored inflash. Filesmustbeexchangedbetweenthetargetandthe developmentworkstation.Thisispossible:
ThroughthenetworkifthetargethasanEthernetconnection,and UBootcontainsadriverfortheEthernetchip.Thisisthefastest andmostefficientsolution. ThroughanUSBkey,ifUBootsupporttheUSBcontrollerofyour platform ThroughaSDormicroSDcard,ifUBootsupportstheMMC controllerofyourplatform Throughtheserialport
15
TFTP
NetworktransferfromthedevelopmentworkstationandUBoot onthetargettakesplacethroughTFTP
TrivialFileTransferProtocol SomewhatsimilartoFTP,butwithoutauthenticationandoverUDP
ATFTPserverisneededonthedevelopmentworkstation
sudoaptgetinstalltftpdhpda Allfilesin/var/lib/tftpbootarethenvisiblethroughTFTP ATFTPclientisavailableinthetftphpapackage,fortesting
ATFTPclientisintegratedintoUBoot
Configuretheipaddrandserveripenvironmentvariables Usetftp<address><filename>toloadafile
16
Ubootmkimage
ThekernelimagethatUBootloadsandbootsmustbeprepared, sothatanUBootspecificheaderisaddedinfrontoftheimage Thisheadergivesdetailssuchastheimagesize,the expectedloadaddress,thecompressiontype,etc. ThisisdonewithatoolthatcomesinUBoot,mkimage Debian/Ubuntu:justinstalltheubootmkimagepackage. Or,compileitbyyourself:simplyconfigureUBootforanyboard ofanyarchitectureandcompileit.Theninstallmkimage: cptools/mkimage/usr/local/bin/ ThespecialtargetuImageofthekernelMakefilecanthenbe usedtogenerateakernelimagesuitableforUBoot.
17
Flashingakernelimage
CompileyourkernelandgeneratetheUBootheader runningmakeuImage CopythekernelimagetothedirectoryexportedbytheTFTPserver Ontheboard,inUBoot,downloadthekernelimagetomemory: uboot#tftp8000uImage UnprotectNORflash uboot#protectoff1:04 EraseNORflash uboot#erase1:04 CopytoNORflash(0x01000000:firstsector) uboot#cp.b${fileaddr}1000000${filesize} RestoreNORflashsectorprotection: uboot#protecton1:04 SeeourpracticallabsfordetailshandlingNANDflash.
18
PracticallabUBoot
Timetostartthepracticallab! Communicatewiththeboard usingaserialconsole Configure,buildandinstallX LoaderandUBoot LearnUBootcommands SetupTFTPcommunication withtheboard
1
FreeElectrons
Linuxkernel introduction
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelintro Corrections,suggestions,contributionsandtranslationsarewelcome!
EmbeddedLinuxdriverdevelopment
Kerneloverview
Linuxfeatures
2
Linuxkernelinthesystem
UserappB LibraryA UserappA Clibrary Calltoservices LinuxKernel Manage hardware Hardware
3
Userspace
Eventnotification, informationexposition
Eventnotification
History
TheLinuxkernelisonecomponentofasystem,whichalso requireslibrariesandapplicationstoprovidefeaturestoend users. TheLinuxkernelwascreatedasahobbyin1991 byaFinnishstudent,LinusTorvalds.
Linuxquicklystartedtobeusedasthekernelforfreesoftware operatingsystems
LinusTorvaldshasbeenabletocreatealargeanddynamic developerandusercommunityaroundLinux. Nowadays,hundredsofpeoplecontributetoeachkernelrelease, individualsorcompaniesbigandsmall.
4
Linuxlicense
ThewholeLinuxsourcesareFreeSoftwarereleased undertheGNUGeneralPublicLicenseversion2(GPLv2).
FortheLinuxkernel,thisbasicallyimpliesthat: WhenyoureceiveorbuyadevicewithLinuxonit, youshouldreceivetheLinuxsources,withtherightto study,modifyandredistributethem. WhenyouproduceLinuxbaseddevices,youmust releasethesourcestotherecipient,withthesamerights, withnorestriction..
5
Linuxkernelkeyfeatures
Portabilityandhardware support Runsonmostarchitectures. Scalability Canrunonsupercomputers aswellasontinydevices (4MBofRAMisenough). Compliancetostandardsand interoperability. Exhaustivenetworking support. Security Itcan'thideitsflaws.Itscode isreviewedbymanyexperts. Stabilityandreliability. Modularity Canincludeonlywhata systemneedsevenatrun time. Easytoprogram Youcanlearnfromexisting code.Manyusefulresources onthenet.
6
Supportedhardwarearchitectures
2.6.31status Seethearch/directoryinthekernelsources Minimum:32bitprocessors,withorwithoutMMU,andgccsupport 32bitarchitectures(arch/subdirectories) arm,avr32,blackfin,cris,frv,h8300,m32r,m68k, m68knommu,microblaze,mips,mn10300,parisc,s390, sparc,um,xtensa 64bitarchitectures: alpha,ia64,sparc64 32/64bitarchitectures powerpc,x86,sh Finddetailsinkernelsources:arch/<arch>/Kconfig, arch/<arch>/README,orDocumentation/<arch>/
7
Systemcalls
Themaininterfacebetweenthekernelanduserspaceistheset ofsystemcalls About~300systemcallsthatprovidesthemainkernelservices
Fileanddeviceoperations,networkingoperations,interprocess communication,processmanagement,memorymapping,timers, threads,synchronizationprimitives,etc.
Thisinterfaceisstableovertime:onlynewsystemcallscanbe addedbythekerneldevelopers ThissystemcallinterfaceiswrappedbytheClibrary,and userspaceapplicationsusuallynevermakeasystemcalldirectly butratherusethecorrespondingClibraryfunction
8
Virtualfilesystems
Linuxmakessystemandkernelinformationavailablein userspacethroughvirtualfilesystems. Virtualfilesystemsallowapplicationstoseedirectoriesand filesthatdonotexistonanyrealstorage:theyarecreated ontheflybythekernel Thetwomostimportantvirtualfilesystemsare proc,forprocessrelatedinformations sysfs,fordevicerelatedinformations
9
EmbeddedLinuxusage
Kerneloverview
Linuxversioningschemeanddevelopmentprocess
10
Until2.6(1)
Onestablemajorbranchevery2or3years Identifiedbyanevenmiddlenumber Examples:1.0,2.0,2.2,2.4 Onedevelopmentbranchtointegratenewfunctionalitiesand majorchanges Identifiedbyanoddmiddlenumber Examples:2.1,2.3,2.5 Aftersometime,adevelopmentversionbecomesthenew baseversionforthestablebranch Minorreleasesonceinwhile:2.2.23,2.5.12,etc.
11
Until2.6(2)
Stableversion 2.4.0 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.4.6 2.4.7 2.4.8
2.5.0
2.5.1 2.5.2
2.5.3
2.5.4
2.6.0
2.6.1 Stable
Development
Note:inreality,manymoreminor versionsexistinsidethestableand developmentbranches
12
ChangessinceLinux2.6(1)
Since2.6.0,kerneldevelopershavebeenableto introducelotsofnewfeaturesonebyoneonasteadypace, withouthavingtomakemajorchangesinexisting subsystems. OpeninganewLinux2.7(or2.9)developmentbranchwill berequiredonlywhenLinux2.6isnolongerableto accommodatekeyfeatureswithoutundergoingtraumatic changes. Thankstothis,morefeaturesarereleasedtousersata fasterpace.
13
ChangessinceLinux2.6(2)
Since2.6.14,thekerneldevelopersagreed onthefollowingdevelopmentmodel: Afterthereleaseofa2.6.xversion,atwoweeksmergewindow opens,duringwhichmajoradditionsaremerged. Themergewindowisclosed bythereleaseoftestversion2.6.(x+1)rc1 Thebugfixingperiodopens,for6to10weeks. Atregularintervalsduringthebugfixingperiod, 2.6.(x+1)rcYtestversionsarereleased. Whenconsideredsufficientlystable, kernel2.6.(x+1)isreleased,andtheprocessstartsagain.
14
Mergeandbugfixingwindows
2weeks
Mergewindow 2.6.21 2.6.22rc1 2.6.22rc2
6to10weeks
Bugfixingperiod 2.6.22rc3 2.6.22rc4 2.6.22rc5 2.6.22
2.6.21.1 2.6.21.2
2.6.21.3 2.6.21.4
2.6.21.5
2.6.22.1
Bugfixupdates
15
Morestabilityforthe2.6kerneltree
Issue:bugandsecurityfixesonlyreleasedfor mostrecentstablekernelversions. Somepeopleneedtohavearecentkernel,but withlongtermsupportforsecurityupdates. Youcouldgetlongtermsupportfroma commercialembeddedLinuxprovider. Youcouldreusesourcesforthekernelusedin UbuntuLongTermSupportreleases(5yearsof freesecurityupdates). Youcouldchooseoneoftheversionsadvertised aslongterminthekernel.orgfrontpage.They willbemaintainedlonger(2or3years),unlike otherversions.
16
Towardsversion3.0
From2003to2011,theofficialkernelversionswerenamed2.6.x. OnMay,29th2011,LinusTorvaldsreleasedversionLinux3.0rc1 AroundSeptember2011,thefinalLinux3.0kernelshouldbe released Thereisnochangetothedevelopmentmodel,onlyachangeto thenumberingscheme
Officialkernelversionswillbenamed3.x(3.0,3.1,3.2,etc.) Stabilizedversionswillbenamed3.x.y(3.0.2,3.4.3,etc.) Iteffectivelyonlyremovesadigitcomparedtotheprevious numberingscheme
17
What'snewineachLinuxrelease?
commit3c92c2ba33cd7d666c5f83cc32aa590e794e91b0 Author:AndiKleen<ak@suse.de> Date:TueOct1101:28:332005+0200 [PATCH]i386:Don'tdiscardupper32bitsofHWCRonK8 Needtouselonglong,notlongwhenRMWingaMSR.Ithink it'sharmlessrightnow,butstillshouldbebetterfixed ifAMDaddsanybitsintheupper32bitofHWCR. BugwasintroducedwiththeTLBflushfilterfixfori386 Signedoffby:AndiKleen<ak@suse.de> Signedoffby:LinusTorvalds<torvalds@osdl.org>
?? !
TheofficiallistofchangesforeachLinuxreleaseisjusta hugelistofindividualpatches!
Verydifficulttofindoutthekeychangesandtogettheglobal pictureoutofindividualchanges.
Fortunately,therearesomeusefulresourcesavailable
http://wiki.kernelnewbies.org/LinuxChanges http://lwn.net http://linuxfr.org,forFrenchreaders
18
EmbeddedLinuxkernelusage
EmbeddedLinux kernelusage
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelusage Corrections,suggestions,contributionsandtranslationsarewelcome!
Contents
Compilingandbooting Linuxkernelsources Kernelconfiguration Compilingthekernel Overallsystemstartup Linuxdevicefiles Crosscompilingthekernel
2
EmbeddedLinuxusage
CompilingandbootingLinux
Linuxkernelsources
3
Locationofkernelsources
TheofficialversionoftheLinuxkernel,asreleasedbyLinus Torvaldsisavailableathttp://www.kernel.org
Thisversionfollowsthewelldefineddevelopmentmodelofthe kernel However,itmaynotcontainthelatestdevelopmentfromaspecific area,duetotheorganizationofthedevelopmentmodeland becausefeaturesindevelopmentmightnotbereadyformainline inclusion
Manykernelsubcommunitiesmaintaintheirownkernel,with usuallynewerbutlessstablefeatures
Architecturecommunities(ARM,MIPS,PowerPC,etc.),device driverscommunities(I2C,SPI,USB,PCI,network,etc.),other communities(realtime,etc.) Theygenerallydon'treleaseofficialversions,onlydevelopment treesareavailable
4
Linuxkernelsize(1)
Linux2.6.37sources: Rawsize:412MB(37,300files,approx14,000,000lines) gzipcompressedtararchive:89MB bzip2compressedtararchive:71MB(better) lzmacompressedtararchive:61MB(best) MinimumLinux2.6.29compiledkernelsizewithCONFIG_EMBEDDED, forakernelthatbootsaQEMUPC(IDEharddrive,ext2filesystem, ELFexecutablesupport): 532KB(compressed),1325KB(raw) Whyarethesesourcessobig? Becausetheyincludethousandsofdevicedrivers,manynetwork protocols,supportmanyarchitecturesandfilesystems... TheLinuxcore(scheduler,memorymanagement...)isprettysmall!
5
Linuxkernelsize(2)
SizeofLinuxsourcedirectories(KB)
arch block crypto Documentation drivers firmware fs include init ipc kernel lib mm net samples scripts security sound tools usr virt 0 50000 100000 150000 200000 250000
Linux2.6.39
6
GettingLinuxsources
Fulltarballs
Containthecompletekernelsources Longtodownloadanduncompress,butmustbedoneatleastonce Example: http://www.kernel.org/pub/linux/kernel/v2.6/linux2.6.38.7.tar.bz2 Incrementalpatchesbetweenversions Itassumesyoualreadyhaveabaseversionandyouapplythe correctpatchesintherightorder Quicktodownloadandapply Examples
http://www.kernel.org/pub/linux/kernel/v2.6/patch2.6.38.bz2(2.6.37to2.6.38) http://www.kernel.org/pub/linux/kernel/v2.6/patch2.6.38.7.bz2(2.6.38to2.6.38.7)
Allpreviouskernelversionsareavailablein http://kernel.org/pub/linux/kernel/
Patch
Apatchisthedifferencebetweentwosourcetrees
Computedwiththedifftool,orwithmoreelaborateversion controlsystems
Theyareverycommonintheopensourcecommunity Excerptfromapatch:
diffNrua/Makefileb/Makefile Filebeingmodified a/Makefile2005030409:27:1508:00 +++b/Makefile2005030409:27:1508:00 Linenumbersinfiles @@1,7+1,7@@ VERSION=2 Contextinfo:3linesbeforethechange Usefultoapplyapatchwhenlinenumbers PATCHLEVEL=6 changed SUBLEVEL=11 Removedline(s)ifany EXTRAVERSION= Addedline(s)ifany +EXTRAVERSION=.1 NAME=WoozyNumbat #*DOCUMENTATION* Contextinfo:3linesafterthechange
8
Usingthepatchcommand
Thepatchcommand: Takesthepatchcontentsonitsstandardinput Appliesthemodificationdescribedbythepatch intothecurrentdirectory patchusageexamples: patchp<n><diff_file catdiff_file|patchp<n> bzcatdiff_file.bz2|patchp<n> zcatdiff_file.gz|patchp<n> n:numberofdirectorylevelstoskipinthefilepaths
Youcantestapatchwith thedryrun option Youcanreverse apatch withtheR option
9
PracticallabKernelsources
Getthesources Applypatches
10
EmbeddedLinuxusage
Kernelconfiguration
11
Kernelconfigurationandbuildsystem
Thekernelconfigurationandbuildsystemisbasedonmultiple Makefiles OneonlyinteractswiththemainMakefile,presentatthetop directoryofthekernelsourcetree Interactiontakesplace
usingthemaketool,whichparsestheMakefile throughvarioustargets,definingwhichactionshouldbedone (configuration,compilation,installation,etc.).Runmakehelpto seeallavailabletargets.
Example
cdlinux2.6.x/ make<target>
12
Kernelconfiguration(1)
Thekernelcontainsthousandsofdevicedrivers,filesystem drivers,networkprotocolsandotherconfigurableitems Thousandsofoptionsareavailable,thatareusedtoselectively compilepartsofthekernelsourcecode Thekernelconfigurationistheprocessofdefiningthesetof optionswithwhichyouwantyourkerneltobecompiled Thesetofoptionsdepends
Onyourhardware(fordevicedrivers,etc.) Onthecapabilitiesyouwouldliketogivetoyourkernel(network capabilities,filesystems,realtime,etc.)
13
Kernelconfiguration(2)
Theconfigurationisstoredinthe.configfileattherootof kernelsources Simpletextfile,key=valuestyle Asoptionshavedependencies,typicallynevereditedbyhand, butthroughgraphicalinterfaces:
makexconfig makegconfig makemenuconfig Youcanswitchfromonetoanother,theyallload/savethesame .configfile,andshowthesamesetofoptions
TomodifyakernelinaGNU/Linuxdistribution: theconfigurationfilesareusuallyreleasedin/boot/,together withkernelimages:/boot/config2.6.1711generic
14
Kernelormodule?
Thekernelimageisasinglefile,resultingfromthelinkingofall objectfilesthatcorrespondtofeaturesenabledinthe configuration
Thisisthefilethatgetsloadedinmemorybythebootloader Allincludedfeaturesarethereforeavailableassoonasthekernel starts,atatimewherenofilesystemexists
Somefeatures(devicedrivers,filesystems,etc.)canhoweverbe compiledasmodules
Thosearepluginsthatcanbeloaded/unloadeddynamicallyto add/removefeaturestothekernel Eachmoduleisstoredasaseparatefileinthefilesystem,and thereforeaccesstoafilesystemismandatorytousemodules Thisisnotpossibleintheearlybootprocedureofthekernel, becausenofilesystemisavailable
15
Kerneloptiontypes
Therearedifferenttypesofoptions
booloptions,theyareeither
true(toincludethefeatureinthekernel)or false(toexcludethefeaturefromthekernel)
tristateoptions,theyareeither
true(toincludethefeatureinthekernelimage)or module(toincludethefeatureasakernelmodule)or false(toexcludethefeature)
intoptions,tospecifyintegervalues stringoptions,tospecifystringvalues
16
Kerneloptiondependencies
Therearedependenciesbetweenkerneloptions Forexample,enablinganetworkdriverrequiresthenetwork stacktobeenabled Twotypesofdependencies
dependsondependencies.Inthiscase,optionAthatdependson optionBisnotvisibleuntiloptionBisenabled selectdependencies.Inthiscase,withoptionAdependingon optionB,whenoptionAisenabled,optionBisautomatically enabled makexconfigallowstoseealloptions,eventhosethatcannotbe selectedbecauseofmissingdependencies.Inthiscase,theyare displayedingray
17
makexconfig
makexconfig Themostcommongraphicalinterface toconfigurethekernel. Makesureyouread help>introduction:usefuloptions! Filebrowser:easiertoloadconfigurationfiles Newsearchinterfacetolookforparameters RequiredDebian/Ubuntupackages: libqt3mtdev,g++
18
makexconfigscreenshot
19
makexconfigsearchinterface
Looksforakeyword inthedescription string Allowstoselect orunselectfound parameters.
20
Kernelconfigurationoptions
Compiledasamodule(separatefile) CONFIG_ISO9660_FS=m
Driveroptions CONFIG_JOLIET=y CONFIG_ZISOFS=y
Compiledstaticallyintothekernel CONFIG_UDF_FS=y
21
Corresponding.configfileexcerpt
# #CDROM/DVDFilesystems # CONFIG_ISO9660_FS=m CONFIG_JOLIET=y CONFIG_ZISOFS=y CONFIG_UDF_FS=y CONFIG_UDF_NLS=y
Sectionname (helpstolocatesettingsintheinterface)
Allparametersareprefixed withCONFIG_
# #DOS/FAT/NTFilesystems # #CONFIG_MSDOS_FSisnotset #CONFIG_VFAT_FSisnotset CONFIG_NTFS_FS=m #CONFIG_NTFS_DEBUGisnotset CONFIG_NTFS_RW=y

22
makegconfig
makegconfig NewGTKbased graphicalconfiguration interface.Functionality similartothatofmake xconfig. Justlackingasearch functionality. RequiredDebian packages: libglade2dev
23
makemenuconfig
makemenuconfig Usefulwhennographics areavailable.Pretty convenienttoo! Sameinterfacefoundin othertools:BusyBox, buildroot... RequiredDebian packages: libncursesdev
24
makeoldconfig
makeoldconfig Neededveryoften! Usefultoupgradea.configfilefromanearlierkernel release Issueswarningsforconfigurationparameters thatnolongerexistinthenewkernel. Asksforvaluesfornewparameters Ifyouedita.configfilebyhand,it'sstrongly recommendedtorunmakeoldconfigafterwards!
25
makeallnoconfig
makeallnoconfig Onlysetsstronglyrecommendedsettingstoy. Setsallothersettingston. Veryusefulinembeddedsystemstoselectonlythe minimumrequiredsetoffeaturesanddrivers. Muchmoreconvenientthanunselectinghundredsof featuresonebyone!
26
Undoingconfigurationchanges
Afrequentproblem: Afterchangingseveralkernelconfigurationsettings, yourkernelnolongerworks. Ifyoudon'trememberallthechangesyoumade, youcangetbacktoyourpreviousconfiguration: >cp.config.old.config Alltheconfigurationinterfacesofthekernel (xconfig,menuconfig,allnoconfig...) keepthis.config.oldbackupcopy.
27
Configurationperarchitecture
Thesetofconfigurationoptionsisarchitecturedependent
Someconfigurationoptionsareveryarchitecturespecific Mostoftheconfigurationoptions(globalkerneloptions,network subsystem,filesystems,mostofthedevicedrivers)arevisibleinall architecture
Bydefault,thekernelbuildsystemassumesthatthekernelis beingbuiltforthehostarchitecture,i.enativecompilation Thearchitectureisnotdefinedinsidetheconfiguration,butatan higherlevel Wewillseelaterhowtooverridethisbehaviour,toallowthe configurationofkernelsforadifferentarchitecture
28
Overviewofkerneloptions(1)
Generalsetup
Promptfordevelopment/incompletecodeallowstobeableto enabledriversorfeaturesthatarenotconsideredascompletely stableyet Localversionappendtokernelreleaseallowstoconcatenatean arbitrarystringtothekernelversionthatanusercangetusing unamer.Veryusefulforsupport! Supportforswap,canusuallybedisabledonmostembedded devices Configurestandardkernelfeatures(forsmallsystems)allowsto removefeaturesfromthekerneltoreduceitssize.Powerful,use withcare!
29
Loadablemodulesupport
Allowstoenableorcompletelydisablemodulesupport.Ifyour systemdoesn'tneedkernelmodules,besttodisablesinceitsaves asignificantamountofspaceandmemory
Enabletheblocklayer
IfCONFIG_EMBEDDEDisenabled,theblocklayercanbe completelyremoved.EmbeddedsystemsusingonlyFlashstorage cansafelydisabletheblocklayer
Processortypeandfeatures(x86)orSystemtype(ARM)orCPU selection(MIPS)
AllowstoselecttheCPUormachineforwhichthekernelmustbe compiled Onx86,onlyoptimizationrelated,onotherarchitecturesvery importantsincethere'snocompatibility
30
Kernelfeatures
Ticklesssystem,whichallowstodisabletheregulartimertickand useondemandticksinstead.Improvespowersavings Highresolutiontimersupport.Bydefault,theresolutionoftimeris thetickresolution.Withhighresolutiontimers,theresolutionisas preciseasthehardwarecangive Preemptiblekernelenablesthepreemptioninsidethekernelcode (theuserspacecodeisalwayspreemptible).Seeourrealtime presentationfordetails
Powermanagement
Globalpowermanagementoptionneededforallpower managementrelatedfeatures SuspendtoRAM,CPUfrequencyscaling,CPUidlecontrol, suspendtodisk
31
Networkingsupport
Thenetworkstack Networkingoptions
Unixsockets,neededforaformofinterprocesscommunication TCP/IPprotocolwithoptionsformulticast,routing,tunneling,Ipsec, Ipv6,congestionalgorithms,etc. OtherprotocolssuchasDCCP,SCTP,TIPC,ATM Ethernetbridging,QoS,etc.
Supportforothertypesofnetwork
CANbus,Infrared,Bluetooth,Wirelessstack,WiMaxstack,etc.
32
Devicedrivers
MTDisthesubsystemforFlash(NOR,NAND,OneNand,battery backedmemory,etc.) Parallelportsupport Blockdevices,afewmiscblockdriverssuchasloopback,NBD, etc. ATA/ATAPI,supportforIDEdisk,CDROMandtapes.Anewstack exists SCSI
TheSCSIcore,needednotonlyforSCSIdevicesbutalsoforUSB massstoragedevices,SATAandPATAharddrives,etc. SCSIcontrollerdrivers
33
Devicedrivers(cont)
SATAandPATA,thenewstackforharddisks,reliesonSCSI RAIDandLVM,toaggregateharddriversanddoreplication Networkdevicesupport,withthenetworkcontrollerdrivers. Ethernet,WirelessbutalsoPPP Inputdevicesupport,foralltypesofinputdevices:keyboards, mices,joysticks,touchscreens,tablets,etc. Characterdevices,containsvariousdevicedrivers,amongstthem
serialportcontrollerdrivers PTYdriver,neededforthingslikeSSHortelnet
I2C,SPI,1wire,supportforthepopularembeddedbuses Hardwaremonitoringsupport,infrastructureanddriversforthermal sensors

34
Devicedrivers(cont)
Watchdogsupport Multifunctiondriversaredriversthatdonotfitinanyothercategory becausethedeviceoffersmultiplefunctionalityatthesametime Multimediasupport,containstheV4LandDVBsubsystems,for videocapture,webcams,AM/FMcards,DVBadapters Graphicssupport,infrastructureanddriversforframebuffers Soundcardsupport,theOSSandALSAsoundinfrastructuresand thecorrespondingdrivers HIDdevices,supportforthedevicesthatconformtotheHID specification(HumanInputDevices)
35
Devicedrivers(cont)
USBsupport
Infrastructure Hostcontrollerdrivers Devicedrivers,fordevicesconnectedtotheembeddedsystem Gadgetcontrollerdrivers Gadgetdrivers,tolettheembeddedsystemactasamassstorage device,aserialportoranEthernetadapter
MMC/SD/SDIOsupport LEDsupport RealTimeClockdrivers Voltageandcurrentregulators Stagingdrivers,crappydriversbeingcleanedup

36
Forsomecategoriesofdevicesthedriverisnotimplemented insidethekernel
Printers Scanners GraphicsdriversusedbyX.org SomeUSBdevices
Forthesedevices,thekernelonlyprovidesamechanismto accessthehardware,thedriverisimplementedinuserspace
37
Filesystems
ThecommonLinuxfilesystemsforblockdevices:ext2,ext3,ext4 Lesscommonfilesystems:XFS,JFS,ReiserFS,GFS2,OCFS2, Btrfs CDROMfilesystems:ISO9660,UDF DOS/Windowsfilesystems:FATandNTFS Pseudofilesystems:procandsysfs Miscellanousfilesystems,withamongstotherFlashfilesystems suchasJFFS2,UBIFS,SquashFS,cramfs Networkfilesystems,withmainlyNFSandSMB/CIFS
Kernelhacking
Debuggingfeaturesusefulforkerneldevelopers
38
EmbeddedLinuxusage
Compilingandinstallingthekernel forthehostsystem
39
Kernelcompilation
make
inthemainkernelsourcedirectory Remembertorunmakej4ifyouhavemultipleCPUcoresto speedupthecompilationprocess Noneedtorunasroot!
Generates
vmlinux,therawuncompressedkernelimage,attheELFformat, usefulfordebuggingpurposes,butcannotbebooted arch/<arch>/boot/*Image,thefinal,usuallycompressed, kernelimagethatcanbebooted
bzImageforx86,zImageforARM,vmImage.gzforBlackfin,etc.
Allkernelmodules,spreadoverthekernelsourcetree,as.kofiles.
40
Kernelinstallation
makeinstall
Doestheinstallationforthehostsystembydefault,soneedstobe runasroot.Generallynotusedwhencompilingforanembedded system,anditinstallsfilesonthedevelopmentworkstation.
Installs /boot/vmlinuz<version> Compressedkernelimage.Sameastheonein arch/<arch>/boot /boot/System.map<version> Storeskernelsymboladdresses /boot/config<version> Kernelconfigurationforthisversion Typicallyrerunsthebootloaderconfigurationutilitytotakeinto accountthenewkernel.
41
Moduleinstallation
makemodules_install
Doestheinstallationforthehostsystembydefault,soneedstobe runasroot
Installsallmodulesin/lib/modules/<version>/ kernel/ Module.ko(KernelObject)files,inthesamedirectory structureasinthesources. modules.alias Modulealiasesformoduleloadingutilities.Exampleline: aliassoundservice?0snd_mixer_oss modules.dep Moduledependencies modules.symbols Tellswhichmoduleagivensymbolbelongsto.
42
Kernelcleanuptargets
Cleanupgeneratedfiles (toforcerecompilingdrivers): makeclean Removeallgeneratedfiles.Neededwhenswitching fromonearchitecturetoanother Caution:alsoremovesyour.configfile! makemrproper Alsoremoveeditorbackupandpatchrejectfiles: (mainlytogeneratepatches): makedistclean
43
EmbeddedLinuxusage
Crosscompilingthekernel
44
Crosscompilingthekernel
WhenyoucompileaLinuxkernelforanotherCPUarchitecture Muchfasterthancompilingnatively,whenthetargetsystemis muchslowerthanyourGNU/Linuxworkstation. MucheasierasdevelopmenttoolsforyourGNU/Linux workstationaremucheasiertofind. Tomakethedifferencewithanativecompiler,crosscompiler executablesareprefixedbythenameofthetargetsystem, architectureandsometimeslibrary.Examples: mipslinuxgcc m68klinuxuclibcgcc armlinuxgnueabigcc
45
Specifyingcrosscompilation
TheCPUarchitectureandcrosscompilerprefixaredefinedthroughthe ARCHandCROSS_COMPILEvariablesinthetoplevelMakefile. ARCHisthenameofthearchitecture.Itisdefinedbythenameofthe subdirectoryinarch/inthekernelsources CROSS_COMPILEistheprefixofthecrosscompilationtools Example:armlinuxifyourcompilerisarmlinuxgcc Threesolutions ForcethesetwovariablesinthemainkernelMakefile ARCH ?=arm CROSS_COMPILE ?=armlinux PassARCHandCROSS_COMPILEonthemakecommandline DefineARCHandCROSS_COMPILEasenvironmentvariables Don'tforgettohavethevaluesproperlysetatallsteps,otherwisethe kernelconfigurationandbuildsystemgetsconfused
46
Predefinedconfigurationfiles
Defaultconfigurationfilesavailable,perboardorperCPUfamily
Theyarestoredinarch/<arch>/configs/,andarejustminimal .configfiles Thisisthemostcommonwayofconfiguringakernelforembedded platforms
Runmakehelptofindifoneisavailableforyourplatform Toloadadefaultconfigurationfile,justrun makeacme_defconfig

Thiswilloverwriteyourexisting.config!
Tocreateyourowndefaultconfigurationfile makesavedefconfig,tocreateaminimalconfigurationfile mvdefconfigarch/<arch>/myown_defconfig

47
Configuringthekernel
Afterloadingadefaultconfigurationfile,youcanadjustthe configurationtoyourneedswiththenormalxconfig,gconfig ormenuconfiginterfaces Youcanalsostarttheconfigurationfromscratchwithoutloading adefaultconfigurationfile Asthearchitectureisdifferentthanyourhostarchitecture
Someoptionswillbedifferentfromthenativeconfiguration (processorandarchitecturespecificoptions,specificdrivers,etc.) Manyoptionswillbeidentical(filesystems,networkprotocol, architectureindependentdrivers,etc.)
MakesureyouhavethesupportfortherightCPU,therightboard andtherightdevicedrivers.
48
Buildingandinstallingthekernel
Runmake Copythefinalkernelimagetothetargetstorage
canbeuImage,zImage,vmlinux,bzImagein arch/<arch>/boot
makeinstallisrarelyusedinembeddeddevelopment,asthe kernelimageisasinglefile,easytohandle Itishoweverpossibletocustomizethemakeinstallbehaviour inarch/<arch>/boot/install.sh makemodules_installisusedeveninembedded development,asitinstallsmanymodulesanddescriptionfiles

makeINSTALL_MOD_PATH=<dir>/modules_install TheINSTALL_MOD_PATHisneededtoinstallthemodulesinthe targetrootfilesysteminsteadofyourhostrootfilesystem.
49
Kernelcommandline
Inadditiontothecompiletimeconfiguration,thekernelbehaviour canbeadjustedwithnorecompilationusingthekernel commandline Thekernelcommandlineisastringthatdefinesvarious argumentstothekernel
Itisveryimportantforsystemconfiguration root=fortherootfilesystem(coveredlater) console=forthedestinationofkernelmessages andmanymore,documentedinDocumentation/kernel parameters.txtinthekernelsources
Thiskernelcommandlineiseither
Passedbythebootloader.InUBoot,thecontentsofthebootargs environmentvariableisautomaticallypassedtothekernel Builtintothekernel,usingtheCONFIG_CMDLINEoption.
50
PracticallabCrosscompiling
Setupacrosscompiling environment ConfigurethekernelMakefile accordingly Crosscompilethekernelforanarm targetplatform Onthisplatform,interactwiththe bootloaderandbootyourkernel.
51
EmbeddedLinuxkernelusage
Usingkernelmodules
52
Advantagesofmodules
Modulesmakeiteasytodevelopdriverswithoutrebooting: load,test,unload,rebuild,load... Usefultokeepthekernelimagesizetotheminimum(essential inGNU/LinuxdistributionsforPCs). Alsousefultoreduceboottime:youdon'tspendtimeinitializing devicesandkernelfeaturesthatyouonlyneedlater. Caution:onceloaded,havefullcontrolandpriviledgesinthe system.Noparticularprotection.That'swhyonlytherootuser canloadandunloadmodules.
53
Moduledependencies
Somekernelmodulescandependonothermodules, whichneedtobeloadedfirst. Example:theusbstoragemoduledependsonthescsi_mod, libusualandusbcoremodules. Dependenciesaredescribed in/lib/modules/<kernelversion>/modules.dep Thisfileisgeneratedwhenyourunmakemodules_install.
54
Kernellog
Whenanewmoduleisloaded, relatedinformationisavailableinthekernellog. Thekernelkeepsitsmessagesinacircularbuffer (sothatitdoesn'tconsumemorememorywithmanymessages) Kernellogmessagesareavailablethroughthedmesgcommand. (diagnosticmessage) Kernellogmessagesarealsodisplayedinthesystemconsole (consolemessagescanbefilteredbylevelusingtheloglevel kernel,orcompletelydisabledwiththequietparameter). Notethatyoucanwritetothekernellogfromuserspacetoo: echoDebuginfo>/dev/kmsg
55
Moduleutilities(1)
modinfo<module_name> modinfo<module_path>.ko Getsinformationaboutamodule:parameters,license, descriptionanddependencies. Veryusefulbeforedecidingtoloadamoduleornot. sudoinsmod<module_path>.ko Triestoloadthegivenmodule.Thefullpathtothemodule objectfilemustbegiven.
56
Understandingmoduleloadingissues
Whenloadingamodulefails, insmodoftendoesn'tgiveyouenoughdetails! Detailsareoftenavailableinthekernellog. Example: >sudoinsmod./intr_monitor.ko insmod:errorinserting'./intr_monitor.ko':1 Deviceorresourcebusy >dmesg [17549774.552000]Failedtoregisterhandlerfor irqchannel2
57
Moduleutilities(2)
sudomodprobe<module_name> Mostcommonusageofmodprobe:triestoloadallthe modulesthegivenmoduledependson,andthenthis module.Lotsofotheroptionsareavailable.Modprobe automaticallylooksin/lib/modules/<version>/forthe objectfilecorrespondingtothegivenmodulename. lsmod Displaysthelistofloadedmodules Compareitsoutputwiththecontentsof /proc/modules!
58
Moduleutilities(3)
sudormmod<module_name> Triestoremovethegivenmodule. Willonlybeallowedifthemoduleisnolongerinuse (forexample,nomoreprocessesopeningadevicefile) sudomodprober<module_name> Triestoremovethegivenmoduleandalldependent modules(whicharenolongerneededafter themoduleremoval)
59
Passingparameterstomodules
Findavailableparameters: modinfosndintel8x0m Throughinsmod: sudoinsmod./sndintel8x0m.koindex=2 Throughmodprobe: Setparametersin/etc/modprobe.conforinanyfilein /etc/modprobe.d/: optionssndintel8x0mindex=2 Throughthekernelcommandline, whenthedriverisbuiltstaticallyintothekernel: sndintel8x0m.index=2 drivername driverparametername driverparametervalue
60
Usefulreading
LinuxKernelinaNutshell,Dec2006 ByGregKroahHartman,O'Reilly http://www.kroah.com/lkn/ Agoodreferencebookandguideonconfiguring, compilingandmanagingtheLinuxkernelsources. Freelyavailableonline! Greatcompaniontotheprintedbook foreasyelectronicsearches! AvailableassinglePDFfileon http://freeelectrons.com/community/kernel/lkn/
61
Rootfilesystem
Linuxrootfilesystem Principleandsolutions
1
Filesystems
Filesystemsareusedtoorganizedataindirectoriesandfileson storagedevicesoronthenetwork.Thedirectoriesandfilesare organizedasahierarchy InUnixsystems,applicationsandusersseeasingleglobal hierarchyoffilesanddirectories,whichcanbecomposedof severalfilesystems. Filesystemsaremountedinaspecificlocationinthishierarchy ofdirectories
Whenafilesystemismountedinadirectory(calledmountpoint), thecontentsofthisdirectoryreflectsthecontentsofthestorage device Whenthefilesystemisunmounted,themountpointisemptyagain.
Thisallowsapplicationstoaccessfilesanddirectorieseasily, regardlessoftheirexactstoragelocation
2
Filesystems(2)
Createamountpoint,whichisjustadirectory $mkdir/mnt/usbkey Itisempty $ls/mnt/usbkey $ Mountastoragedeviceinthismountpoint $mounttvfat/dev/sda1/mnt/usbkey $ YoucanaccessthecontentsoftheUSBkey $ls/mnt/usbkey docsprog.cpicture.pngmovie.avi $
3
mount/umount
mountallowstomountfilesystems
mountttypedevicemountpoint typeisthetypeoffilesystem deviceisthestoragedevice,ornetworklocationtomount mountpointisthedirectorywherefilesofthestoragedeviceor networklocationwillbeaccessible mountwithnoargumentsshowsthecurrentlymountedfilesystems
umountallowstounmountfilesystems
Thisisneededbeforerebooting,orbeforeunplugginganUSBkey, becausetheLinuxkernelcacheswritesinmemorytoincrease performances.umountmakessurethatthosewritesarecommitted tothestorage.
4
Rootfilesystem
Aparticularfilesystemismountedattherootofthehierarchy, identifiedby/ Thisfilesystemiscalledtherootfilesystem Asmountandumountareprograms,theyarefilesinsidea filesystem.
Theyarenotaccessiblebeforemountingatleastonefilesystem.
Astherootfilesystemisthefirstmountedfilesystem,itcannotbe mountedwiththenormalmountcommand Itismounteddirectlybythekernel,accordingtotheroot= kerneloption Whennorootfilesystemisavailable,thekernelpanics

Pleaseappendacorrect"root="bootoption Kernelpanicnotsyncing:VFS:Unabletomountrootfsonunknown block(0,0) 5
Locationoftherootfilesystem
Itcanbemountedfromdifferentlocations
Fromthepartitionofanharddisk FromthepartitionofanUSBkey FromthepartitionofaSDcard FromthepartitionofaNANDFlashorsimilartypeofstoragedevice Fromthenetwork,usingtheNFSprotocol Frommemory,usingapreloadedfilesystem(bythebootloader) etc.
Itisuptothesystemdesignertochoosetheconfigurationforthe system,andconfigurethekernelbehaviourwithroot=
6
Mountingrootfsfromstoragedevices
PartitionsofaharddiskorUSBkey
root=/dev/sdXY,whereXisaletterindicatingthedevice,andY anumberindicatingthepartition /dev/sdb2isthesecondpartitionoftheseconddiskdrive(either USBkeyorATAharddrive)
PartitionsofaSDcard
root=/dev/mmcblkXpY,whereXisanumberindicatingthe deviceandYanumberindicatingthepartition /dev/mmcblk0p2isthesecondpartitionofthefirstdevice
PartitionsofaNANDFlash
root=/dev/mtdblockX,whereXisthepartitionnumber /dev/mtdblock3isthefourthpartitionoftheNANDflash(ifonly oneNANDflashispresent)
7
Mountingrootfsoverthenetwork(1)
Oncenetworkingworks,yourrootfilesystemcouldbeadirectory onyourGNU/Linuxdevelopmenthost,exportedbyNFS(Network FileSystem).Thisisveryconvenientforsystemdevelopment: Makesitveryeasytoupdatefilesontherootfilesystem, withoutrebooting.Muchfasterthanthroughtheserialport. Canhaveabigrootfilesystemevenifyoudon'thavesupport forinternalorexternalstorageyet. Therootfilesystemcanbehuge.Youcanevenbuildnative compilertoolsandbuildallthetoolsyouneedonthetarget itself(bettertocrosscompilethough).
Host Ethernet NFS server Target NFS client built into the kernel
8
Onthedevelopmentworkstationside,aNFSserverisneeded InstallanNFSserver(example:Debian,Ubuntu)
sudoaptgetinstallnfskernelserver
Addtheexporteddirectorytoyour/etc/exportsfile:
/home/tux/rootfs192.168.1.111(rw,no_root_squash,no_subtree_check) clientaddress NFSserveroptions
StartorrestartyourNFSserver(example:Debian,Ubuntu)
sudo/etc/init.d/nfskernelserverrestart
9
Onthetargetsystem Thekernelmustbecompiledwith CONFIG_NFS_FS=y (NFSsupport) CONFIG_IP_PNP=y (configureIPatboottime) CONFIG_ROOT_NFS=y (supportforNFSasrootfs) Thekernelmustbebootedwiththefollowingparameters root=/dev/nfs (wewantrootfsoverNFS) ip=192.168.1.111 (targetIPaddress) nfsroot=192.168.1.110:/home/tux/rootfs/ (NFSserverdetails)
10
Host NFS server /home/tux/rootfs/ /home/tux/rootfs/root/ /home/tux/rootfs/root/README /home/tux/rootfs/usr/ /home/tux/rootfs/bin/ /home/tux/rootfs/bin/ls
Target NFS client built into the kernel / /root /root/README /usr /bin /bin/ls
Ethernet
11
rootfsinmeory:initramfs(1)
Itisalsopossibletohavetherootfilesystemintegratedintothe kernelimage Itisthereforeloadedintomemorytogetherwiththekernel Thismechanismiscalledinitramfs
Itintegratesacompressedarchiveofthefilesystemintothekernel image
Itisusefulfortwocases
Fastbootingofverysmallrootfilesystems.Asthefilesystemis completelyloadedatboottime,applicationstartupisveryfast. Asanintermediatestepbeforeswitchingtoarealrootfilesystem, locatedondevicesforwhichdriversnotpartofthekernelimageare needed(storagedrivers,filesystemdrivers,networkdrivers).Thisis alwaysusedonthekernelofdesktop/serverdistributionstokeep thekernelimagesizereasonable.
12
rootfsinmemory:initramfs(2)
Kernel code and data
Root filesystem stored as a compressed cpio archive
Kernel image (uImage, zImage, bzImage)
13
rootfsinmemory:initramfs(3)
Thecontentsofaninitramfsaredefinedatthekernel configurationlevel,withtheCONFIG_INITRAMFS_SOURCE option
Canbethepathtoadirectorycontainingtherootfilesystem contents Canbethepathtoacpioarchive Canbeatextfiledescribingthecontentsoftheinitramfs(see documentationfordetails)
Thekernelbuildprocesswillautomaticallytakethecontentsof theCONFIG_INITRAMFS_SOURCEoptionandintegratetheroot filesystemintothekernelimage

Documentation/filesystems/ramfsrootfsinitramfs.txt Documentation/earlyuserspace/README
14
Rootfilesystem
Linuxrootfilesystem Contents
15
Rootfilesystemorganization
TheorganizationofaLinuxrootfilesystemintermsofdirectories iswelldefinedbytheFilesystemHierarchyStandard http://www.pathname.com/fhs/ MostLinuxsystemsconformtothisspecification

Applicationsexpectthisorganization Itmakesiteasierfordevelopersandusersasthefilesystem organizationissimilarinallsystems
16
Importantdirectories(1)
/bin /boot Basicprograms Kernelimage(onlywhenthekernelisloaded fromafilesystem,notcommononnonx86 architectures) Devicefiles(coveredlater) Systemwideconfiguration Directoryfortheusershomedirectories Basiclibraries Mountpointsforremovablemedia Mountpointsforstaticmedia Mountpointfortheprocvirtualfilesystem
17
/dev /etc /home /lib /media /mnt /proc
Importantdirectories(2)
/root /sbin /sys /tmp /usr
/usr/bin /usr/lib /usr/sbin Nonbasicprograms Nonbasiclibraries Nonbasicsystemprograms
Homedirectoryoftherootuser Basicsystemprograms Mountpointofthesysfsvirtualfilesystem Temporaryfiles
/var
Variabledatafiles.Thisincludesspooldirectories andfiles,administrativeandloggingdata,and transientandtemporaryfiles

18
Separationofprogramsandlibraries
Basicprogramsareinstalledin/binand/sbinandbasic librariesin/lib Allotherprogramsareinstalledin/usr/binand/usr/sbin andallotherlibrariesin/usr/lib Inthepast,onUnixsystems,/usrwasveryoftenmountedover thenetwork,throughNFS Inordertoallowthesystemtobootwhenthenetworkwasdown, somebinariesandlibrariesarestoredin/bin,/sbinand/lib /binand/sbincontainprogramslikels,ifconfig,cp,bash,etc. /libcontainstheClibraryandsometimesafewotherbasic libraries Allotherprogramsandlibrariesarein/usr
19
Devices
Oneofthekernelimportantroleistoallowapplicationstoaccess hardwaredevices IntheLinuxkernel,mostdevicesarepresentedtouserspace applicationsthroughtwodifferentabstractions
Characterdevice Blockdevice
Internally,thekernelidentifieseachdevicebyatripletof information
Type(characterorblock) Major(typicallythecategoryofdevice) Minor(typicallytheidentifierofthedevice)
20
Typesofdevices
Blockdevices
Adevicecomposedoffixedsizedblocks,thatcanbereadand writtentostoredata Usedforharddisks,USBkeys,SDcards,etc.
Characterdevices
Originally,aninfinitestreamofbytes,withnobeginning,noend,no size.Thepureexample:aserialport. Usedforserialports,terminals,butalsosoundcards,video acquisitionsdevices,framebuffers Mostofthedevicesthatarenotblockdevicesarerepresentedas characterdevicesbytheLinuxkernel
21
Devices:everythingisafile
AveryimportantUnixdesigndecisionwastorepresentmostof thesystemobjectsasfiles Itallowsapplicationstomanipulateallsystemobjectswiththe normalfileAPI(open,read,write,close,etc.) So,deviceshadtoberepresentedasfilestotheapplications Thisisdonethroughaspecialartefactcalledadevicefile Itaspecialtypeoffile,thatassociatesafilenamevisibleto userspaceapplicationstothetriplet(type,major,minor)thatthe kernelunderstands Alldevicefilesarebyconventionstoredinthe/devdirectory
22
Devicefilesexamples
Example of device files in a Linux system
$lsl/dev/ttyS0/dev/tty1/dev/sda1/dev/zero brwrw1rootdisk8,12011052708:56/dev/sda1 crw1rootroot4,12011052708:57/dev/tty1 crwrw1rootdialout4,642011052708:56/dev/ttyS0 crwrwrw1rootroot1,52011052708:56/dev/zero
Example C code that uses the usual file API to write data to a serial port
intfd; fd=open(/dev/ttyS0,O_RDWR); write(fd,Hello,5); close(fd);
23
Creatingdevicefiles
OnabasicLinuxsystem,thedevicefileshavetobecreated manuallyusingthemknodcommand
mknod/dev/<device>[c|b]majorminor Needsrootpriviledges Coherencybetweendevicefilesanddeviceshandledbythekernel islefttothesystemdeveloper
OnmoreelaborateLinuxsystems,mechanismscanbeaddedto create/removethemautomaticallywhendevicesappearand disappear

devtmpfsvirtualfilesystem,sincekernel2.6.32 udevdaemon,solutionusedbydesktopandserverLinuxsystems mdevprogram,alightersolutionthanudev
24
procvirtualfilesystem
TheprocvirtualfilesystemexistssincethebeginningofLinux Itallows
Thekerneltoexposestatisticsaboutrunningprocessesinthesystem Theusertoadjustatruntimevarioussystemparametersabout processmanagement,memorymanagement,etc.
Theprocfilesystemisusedbymanystandarduserspace applications,andtheyexpectittobemountedin/proc Applicationssuchaspsortopwouldnotworkwithouttheproc filesystem Documentation/filesystems/proc.txtinthekernelsources manproc

25
proccontents
Onedirectoryforeachrunningprocessinthesystem
/proc/<pid> cat/proc/3840/cmdline Itcontainsdetailsaboutthefilesopenedbytheprocess,theCPU andmemoryusage,etc.
/proc/interrupts,/proc/devices,/proc/iomem, /proc/ioportscontaingeneraldevicerelatedinformations /proc/cmdlinecontainsthekernelcommandline /proc/syscontainsmanyfilesthatcanbewrittentotoadjust kernelparameters

Theyarecalledsysctl.SeeDocumentation/sysctl/inkernel sources. Example echo3>/proc/sys/vm/drop_caches
26
sysfsfilesystem
Thesysfsfilesystemisafeatureintegratedinthe2.6Linuxkernel Itallowstorepresentinuserspacethevisionthatthekernelhas ofthebusses,devicesanddriversinthesystem Itisusefulforvarioususerspaceapplicationsthatneedtolistand querytheavailablehardware,forexampleudevormdev Allapplicationsusingsysfsexpectittobemountedinthe/sys directory $ls/sys/ block bus class dev fs kernel module power devicesfirmware
27
Basicapplications
Inordertowork,aLinuxsystemneedsatleastafewapplications Aninitapplication,whichisthefirstuserspaceapplication startedbythekernelaftermountingtherootfilesystem
Thekerneltriestorun/sbin/init,/bin/init,/etc/init and/bin/sh. Ifnoneofthemarefound,thekernelpanicsandthebootprocessis stopped. Theinitapplicationisresponsibleforstartingallotheruserspace applicationsandservices
Usuallyashell,toallowanusertointeractwiththesystem BasicUnixapplications,tocopyfiles,movefiles,listfiles (commandslikemv,cp,mkdir,cat,etc.) Thosebasiccomponentshavetobeintegratedintotheroot filesystemtomakeitusable
28
Overallbootingprocess
Bootloader loads the kernel to RAM and starts it
Kernel initialize hardware devices and kernel subsystems mount the root filesystem indicated by root= starts the init application, /sbin/init by default
/sbin/init starts other userspace services and applications
shell
other application Root filesystem

29
BusyBox
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/busybox Corrections,suggestions,contributionsandtranslationsarewelcome!
WhyBusybox?
ALinuxsystemneedsabasicsetofprogramstowork Aninitprogram Ashell Variousbasicutilitiesforfilemanipulationandsystemconfiguration InnormalLinuxsystems,thoseprogramsareprovidedbydifferent projects coreutils,bash,grep,sed,tar,wget,modutils, etc.arealldifferentprojects Alotofdifferentcomponentstointegrate Componentsnotdesignedwithembeddedsystemsconstraintsin mind:theyarenotveryconfigurableandhaveawiderangeof features Busyboxisanalternativesolution,extremelycommononembedded systems
Generalpurposetoolbox:BusyBox
http://www.busybox.net/ RewriteofmanyusefulUnixcommandlineutilities Integratedintoasingleproject,whichmakesiteasytoworkwith Designedwithembeddedsystemsinmind:highlyconfigurable,no unnecessaryfeatures Alltheutilitiesarecompiledintoasingleexecutable,/bin/busybox Symboliclinksto/bin/busyboxarecreatedforeachapplication integratedintoBusybox Forafairlyfeaturefulconfiguration,lessthan<500KB(statically compiledwithuClibc)orlessthan1MB(staticallycompiledwith glibc).
3
BusyBoxcommands!
[,[[,addgroup,adduser,adjtimex,ar,arp,arping,ash,awk,basename,bbconfig,bbsh, brctl,bunzip2,busybox,bzcat,bzip2,cal,cat,catv,chat,chattr,chcon,chgrp,chmod, chown,chpasswd,chpst,chroot,chrt,chvt,cksum,clear,cmp,comm,cp,cpio,crond,crontab, cryptpw,cttyhack,cut,date,dc,dd,deallocvt,delgroup,deluser,depmod,devfsd,df, dhcprelay,diff,dirname,dmesg,dnsd,dos2unix,dpkg,dpkg_deb,du,dumpkmap,dumpleases, e2fsck,echo,ed,egrep,eject,env,envdir,envuidgid,ether_wake,expand,expr,fakeidentd, false,fbset,fbsplash,fdflush,fdformat,fdisk,fetchmail,fgrep,find,findfs,fold,free, freeramdisk,fsck,fsck_minix,ftpget,ftpput,fuser,getenforce,getopt,getsebool,getty, grep,gunzip,gzip,halt,hd,hdparm,head,hexdump,hostid,hostname,httpd,hush,hwclock, id,ifconfig,ifdown,ifenslave,ifup,inetd,init,inotifyd,insmod,install,ip,ipaddr, ipcalc,ipcrm,ipcs,iplink,iproute,iprule,iptunnel,kbd_mode,kill,killall,killall5, klogd,lash,last,length,less,linux32,linux64,linuxrc,ln,load_policy,loadfont, loadkmap,logger,login,logname,logread,losetup,lpd,lpq,lpr,ls,lsattr,lsmod,lzmacat, makedevs,man,matchpathcon,md5sum,mdev,mesg,microcom,mkdir,mke2fs,mkfifo,mkfs_minix, mknod,mkswap,mktemp,modprobe,more,mount,mountpoint,msh,mt,mv,nameif,nc,netstat, nice,nmeter,nohup,nslookup,od,openvt,parse,passwd,patch,pgrep,pidof,ping,ping6, pipe_progress,pivot_root,pkill,poweroff,printenv,printf,ps,pscan,pwd,raidautorun, rdate,rdev,readahead,readlink,readprofile,realpath,reboot,renice,reset,resize, restorecon,rm,rmdir,rmmod,route,rpm,rpm2cpio,rtcwake,run_parts,runcon,runlevel, runsv,runsvdir,rx,script,sed,selinuxenabled,sendmail,seq,sestatus,setarch, setconsole,setenforce,setfiles,setfont,setkeycodes,setlogcons,setsebool,setsid, setuidgid,sh,sha1sum,showkey,slattach,sleep,softlimit,sort,split,start_stop_daemon, stat,strings,stty,su,sulogin,sum,sv,svlogd,swapoff,swapon,switch_root,sync,sysctl, syslogd,tac,tail,tar,taskset,tcpsvd,tee,telnet,telnetd,test,tftp,tftpd,time,top, touch,tr,traceroute,true,tty,ttysize,tune2fs,udhcpc,udhcpd,udpsvd,umount,uname, uncompress,unexpand,uniq,unix2dos,unlzma,unzip,uptime,usleep,uudecode,uuencode, vconfig,vi,vlock,watch,watchdog,wc,wget,which,who,whoami,xargs,yes,zcat,zcip
CommandsavailableinBusyBox1.13
Applethighlight:Busyboxinit
Busyboxprovidesanimplementationofaninitprogram Simplerthantheinitimplementationfoundondesktop/server systems:norunlevelsareimplemented Asingleconfigurationfile:/etc/inittab
Eachlinehastheform<id>::<action>:<process>
Allowstorunservicesatstartup,andtomakesurethatcertain servicesarealwaysrunningonthesystem Seeexamples/inittabinBusyboxfordetailsonthe configuration
5
ApplethighlightBusyBoxvi
IfyouareusingBusyBox,addingvisupportsonlyadds20K. (builtwithsharedlibraries,usinguClibc). Youcanselectwhichexactfeaturestocompilein. Usershardlyrealizethattheyareusingalightweightvi version! Tip:youcanlearnvionthedesktop, byrunningthevimtutorcommand.
6
ConfiguringBusyBox
Getthelateststablesourcesfromhttp://busybox.net ConfigureBusyBox(createsa.configfile): makedefconfig GoodtobeginwithBusyBox. ConfiguresBusyBoxwithalloptionsforregularusers. makeallnoconfig Unselectsalloptions.Goodtoconfigureonlywhatyouneed. makexconfig(graphical)ormakemenuconfig(text) SameconfigurationinterfacesastheonesusedbytheLinux kernel.
7
BusyBoxmakexconfig
Youcanchoose: thecommands tocompile, andeventhe command optionsand featuresthatyou need!
8
CompilingBusyBox
Setthecrosscompilerprefixintheconfigurationinterface: BusyBoxSettings>BuildOptions>CrossCompiler prefix Example:armlinux Settheinstallationdirectoryintheconfigurationinterface: BusyBoxSettings>InstallationOptions>BusyBox installationprefix AddthecrosscompilerpathtothePATHenvironmentvariable:
exportPATH=/usr/xtools/armunknownlinuxuclibcgnueabi/bin:$PATH
CompileBusyBox: make Installit(thiscreatesaUnixdirectorystructuresymboliclinkstothebusybox executable): makeinstall
9
PracticallabAtinyembeddedsystem
MakeLinuxbootonadirectoryon yourworkstation,sharedbyNFS Createandconfigurea minimalisticLinuxembedded system InstallanduseBusyBox Systemstartupwith/sbin/init Setupasimplewebinterface Usesharedlibraries
10
Blockfilesystems
Blockfilesystems
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/blockfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!
Blockvs.flash
Storagedevicesareclassifiedintwomaintypes:blockdevices andflashdevices
Theyarehandledbydifferentsubsystemsanddifferentfilesystems
Blockdevicescanbereadandwrittentoonaperblockbasis, withouterasing,anddonotwearoutwhenbeingusedforalong time

Harddisks,floppydisks,RAMdisks USBkeys,CompactFlash,SDcard,thesearebasedonflashes, buthaveanintegratedcontrollerthatemulatesablockdevice
Flashdevicescanberead,butwritingrequireserasing,and oftenoccursonalargersizethantheblocksize
NORflash,NANDflash
2
Blockdevicelist
Thelistofallblockdevicesavailableinthesystemcanbefound in/proc/partitions $cat/proc/partitions majorminor#blocksname 80312571224sda 81303903744sda1 821sda2 858665088sda5 Andalsoin/sys/block/
3
Traditionalblockfilesystems
Traditionalfilesystems Canbeleftinanoncoherentstateafterasystemcrash orsuddenpoweroff,whichrequiresafullfilesystem checkafterreboot. ext2:traditionalLinuxfilesystem (repairitwithfsck.ext2) vfat:traditionalWindowsfilesystem (repairitwithfsck.vfatonGNU/LinuxorScandisk onWindows)
4
Journaledfilesystems
Designedtostayina correctstateeven aftersystemcrashes orasuddenpoweroff Allwritesarefirst describedinthe journalbeforebeing committedtofiles
Application
Userspace Kernelspace (filesystem) Writetofile
Writeanentry inthejournal Write tofile Clear journalentry
5
Filesystemrecoveryaftercrashes
Reboot Reboot
No Discard Discard incomplete incomplete journalentries journalentries
Journal Journal empty? empty?
Thankstothejournal, thefilesystemisnever leftinacorrupted state Recentlysaveddata couldstillbelost
Yes Execute Execute journal journal FilesystemOK FilesystemOK
6
Journaledblockfilesystems
Journaledfilesystems ext3:ext2withjournalextension ext4:thenewgenerationwithmanyimprovements. Readyforproduction.Theyarethedefaultfilesystemsforall Linuxsystemsintheworld. TheLinuxkernelsupportsmanyotherfilesystems:reiserFS, JFS,XFS,etc.Eachofthemhavetheirowncharacteristics,but aremoreorientedtowardsserverorscientificworkloads btrfs(ButterFS) Thenextgeneration.Inmainlinebutstillexperimental.
7
Creatingext2/ext3volumes
Tocreateanemptyext2/ext3filesystemonablockdeviceor insideanalreadyexistingimagefile
mkfs.ext2/dev/hda3 mkfs.ext3/dev/sda2 mkfs.ext2disk.img
Tocreateafilesystemimagefromadirectorycontainingallyour filesanddirections
Usethegenext2fstool,fromthepackageofthesamename genext2fsdrootfs/rootfs.img Yourimageisthenreadytobetransferredtoyourblockdevice
8
Mountingfilesystemimages
Onceafilesystemimagehasbeencreated,onecanaccessand modifiesitscontentsfromthedevelopmentworkstation,usingthe loopmechanism Example: genext2fsdrootfs/rootfs.img mkdir/tmp/tst mounttext2olooprootfs.img/tmp/tst Inthe/tmp/tst,onecanaccessandmodifythecontentsofthe rootfs.imgfile. Thisispossiblethankstoloop,whichisakerneldriverthat emulatesablockdevicewiththecontentsofafile. Donotforgettoumountbeforeusingthefilesystemimage!
9
Squashfs
Squashfs:http://squashfs.sourceforge.net Readonly,compressedfilesystemforblockdevices> Fineforpartsofafilesystemwhichcanbereadonly(kernel,binaries...) Greatcompressionrateandreadaccessperformance UsedinmostliveCDsandliveUSBdistributions SupportsLZOcompressionforbetterperformance onembeddedsystemswithslowCPUs (attheexpenseofaslightlydegradedcompressionrate) AvailableinmainstreamLinuxsinceversion2.6.29. Patchesavailableforallearlierversions. Benchmarks:(roughly3timessmallerthanext3,and24timesfaster) http://elinux.org/Squash_Fs_Comparisons
10
SquashfsHowtouse
Needtoinstallthesquashfstoolspackage Creationoftheimage Onyourworkstation,createyourfilesystemimage: mksquashfsrootfs/rootfs.sqfs
Caution:iftheimagealreadyexistsremoveitfirst, orusethenoappendoption.
Installationoftheimage Let'sassumeyourpartitiononthetargetisin/dev/sdc1 Copythefilesystemimageonthedevice ddif=rootfs.sqfsof=/dev/sdc1

Becarefulwhenusingddtonotoverwritetheincorrectpartition!
Mountyourfilesystem: mounttsquashfs/dev/sdc1/mnt/root
11
tmpfs
UsefultostoretemporarydatainRAM: systemlogfiles,connectiondata,temporaryfiles... Don'tuseramdisks!Theyhavemanydrawbacks:fixedinsize, RemainingspacenotusableasRAM,filesduplicatedinRAM(inthe blockdeviceandfilecache)! tmpfsconfiguration:Filesystems>Pseudofilesystems LivesintheLinuxfilecache.Doesn'twasteRAM:growsandshrinksto accommodatestoredfiles.SavesRAM:noduplication;canswapout pagestodiskwhenneeded. Howtouse:chooseanametodistinguishthevarioustmpfsinstances youcouldhave.Examples: mountttmpfsvarrun/var/run mountttmpfsudev/dev SeeDocumentation/filesystems/tmpfs.txtinkernelsources.
12
Mixingreadonlyandreadwritefilesystems
Goodideatosplityourblockstorageinto
Areadwritepartitionwithajournaledfilesystem(likeext3) Usedtostoreuserorconfigurationdata. Guaranteesfilesystemintegrityafterpowerofforcrashes. Ramstoragefortemporaryfiles(tmpfs)
readwrite userand configuration data
ext3
readwrite volatiledata
RAM
13
tmpfs
BlockStorage
readonly Acompressedreadonlypartition(Squashfs) compressed Typicallyusedfortherootfilesystem(binaries,kernel...). root Compressionsavesspace.Readonlyaccessprotectsyour filesystem systemfrommistakesanddatacorruption.
Squashfs
PracticallabBlockfilesystems
Creatingpartitionsonyourblock storage Bootingyoursystemwithamixof filesystems:SquashFSfor applications,ext3forconfiguration anduserdata,andtmpfsfor temporarysystemfiles.
14
Flashfilesystems
Flash filesystems
MichaelOpdenacker ThomasPetazzoni GregoryClement FreeElectrons
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/flashfilesystems Corrections,suggestions,contributionsandtranslationsarewelcome!
TheMTDsubsystem
MTD:MemoryTechnologyDevices(flash,ROM,RAM)
Linuxfilesysteminterface MTDUsermodules
UBI yaffs2 jffs2 Chardevice Blockdevice
FlashTranslationLayers forblockdeviceemulation Caution:patentedalgorithms!
Readonlyblockdevice
FTL
NFTL
INFTL
MTDChipdrivers
NORflash NANDflash DiskOnChipflash
RAMchips Blockdevice ROMchips Virtualmemory Virtualdevicesappearing asMTDdevices
Memorydevices hardware
MTDdevices
MTDdevicesarevisiblein/proc/mtd ThemtdchardrivercreatesacharacterdeviceforeachMTD deviceofthesystem
Usuallynamed/dev/mtdX,major90.Evenminorsforreadwrite access,oddminorsforreadonlyaccess Provideioctl()toeraseandmanagetheflash Usedbythemtdutils
ThemtdblockdrivercreatesablockdeviceforeachMTDdevice ofthesystem
Usuallynamed/dev/mtdblockX,major31.Minoristhenumber oftheMTDdevice Allowsread/writeblocklevelaccess.Butbadblocksarenot handled,andnowearlevelingisdoneforwrites.
3
MTDpartitioning
MTDdevicesareusuallypartitioned
Itallowstousedifferentareasoftheflashfordifferentpurposes: readonlyfilesystem,readwritefilesystem,backupareas, bootloaderarea,kernelarea,etc.
Unlikeblockdevices,whichcontainstheirownpartitiontable,the partitioningofMTDdevicesisdescribedexternally
Hardcodedintothekernelcode Specifiedthroughthekernelcommandline
EachpartitionbecomesaseparateMTDdevice
Differentfromblockdevicelabeling(hda3,sda2) /dev/mtd1iseitherthesecondpartitionofthefirstflashdevice,or thefirstpartitionofthesecondflashdevice
4
DefinitionofMTDpartitions
MTDpartitionsaredefinedinthekernel,intheboarddefinitions: arch/arm/machomap2/boardigep0020.cexample:
staticstructmtd_partitionigep2_onenand_partitions[]={ { .name="XLoader", .offset=0, .size=2*(64*(2*2048)) }, { .name="UBoot", .offset=MTDPART_OFS_APPEND, .size=6*(64*(2*2048)), }, [...] { .name="FileSystem", .offset=MTDPART_OFS_APPEND, .size=MTDPART_SIZ_FULL, }, }; 5
ModifyingMTDpartitions(1)
MTDpartitionscanfortunatelybedefined throughthekernelcommandline. FirstneedtofindthenameoftheMTDdevice. Lookatthekernellogatboottime: MuxedOneNAND(DDP)512MB1.8V16bit(0x58) OneNANDversion=0x0031 Scanningdeviceforbadblocks OneNANDeraseblock1812isaninitialbadblock [] Creating5MTDpartitionson"omap2onenand": 0x0000000000000x000000080000:"XLoader" 0x0000000800000x000000200000:"UBoot" 0x0000002000000x000000280000:"Environment" 0x0000002800000x000000580000:"Kernel" 0x0000005800000x000020000000:"FileSystem"
6
Youcannowusethemtdpartskernelbootparameter Example:
mtdparts=omap2onenand:512k(XLoader),1536k(U Boot)ro,512k(Environment),3m(Kernel),(RootFS)
We'vejustdefined3partitionsintheomap2onenanddevice:
2ndstagebootloader(512KiB) UBoot(1536KiB) UBootEnvironment(512KiB) Kernel(3MiB) data
7
Partitionsizesmustbemultipleoftheeraseblocksize. Youcanusesizesinhexadecimaltoo.Rememberthebelow sizes: 0x20000=128k,0x100000=1m,0x1000000=16m roliststhepartitionasreadonly isusedtousealltheremainingspace.
8
mtdutils
AsetofutilitiestomanipulateMTDdevices
mtdinfotogetdetailedinformationaboutaMTDdevice flash_erasealltocompletelyeraseagivenMTDdevice flashcptowritetoNORflash nandwritetowritetoNANDflash UBIutilities Flashfilesystemimagecreationtools:mkfs.jffs2, mkfs.ubifs
Usuallyavailableasthemtdutilspackageinyourdistribution Seehttp://www.linuxmtd.infradead.org/
9
jffs2
http://www.linuxmtd.infradead.org/doc/jffs2.html Today'sstandardfilesystemforMTDflash Nicefeatures:ontheflycompression(savesstorage spaceandreducesI/O),powerdownreliable,wear levelingandECC. Drawbacks:doesn'tscalewell Mounttimedependingonfilesystemsize: thekernelhastoscanthewholefilesystemat mounttime,toreadwhichblockbelongstoeach file. NeedtousetheCONFIG_JFFS2_SUMMARYkernel optiontostoresuchinformationinflash.This dramaticallyreducesmounttime(from16sto0.8s fora128MBpartition). MTDdriver Standardfile API JFFS2 filesystem
Flashchip
10
jffs2Howtouse
OntheLinuxtarget NeedeitherthemtdutilsfromtheMTDproject,ortheir embeddedvariantsfromBusybox Eraseandformatapartitionwithjffs2: flash_eraseallj/dev/mtd2 Mountthepartition: mounttjffs2/dev/mtdblock2/mnt/flash Fillthecontentsbywriting (copyingfromNFSorfromexternalstorage) Otherpossibility:useajffs2image(seenextpagetoproduceit): flash_eraseall/dev/mtd2 nandwritep/dev/mtd2rootfs.jffs2
11
Howtocreateajffs2image(1)
mkfs.jffs2commandavailableinthemtdutilspackage. Caution:unlikesomemkfscommands,itdoesn'tcreatea filesystem,butafilesystemimage. First,findtheeraseblocksizefrommtdinfom4:
[...] Eraseblocksize:262144bytes,256.0KiB
Thencreatetheimageonyourworkstation: mkfs.jffs2padnocleanmarkers eraseblock=256drootfs/orootfs.jffs2 Thepadoptionpadsthejffs2imagecontents untiltheendofthefinaleraseblock. Itisfineifthejffs2imageissmallerthantheMTDpartition. Thejffs2filesystemwillusetheentirepartitionanyway. ThenocleanmarkersoptionisforNANDflashonly.

12
Mountingajffs2imageonyourhost
Usefultoeditjffs2imagesonyourdevelopmentsystem MountinganMTDdeviceasaloopdeviceisabitcomplextask. Here'sanexampleforjffs2,foryourreference: Firstfindtheeraseblocksizeusedtocreatethejffs2image. Let'sassumeitis256KiB(262144bytes). Createablockdevicefromtheimage losetup/dev/loop0root.jffs2 EmulateanMTDdevicefromablockdevice, usingtheblock2mtdkernelmodule modprobeblock2mtdblock2mtd=/dev/loop0,262144 Finally,mountthefilesystem(create/mnt/jffs2ifneeded) mounttjffs2/dev/mtdblock0/mnt/jffs2
13
Initializingjffs2partitionsfromUboot
Youmaynotwanttohavemtdutilsonyourtarget! CreateaJFFS2imageonyourworkstation IntheUBootprompt: Downloadthejffs2imagetoRAMwithtftp OrcopythisimagetoRAMfromexternalstorage (UbootunderstandsFATfilesystemsandsupportsUSBstorage) FlashitinsideanMTDpartition (exactinstructionsdependingonflashtype,NORorNAND, reusetheinstructionsusedtoflashyourkernel).Makesuretowriteonly thesizeoftheimage,notmore! Ifyoubootonajffs2rootfilesystem,addroot=/dev/mtdblock<x>and rootfstype=jffs2totheLinuxcommandlinearguments. Limitation:needtosplitthejffs2imageinseveralchunks ifbiggerthantheRAMsize.
14
yaffs2
http://www.yaffs.net/ MainlysupportsNANDflash Nocompression Wearleveling,ECC,powerfailureresistant Fastboottime Codeavailableseparatelythroughgit (DualGPL/Proprietarylicense fornonLinuxoperatingsystems)
MTDdriver Standardfile API YAFFS2 filesystem
Flashchip
15
yaffs2Howtouse
Eraseapartition: flash_eraseall/dev/mtd2 Thefilesystemisautomaticallyformattedatthefirstmount: mounttyaffs2/dev/mtdblock2/mnt/flash Imagescanbecreatedwiththemkyaffstool,fromyaffsutils http://www.aleph1.co.uk/cgibin/viewvc.cgi/yaffs/utils/
16
UBI(1)
UnsortedBlockImages http://www.linuxmtd.infradead.org/doc/ubi.html VolumemanagementsystemontopofMTDdevices. Allowstocreatemultiplelogicalvolumes andspreadwritesacrossallphysicalblocks. Takescareofmanagingtheeraseblocksandwear leveling.Makesfilesystemeasiertoimplement. Volumescanbedynamicallyresizedor,onthe opposite,canbereadonly(static).
17
UBI(2)
UBI Logical EraseBlocks MTD Physical EraseBlocks
Volume1
Volume2
LEB
LEB
LEB
LEB
LEB
LEB
LEB
PEB PEB PEB PEB PEB PEB PEB PEB PEB

Freeblock Freeblock
18
UBIFS
http://www.linuxmtd.infradead.org/doc/ubifs.html Thenextgenerationofthejffs2filesystem, fromthesamelinuxmtddevelopers. AvailableinLinux2.6.27 WorksontopofUBIvolumes Hasanoticeablemetadataoverheadonvery smallpartitions(4M,8M)
Standardfile API UBIFS UBI MTDdriver
Flashchip
19
UBIlayout
UBIFSmountedon/ (fromubi0:rootfs)
UBIFS mountedon /myapp
JFFS2 mounted on
/appsettings
UBIFSmountedon /var/log (fromubi1:log)
UBI Volume: raw
(from
ubi0:data)
(from
/dev/mtd1)
UBIVolume:rootfs
UBIVolume: data
UBIVolume:log
UBIdeviceubi0
MTDpartition0
MTD partition1
UBIdeviceubi1
MTDpartition2
Flashdevice
20
UBIPreparation
Eraseyourflashpartitionwhilepreservingyourerasecounters ubiformat/dev/mtd1 Seehttp://www.linuxmtd.infradead.org/faq/ubi.htmlifyoufaceproblems Needtocreatea/dev/ubi_ctrlchardevice(ifyoudon'thaveudev) ThisspecialcharacterdeviceisusedbyotherUBIutilities Majorandminornumberallocatedinthekernel.Findthesenumbers in/sys/class/misc/ubi_ctrl/dev(e.g.:10:63) Orrunubinfo: UBIversion:1 CountofUBIdevices:1 UBIcontroldevicemajor/minor:10:63 PresentUBIdevices:ubi0
Thesestepsaredoneonceforall
21
UBIAttaching
AttachUBItoone(ofseveral)oftheMTDpartitions: ubiattach/dev/ubi_ctrlm1 Thiscommandcreatestheubi0device,whichrepresentthefull UBIspacestoredonMTDdevice1
FindthemajorandminornumbersusedbyUBI: cat/sys/class/ubi/ubi0/dev(e.g.250:0) CreatetheUBIdevicefile: mknod/dev/ubi0c2500 ThisUBIspacecancontainseveralvolumes
22
UBIVolumemanagement
Volumecreationwithubimkvol
ubimkvol/dev/ubi0Ntests116MiB ubimkvol/dev/ubi0Ntestm(maxavailablesize) Thevolumeisthenidentifiedasubi0:testforthemount/umount commands
Volumeremovalwithubirmvol
ubirmvol/dev/ubi0Ntest
23
UBIFSHowtouse
WhenaUBIvolumeiscreated,creatinganemptyUBIFS filesystemisjustamatterofmountingit
mounttubifsubi0:test/mnt/flash
ImagesofUBIFSfilesystemscanbecreatedusingthe mkfs.ubifsutility
mkfs.ubifsm4096e252KiBc1000r/opt/img ubifs.img CanbewrittentoaUBIvolumeusingubiupdatevolandthe /dev/ubiX_Ydevices
24
Ubinize
ImagesofafullUBIspace,containingseveralvolumescanbe createdusingtheubinizeutility
CanbewrittentoarawMTDusingonenandwrite ButonenanderasewillalsoerasetheEraseCounters
ubinizeoubi.imgp256KiBm4096 rootubifs.ini
Createsubi.img,with256KiBphysicaleraseblocks,4096minimum I/Osize(m).
25
UBIFSHowtopreparearootfs
CreatetheUBIFSimagefromthetargetdirectory WritetheconfigurationfilefortheUBIdevice
[RFSvolume] mode=ubi image=rootfs.ubifs vol_id=1 vol_size=30MiB vol_type=dynamic vol_name=rootfs vol_flags=autoresize vol_alignment=1
CreatetheUBIdeviceimage Flashitusingabadblockawarecommandfromthebootloader PassUBIlayoutinformationtothekernel:

rootfstype=ubifsubi.mtd=1root=ubi0:rootfs
26
Ourbenchmarks
jffs2 Dramaticallyoutperformedbyubifsin mostaspects. Hugemount/boottimeunless CONFIG_SUMMARYisused. yaffs2 Alsooutperformedbyubifs. Maynotfitallyourdata Uglyfileremovaltime (poordirectoryupdate performance?) Memoryusagenotscaling ubifsleavesnoreason tosticktoyaffs2.
27
ubifs Greatperformanceinallcorner cases.
Fullbenchmarkdetailson http://freeelectrons.com/pub/conferences/2008/elce/flashfilesystems.pdf
Issueswithflashbasedblockstorage
Flashstoragemadeavailableonlythroughablockinterface. Hence,nowaytoaccessalowlevelflashinterface andusetheLinuxfilesystemsdoingwearleveling. Nodetailsaboutthelayer(FlashTranslationLayer)theyuse. Detailsarekeptastradesecrets,andmayhidepoor implementations. Hence,itishighlyrecommendedtolimitthenumberofwritesto thesedevices.
28
Reducingthenumberofwrites
Ofcourse,donotuseyourflashstorageasswaparea (rareinembeddedsystemsanyway) Mountyourfilesystemsasreadonly,orusereadonlyfilesystems (SquashFS),wheneverpossible. KeepvolatilefilesinRAM(tmpfs) Usethenoatimemountoption,toavoidupdatingthefilesystem everytimeyouaccessafile.Oratleast,ifyouneedtoknowwhether fileswerereadaftertheirlastchange,usetherelatimeoption (defaultsettingsinceLinux2.6.30). Don'tusethesyncmountoption(commitswritesimmediately).Use thefsync()systemcallforperfilesynchronization. Youmaydecidetodowithoutjournaledfilesystems.Theycausemore writes,butarealsomuchmorepowerdownresistant(tradeoff).
29
Usefulreading
ArndBergmann:OptimizingLinuxwithcheapflashdrives Indepthcoverageofflashstoragewithablockinterface. http://lwn.net/Articles/428584/ IntroductiontoJFFS2andLogFS: http://lwn.net/Articles/234441/ NiceUBIpresentationfromToshiba: http://freeelectrons.com/redirect/celfubi.html Documentationonthelinuxmtdwebsite: http://www.linuxmtd.infradead.org/
30
Filesystemchoicesummary
Volatile Volatile data? data?
No
Block Storagetype Storagetype
Readonly Readonly data? data?
No
Contains Contains flash? flash?
MTD Yes Yes chooseUBIFS chooseUBIFS orJFFS2 orJFFS2
Yes
No
noatimeoption noatimeoption
chooseext2 chooseext2
Choosetmpfs Choosetmpfs chooseSquashFS chooseSquashFS
Chooseext3orext4 Chooseext3orext4
SeeDocumentation/filesystems/inkernelsourcesfordetails aboutallavailablefilesystems.
31
PracticallabFlashfilesysetms
Creatingpartitionsinyour internalflashstorage. UseareadonlyJFFS2partition forthesystem UseareadwriteJFFS2partition fordata
32
EmbeddedLinux system development

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/sysdev Corrections,suggestions,contributionsandtranslationsarewelcome!
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http://freeelectrons.com
Contents
Usingopensourcecomponents Toolsforthetargetdevice Networking Systemutilities Languageinterpreters Audio,videoandmultimedia Graphicaltoolkits Databases Webbrowsers Systembuilding Emulators Commercialtoolsetsanddistributions
2
Leveragingopensourcecomponentsinan EmbeddedLinuxsystem
3
Thirdpartylibrariesandapplications
OneoftheadvantagesofembeddedLinuxisthewiderangeof thirdpartylibrariesandapplicationsthatonecanleverageinits product
Theyarefreelyavailable,freelydistributable,andthankstotheir opensourcenature,theycanbeanalyzedandmodifiedaccording totheneedsoftheproject
However,efficientlyreusingthesecomponentsisnotalways easy.Onemust:
Findthesecomponents Choosethemostappropriateones Crosscompilethem Integratethemintheembeddedsystemandwiththeother applications
4
Findexistingcomponents
Freshmeat,awebsitereferencingmostopensourceprojects http://www.freshmeat.net FreeSoftwareDirectory http://directory.fsf.org LookatotherembeddedLinuxproducts,andseewhattheir componentsare Lookatthelistofsoftwarepackaged byembeddedLinuxbuildsystems
Thesearetypicallychosenfortheirsuitabilitytoembeddedsystems
AskthecommunityorGoogle Thispresentationwillalsofeaturealistofcomponentsfor commonneeds

5
Choosingcomponents
Notallfreesoftwarecomponentsarenecessarilygoodtoreuse. Onemustpayattentionto:
Vitalityofthedeveloperandusercommunities.Thisvitalityensures longtermmaintenanceofthecomponent,andrelativelygoodsupport. Itcanbemeasuredbylookingatthemailinglisttrafficandtheversion controlsystemactivity. Qualityofthecomponent.Typically,ifacomponentisalready availablethroughembeddedbuildsystems,andhasadynamicuser community,itprobablymeansthatthequalityisrelativelygood. License.Thelicenseofthecomponentmustmatchyourlicensing constraints.Forexample,GPLlibrariescannotbeusedinproprietary applications. Technicalrequirements.Ofcourse,thecomponentmustmatchyour technicalrequirements.Butdon'tforgetthatyoucanimprovethe existingcomponentsifafeatureismissing!
6
Licenses(1)
Allsoftwarethatareunderafreesoftwarelicensegivefour freedomstoallusers
Freedomtouse Freedomtostudy Freedomtocopy Freedomtomodifyanddistributemodifiedcopies
Seehttp://www.gnu.org/philosophy/freesw.htmlforadefinitionof FreeSoftware OpenSourcesoftware,asperthedefinitionoftheOpenSource Initiative,aretechnicallysimilartoFreeSoftwareintermsof freedoms Seehttp://www.opensource.org/docs/osdforthedefinitionof OpenSourceSoftware

7
Licenses(2)
FreeSoftwarelicensesfallintwomaincategories
Thecopyleftlicenses Thenoncopyleftlicenses
Theconceptofcopyleftistoaskforreciprocityinthe freedomsgiventoanuser. Theresultisthatwhenyoureceiveasoftwareunderacopyleft freesoftwarelicenseanddistributemodifiedversionsofit,you mustdosounderthesamelicense

Samefreedomstothenewusers It'sanincentivetocontributebackyourchangesinsteadofkeeping themsecret
Noncopyleftlicenseshavenosuchrequirements,andmodified versionscanbekeptproprietary,buttheystillrequireattribution
8
GPL
GNUGeneralPublicLicense Covers~55%ofthefreesoftwareprojects
IncludingtheLinuxkernel,Busyboxandmanyapplications
Isacopyleftlicense
Requiresderivativeworkstobereleasedunderthesamelicense ProgramslinkedwithalibraryreleasedundertheGPLmustalsobe releasedundertheGPL
Someprogramscoveredbyversion2(Linuxkernel,Busyboxand others) Moreandmoreprogramscoveredbyversion3,releasedin2007

Majorchangefortheembeddedmarket:therequirementthatthe usermustbeabletorunthemodifiedversionsonthedevice,ifthe deviceisaconsumerdevice
GPL:redistribution
Noobligationwhenthesoftwareisnotdistributed
Youcankeepyourmodificationssecretuntiltheproductdelivery
Itisthenauthorizedtodistributebinaryversions,ifoneofthe followingconditionsismet:
Conveythebinarywithacopyofthesourceonaphysicalmedium Conveythebinarywithawrittenoffervalidfor3yearsthatindicates howtofetchthesourcecode Conveythebinarywiththenetworkaddressofalocationwherethe sourcecodecanbefound Seesection6.oftheGPLlicense
Inallcases,theattributionandthelicensemustbepreserved
Seesection4.and5.
10
LGPL
GNULesserGeneralPublicLicense Covers~10%ofthefreesoftwareprojects Acopyleftlicense
Modifiedversionsmustbereleasedunderthesamelicense But,programslinkedagainstalibraryundertheLGPLdonotneed tobereleasedundertheLGPLandcanbekeptproprietary However,theusermustkeeptheabilitytoupdatethelibrary independentlyfromtheprogram,sodynamiclinkingmustbeused
UsedinsteadoftheGPLformostofthelibraries,includingtheC libraries
Someexceptions:MySQL,orQt<=4.4
Alsoavailableintwoversions,v2andv3
11
Licensing:examples
YoumakemodificationstotheLinuxkernel(toadddriversor adapttoyourboard),toBusybox,UBootorotherGPLsoftware
Youmustreleasethemodifiedversionsunderthesamelicense, andbereadytodistributethesourcecodetoyourcustomers
YoumakemodificationstotheClibraryoranyotherLGPLlibrary
Youmustreleasethemodifiedversionsunderthesamelicense
YoucreateanapplicationthatreliesonLGPLlibraries
Youcankeepyourapplicationproprietary,butyoumustlink dynamicallywiththeLGPLlibraries
Youmakemodificationstoanoncopyleftlicensedsoftware
Youcankeepyourmodificationsproprietary,butyoumuststill credittheauthors
12
Noncopyleftlicenses
Alargefamilyofnoncopyleftlicensesthatarerelativelysimilarin theirrequirements Afewexamples
Apachelicense(~4%) BSDlicense(~6%) MITlicense(~4%) X11license Artisticlicense(~9%)
13
BSDlicense
Copyright(c)<year>,<copyrightholder> Allrightsreserved. Redistributionanduseinsourceandbinaryforms,withor withoutmodification,arepermittedprovidedthatthefollowing conditionsaremet: *Redistributionsofsourcecodemustretaintheabove copyrightnotice,thislistofconditionsandthefollowing disclaimer. *Redistributionsinbinaryformmustreproducetheabove copyrightnotice,thislistofconditionsandthefollowing disclaimerinthedocumentationand/orothermaterialsprovided withthedistribution. *Neitherthenameofthe<organization>northenamesofits contributorsmaybeusedtoendorseorpromoteproductsderived fromthissoftwarewithoutspecificpriorwrittenpermission. [...]
14
Isthisfreesoftware?
Mostofthefreesoftwareprojectsarecoveredby~10wellknown licenses,soitisfairlyeasyforthemajorityofprojecttogeta goodunderstandingofthelicense Otherwise,readthelicensetext CheckFreeSoftwareFoundation'sopinion http://www.fsf.org/licensing/licenses/ CheckOpenSourceInitiative'sopinion http://www.opensource.org/licenses
15
Respectfreesoftwarelicenses
FreeSoftwareisnotpublicdomainsoftware,thedistributorshave obligationsduetothelicenses
Beforeusingafreesoftwarecomponent,makesurethelicense matchesyourprojectconstraints Makesuretokeepacompletelistofthefreesoftwarepackagesyou use,theoriginalversionyouusedandtokeepyourmodificationsand adaptationswellseparatedfromtheoriginalversion Conformtothelicenserequirementsbeforeshippingtheproductto thecustomers
FreeSoftwarelicenseshavebeenenforcedsuccessfullyincourts
GPLviolations.org,http://www.gplviolations.org SoftwareFreedomLawCenter,http://www.softwarefreedom.org/ Askyourlegaldepartment!
16
Keepingchangesseparate(1)
Whenintegratingexistingopensourcecomponentsinyour project,itissometimesneededtomakemodificationstothem
Betterintegration,reducedfootprint,bugfixes,newfeatures,etc.
Insteadofmixingthesechanges,itismuchbettertokeepthem separatefromtheoriginalcomponentversion
Ifthecomponentneedstobeupgraded,easiertoknowwhat modificationsweremadetothecomponent Ifsupportfromthecommunityisrequested,importanttoknowhow differentthecomponentwe'reusingisfromtheupstreamversion Makescontributingthechangesbacktothecommunitypossible
Itisevenbettertokeepthevariouschangesmadeonagiven componentseparate
Easiertoreviewandtoupdatetonewerversions
17
Keepingchangesseparate(2)
ThesimplestsolutionistouseQuilt
Quiltisatoolthatallowstomaintainastackofpatchesoversource code Makesiteasytoadd,removemodificationsfromapatch,toadd andremovepatchesfromstackandtoupdatethem Thestackofpatchescanbeintegratedintoyourversioncontrol system https://savannah.nongnu.org/projects/quilt/
Anothersolutionistouseaversioncontrolsystem
Importtheoriginalcomponentversionintoyourversioncontrol system Maintainyourchangesinaseparatebranch
18
Toolsforthetargetdevice
Networking
19
sshserverandclient:dropbear
http://matt.ucc.asn.au/dropbear/dropbear.html Verysmallmemoryfootprintsshserverforembeddedsystems Satisfiesmostneeds.Bothclientandserver! Size:110KB,staticallycompiledwithuClibconi386. (OpenSSHclientandserver:approx1200KB, dynamicallycompiledwithglibconi386) Usefulto: Getaremoteconsoleonthetargetdevice Copyfilestoandfromthetargetdevice(scporrsynce ssh). AnalternativetoOpenSSH,usedondesktopandserversystems.
20
Benefitsofawebserverinterface
Manynetworkenableddevicescanjusthaveanetworkinterface Examples:modems/routers,IPcameras,printers... Noneedtodevelopdriversandapplicationsforcomputers connectedtothedevice.Noneedtosupportmultipleoperating systems! JustneedtodevelopstaticordynamicHTMLpages (possiblywithpowerfulclientsideJavaScript). Easywayofprovidingaccesstodeviceinformationand parameters. Reducedhardwarecosts(noLCD,verylittlestoragespace needed)
21
Webservers
BusyBoxhttpserver:http://busybox.net Tiny:onlyadds9KtoBusyBox(dynamicallylinkedwithglibcon i386,withallfeaturesenabled.) Sufficientfeaturesformanydeviceswithawebinterface,including CGI,httpauthenticationandscriptsupport(likePHP,withasepate interpreter). License:GPL lighthttpd:http://lighttpd.net
Lowfootprintservergoodatmanaginghighloads. Maybeusefulinembeddedsystemstoo
Otherpossibilities:Boa,thttpd,etc.
22
Networkutilities(1)
avahiisanimplementationofMulticastDNSServiceDiscovery, thatallowsprogramstopublishanddiscoverservicesonalocal network bind,aDNSserver iptables,theuserspacetoolsassociatedtotheLinuxfirewall, Netfilter iwandwirelesstools,theuserspacetoolsassociatedto Wirelessdevices netsnmp,implementationoftheSNMPprotocol openntpd,implementationoftheNetworkTimeProtocol,for clocksynchronization openssl,atoolkitforSSLandTLSconnections
23
Networkutilities(2)
pppd,implementationofthePointtoPointProtocol,usedfor dialupconnections samba,implementstheSMBandCIFSprotocols,usedby Windowstosharefilesandprinters coherence,aUPnP/DLNAimplementation vsftpd,proftpd,FTPservers
24
Systemutilities
25
Systemutilities
dbus,aninterapplicationobjectorientedcommunicationbus gpsd,adaemontointerpretandshareGPSdata hal,theHardwareAbstractionLayersuite.Adaemonthat receiveshardwarenotifications,maintainsadatabaseof availablehardwaredevicesandoffersaDBusinterface libraw1394,rawaccesstoFirewiredevices libusb,auserspacelibraryforaccessingUSBdeviceswithout writinganinkerneldriver Utilitiesforkernelsubsystems:i2ctoolsforI2C,inputtoolsfor input,mtdutilsforMTDdevices,usbutilsforUSBdevices
26
Languageinterpreters
27
Languageinterpreters
Interpretersforthemostcommonscriptinglanguagesare available.Usefulfor
Applicationdevelopment Webservicesdevelopment Scripting
Languagessupported
Lua Python Perl Ruby TCL PHP
28
Audio,videoandmultimedia
29
Audio,videoandmultimedia
gstreamer,amultimediaframework
Allowstodecode/encodeawidevarietyofformats Supportshardwareencodersanddecodersthroughplugins
alsalib,theuserspacetoolsassociatedtotheALSAsound kernelsubsystem Encodinganddecodinglibrariessuchasflac,libogg,libtheora, libvorbis,libmad,libsndfile,speex,etc.
30
Graphicaltoolkits
31
Graphicaltoolkits Lowlevelsolutionsandlayers
32
DirectFB(1)
Lowlevelgraphicallibrary
Lines,rectangles,trianglesdrawingandfilling Blitting,flipping Textdrawing Windowsandtransparency Imageloadingandvideodisplay
Butalsohandlesinputeventhandling:mouse,keyboard, joystick,touchscreen,etc. Providesacceleratedgraphicoperationsonvarious hardware,morecanbeaddedinaneasyway Integratedwindowinginfrastructure

33
DirectFB(2)
Singleapplicationbydefault,butmultipleapplicationscan sharetheframebufferthankstofusion Developmentandcommunity:veryactive License:LGPL2.1 http://www.directfb.org
34
DirectFB:sizeanddependencies
DirectFB library ~450k plugins ~200k Total:~650k Total:~1.4m
Utils freetype ~380k libpng ~130k libjpeg ~130k libts ~6k zlib ~64k
Total:~710k Someofthese dependenciesareoptional. Thisisatypicalsetup.

35
DirectFB:architecture
Application
DirectFB fbdev devmem
FB driver
Kernel
/dev/ mem
Hardware
36
DirectFB:usage(1)
Multimediaapplications
ForexampletheDiskoframework,forsettopboxrelated applications
Simplegraphicalapplications
Industrialcontrol Devicecontrolwithlimitednumberofwidgets
Visualizationapplications Asalowerlayerforhigherlevelgraphicallibraries
37
DirectFB:usage(2)
38
X.orgKDrive
StandalonesimplifiedversionoftheXserver,forembedded systems
FormerlyknowasTinyX KdriveisintegratedintheofficialX.orgserver
WorksontopoftheLinuxframebuffer,thankstotheXfbdev variantoftheserver RealXserver

FullysupportstheX11protocol:drawing,inputeventhandling,etc. AllowstouseanyexistingX11applicationorlibrary
Activelydevelopedandmaintained X11license http://www.x.org

39
Kdrive:sizeanddependencies
Xserver Xfbdev ~1.2m Fonts
fromafewkb toseveralmb
Utils dbus lib:~200k bin:~350k libsysfs ~27k libpng ~130k zlib ~64k pixman ~130k
Xlibraries libxcb ~300k liblbxutil ~156k libX11 ~920k libXfont ~380k Misclibs ~770k Total:2.5m
expat ~120k freetype ~380k fontconfig ~165k Total:1.5m Total,withoutX toolkit:5.4m

40
Xtoolkit(optional) libXaw6,7,8 libXt ~900k ~330k
Kdrive:architecture
Application Application Toolkit
X.orgKdriveserver fbdev
FB driver
Kernel
Hardware
41
Kdrive:usage
CanbedirectlyprogrammedusingXlib/XCB
Lowlevelgraphiclibrary Probablydoesn'tmakesensesinceDirectFBisamorelightweight solutionforanAPIofroughlythesamelevel(nowidgets)
Or,usuallyusedwithatoolkitontopofit
Gtk Qt Fltk WxEmbedded
42
Graphicaltoolkits Highlevelsolutions
43
Gtk
Thefamoustoolkit,providingwidgetbased highlevelAPIstodevelopgraphical applications StandardAPIinC,butbindingsexistfor variouslanguages:C++,Python,etc. TwoGDKbackends
TheclassicalXorgbackend TheDirectFBbackend,whichremovesthe needforanXorgserver
GtkDFB DirectFB Kernel Hardware
Gtk X.orgKdrive Kernel Hardware

44
Nowindowingsystem,alightweightwindow managerneededtorunseveral applications.Possiblesolution:Matchbox. License:LGPL http://www.gtk.org
GtkDFB:dependenciesandsize
Gtkgalaxy libgtk2 ~3.2m atk ~100k pango ~400k cairo ~350k libglib2 ~1.4m
Imagelibraries
libpng ~130k jpeg ~130k tiff ~320k
directfb ~750k Total:~9MB
Utils gettext ~510k libintl ~30k fontconfig ~165k zlib ~64k freetype ~365k expat ~120k pixman ~130k
45
Gtkstackcomponents
Glib,coreinfrastructure
ObjectorientedinfrastructureGObject Eventloop,threads,asynchronousqueues,plugins,memory allocation,I/Ochannels,stringutilities,timers,dateandtime, internationalization,simpleXMLparser,regularexpressions Datatypes:memoryslicesandchunks,linkedlists,arrays,trees, hashtables,etc.
Pango,internationalizationoftexthandling ATK,accessibilitytoolkit Cairo,vectorgraphicslibrary Gtk+,thewidgetlibraryitself TheGtkstackisacompleteframeworktodevelopapplications

46
GNOMEMobile
TheGtkstackispartofthelargerGNOMEMobileplatform
47
Gtkexamples
OM2007.2platformon OpenMokophone
Maemoplatformon NokiaInternettablets
InterfaceofVernierdata acquisitionand visualizationsystems

48
Qt(1)
Theotherfamoustoolkit,providingwidgetbasedhighlevel APIstodevelopgraphicalapplications
QtforEmbeddedLinux,formerlyknownasQtopiaCore,is theversionofQtthatrunsontopofaframebuffer,onembedded devices.Itincludesawindowingsystem QtExtended,formerlyknownasQtopia,extendsQtfor EmbeddedLinuxwithusefulcomponentsonembedded devices:communication,contents,applicationspecificanduser experiencecomponents.
ImplementedinC++
theC++libraryisrequiredonthetargetsystem standardAPIinC++,butbindingsarealsoavailableforother languages http://www.qtsoftware.com/products/platform/qtforembeddedlinux
49
Qt(2)
Workseitherontopof
Framebuffer X11 DirectFBbackendintegratedinversion4.4,whichallowstotake advantageoftheaccelerationprovidedbyDirectFBdrivers
Qtismorethanjustagraphicaltoolkit,italsooffersacomplete developmentframework:datastructures,threads,network, databases,XML,etc. QtEmbeddedhasanintegratedwindowingsystem,allowing severalapplicationstosharethesamescreen Verywelldocumented Sinceversion4.5,availableundertheLGPL,allowingproprietary applications

50
Qt:sizeanddependencies
Qtgalaxy QtCore ~2.6m QtGui ~8.6m QtNetwork ~740k QtScript ~1.1m QtTest ~64k QtXml ~290k Imgplugins ~750k Libraries jpeg ~130k libpng ~130k zlib ~64k freetype ~365k libstdc++ ~675k
Fonts ~160k
Total:~15.5MB
51
Qt'susage
QtontheDash Expressnavigation system
QtontheNetflix PlayerbyRoku
QtontheOpenMoko phone
52
Otherlessusedsolutions
Enlightenmentfoundationlibraries
Verypowerful,butcomplicatedtouseduetothelackof documentation http://www.enlightenment.org/p.php?p=about/efl
FLTK
LightweightC++toolkit.Version2.xworksonlyontopofX.org. Usedbyrelativelyfewapplications. http://www.fltk.org
WxEmbedded
TheembeddedversionofWxWindows.WorksontopofX.organd DirectFB http://www.wxwidgets.org/docs/embedded.htm
53
Databases
54
LightweightdatabaseSQLite
http://www.sqlite.org SQLiteisasmallClibrarythatimplementsaselfcontained, embeddable,lightweight,zeroconfigurationSQLdatabaseengine ThedatabaseengineofchoiceforembeddedLinuxsystems Canbeusedasanormallibrary Canbedirectlyembeddedintoaapplication,evenaproprietaryone sinceSQLiteisreleasedinthepublicdomain
55
Webbrowsers
56
Fastandtinywebbrowser:Dillo
http://www.dillo.org/ Veryfast,lightweightwebbrowserwritten inC/C++,withaFLTK2interface TheDillobinaryfitsin940KBonx86 (doesn'tincludetheFLTK2sharedlibraries) License:GPL Supportsmanystandardfeatures:cookies,images,tables,CSS... Extensiblethroughplugins(e.g.ftp) Fitswellonsmallscreens Stillmissing:frames,javascript AgoodsolutionifyourproductjustneedstodisplayitsownHTML pages.Notsufficientasageneralpurposewebbrowser.
57
Fullfeaturedbrowser:MozillaFirefox
http://www.mozilla.org/products/firefox/ LightweightandfastbrowserbasedonMozilla Fullfeatured:CSS,SSL,Javascript,tabbedbrowsing,pop upblocking...,butveryeasytoconfigure. Takesaround40MBofRAMwith8tabsopen. Need25MBofstoragespace(SharpZaurus) Designedtobecrossplatform.Alreadyusedinembedded systemswithsufficientscreenresolution(webpads,high endPDAs) Greatforconsumersappliances.Looksfamiliarto consumers:thedefaultthemerecallsIE.
58
WebKit
http://webkit.org/ Webbrowserengine. Applicationframeworkthatcanbeused todevelopwebbrowsers. License:portionsinLGPLandothersinBSD. Proprietaryapplicationsallowed. Usedeverywhere(MacOSX,iPhone,GoogleAndroidandChrome...) Manyapplications(browsers,emailclients...)arealreadyusingWebKit: http://trac.webkit.org/projects/webkit/wiki/Applications%20using%20WebKit Multiplegraphicalbackends:Qt4,GTK... Lightweightwebbrowsers:Midori(GTK),Arora(Qt) Youcoulduseittocreateyourcustombrowser.
59
Exampleofcomponentsusedinrealdevices
60
Industrialapplications
Inmanyindustrialapplications,thesystemisonlyresponsiblefor monitoringandcontroladevice Suchasystemisusuallyrelativelysimpleintermsofcomponents
Kernel BusyBox Clibrary ApplicationsrelyingdirectlyontheClibrary,sometimesusingthe realtimecapabilitiesoftheLinuxkernel SometimesaWebserverforremotecontrol,oranotherserver implementingacustomprotocol
61
DigitalPhotoFrame:requirements
ExampletakenfromaconferenceofMattPorter,Embedded AlleyatELC2008 Hardware:ARMSoCwithDSP,audio,800x600LCD,MMC/SD, NAND,buttons,speakers Thephotoframemustbeableto
DisplaytotheLCD DetectSDcardinsertion,notifyapplicationsoftheinsertionsothat applicationscanbuildacatalogofthepicturesontheSDcard Modern3DGUIwithnicetransitions Navigationthroughbuttons Supportaudioplayback(MP3,playlists,ID3tag) JPEGresizeandrotation
62
DigitalPhotoFrame:components(1)
Basesystem
ComponentspresentinvirtuallyallembeddedLinuxsystems TheUBootbootloader LinuxKernel
DriversforSD/MMC,framebuffer,sound,inputdevices
Busybox Buildsystem,inthiscasewasOpenEmbedded Components:uboot,linux,busybox
63
EventhandlingtodetectSDcardinsertion
udev,thatreceiveseventsfromthekernel,createsdevicenodes, andsendseventstoHAL HAL,whichmaintainsadatabaseofavailabledevicesandprovides aDBusAPI DBustoconnectHALwiththeapplication.Theapplication subscribestoHALeventthroughDBusandgetsnotifiedwhenthey aretriggered Components:udev,hal,dbus
64
JPEGdisplay
libjpegtodecodethepictures jpegtrantoresizeandrotatethem FIM(FbiImproved)fordithering
MP3support
libmadforplaying libid3forID3tagsreading libm3utosupportplaylists UsedvendorprovidedcomponentstoleveragetheDSPtoplay MP3
65
3Dinterface
Vincent,anopensourceimplementationofOpenGLES Clutter,higherlevelAPItodevelop3Dapplications
Applicationitself
Managesmediaevents UsestheJPEGlibrariestodecodeandrenderpictures ReceivesLinuxinputeventsfrombuttonsanddrawsOpenGL basedUIdevelopedwithClutter Manageauserdefinedconfiguration PlaythemusicwiththeMP3relatedlibraries Displayphotoslideshow
66
Systembuilding
67
Systembuilding:goalandsolutions
Goal
Integrateallthesoftwarecomponents,boththirdpartyandin house,intoaworkingrootfilesystem Itinvolvesthedownload,extraction,configuration,compilation andinstallationofallcomponents,andpossiblyfixingissues andadaptingconfigurationfiles
Severalsolutions
Manually Systembuildingtools Distributionsorreadymadefilesystems
68
Systembuilding:manually
Manuallybuildingatargetsysteminvolvesdownloading, configuring,compilingandinstallingallthecomponentsofthe system. Allthelibrariesanddependenciesmustbeconfigured,compiled andinstalledintherightorder. Sometimes,thebuildsystemusedbylibrariesorapplicationsis notverycrosscompilefriendly,sosomeadaptationsare necessary. Thereisnoinfrastructuretoreproducethebuildfromscratch, whichmightcauseproblemsifonecomponentneedstobe changed,ifsomebodyelsetakesovertheproject,etc.
69
Systembuilding:manually(2)
Manualsystembuildingisnotrecommendedforproduction projects However,usingautomatedtoolsoftenrequiresthedeveloperto digintospecificissues Havingabasicunderstandingofhowasystemcanbebuilt manuallyisthereforeveryusefultofixissuesencounteredwith automatedtools
Wewillfirststudymanualsystembuilding,andduringapractical lab,createasystemusingthismethod Then,wewillstudytheautomatedtoolsavailable,anduseoneof themduringalab
70
Systemfoundations
Abasicrootfilesystemneedsatleast
Atraditionaldirectoryhierarchy,with/bin,/etc,/lib,/root,
/usr/bin,/usr/lib,/usr/share,/usr/sbin,/var,/sbin
Asetofbasicutilities,providingatleasttheinitprogram,ashelland othertraditionalUnixcommandlinetools.Thisisusuallyprovided byBusybox TheClibraryandtherelatedlibraries(thread,math,etc.)installed in/lib Afewconfigurationfiles,suchas/etc/inittab,andinitialization scriptsin/etc/init.d
OntopofthisfoundationcommontomostembeddedLinux system,wecanaddthirdpartyorinhousecomponents
71
Targetandbuildspaces
Thesystemfoundation,BusyboxandClibrary,arethecoreof thetargetrootfilesystem However,whenbuildingothercomponents,onemustdistinguish twodirectories
Thetargetspace,whichcontainsthetargetrootfilesystem, everythingthatisneededforexecutionoftheapplication Thebuildspace,whichwillcontainalotmorefilesthanthe targetspace,sinceitisusedtokeepeverythingneededto compilelibrariesandapplications.Sowemustkeeptheheaders, documentation,andotherconfigurationfiles
72
Buildsystems
Eachopensourcecomponentcomeswithamechanismto configure,compileandinstallit
AbasicsimpleMakefile
NeedtoreadtheMakefiletounderstandhowitworksandhowto tweakitforcrosscompilation
AbuildsystembasedontheAutotools
Asthisisthemostcommonbuildsystem,wewillstudyitindetails
CMake,http://www.cmake.org/
Newerandsimplerthantheautotools.Usedbylargeprojectssuchas KDEorSecondLife
Scons,http://www.scons.org/ Waf,http://code.google.com/p/waf/ Othermanualbuildsystems

73
Autotoolsandfriends
Afamilyoftools,whichassociatedtogetherformacompleteand extensiblebuildsystem
autoconfisusedtohandletheconfigurationofthesoftware package automakeisusedtogeneratetheMakefilesneededtobuildthe softwarepackage pkgconfigisusedtoeasecompilationagainstalreadyinstalled sharedlibraries libtoolisusedtohandlethegenerationofsharedlibrariesina systemindependentway
Mostofthesetoolsareoldandrelativelycomplicatedtouse,but theyareusedbyamajorityoffreesoftwarepackagestoday.One musthaveabasicunderstandingofwhattheydoandhowthey work.
74
automake/autoconf/autoheader
configure.in
autoconf autoheader
configure config.h.in
Makefile.am
automake
Makefile.in
configure
Makefile
Writtenbythedeveloper Generatedbythedeveloperusingtheautotools Generatedbytheuserbyrunningtheconfigurescript
config.h
75
automake/autoconf
Fileswrittenbythedeveloper
configure.indescribestheconfigurationoptionsandthechecks doneatconfiguretime Makefile.amdescribeshowthesoftwareshouldbebuilt
TheconfigurescriptandtheMakefile.infilesare generatedbyautoconfandautomakerespectively.
Theyshouldnevermodifieddirectly Theyareusuallyshippedpregeneratedinthesoftwarepackage, becausethereareseveralversionsofautoconfandautomake, andtheyarenotcompletelycompatible
TheMakefilefilesaregeneratedatconfiguretime,before compiling
Theyarenevershippedinthesoftwarepackage.
76
Configuringandcompiling:nativecase
Thetraditionalstepstoconfigureandcompileanautotoolsbased packageare
Configurationofthepackage ./configure Compilationofthepackage make Installationofthepackage makeinstall
Additionalargumentscanbepassedtothe./configurescriptto adjustthecomponentconfiguration. Onlythemakeinstallneedstobedoneasrootiftheinstallation shouldtakeplacesystemwide

77
Configuringandcompiling:crosscase(1)
Forcrosscompilation,thingsarealittlebitmorecomplicated. AtleastsomeoftheenvironmentvariablesAR,AS,LD,NM, CC,GCC,CPP,CXX,STRIP,OBJCOPYmustbedefinedto pointtothepropercrosscompilationtools.Thehosttupleisalso bydefaultusedasprefix. Thehostargumentmustbepassedtotheconfigurescript. buildargumentisautomaticallydetected targetisonlyfortoolsgeneratingcode. Itisrecommendedtopasstheprefixargument.Itdefines fromwhichlocationthesoftwarewillruninthetarget environment.Usually,/usrisfine.
78
Configuringandcompiling:crosscase(2)
Ifonesimplyrunsmakeinstall,thesoftwarewillbeinstalledin thedirectorypassedasprefix.Forcrosscompiling,one mustpasstheDESTDIRargumenttospecifywherethesoftware mustbeinstalled. Makingthedistinctionbetweentheprefix(aspassedwith prefixatconfiguretime)andthedestinationdirectory(as passedwithDESTDIRatinstallationtime)isveryimportant. Example:
exportPATH=/usr/local/armlinux/bin:$PATH exportCC=armlinuxgcc exportSTRIP=armlinuxstrip ./configurehost=armlinux make makeDESTDIR=/home/<user>/work/rootfsinstall
79
Installation
Theautotoolsbasedsoftwarepackagesprovidebothainstall andinstallstripmaketargets,usedtoinstallthesoftware, eitherstrippedorunstripped. Forapplications,thesoftwareisusuallyinstalledin <prefix>/bin,withconfigurationfilesin<prefix>/etcand datain<prefix>/share/<application>/. Thecaseoflibrariesisalittlemorecomplicated:
In<prefix>/lib,thelibraryitself(a.so.<version>),afew symboliclinks,andthelibtooldescriptionfile(a.lafile) Thepkgconfigdescriptionfilein<prefix>/lib/pkgconfig Includefilesin<prefix>/include/ Sometimesa<libname>configprogramin<prefix>/bin Documentationin<prefix>/share/manor <prefix>/share/doc/
80
Installation(2)
Contentsof/usrafterinstallationofzlibandlibpng
./lib ./lib/libpng12.la ./lib/libpng.la ./lib/libpng12.a ./lib/libpng.a ./lib/libpng.so.3.32.0 ./lib/libpng12.so.0.32.0 ./lib/libpng12.so.0 ./lib/libpng12.so ./lib/libpng.so ./lib/libpng.so.3 ./lib/pkgconfig ./lib/pkgconfig/libpng.pc ./lib/pkgconfig/libpng12.pc ./lib/libz.so.1.2.3 ./lib/libz.so ./lib/libz.so.1 >libpng12.la >libpng12.a >libpng12.so.0.32.0 >libpng12.so.0.32.0 >libpng12.so >libpng.so.3.32.0 >libpng12.pc >libz.so.1.2.3 >libz.so.1.2.3 Libtool description file Static versionof thelibrary Dynamic versionof thelibrary
Pkgconfig description file Zlib dynamic library
81
Installationinthebuildandtargetspaces
Fromallthesefiles,everythingexceptdocumentationis necessarytobuildanapplicationthatreliesonlibpng.
Thesefileswillgointothebuildspace
However,onlythelibrarybinaryin<prefix>/libandsome symboliclinksareneededtoexecutetheapplicationonthe target.

Onlythesefileswillgointhetargetspace
Thebuildspacemustbekeptinordertobuildotherapplications orrecompileexistingapplications.
82
Let'sfindthelibraries
Whencompilinganapplicationoralibrarythatreliesonother libraries,thebuildprocessbydefaultlooksin/usr/libfor librariesand/usr/includeforheaders. ThefirstthingtodoistosettheCFLAGSandLDFLAGS environmentvariables: exportCFLAGS=I/my/build/space/usr/include/ exportLDFLAGS=L/my/build/space/usr/lib Thelibtoolfiles(.lafiles)mustbemodifiedbecausetheyinclude theabsolutepathsofthelibraries: libdir='/usr/lib' +libdir='/my/build/space/usr/lib' ThePKG_CONFIG_PATHenvironmentvariablemustbesettothe locationofthe.pcfilesandthePKG_CONFIG_SYSROOT_DIR variablemustbesettothebuildspacedirectory.
83
pkgconfig
pkgconfigisatoolthatallowstoqueryasmalldatabasetoget informationonhowtocompileprogramsthatdependonlibraries Thedatabaseismadeof.pcfiles,installedbydefaultin <prefix>/lib/pkgconfig/. pkgconfigisusedbytheconfigurescripts togetthelibraryconfigurations Itcanalsobeusedmanuallytocompileanapplication:
armlinuxgccotesttest.c$(pkgconfiglibscflagsthelib)
Bydefault,pkgconfiglooksin/usr/lib/pkgconfigforthe *.pcfiles,andassumesthatthepathsinthesefilesarecorrect. PKG_CONFIG_PATHallowstosetanotherlocationforthe*.pc filesandPKG_CONFIG_SYSROOT_DIRtoprependaprefixtothe pathsmentionedinthe.pcfiles.

84
PracticallabManualcrosscompiling
Manuallycrosscompiling applicationsandlibraries Learningaboutcommon techniquesandissues.
85
Systembuildingtools:principle
Differenttoolsareavailabletoautomatetheprocessof buildingatargetsystem,includingthekernel,and sometimesthetoolchain. Theyautomaticallydownload,configure,compileandinstall allthecomponentsintherightorder,sometimesafter applyingpatchestofixcrosscompilingissues. Theyalreadycontainalargenumberofpackages,that shouldfityourmainrequirements,andareeasilyextensible. Thebuildbecomesreproducible,whichallowstoeasily changetheconfigurationofsomecomponents,upgrade them,fixbugs,etc.
86
Availablesystembuildingtools
Largechoiceoftools Buildroot,developedbythecommunity http://www.buildroot.net PTXdist,developedbyPengutronix http://www.pengutronix.de/software/ptxdist/index_en.html OpenWRT,originallyaforkofBuildrootforwirelessrouters,nota moregenericproject http://www.openwrt.org LTIB,developedmainlybyFreescale.GoodsupportforFreescale boards,butsmallcommunity http://www.bitshrine.org/ OpenEmbedded,moreflexiblebutalsofarmorecomplicated http://www.openembedded.org Vendorspecifictools(siliconvendororembeddedLinuxvendor)
87
Buildroot(1)
Allowstobuildatoolchain,arootfilesystemimagewithmany applicationsandlibraries,abootloaderandakernelimage
Oranycombinationofthepreviousitems
SupportsbuildinguClibctoolchainsonly,butcanuseexternal uClibcorglibctoolchains Over500+applicationsorlibrariesintegrated,frombasicutilities tomoreelaboratesoftwarestacks:X.org,Gstreamer,Qt,Gtk, Webkit,etc. Goodforsmalltomediumembeddedsystems,withafixedsetof features

Nosupportforgeneratingpackages(.debor.ipk) Needscompleterebuildformostconfigurationchanges.
Activecommunity,releasespublishedevery3months.
88
Buildroot(2)
Configurationtakesplacethrough a*configinterfacesimilartothe kernel makemenuconfig Allowstodefine
ArchitectureandspecificCPU Toolchainconfiguration Setofapplicationsandlibraries tointegrate Filesystemimagestogenerate Kernelandbootloader configuration
Buildbyjustrunning make
89
Buildroot:addinganewpackage(1)
Apackageallowstointegrateauserapplicationorlibraryto Buildroot Eachpackagehasitsowndirectory(suchas package/gqview).Thisdirectorycontains:
AConfig.infile(mandatory),describingtheconfigurationoptions forthepackage.Atleastoneisneededtoenablethepackage.This filemustbesourcedfrompackage/Config.in Agqview.mkfile(mandatory),describinghowthepackageisbuilt. Patches(optional).Eachfileoftheformgqview*.patchwillbe appliedasapatch.
90
Buildroot:addinganewpackage(2)
Forasimplepackagewithasingleconfigurationoptionto enable/disableit,theConfig.infilelookslike:
configBR2_PACKAGE_GQVIEW bool"gqview" dependsonBR2_PACKAGE_LIBGTK2 help GQviewisanimageviewerforUnixoperatingsystems http://prdownloads.sourceforge.net/gqview
Itmustbesourcedfrompackage/Config.in: source"package/gqview/Config.in"
91
Buildroot:addingnewpackage(3)
Createthegqview.mkfiletodescribethebuildsteps
GQVIEW_VERSION=2.1.5 GQVIEW_SOURCE=gqview$(GQVIEW_VERSION).tar.gz GQVIEW_SITE=http://prdownloads.sourceforge.net/gqview GQVIEW_AUTORECONF=NO GQVIEW_INSTALL_STAGING=NO GQVIEW_INSTALL_TARGET=YES GQVIEW_DEPENDENCIES=hostpkgconfiglibgtk2 $(eval$(callAUTOTARGETS,package,gqview))
ThelastargumentoftheAUTOTARGETSmacro,theprefixofall variablesmustbeidenticaltothesuffixofthemainconfiguration optionBR2_PACKAGE_GQVIEW TheAUTOTARGETSinfrastructureknowshowtobuildautotools packages.AmoregenericGENTARGETSinfrastructureis availableforpackagesnotusingtheautotoolsastheirbuild system.
92
OpenEmbedded
ThemostversatileandpowerfulembeddedLinuxbuildsystem
Acollectionofrecipes(.bbfiles) Atoolthatprocessestherecipes:bitbake
Integrates2000+applicationandlibraries,ishighlyconfigurable, cangeneratepackages(.ipk)tomakethesystem customizable,supportsmultipleversions/variantsofthesame package,noneedforfullrebuildwhentheconfigurationis changed. Configurationtakesplacebyeditingvariousconfigurationfiles GoodforlargerembeddedLinuxsystems,orpeoplelookingfor moreconfigurabilityandextensibility Drawbacks:nostablereleases,verysteeplearningcurve,very longfirstbuild.
93
Distributions(1)
DebianGNU/Linux,http://www.debian.org AvailableonARM,MIPSandPowerPCarchitectures Providesareadytousefilesystemwithallthesoftwareyouneed. Hugeflexibilitythankstothepackagemanagementsystem,butonly worksonlysystemswithenoughstoragesize(>300MB) andRAM(>64MB). Softwareiscompilednativelybydefault. Youcanbuildyourownrootfilesystemimagesonx86 byusingthedebootstrapcommand. EmdebianisaprojecttomakeDebianbetterforembeddedsystems: leverageDebianpackagedescriptions,butreducesdependencies, smallerconfiguration,removesdocumentation,supportsuClibc...See http://emdebian.org.
94
Distributions(2)
UbuntuGNU/Linux BasedonDebian,samebenefits Newreleaseevery6months, supportedfor18monthsoreven3years. SupportedonARM,butonlyonCortexA8andbeyond. SuppliesThumb2binaries.Neonnotsupported. Goodsolutionformobilemultimediadevices. Others FedoraalsohassupportforARM,butnotactivelymaintained.
95
Embeddeddistributions
Distributionsdesignedforspecifictypesofdevices Meego:http://meego.com/ Distributiontargetingphones,mediaplayers, netbooks,TVsandInVehicleInfotainment. BackedbyIntelandNokia. Android:http://www.android.com/ Google'sdistributionforphonesandtabletPCs. ExcepttheLinuxkernel,verydifferentuserspace thanotherLinuxdistributions.Verysuccessful, lotsofapplicationsavailable(manyproprietary). ngstrm:http://www.angstromdistribution.org/ TargetsPDAsandwebpads(SiemensSimpad...) Binariesavailableforarmlittleendian.
96
PracticallabBuildroot
Rebuildthesamesystem, thistimewithBuildroot. Seehoweasieritgets! AddingyourownDirectFBbased application.
97
GNU/Linuxworkstation
Emulators
98
qemu
http://fabrice.bellard.free.fr/qemu/ Fastprocessoremulator usingaportabledynamictranslator. Fullsystememulation Emulatestheprocessorandvariousperipherals Supported:x86,x86_64,ppc,arm,sparc,mips,m68k Toknowwhichmachinetypesaresupported: qemusystemarmM? i386,x86_64systememulation:nowclosetonativespeeds thankstothekqemukernelmodule(nowGPLv2!).
99
Otheremulators
ARMplatform SkyEye:http://skyeye.sourceforge.net
EmulatesseveralARMplatforms(AT91,Xscale...)andcan bootseveraloperatingsystems(Linux,uClinux,andothers)
Softgun:http://softgun.sourceforge.net
VirtualARMsystemwithmanyvirtualonboardperipherals. BootsLinux.
SWARMSoftwareARMarm7emulator http://www.cl.cam.ac.uk/~mwd24/phd/swarm.html
CanrunuClinux
ColdFireemulator http://www.slicer.ca/coldfire/
CanbootuClinux
100
GNU/Linuxworkstation
Commercialtoolsets
101
Commercialtoolsets
Caution:commercialdoesn'tmeanproprietary! VendorsplayfairwiththeGPLanddomaketheirsourcecode availabletotheirusers,andmostofthetime,tothecommunity. Aslongastheydistributethesourcestotheirusers,theGPL doesn'trequirevendorstosharetheirsourceswithanythird party. NoissuewithalltheGPLsourcesdevelopedbyorwiththe community. Graphicaltoolkitsdevelopedbythevendorslookproprietary. Theirlicensesarenotadvertisedontheirwebsites!Youhaveto beacustomertoknoworgetafreepreviewkittoknow.
102
Commercialtoolsetstrengths
Technicaladvantages Welltestedandsupported kernelandtoolversions Includingearlypatchesnot supportedbythemainstream kernelyet Completedevelopmenttoolsets: kernels,toolchains,utilities,binaries forimpressivelistsoftarget platforms Integratedutilitiesforautomatic kernelimage,initrdandfilesystem generation. Graphicaldevelopmentstools Developmenttoolsavailableon multipleplatforms:GNU/Linux, Solaris,Windows... Supportservices Usefulifyoudon'thaveyour ownsupportresources Longtermsupport commitment,evenfor versionsconsideredas obsoletebythecommunity, butnotbyyourusers!
103
WindRiver
WindRiverLinux: http://www.windriver.com/products/linux/ Newmarketleader,recentlyacquiredbyIntel AlotofembeddedandrealtimeexperiencefromVxWorks. Nowsaytheyintegrate,testandsupportLinux asrigorouslyastheydowithVxWorks. LinuxdevelopmentsupportedwiththeirWorkbenchintegrated developmentenvironment,alreadyusedforVxWorks. SupportstandardandrecentLinuxkernelsources,includingreal timepreemptpatches(Linux2.6.27).Alsoofferhardrealtime Linux(RealTimeCore:formerlyRTLinux).
104
Montavista
http://www.mvista.com/
Thesecondmarketleader Employssomeofthemostactivekernelhackers, inparticularonthearmplatform. Kerneldevelopmenteventuallysharedwiththe community.kernel.Manydriversmergedinmainstream Linux. Graphicaldevelopmenttoolsareproprietary.
105
TimeSys
http://timesys.com Similartoolsetofferingasothervendors. GreatflexibilityavailabletotheirLinuxLinkTM subscribers Communityfriendly:theyshareveryinterestingand generictechnicalwhitepapersandarticles.Theyalso employkeycommunityhackers(ThomasGleixner,Rob Landley...). FreeSoftwareBSPs(BoardSupportPackages) available. LinuxsoftandhardrealtimeOSproduct. Developmenttoolsseemtobeproprietary.
106
SysgoKoanSoftware
http://sysgo.com ELinOSdevelopmenttoolset, inparticularbasedonEclipse andtheLinuxTraceToolkit. IncludesFreeToolBox,afreely downloadablecompilingandrootfs creatingtoolchain. Supportsi386,armandppc. Hardrealtimesupportwiththeirown microkernel(PikeOS), anapproachsimilartoRTAI. http://koansoftware.com MakersofKaeilOS (http://koansoftware.com/kaeilos/index_en.htm), aGPLembeddedLinuxdistributionfor industrialapplications. KaeilOSsupportsi386andpopulararm platforms.Otherplatformssupportedupon request. Includesseveralgraphicaltoolkitsand supportshardrealtime(RTAI,Xenomai, preemptionpatches). Unfortunately,KaeilOSisGPLbutnot availableforpublicdownload.
107
DenxSoftwareEngineering
http://denx.de CreatedbyWolfgangDenk, theauthoroftheUBootbootloader. CreateandsupporttheEmbeddedLinuxDevelopmentKit (ELDK),acompleteandwelldocumenteddevelopment environment. ThiskitisnotonlyFreeSoftware, itcanbedownloadedfreelybyanyone. Agreatcommunitymemberandcontributor!
108
CommercialtoolsetsSummary
Majorvendors:MontaVista,WindRiver,TimeSys InvolvedinLinuxdevelopment. Smallervendors:Koan,Sysgo,Denx... Tryingtodifferentiatetheirproducts. Communitybasedcompanies:Denx,CodeSourcery Contributetocommunitytools. Mainlyoffersupportanddevelopment.
109
Commercialorcommunitysolutions?
Commercialdistributionsandtoolsets Bestifyoudon'thaveyourownsupportresourcesandhavea sufficientbudget Reallyhelpfocusingonyourrealjob:makinganembedded device. Youcanevensubcontractdriverdevelopmenttothevendor Communitydistributionsandtools Bestifyouareonatightbudget BestifyouarewillingtobuildyourownembeddedLinuxexpertise andtrainyourownsupportresources. Inanycase,yourproductsarebasedonFreeSoftware!
110
References
111
Usefulreading(1)
EmbeddedLinuxSystemDesignandDevelopment P.Raghavan,A.Lad,S.Neelakandan,Auerbach,Dec.2005. http://freeelectrons.com/redirect/elsddbook.html UsefulbookcoveringmostaspectsofembeddedLinux systemdevelopment(kernelandtools). BuildingEmbeddedLinuxSystems,O'Reilly ByKarimYaghmour,JonMasters,GiladBenYossefand PhilippeGerum,andothers(includingMichaelOpdenacker), August2008 http://oreilly.com/catalog/9780596529680/ EmbeddedLinuxPrimer,PrenticeHall ByChristopherHallinan,September2006 Coversaverywiderangeofinterestingtopics.
112
Usefulreading(2)
http://www.denx.de/wiki/DULG/Manual Lotsofusefulcommandexamples,generichelpandadvice forembeddedLinuxsystems. Seehttp://www.linuxdevices.com/articles/AT2969812114.html formorebooksonLinuxforembeddedsystems.
113
Usefulwebsites
LinuxDevices.com:http://linuxdevices.com Weeklynewsletterwithnewsandannouncementsabout embeddeddevicesrunningLinux. Articles,whitepapers,andLinuxembeddeddevicescatalog. Anexcellentsitetofollowindustrynews!
114
Internationalconferences
UsefulconferencesfeaturingembeddedLinuxandkerneltopics EmbeddedLinuxConference:http://embeddedlinuxconference.com/ OrganizedbytheCELinuxForum:California (SanFrancisco,April),inEurope(OctoberNovember). Veryinterestingkernelanduserspacetopicsforembeddedsystems developers.Presentationslidesfreelyavailable LinuxPlumbers http://linuxplumbersconf.org ConferenceonthelowlevelplumbingofLinux:kernel,audio,power management,devicemanagement,multimedia,etc. Fosdem:http://fosdem.org(Brussels,February) Fordevelopers.Presentationsaboutsystemdevelopment. Don'tmissourfreeconferencevideoson http://freeelectrons.com/community/videos/conferences/!
115
EmbeddedLinux application development
.S
make
.h
.c++
.c
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/appdev Corrections,suggestions,contributionsandtranslationsarewelcome!
1
Contents
Applicationdevelopment DevelopingapplicationsonembeddedLinux Buildingyourapplications Sourcemanagement Integrateddevelopmentenvironments(IDEs) Versioncontrolsystems Debuggingandanalysistools Debuggers Memorycheckers Systemanalysis Developmentenvironments DevelopingonWindows
2
Applicationdevelopment
DevelopingapplicationsonembeddedLinux
3
AnembeddedLinuxsystemisjustanormalLinuxsystem,with usuallyasmallerselectionofcomponents Intermsofapplicationdevelopment,developingonembedded LinuxisexactlythesameasdevelopingonadesktopLinux system Allexistingskillscanbereused,withoutanyparticular adaptation Allexistinglibraries,eitherthirdpartyorinhouse,canbe integratedintotheembeddedLinuxsystem
Takingintoaccount,ofcourse,thelimitationoftheembedded systemsintermsofperformance,storageandmemory
4
Programminglanguage
Thedefaultprogramminglanguageforsystemlevelapplicationin LinuxisusuallyC
TheClibraryisalreadypresentonyoursystem,nothingtoadd
C++canbeusedforlargerapplications
TheC++librarymustbeaddedtothesystem Somelibraries,includingQt,aredevelopedinC++sotheyneedthe C++libraryonthesystemanyway
Scriptinglanguagescanalsobeusefulforquickapplication development,webapplicationsorscripts
Buttheyrequireaninterpreterontheembeddedsystemandhave usuallyhighermemoryconsumptionandslightlylower performances
Languages:Python,Perl,Lua,Ada,Fortran,etc.
5
Clibraryorhigherlevellibraries?
Formanyapplications,theClibraryalreadyprovidesarelatively largesetoffeatures
fileanddeviceI/O,networking,threadsandsynchronization,inter processcommunication Thoroughlydescribedintheglibcmanual,orinanyLinuxsystem programmingbook However,theAPIcarriesalotofhistoryandisnotnecessarilyeasy tograspfornewcomers
Therefore,usingahigherlevelframework,suchasQtortheGtk stack,mightbeagoodidea
Theseframeworksarenotonlygraphicallibraries,theircoreis separatefromthegraphicalpart Butofcourse,theselibrarieshavesomememoryandstorage footprint,intheorderofafewmegabytes
6
Buildingyourapplications
Forsimpleapplicationsthatdonotneedtobereallyportableor providecompiletimeconfigurationoptions,asimpleMakefilewill besufficient Formorecomplicatedapplications,orifyouwanttobeableto runyourapplicationonadesktopLinuxPCandonthetarget device,usingabuildsystemisrecommended
Lookattheautotools(ancient,complicatedbutverywidelyused)or CMake(modern,simpler,smallerbutgrowinguserbase)
TheQTlibraryisaspecialcase,sinceitcomeswithitsownbuild systemforapplications,calledqmake
7
SimpleMakefile(1)
CaseofanapplicationthatonlyusestheClibrary,containstwo sourcefilesandgeneratesasinglebinary CROSS_COMPILE?=armlinux CC=$(CROSS_COMPILE)gcc OBJS=foo.obar.o all:foobar foobar:$(OBJS) $(CC)o$@$^ clean: $(RM)ffoobar$(OBJS)
8
SimpleMakefile(2)
CaseofanapplicationthatusestheGlibandtheGPSlibraries CROSS_COMPILE?=armlinux LIBS=libgpsglib2.0 OBJS=foo.obar.o CC=$(CROSS_COMPILE)gcc CFLAGS=$(shellpkgconfigcflags$(LIBS)) LDFLAGS=$(shellpkgconfiglibs$(LIBS)) all:foobar foobar:$(OBJS) $(CC)o$@$^$(LDFLAGS) clean: $(RM)ffoobar$(OBJS)
Sourcemanagement
IntegratedDevelopmentEnvironments(IDE)
10
Sourcebrowsers
LXR:LinuxCrossReference Allowstobrowsecodethroughawebbrowser. cscope Consolemodesourcebrowsingtool. Integrateswitheditorslikeviandemacs. SeeourLinuxkernelanddriverdevelopmenttrainingmaterials (http://freeelectrons.com/docs/kernel)formoredetails.
11
KDevelop
http://kdevelop.org AfullfeaturedIDE! License:GPL Supportsmanylanguages: Ada,C,C++,Database,Java,Perl,PHP,Python,Ruby,Shell Supportsmanykindsofprojects: KDE,butalsoGTK,Gnome,kerneldrivers,embedded(Opie)... Manyfeatures:editor,syntaxhighlighting,codecompletion, compilerinterface,debuggerinterface,filemanager,class browser... Niceoverview:http://en.wikipedia.org/wiki/Kdevelop
12
KDevelopscreenshot
Ruby debugger
13
Eclipse(1)
http://www.eclipse.org/ Anextensible,pluginbased softwaredevelopmentkit,typicallyusedforcreatingIDEs. SupportedbytheEclipsefoundation,anonprofit consortiumofmajorsoftwareindustryvendors(IBM,Intel, Borland,Nokia,WindRiver,Zend,ComputerAssociates...). FreeSoftwarelicense(EclipsePublicLicense). IncompatiblewiththeGPL. Supportedplatforms:GNU/Linux,Unix,Windows Extremelypopular:createdalotofattraction.
14
Eclipse(2)
Eclipseisactuallyaplatformcomposedofmanyprojects: http://www.eclipse.org/projects/
SomeprojectsarededicatedatintegratingintoEclipsefeatures usefulforembeddeddevelopers(crosscompilation,remote development,remotedebugging,etc.)
TheplatformisusedbymajorembeddedLinuxsoftwarevendors fortheir(proprietary)systemdevelopmentkits: MontaVistaDevRocket,TimeSysTimeStorm,Windriver Workbench,SysgoELinOS. Eclipseisahugeproject. Itwouldrequireanentiretrainingsession!
15
Sourcemanagement
Versioncontrolsystems
16
Versioncontrolsystems
Realprojectscan'tdowithoutthem Allowmultipledeveloperstocontributeonthesameproject.Each developercanseethelatestchangesfromtheothers,orchoose tostickwitholderversionsofsomecomponents. Allowtokeeptrackofchanges,andrevertthemifneeded. Allowdeveloperstohavetheirowndevelopmentbranch (branching) Supposedtohelpdevelopersresolvingconflictswithdifferent branches(merging)
17
Traditionalversioncontrolsystems
Relyonacentralrepository.Themostpopularopensourceones: CVSConcurrentVersionsSystem
Stillquitepopular Shouldnolongerbeusedfornewprojects http://en.wikipedia.org/wiki/Concurrent_Versions_System
Subversion
CreatedasareplacementofCVS,removingmanyofitslimitations. Commitsonseveralfiles,properrenamingsupport,better performances,etc. TheuserinterfaceisverysimilartoCVS http://en.wikipedia.org/wiki/Subversion_(software)
18
Distributedsourcecontrolsystems(1)
Nolongerhaveacentralrepository MoreadaptedtothewaytheFreeSoftwarecommunitydevelops softwareandorganizes Alloweachdevelopertohavealocalfullhistoryoftheproject,to createlocalbranches.Makeseachdeveloper'sworkeasier. Peoplegetworkingcopiesfromotherpeople'sworkingcopies, andexchangechangesbetweenthemselves.Branchingand mergingismadeeasier. Makeiteasierfornewdeveloperstojoin,makingtheirown experimentswithouthavingtoapplyforrepositoryaccess.
19
Distributedsourcecontrolsystems(2)
Git
InitiallydesignedanddevelopedbyLinusTorvalds forLinuxkerneldevelopment Extremelypopularinthecommunity,andusedbymoreandmore projects(kernel,UBoot,Barebox,uClibc,GNOME,X.org,etc.) Outstandingperformance,inparticularinbigprojects http://en.wikipedia.org/wiki/Git_(software)
Mercurial
Anothersystem,createdwiththesamegoalsasGit. Usedbysomebigprojectstoo http://en.wikipedia.org/wiki/Mercurial
http://en.wikipedia.org/wiki/Version_control_systems#Distributed_revision_control
20
Softwaredevelopmenttools
Debuggingandanalysistools
Debuggers
21
GDB
TheGNUProjectDebugger http://www.gnu.org/software/gdb/ ThedebuggeronGNU/Linux, availableformostembeddedarchitectures. Supportedlanguages:C,C++,Pascal,ObjectiveC,Fortran, Ada... Consoleinterface(usefulforremotedebugging). Graphicalfrontendsavailable. Canbeusedtocontroltheexecutionofaprogram,set breakpointsorchangeinternalvariables.Youcanalsouseitto seewhataprogramwasdoingwhenitcrashed(byloadingits memoryimage,dumpedintoacorefile). Seealsohttp://en.wikipedia.org/wiki/Gdb
22
GDBcrashcourse
AfewusefulGDBcommands
breakfoobar putsabreakpointattheentryoffunctionfoobar() breakfoobar.c:42 putsabreakpointinfoobar.c,line42 printvarorprinttask>files[0].fd printsthevariablevar,oramorecomplicatedreference.GDBcan alsonicelydisplaystructureswithalltheirmembers continue continuetheexecution next continuetothenextline,steppingoverfunctioncalls step continuetothenextline,enteringintosubfunctions
23
GDBgraphicalfrontends
DDDDataDisplayDebugger http://www.gnu.org/software/ddd/ Apopulargraphicalfrontend,withadvanceddataplotting capabilities. GDB/Insight http://sourceware.org/insight/ FromtheGDBmaintainers. KDbg http://www.kdbg.org/ Anotherfrontend,fortheKDisplayEnvironment. IntegrationwithotherIDEs:Eclipse,Emacs,KDevelop,etc.
24
Introductiontosoftwaredevelopmenttools
Remotedebugging
25
Remotedebugging
Inanonembeddedenvironment,debuggingtakesplaceusinggdbor oneofitsfrontend. gdbhasdirectaccesstothebinaryandlibrariescompiledwith debuggingsymbols. However,inanembeddedcontext,thetargetplatformenvironmentis oftentoolimitedtoallowdirectdebuggingwithgdb(2.4MBonx86). Remotedebuggingispreferred gdbisusedonthedevelopmentworkstation,offeringallitsfeatures. gdbserverisusedonthetargetsystem(only100KBonarm).
gdb
gdbserver
26
Remotedebugging:architecture
Host Target gdbserver Serialor Ethernet connection
Binariesandlibraries withdebugging symbolsnotstripped Runningprogram withbinariesand librariesthat canbestripped
ARCHlinuxgdb
27
Remotedebugging:usage
Onthetarget,runaprogramthroughgdbserver. Programexecutionwillnotstartimmediately. gdbserverlocalhost:<port><executable><args> gdbserver/dev/ttyS0<executable><args> Otherwise,attachgdbservertoanalreadyrunningprogram: gdbserverattachlocalhost:<port><pid> Then,onthehost,runARCHlinuxgdbprogram, andusethefollowinggdbcommands: Toconnecttothetarget: gdb>targetremote<target>:<port> gdb>targetremote/dev/ttyS0 Totellgdbwheresharedlibrariesare: gdb>setsysroot<librarypath> (networking) (seriallink) (withoutlib/)
28
Postmortemanalysis
Whenanapplicationcrashesduetoasegmentationfaultandthe applicationwasnotundercontrolofadebugger,wegetno informationsaboutthecrash Fortunately,Linuxcangenerateacorefilethatcontainsthe imageoftheapplicationmemoryatthemomentofthecrash,and gdbcanusethiscorefiletoletusanalyzethestateofthe crashedapplication Onthetarget
Useulimitcunlimitedtoenablethegenerationofacore filewhenacrashoccurs
Onthehost
Afterthecrash,transferthecorefilefromthetargettothehost,and runARCHlinuxgdbccorefileapplicationbinary
29
Debuggingandanalysistools Memorycheckers
30
memcheck
http://hald.dnsalias.net/projects/memcheck/ GNUGPLtoolfordynamicmemorychecking Worksbyreplacingglibc'smemorymanagementfunctions byitsown. SupportsmostusefulCPUarchitectures.
31
DUMA
DetectUnintendedMemoryAccess http://duma.sourceforge.net/ ForkandreplacementforElectricFence Stopsyourprogramontheexactinstructionthatoverrunsor underrunsamalloc()memorybuffer. GDBwillthendisplaythesourcecodelinethatcausesthe bug. Worksbyusingthevirtualmemoryhardwaretocreatea redzoneattheborderofeachbuffertouchthat,andyour programstops. WorksonanyplatformsupportedbyLinux,whateverthe CPU(providedvirtualmemorysupportisavailable).
32
Valgrind(1)
http://valgrind.org/ GNUGPLSoftwaresuitefordebugging andprofilingprograms. Supportedplatforms:Linuxonx86,x86_64,ppc32,ppc64 andarm(armv7only:CortexA8,A9andA5) Candetectmanymemorymanagementandthreadingbugs. Profiler:providesinformationhelpfultospeedupyourprogram andreduceitsmemoryusage. Themostpopulartoolforthisusage. Evenusedbyprojectswithhundredsofprogrammers.
33
Valgrind(2)
Canbeusedtorunanyprogram,withouttheneedto recompileit. Exampleusage valgrindleakcheck=yeslsla Worksbyaddingitsowninstrumentationtoyourcodeand thenrunninginonitsownvirtualcpucore. Significantlyslowsdownexecution,butstillfinefortesting! Moredetailsonhttp://valgrind.org/info/and http://valgrind.org/docs/manual/coregrind_core.html#howworks
34
Systemanalysis
35
strace
Systemcalltracer http://sourceforge.net/projects/strace/ AvailableonallGNU/Linuxsystems Canbebuiltbyyourcrosscompilingtoolchaingenerator. Allowstoseewhatanyofyourprocessesisdoing: accessingfiles,allocatingmemory... Oftensufficienttofindsimplebugs. Usage: strace<command> stracep<pid> Seemanstracefordetails. (startinganewprocess) (tracinganexistingprocess)
36
straceexampleoutput
>stracecatMakefile execve("/bin/cat",["cat","Makefile"],[/*38vars*/])=0 brk(0)=0x98b4000 access("/etc/ld.so.nohwcap",F_OK)=1ENOENT(Nosuchfileordirectory) mmap2(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,1,0)= 0xb7f85000 access("/etc/ld.so.preload",R_OK)=1ENOENT(Nosuchfileordirectory) open("/etc/ld.so.cache",O_RDONLY)=3 fstat64(3,{st_mode=S_IFREG|0644,st_size=111585,...})=0 mmap2(NULL,111585,PROT_READ,MAP_PRIVATE,3,0)=0xb7f69000 close(3)=0 access("/etc/ld.so.nohwcap",F_OK)=1ENOENT(Nosuchfileordirectory) open("/lib/tls/i686/cmov/libc.so.6",O_RDONLY)=3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320h\1\0004\0\0\0\344"...,512) =512 fstat64(3,{st_mode=S_IFREG|0755,st_size=1442180,...})=0 mmap2(NULL,1451632,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_DENYWRITE,3,0)= 0xb7e06000 mprotect(0xb7f62000,4096,PROT_NONE)=0 mmap2(0xb7f63000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED| MAP_DENYWRITE,3,0x15c)=0xb7f63000 mmap2(0xb7f66000,9840,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED| MAP_ANONYMOUS,1,0)=0xb7f66000 close(3)=0
37
ltrace
Atooltotracelibrarycallsusedbyaprogram andallthesignalsitreceives Veryusefulcomplementtostrace, whichshowsonlysystemcalls. Ofcourse,worksevenifyoudon'thavethesources Allowstofilterlibrarycallswithregularexpressions, orjustbyalistoffunctionnames. Manualpage:http://linux.die.net/man/1/ltrace Seehttp://en.wikipedia.org/wiki/Ltracefordetails
38
ltraceexampleoutput
ltraceneditindex.html sscanf(0x8274af1,0x8132618,0x8248640,0xbfaadfe8,0)=1 sprintf("const0","const%d",0)=7 strcmp("startScan","const0")=1 strcmp("ScanDistance","const0")=1 strcmp("const200","const0")=1 strcmp("$list_dialog_button","const0")=1 strcmp("$shell_cmd_status","const0")=1 strcmp("$read_status","const0")=1 strcmp("$search_end","const0")=1 strcmp("$string_dialog_button","const0")=1 strcmp("$rangeset_list","const0")=1 strcmp("$calltip_ID","const0")=1 ...
39
ltracesummary
Examplesummaryattheendoftheltraceoutput(coption)
Process17019detached %timesecondsusecs/callcallserrorssyscall 100.000.000050501set_thread_area 0.000.000000048read 0.000.000000044write 0.000.00000008063open 0.000.000000019close 0.000.00000001execve 0.000.000000022access 0.000.00000003brk 0.000.00000001munmap 0.000.00000001uname 0.000.00000001mprotect 0.000.000000019mmap2 0.000.00000005046stat64 0.000.000000018fstat64 100.000.000050288111total
40
Oprofile
http://oprofile.sourceforge.net Asystemwideprofilingtool CancollectstatisticslikethetopusersoftheCPU. Workswithouthavingthesources. Requiresakernelpatchtoaccessallfeatures, butisalreadyavailableinastandardkernel. Requiresmoreinvestigationtoseehowitworks. Ubuntu/Debianpackages: oprofile,oprofilegui
41
Callgrind/KCachegrind
Cachegrind/Callgrind:partoftheValgrindtoolsuite Collectsfunctioncallstatisticsandcallgraphs. Usefultoknowinwhichfunctionsmosttimeisspent. KCachegrind:http://kcachegrind.sourceforge.net/ AnamazingvisualizerforCachegrind/Callgrinddata. KCachegrindcanalsoimportdatafromotherprofilers (suchasOProfile),andfromprofilingoutputfromPython,Perl andPHP.
42
KCachegrindscreenshot
43
PracticallabRemotedebugging
TimetostartLab! Setupremotedebuggingtools onthetarget:strace,ltrace andgdbserver. Debugasimpleapplication runningonthetargetusing remotedebugging
44
DevelopingonWindows
45
DevelopingonWindows!?
UsingaGNU/Linuxworkstationistheeasiestwaytocreate softwareforGNU/LinuxorembeddedLinux Youusethesametoolsandenvironmentasallcommunity developersdo.Muchfewerissuesyouaretheonlyoneto face. Yougetfamiliarwiththesystem. Essentialforunderstandingissues. However,somedevelopershavenochoice: WindowsistheonlydesktopOSallowedintheircompany.
46
Cygwin
http://cygwin.com/ Linux(POSIX)likeenvironmentforWindows 2components: LinuxAPIemulationlayer:cygwin1.dll AcollectionoftoolsoriginallyfoundinGNU/Linux AllowstocompileandrunmanyGNU/LinuxprogramsonWindows: shells,compiler,httpservers,XWindow,GTK... Veryeasytoinstall.Canchoosewhichtoolstodownloadandinstall. ForembeddedLinuxsystemdevelopers: makesitpossibletouseGNUtoolchains(compiledforWindows) requiredtobuildLinuxbinaries(kernel,librariesorapplications).
47
Cygwinlimitations
CygwinisnotacompletesubstituteforarealGNU/Linuxsystem. AlmostalldevelopersworkonGNU/LinuxoronanotherUnix platform(typicallyBSD).Don'texpectthemtotestthattheir toolsbuildonWindowswithCygwin. ThenumberofCygwinusersisquitesmall. Youmaybethefirsttofaceorreportbuildingissuesonthis platformforagivencompilerortoolversion. Cygwinisveryslow. So,thebestsolutionistorunLinuxinsideWindows!
48
VMware
http://en.wikipedia.org/wiki/VMware License:proprietary CanrunaGNU/LinuxPCfromWindows, almostatthehostspeed. VMwarePlayerisnowavailablefreeofcharge. ManyFreeSoftwaresystemimagesavailablefordownload. Themostpopularsolutioninthecorporateworld.
49
VirtualBox
http://virtualbox.orgfromSunMicrosystems PCemulationsolutionavailable onbothWindowsandGNU/Linux 2licenses:
Proprietary:freeofcostforpersonaluseandevaluation. BinariesavailableforWindows.Fullfeatures. OpenSourceEdition(OSE):GPLlicense. Mostfeatures(exceptinparticularUSBsupport). NobinariesreleasedforWindowssofar(butpossible).
BasedonQEMU'scoreengine. PerformancesimilartothatofVMware. Seehttp://en.wikipedia.org/wiki/VirtualBox

50
Realtimein embeddedLinux systems

MichaelOpdenacker ThomasPetazzoni GillesChanteperdrix FreeElectrons
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/realtime Corrections,suggestions,contributionsandtranslationsarewelcome!
1
RealTimeinEmbeddedLinuxSystems
Introduction
2
EmbeddedLinuxandrealtime
Duetoitsadvantages,Linuxandtheopensourcesoftwaresare moreandmorecommonlyusedinembeddedapplications However,someapplicationsalsohaverealtimeconstraints They,atthesametime,wantto
GetalltheniceadvantagesofLinux:hardwaresupport, componentsreuse,lowcost,etc. Gettheirrealtimeconstraintsmet
?
3
EmbeddedLinuxandrealtime
LinuxisanoperatingsystempartofthelargeUnixfamily Itwasoriginallydesignedasatimesharingsystem
Themaingoalistogetthebestthroughputfromtheavailable hardware,bymakingthebestpossibleusageofresources(CPU, memory,I/O) Timedeterminismisnottakenintoaccount
Ontheopposite,realtimeconstraintsimplytimedeterminism, evenattheexpenseoflowerglobalthroughput Bestthroughputandtimedeterminismarecontradictory requirements
4
Linuxandrealtimeapproaches
Overtime,twomajorapproacheshavebeentakentobringreal timerequirementsintoLinux Approach1
ImprovetheLinuxkernelitselfsothatitmatchesrealtime requirements,byprovidingboundedlatencies,realtimeAPIs,etc. ApproachtakenbythemainlineLinuxkernelandthe PREEMPT_RTproject.
Approach2
AddalayerbelowtheLinuxkernelthatwillhandlealltherealtime requirements,sothatthebehaviourofLinuxdoesn'taffectrealtime tasks. ApproachtakenbyRTLinux,RTAIandXenomai
5
Approach1 ImprovingthemainLinuxkernelwith PREEMPT_RT
6
Understandinglatency
Whendevelopingrealtimeapplicationswithasystemsuchas Linux,thetypicalscenarioisthefollowing
Aneventfromthephysicalworldhappensandgetsnotifiedtothe CPUbymeansofaninterrupt Theinterrupthandlerrecognizesandhandlestheevent,andthen wakeuptheuserspacetaskthatwillreacttothisevent Sometimelater,theuserspacetaskwillrunandbeabletoreactto thephysicalworldevent
Realtimeisaboutprovidingguaranteedworstcaselatenciesfor thisreactiontime,calledlatency
Somethingnotveryimportant... Yourimportant realtimetask!
Interrupt!
Linuxkernellatencycomponents
Waiting task
Makesthe taskrunnable interrupt latency Interrupt handler duration scheduler latency scheduler duration Interrupt handler Scheduler
Runningtask
Process context
Interrupt context
Schedulinglatency
kernellatency=interruptlatency+handlerduration +schedulerlatency+schedulerduration
8
Interruptlatency
Waiting task Runningtask
Schedulinglatency
Timeelapsedbeforeexecutingtheinterrupthandler
9
Sourceofinterruptlatency
Oneoftheconcurrencypreventionmechanismusedinthekernel isthespinlock Ithasseveralvariants,butoneofthevariantcommonlyusedto preventconcurrentaccessesbetweenaprocesscontextandan interruptcontextworksbydisablinginterrupts Criticalsectionsprotectedbyspinlocks,orothersectioninwhich interruptsareexplictlydisabledwilldelaythebeginningofthe executionoftheinterrupthandler
Thedurationofthesecriticalsectionsisunbounded
Otherpossiblesource:sharedinterrupts
Kernel code Criticalsection protectedbyspinlock Interrupt Interrupt handler
10
Interrupthandlerduration
Schedulinglatency
Timetakentoexecutetheinterrupthandler
11
Interrupthandlerimplementation
InLinux,manyinterrupthandlersaresplitintwoparts
Atophalf,startedbytheCPUassoonasinterruptare enabled.Itrunswiththeinterruptlinedisabledandis supposedtocompleteasquicklyaspossible. Abottomhalf,scheduledbythetophalf,whichstartsafterall pendingtophalfhavecompletedtheirexecution.
Therefore,forrealtimecriticalinterrupts,bottomhalf shouldn'tbeused:theirexecutionisdelayedbyallother interruptsinthesystem.

Tophalf Interrupt ACK Schedule Exit bottom half Otherinterrupt handlers... Bottomhalf Handle device data... Wakeup waiting tasks
Userspace...
12
Schedulerlatency
Schedulinglatency
Timeelapsedbeforeexecutingthescheduler
13
Understandingpreemption(1)
TheLinuxkernelisapreemptiveoperatingsystem Whenataskrunsinuserspacemodeandgetsinterruptedbyan interruption,iftheinterrupthandlerwakesupanothertask,this taskcanbescheduledassoonaswereturnfromtheinterrupt handler.
Interrupthandler WakesupTaskB
TaskA (runninginusermode)
TaskB (runninginusermode)
Interrupt
14
Understandingpreemption(2)
However,whentheinterruptcomeswhilethetaskisexecutinga systemcall,thissystemcallhastofinishbeforeanothertaskcan bescheduled. Bydefault,theLinuxkerneldoesnotdokernelpreemption. Thismeansthatthetimebeforewhichtheschedulerwillbe calledtoscheduleanothertaskisunbounded.
TaskA TaskA Interrupthandler (kernelmode) WakesupTaskB (kernelmode)
TaskA (usermode)
Returnfromsyscall
TaskB (usermode)
Systemcall
Interrupt
15
Schedulerduration
Schedulinglatency
Timetakentoexecutethescheduler andswitchtothenewtask.
16
Othernondeterministicmechanisms
Outsideofthecriticalpathdetailedpreviously,othernon deterministicmechanismsofLinuxcanaffecttheexecutiontime ofrealtimetasks Linuxishighlybasedonvirtualmemory,asprovidedbyanMMU, sothatmemoryisallocatedondemand.Wheneveranapplication accessescodeordataforthefirsttime,itisloadedondemand, whichcancreateshugedelays. ManyClibraryservicesorkernelservicesarenotdesignedwith realtimeconstraintsinmind.
17
Priorityinversion
Aprocesswithalowprioritymightholdalockneededbyahigher priorityprocess,effectivelyreducingthepriorityofthisprocess. ThingscanbeevenworseifamiddlepriorityprocessusestheCPU.
Priority
Triestoget thesame lock
waits preempted
Acquires alock
Time
18
Interrupthandlerpriority
InLinux,interrupthandlersareexecuteddirectlybytheCPU interruptmechanisms,andnotundercontroloftheLinux scheduler.Therefore,allinterrupthandlershaveanhigher prioritythanalltasksrunningonthesystem.
Anyinterrupt Anyinterrupt...
topprioritytask
topprioritytask
19
ThePREEMPT_RTproject
LongtermprojectleadbyLinuxkerneldevelopersIngoMolnar, ThomasGleixnerandStevenRostedt
https://rt.wiki.kernel.org
ThegoalistograduallyimprovetheLinuxkernelregardingreal timerequirementsandtogettheseimprovementsmergedinto themainlinekernel

PREEMPT_RTdevelopmentworksverycloselywiththemainline development
Manyoftheimprovementsdesigned,developedanddebugged insidePREEMPT_RTovertheyearsarenowpartofthemainline Linuxkernel

TheprojectisalongtermbranchoftheLinuxkernelthatultimately shoulddisappearaseverythingwillhavebeenmerged
20
Improvementsinthemainlinekernel
Comingfromthe PREEMPT_RTproject Sincethebeginningof2.6
O(1)scheduler Kernelpreemption BetterPOSIXrealtimeAPI support
Since2.6.30
Threadedinterrupts
Since2.6.33
Spinlockannotations
Since2.6.18
Priorityinheritancesupport formutexes
Since2.6.21
Highresolutiontimers
21
NewpreemptionoptionsinLinux2.6
2newpreemptionmodelsofferedbystandardLinux2.6:
22
1 option:noforcedpreemption
CONFIG_PREEMPT_NONE Kernelcode(interrupts,exceptions,systemcalls)neverpreempted. Defaultbehaviorinstandardkernels. Bestforsystemsmakingintensecomputations, onwhichoverallthroughputiskey. BesttoreducetaskswitchingtomaximizeCPUandcacheusage (byreducingcontextswitching). StillbenefitsfromsomeLinux2.6improvements: O(1)scheduler,increasedmultiprocessorsafety(workonRT preemptionwasusefultoidentifyhardtofindSMPbugs). Canalsobenefitfromalowertimerfrequency (100Hzinsteadof250or1000).
st
23
2 option:voluntarykernelpreemption
CONFIG_PREEMPT_VOLUNTARY Kernelcodecanpreemptitself Typicallyfordesktopsystems,forquickerapplicationreactionto userinput. Addsexplicitreschedulingpointsthroughoutkernelcode. Minorimpactonthroughput.
nd
24
3 option:preemptiblekernel
CONFIG_PREEMPT Mostkernelcodecanbeinvoluntarilypreemptedatanytime. Whenaprocessbecomesrunnable,nomoreneedtowaitfor kernelcode(typicallyasystemcall)toreturnbeforerunningthe scheduler. Exception:kernelcriticalsections(holdingspinlocks),buta reschedulingpointoccurswhenexitingtheoutercriticalsection, incaseapreemptionopportunitywouldhavebeensignaledwhile inthecriticalsection. Typicallyfordesktoporembeddedsystemswithlatency requirementsinthemillisecondsrange. Stillarelativelyminorimpactonthroughput.
25
rd
Priorityinheritance
Oneclassicalsolutiontothepriorityinversionproblemiscalled priorityinheritance
Theideaisthatwhenataskofalowpriorityholdsalockrequested byanhigherprioritytask,thepriorityofthefirsttaskgetstemporarly raisedtothepriorityofthesecondtask:ithasinheriteditspriority.
InLinux,since2.6.18,mutexessupportpriorityinheritance Inuserspace,priorityinheritancemustbeexplictlyenabledona permutexbasis.
26
Highresolutiontimers
Theresolutionofthetimersusedtobeboundtotheresolutionof theregularsystemtick
Usually100Hzor250Hz,dependingonthearchitectureandthe configuration Aresolutionofonly10msor4ms. Increasingtheregularsystemtickfrequencyisnotanoptionasit wouldconsumetoomuchresources
Thehighresolutiontimersinfrastructure,mergedin2.6.21, allowstousetheavailablehardwaretimerstoprograminterrupts attherightmoment.

Hardwaretimersaremultiplexed,sothatasinglehardwaretimeris sufficienttohandlealargenumberofsoftwareprogrammedtimers. UsabledirectlyfromuserspaceusingtheusualtimerAPIs
27
Threadedinterrupts
Tosolvetheinterruptinversionproblem,PREEMPT_RThas introducedtheconceptofthreadedinterrupts Theinterrupthandlersruninnormalkernelthreads,sothatthe prioritiesofthedifferentinterrupthandlerscanbeconfigured Therealinterrupthandler,asexecutedbytheCPU,isonlyin chargeofmaskingtheinterruptandwakingupthecorresponding thread Theideaofthreadedinterruptsalsoallowstousesleeping spinlocks(seelater) Mergedsince2.6.30,theconversionofinterrupthandlersto threadedinterruptsisnotautomatic:driversmustbemodified InPREEMPT_RT,allinterrupthandlersareswitchedtothreaded interrupts
28
PREEMPT_RTspecifics
29
CONFIG_PREEMPT_RT(1)
ThePREEMPT_RTpatchaddsanewlevelofpreemption, calledCONFIG_PREEMPT_RT Thislevelofpreemptionreplacesallkernelspinlocksbymutexes (orsocalledsleepingspinlocks)
Insteadofprovidingmutualexclusionbydisablinginterruptsand preemption,theyarejustnormallocks:whencontentionhappens, theprocessisblockedandanotheroneisselectedbythescheduler Workswellwiththreadedinterrupts,sincethreadscanblock,while usualinterrupthandlerscouldnot Somecore,carefullycontrolled,kernelspinlocksremainasnormal spinlocks
30
CONFIG_PREEMPT_RT(2)
WithCONFIG_PREEMPT_RT,virtuallyallkernelcodebecomes preemptible
Aninterruptcanoccuratanytime,whenreturningfromtheinterrupt handler,thewokenupprocesscanstartimmediately
ThisisthelastbigpartofPREEMPT_RTthatisn'tfullyinthe mainlinekernelyet
Partofithasbeenmergedin2.6.33:thespinlockannotations.The spinlocksthatmustremainasspinningspinlocksarenow differentiatedfromspinlocksthatcanbeconvertedtosleeping spinlocks.ThishasreducedalotthePREEMPT_RTpatchsize!
31
Threadedinterrupts
ThemechanismofthreadedinterruptsinPREEMPT_RTisstill differentfromtheonemergedinmainline InPREEMPT_RT,allinterrupthandlersareunconditionally convertedtothreadedinterrupts. Thisisatemporarysolution,untilinterestingdriversinmainline getgraduallyconvertedtothenewthreadedinterruptAPIthat hasbeenmergedin2.6.30.
32
SettingupPREEMPT_RT
33
PREEMPT_RTsetup(1)
PREEMPT_RTisdeliveredasapatchagainstthemainline kernel
Besttohaveaboardsupportedbythemainlinekernel,otherwise thePREEMPT_RTpatchmaynotapplyandmayrequiresome adaptations
Manyofficialkernelreleasesaresupported,butnotall.For example,2.6.31and2.6.33aresupported,butnot2.6.32. Quicksetup

Downloadandextractmainlinekernel DownloadthecorrespondingPREEMPT_RTpatch Applyittothemainlinekerneltree
34
PREEMPT_RTsetup(2)
Inthekernelconfiguration,besuretoenable
CONFIG_PREEMPT_RT Highresolutiontimers
Compileyourkernel,andboot YouarenowrunningtherealtimeLinuxkernel Ofcourse,somesystemconfigurationremainstobedone,in particularsettingappropriateprioritiestotheinterruptthreads, whichdependonyourapplication.
35
Realtimeapplicationdevelopment
36
Developmentandcompilation
Nospeciallibraryisneeded,thePOSIXrealtimeAPIispartof thestandardClibrary TheglibcoreglibcClibrariesarerecommended,asthesupport ofsomerealtimefeaturesisnotavailableyetinuClibc
PriorityinheritancemutexesorNPTLonsomearchitectures,for example
Compileaprogram
ARCHlinuxgccomyprogmyprog.clrt
TogetthedocumentationofthePOSIXAPI
Installthemanpagesposixdevpackage Runmanfunctioname
37
Process,thread?
Confusionaboutthetermsprocess,threadandtask InUnix,aprocessiscreatedusingfork()andiscomposedof
Anaddressspace,whichcontainstheprogramcode,data,stack, sharedlibraries,etc. Onethread,thatstartsexecutingthemain()function. Uponcreation,aprocesscontainsonethread
Additionalthreadscanbecreatedinsideanexistingprocess, usingpthread_create()
Theyruninthesameaddressspaceastheinitialthreadofthe process Theystartexecutingafunctionpassedasargumentto pthread_create()
38
Process,thread:kernelpointofview
Thekernelrepresentseachthreadrunninginthesystembya structureoftypetask_struct Fromaschedulingpointofview,itmakesnodifferencebetween theinitialthreadofaprocessandalladditionalthreadscreated dynamicallyusingpthread_create()
Thread A Addressspace
Processafterfork()
Thread A
Thread B
Addressspace
Sameprocessafterpthread_create() 39
Creatingthreads
LinuxsupportthePOSIXthreadAPI Tocreateanewthread
pthread_create(pthread_t*thread, pthread_attr_t*attr, void*(*routine)(*void*), void*arg); Thenewthreadwillruninthesameaddressspace,butwillbe scheduledindependently
Exitingfromathread
pthread_exit(void*value_ptr);
Waitingforathreadtermination
pthread_join(pthread_t*thread,void**value_ptr);
40
Schedulingclasses(1)
TheLinuxkernelschedulersupportdifferentschedulingclasses Thedefaultclass,inwhichprocessesarestartedbydefaultisa timesharingclass
Allprocesses,regardlessoftheirpriority,getsomeCPUtime TheproportionofCPUtimetheygetisdynamicandaffectedbythe nicevalue,whichrangesfrom20(highest)to19(lowest).Canbe setusingtheniceorrenicecommands
TherealtimeclassesSCHED_FIFOandSCHED_RR
ThehighestpriorityprocessgetsalltheCPUtime,untilitblocks. InSCHED_RR,roundrobinschedulingbetweentheprocessesof thesamepriority.Allmustblockbeforelowerpriorityprocessesget CPUtime. Prioritiesrangingfrom0(lowest)to99(highest)
41
Anexistingprogramcanbestartedinaspecificschedulingclass withaspecificpriorityusingthechrtcommandlinetool
Example:chrtf99./myprog
Thesched_setscheduler()APIcanbeusedtochangethe schedulingclassandpriorityofaprocess
intsched_setscheduler(pid_tpid,intpolicy, conststructsched_param*param); policycanbeSCHED_OTHER,SCHED_FIFO,SCHED_RR,etc. paramisastructurecontainingthepriority
42
Theprioritycanbesetonaperthreadbasiswhenathreadis created:
structsched_paramparm; pthread_attr_tattr; pthread_attr_init(&attr); pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED); pthread_attr_setschedpolicy(&attr,SCHED_FIFO); parm.sched_priority=42; pthread_attr_setschedparam(&attr,&parm);
Thenthethreadcanbecreatedusingpthread_create(), passingtheattrstructure. Severalotherattributescanbedefinedthisway:stacksize,etc.

43
Memorylocking
Inordertosolvethenondeterminismintroducedbyvirtual memory,memorycanbelocked
Guaranteethatthesystemwillkeepitallocated Guaranteethatthesystemhaspreloadedeverythingintomemory
mlockall(MCL_CURRENT|MCL_FUTURE);
Locksallthememoryofthecurrentaddressspace,forcurrently mappedpagesandpagesmappedinthefuture
Other,lessusefulpartsoftheAPI:munlockall,mock, munlock. Watchoutfornoncurrentlymappedpages

Stackpages Dynamicallyallocatedmemory
44
Mutexes
Allowsmutualexclusionbetweentwothreadsinthesame addressspace
Initialization/destruction pthread_mutex_init(pthread_mutex_t*mutex,const pthread_mutexattr_t*mutexattr); pthread_mutex_destroy(pthread_mutex_t*mutex); Lock/unlock pthread_mutex_lock(pthread_mutex_t*mutex); pthread_mutex_unlock(pthread_mutex_t*mutex);
Priorityinheritancemustexplictlybeactivated
pthread_mutexattr_tattr; pthread_mutexattr_init(&attr); pthread_mutexattr_getprotocol (&attr,PTHREAD_PRIO_INHERIT);
45
Timers
timer_create(clockid_tclockid, structsigevent*evp, timer_t*timerid)
Createatimer.clockidisusuallyCLOCK_MONOTONIC. sigeventdefineswhathappensupontimerexpiration:senda signalorstartafunctioninanewthread.timeridisthereturned timeridentifier.
timer_settime(timer_ttimerid,intflags, structitimerspec*newvalue, structitimerspec*oldvalue)

Configuresthetimerforexpirationatagiventime.
timer_delete(timer_ttimerid),deleteatimer clock_getres(),gettheresolutionofaclock Otherfunctions:timer_getoverrun(),timer_gettime()

46
Signals
Signalsareanasynchronousnotificationmechanism Notificationoccurseither
Bythecallofasignalhandler.Becarefulwiththelimitationsof signalhandlers! Bybeingunblockedfromthesigwait(),sigtimedwait()or sigwaitinfo()functions.Usuallybetter.
Signalbehaviourcanbeconfiguredusingsigaction() Maskofblockedsignalscanbechangedwith pthread_sigmask() Deliveryofasignalusingpthread_kill()ortgkill() AllsignalsbetweenSIGRTMINandSIGRTMAX,32signalsunder Linux.

47
Interprocesscommunication
Semaphores
Usablebetweendifferentprocessesusingnamedsemaphores sem_open(),sem_close(),sem_unlink(),sem_init(), sem_destroy(),sem_wait(),sem_post(),etc.
Messagequeues
Allowsprocessestoexchangedataintheformofmessages. mq_open(),mq_close(),mq_unlink(),mq_send(), mq_receive(),etc.
Sharedmemory
Allowsprocessestocommunicatebysharingasegmentofmemory shm_open(),ftruncate(),mmap(),munmap(), close(),shm_unlink()
48
Debuggingrealtimelatencies
49
ftraceKernelfunctiontracer
Newinfrastructurethatcanbeusedfordebuggingoranalyzing latenciesandperformanceissuesinthekernel. DevelopedbyStevenRostedt.Mergedin2.6.27. Forearlierkernels,canbefoundfromthertpreemptpatches. VerywelldocumentedinDocumentation/ftrace.txt Negligibleoverheadwhentracingisnotenabledatruntime. Canbeusedtotraceanykernelfunction! SeeourvideoofSteven'stutorialatOLS2008: http://freeelectrons.com/community/videos/conferences/
50
Usingftrace
Tracinginformationavailablethroughthedebugfsvirtualfs (CONFIG_DEBUG_FSintheKernelHackingsection) Mountthisfilesystemasfollows: mounttdebugfsnodev/debug Whentracingisenabled(seethenextslides), tracinginformationisavailablein/debug/tracing. Checkavailabletracers in/debug/tracing/available_tracers
51
Schedulinglatencytracer
CONFIG_SCHED_TRACER(KernelHackingsection) Maximumrecordedtimebetweenwakingupatopprioritytask anditsschedulingonaCPU,expressedins. Checkthatwakeupislistedin /debug/tracing/available_tracers Toselect,resetandenablethistracer: echowakeup>/debug/tracing/current_tracer echo0>/debug/tracing/tracing_max_latency echo1>/debug/tracing/tracing_enabled Letyoursystemrun,inparticularrealtimetasks. Example:chrtf5sleep1 Disabletracing: echo0>/debug/tracing/tracing_enabled Readthemaximumrecordedlatencyandthecorrespondingtrace: cat/debug/tracing/tracing_max_latency
52
Usefulreading
AboutrealtimesupportinthestandardLinuxkernel InternalsoftheRTPatch,StevenRostedt,RedHat,June2007 http://www.kernel.org/doc/ols/2007/ols2007v2pages161172.pdf Definitelyworthreading. TheRealTimeLinuxWiki:http://rt.wiki.kernel.org TheWikiWebfortheCONFIG_PREEMPT_RTcommunity, andrealtimeLinuxingeneral. Containsniceandusefuldocuments! Seealsoourbookspage.
53
Approach2 RealtimeextensionstotheLinuxkernel
54
Linuxrealtimeextensions
Threegenerations RTLinux RTAI Xenomai
realtime realtime tasks tasks
Acommonprinciple Addaextralayerbetweenthe hardwareandtheLinuxkernel, tomanagerealtimetasks separately.

Linux kernel
Microkernel
Hardware
55
RTLinux
FirstrealtimeextensionforLinux,createdbyVictorYodaiken. Nice,buttheauthorfiledasoftwarepatentcoveringtheadditionofreal timesupporttogeneraloperatingsystemsasimplementedinRTLinux! ItsOpenPatentLicensedrewmanydevelopersawayandfrightened users.CommunityprojectslikeRTAIandXenomainowattractmost developersandusers. February,2007:RTLinuxrightssoldtoWindRiver. NowsupportedbyWindRiverasRealTimeCoreforWindRiverLinux. FreeversionstilladvertisedbyWindRiveronhttp://www.rtlinuxfree.com, butnolongeracommunityproject.
56
RTAI
http://www.rtai.org/RealTimeApplicationInterfaceforLinux Createdin1999,byProf.PaoloMontegazza(longtime contributortoRTLinux),DipartimentodiIngegneria AerospazialePolitecnicodiMilano(DIAPM). Communityproject.Significantuserbase. AttractedcontributorsfrustratedbytheRTLinuxlegalissues. Onlyreallyactivelymaintainedonx86 MayofferslightlybetterlatenciesthanXenomai,atthe expenseofalessmaintainableandlessportablecodebase SinceRTAIisnotreallymaintainedonARMandother embeddedarchitectures,ourpresentationisfocusedon Xenomai.
57
Xenomaiproject
http://www.xenomai.org/ Startedin2001asaprojectaimingatemulating traditionalRTOS. Initialgoals:facilitatetheportingofprogramstoGNU/Linux. InitiallyrelatedtotheRTAIproject(astheRTAI/fusion branch),nowindependent. SkinsmimickingtheAPIsoftraditional RTOSsuchasVxWorks,pSOS+,andVRTXsaaswellasthe POSIXAPI,andanativeAPI. Aimsatworkingbothasacokernelandontopof PREEMPT_RTintheupcoming3.0branch. Willneverbemergedinthemainlinekernel.
58
Xenomaiarchitecture
Linuxapplication glibc VxWorksapplication glibc Xenomai libvxworks POSIXapplication glibc Xenomai libpthread_rt
Systemcalls
VFS Memory
Network ...
XenomaiRTOS (nucleus)
Linux kernelspace
Piecesadded byXenomai Xenomai skins
AdeosIPipe
59
TheAdeosinterruptpipelineabstraction
FromAdeospointofview,guestOSesareprioritizeddomains. Foreachevent(interrupts,exceptions,syscalls,etc...),the variousdomainsmayhandletheeventorpassitdownthe pipeline.
60
Adeosvirtualizedinterruptsdisabling
Eachdomainmaybestalled,meaningthatitdoesnotaccept interrupts. Hardwareinterrupts arenotdisabled however(except forthedomain leadingthepipeline), insteadtheinterrupts receivedduringthat timeareloggedand replayedwhenthe domainisunstalled.
61
Adeosadditionalfeatures
TheAdeosIpipepatchimplementadditionalfeatures,essential fortheimplementationoftheXenomairealtimeextension:
Disablesondemandmappingofkernelspacevmalloc/ioremap areas. Disablescopyonwritewhenrealtimeprocessesareforking. AllowsubscribingtoeventallowingtofollowprogressoftheLinux kernel,suchasLinuxsystemcalls,contextswitches,process destructions,POSIXsignals,FPUfaults. OntheARMarchitectures,integratestheFCSEpatch,whichallows toreducethelatencyinducedbycacheflushesduringcontext switches.
62
Xenomaifeatures
Factoredrealtimecorewithskinsimplementingvariousrealtime APIs Seamlesssupportforhardrealtimeinuserspace Nosecondclasscitizen,allportsareequivalentfeaturewise Xenomaisupportisasmuchaspossibleindependentfromthe Linuxkernelversion(backwardandforwardcompatiblewhen reasonable) EachXenomaibranchhasastableuser/kernelABI Timersystembasedonhardwarehighresolutiontimers Perskintimebasewhichmaybeperiodic RTDMskinallowingtowriterealtimedrivers
63
Xenomaiuserspacerealtimesupport.
Xenomaisupportsrealtimeinuserspaceon5architectures, including32and64bitsvariants. Twomodesaredefinedforathread
theprimarymode,wherethethreadishandledbyXenomai scheduler thesecondarymode,whenitishandledbyLinuxscheduler.
ThankstotheservicesoftheAdeosIpipeservice,Xenomai systemcallsaredefined.
Athreadmigratesfromsecondarymodetoprimarymodewhen suchasystemcallisissued ItmigratesfromprimarymodetosecondarymodewhenaLinux systemcallisissued,ortohandlegracefullyexceptionalevents suchasexceptionsorLinuxsignals.
64
LifeofaXenomaiapplication
XenomaiapplicationsarestartedlikenormalLinuxprocesses, theyareinitiallyhandledbytheLinuxschedulerandhaveaccess toallLinuxservices Aftertheirinitialization,theydeclarethemselvesasrealtime application,whichmigratesthemtoprimarymode.Inthismode:
TheyarescheduleddirectlybytheXenomaischeduler,sothey havetherealtimepropertiesofferedbyXenomai Theydon'thaveaccesstoanyLinuxservice,otherwisetheyget migratedbacktosecondarymodeandloosesallrealtime properties TheycanonlyusedevicedriversthatareimplementedinXenomai, nottheonesoftheLinuxkernel
NeedtoimplementdevicedriversinXenomai,andtosplitreal timeandnonrealtimepartsofyourapplications.
65
RealTimeDriverModel(RTDM)
Anapproachtounifytheinterfacesfordevelopingdevicedrivers andassociatedapplicationsunderrealtimeLinux
AnAPIverysimilartothenativeLinuxkerneldriverAPI
Allowsthedevelopment,inkernelspace,of
Characterstyledevicedrivers Networkstyledevicedrivers
Seethewhitepaperon
http://www.xenomai.org/documentation/xenomai2.4/pdf/RTDMandApplications.pdf
CurrentnotableRTDMbaseddrivers:
Serialportcontrollers; RTnetUDP/IPstack; RTsocketCAN,driversforCANcontrollers; Analogy,forkoftheComedyproject,driversforacquisitioncards.
66
SettingupXenomai
67
HowtobuildXenomai
DownloadXenomaisourcesat http://download.gna.org/xenomai/stable/ DownloadoneoftheLinuxversionssupportedbythisrelease (seeksrc/arch/<arch>/patches/) Sinceversion2.0,splitkernel/userbuildingmodel. Kernelusesascriptcalledscript/preparekernel.shwhich integratesXenomaikernelspacesupportintheLinuxsources. Runthekernelconfigurationmenu.
68
LinuxoptionsforXenomaiconfiguration
69
Xenomaiuserspacesupport
Userspacelibrariesarecompiledusingthetraditionalautotools ./configuretarget=armlinux&&make&& makeDESTDIR=/your/rootfs/install Thexenoconfigscript,installedwheninstallingXenomaiuser spacesupporthelpsyoucompilingyourownprograms. SeeXenomai'sexamplesdirectory. InstallationdetailsmaybefoundintheREADME.INSTALLguide. ForanintroductiononprogrammingwiththenativeAPI,see:
http://www.xenomai.org/documentation/branches/v2.3.x/pdf/NativeAPITourrevC.pdf
ForanintroductiononprogrammingwiththePOSIXAPI,see:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai
70
DevelopingapplicationsonXenomai
71
ThePOSIXskin
ThePOSIXskinallowstorecompilewithoutchangesatraditional POSIXapplicationsothatinsteadofusingLinuxrealtime services,itusesXenomaiservices
Clocksandtimers,conditionvariables,messagequeues,mutexes, semaphores,sharedmemory,signals,threadmanagement GoodforexistingcodeorprogrammersfamiliarwiththePOSIXAPI
Ofcourse,iftheapplicationusesanyLinuxservicethatisn't availableinXenomai,itwillswitchbacktosecondarymode TolinkanapplicationagainstthePOSIXskin

DESTDIR=/path/to/xenomai/ exportDESTDIR CFL=`$DESTDIR/bin/xenoconfigposixcflags` LDF=`$DESTDIR/bin/xenoconfigposixldflags` ARCHgcc$CFLorttestrttest.c$LDF
72
Communicationwithanormaltask
IfaXenomairealtimeapplicationusingthePOSIXskinwishesto communicatewithaseparatenonrealtimeapplication,itmust usethertipcmechanism IntheXenomaiapplication,createanIPCPROTO_XDDPsocket
socket(AF_RTIPC,SOCK_DGRAM,IPCPROTO_XDDP); setsockopt(s,SOL_RTIPC,XDDP_SETLOCALPOOL,&poolsz, sizeof(poolsz)); memset(&saddr,0,sizeof(saddr)); saddr.sipc_family=AF_RTIPC; saddr.sipc_port=MYAPPIDENTIFIER; ret=bind(s,(structsockaddr*)&saddr,sizeof(saddr));
AndthenthenormalsocketAPIsendto()/recvfrom()
IntheLinuxapplication
Open/dev/rtpX,whereXistheXDDPport Useread()andwrite()
73
ThenativeAPI(1)
AXenomaispecificAPIfordevelopingrealtimetasks
Usablebothinuserspaceandkernelspace.Developmentoftasks inuserspaceisthepreferredway. MorecoherentandmoreflexibleAPIthanthePOSIXAPI.Easierto learnandunderstand.Certainlythewaytogofornewapplications.
Applicationsshouldinclude<native/service.h>,where servicecanbealarm,buffer,cond,event,heap, intr,misc,mutex,pipe,queue,sem,task,timer Tocompileapplications:

DESTDIR=/path/to/xenomai/ exportDESTDIR CFL=`$DESTDIR/bin/xenoconfigxenocflags` LDF=`$DESTDIR/bin/xenoconfigxenoldflags` ARCHgcc$CFLorttestrttest.c$LDFlnative
74
ThenativeAPI(2)
Taskmanagementservices
rt_task_create(),rt_task_start(), rt_task_suspend(),rt_task_resume(), rt_task_delete(),rt_task_join(),etc.
Countingsemaphoreservices
rt_sem_create(),rt_sem_delete(),rt_sem_p(), rt_sem_v(),etc.
Messagequeueservices
rt_queue_create(),rt_queue_delete(), rt_queue_alloc(),rt_queue_free(), rt_queue_send(),rt_queue_receive(),etc.
Mutexservices
rt_mutex_create(),rt_mutex_delete(), rt_mutex_acquire(),rt_mutex_release(),etc.
75
ThenativeAPI(3)
Alarmservices
rt_alarm_create(),rt_alarm_delete(), rt_alarm_start(),rt_alarm_stop(), rt_alarm_wait(),etc.
Memoryheapservices
Allowstosharememorybetweenprocessesand/ortopreallocate apoolofmemory rt_heap_create(),rt_heap_delete(), rt_heap_alloc(),rt_heap_bind()
Conditionvariableservices
rt_cond_create(),rt_cond_delete(), rt_cond_signal(),rt_cond_broadcast(), rt_cond_wait(),etc.
76
Xenomaiandnormaltaskcommunication
Usingrt_pipes InthenativeXenomaiapplication,usethePipeAPI
rt_pipe_create(),rt_pipe_delete(), rt_pipe_receive(),rt_pipe_send(), rt_pipe_alloc(),rt_pipe_free()
InthenormalLinuxapplication
Openthecorresponding/dev/rtpXfile,theminorisspecifiedat rt_pipe_create()time Then,justread()andwrite()totheopenedfile
Xenomaiapplication
Usesthert_pipe_*()API
Linuxapplication
open(/dev/rtpX)
77
Realtimeapproaches
ThefollowingtableisPaulMacKenney'ssummaryofhisown articledescribingthevariousapproachesforrealtimeonLinux:
Approach VanillaLinux PREEMPT NestedOS (cokernel) DualOS/DualCore (ASMP) Quality 10sofms allservices 100sofus Schd,Int ~10us RTOSsvcs <1us RTOSsvcs Inspection All preemptor irqdisable RTOS, hwirqdisable API POSIX+RT Complexity N/A Fault isolation None None Good HW/SW Configs All All All
RTOS preemptandirq disable(most intsinprocessctx), 10sofus (mostlydrivers) PREEMPT_RT Schd,Int POSIX+RT MigrationbetweenOSes ?us RTOS, RTOS(can RTOSsvcs hwirqdisable bePOSIXRT) ?us Sched, RTOSsvcs RTOSsvcs MigrationwithinOS POSIX+RT (additionsinblue)
POSIX+RT N/A RTOS(can bePOSIXRT) Dualenv. RTOS(can bePOSIXRT) Dualenv.
Excellent Specialized All(except some drivers) All All?
"Modest"patch (carefultuning) None Dualenv.(easy mix) OK Smallpatch None
Fullstoryathttp://lwn.net/Articles/143323
78
Books
BuildingEmbeddedLinuxSystems,O'Reilly ByKarimYaghmour,JonMasters, GiladBenYossef,PhilippeGerumandothers (includingMichaelOpdenacker),August2008 AnicecoverageofXenomai(PhilippeGerum) andtheRTpatch(StevenRostedt) http://oreilly.com/catalog/9780596529680/
79
Organizations
http://www.realtimelinuxfoundation.org/ CommunityportalforrealtimeLinux. Organizesayearlyworkshop. http://www.osadl.org OpenSourceAutomationDevelopmentLab(OSADL) CreatedasanequivalentofOSDLformachineandplantcontrol systems.MembercompaniesareGermansofar(ThomasGleixner isonboard).Oneoftheirgoalsistosupportsthedevelopmentof RTpreemptpatchesinthemainlineLinuxkernel(HOWTOs,live CD,patches).
80
Hotpluggingwithudev
Hotplugging withudev
MichaelOpdenacker FreeElectrons
Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/udev Corrections,suggestions,contributionsandtranslationsarewelcome!
/devissuesandlimitations
OnRedHat9,18000entriesin/dev! Allentriesforallpossibledevices hadtobecreatedatsysteminstallation. Neededanauthoritytoassignmajornumbers http://lanana.org/:LinuxAssignedNamesandNumbers Authority Notenoughnumbersin2.4,limitsextendedin2.6. Userspaceneitherknewwhatdeviceswerepresentinthe system,norwhichrealdevicecorrespondedtoeach/dev entry.
2
Theudevsolution
TakesadvantageofsysfsintroducedbyLinux2.6. CreatedbyGregKroahHartman,ahugecontributor. Otherkeycontributors:KaySievers,DanStekloff. Entirelyinuserspace. Automaticallycreates/removesdeviceentries in/dev/accordingtoinserted/removeddevices. Majorandminordevicetransmittedbythekernel. Requiresnochangetodrivercode. Fast:writteninC Smallsize:udevdversion108:61KBinUbuntu7.04
3
Startingudev(1)
Attheverybeginningofuserspacestartup, mountthe/dev/directoryasatmpfsfilesystem: sudomountttmpfsudev/dev /dev/ispopulatedwithstaticdevicesavailablein /lib/udev/devices/:
Ubuntu6.10example:
crw1rootroot5,12007013104:18console lrwxrwxrwx1rootroot112007013104:18core>/proc/kcore lrwxrwxrwx1rootroot132007013104:18fd>/proc/self/fd crwr1rootkmem1,22007013104:18kmem brw1rootroot7,02007013104:18loop0 lrwxrwxrwx1rootroot132007013104:18MAKEDEV>/sbin/MAKEDEV drwxrxrx2rootroot40962007013104:18net crw1rootroot1,32007013104:18null crw1rootroot108,02007013104:18ppp drwxrxrx2rootroot40962006101614:39pts drwxrxrx2rootroot40962006101614:39shm lrwxrwxrwx1rootroot242007013104:18sndstat>/proc/asound/oss/sndstat lrwxrwxrwx1rootroot152007013104:18stderr>/proc/self/fd/2 lrwxrwxrwx1rootroot152007013104:18stdin>/proc/self/fd/0 lrwxrwxrwx1rootroot152007013104:18stdout>/proc/self/fd/1
4
Startingudev(2)
Theudevddaemonisstarted. Itlistenstoueventsfromthedrivercore, whicharesentwheneverdevicesareinsertedorremoved. Theudevddaemonreadsandparsesalltherulesfoundin /etc/udev/rules.d/ andkeepstheminmemory. Wheneverrulesareadded,removedormodified, udevdreceivesaninotifyeventandupdatesits rulesetinmemory. Whenaneventisreceived,udevdstartsaprocessto: trytomatchtheeventagainstudevrules, create/removedevicefiles, andrunprograms(toload/removeadriver,tonotifyuserspace...)
Theinotifymechanismlets userspaceprogramssubscribe tonotificationsoffilesystem changes.Possibilitytowatch individualfilesordirectories.
5
ueventmessageexample
ExampleinsertingaUSBmouse
recv(4, //socketid "add@/class/input/input9/mouse2\0 //message ACTION=add\0 //actiontype DEVPATH=/class/input/input9/mouse2\0 //pathin/sys SUBSYSTEM=input\0 //subsystem(class) SEQNUM=1064\0 //sequencenumber PHYSDEVPATH=/devices/pci0000:00/0000:00:1d.1/usb2/22/22:1.0\0 //devicepathin/sys PHYSDEVBUS=usb\0 //bus PHYSDEVDRIVER=usbhid\0 //driver MAJOR=13\0 //majornumber MINOR=34\0", //minornumber 2048, //messagebuffersize 0) //flags =221 //actualmessagesize
6
udevrules
Whenaudevrulematchingeventinformationisfound, itcanbeused: Todefinethenameandpathofadevicefile. Todefinetheowner,groupandpermissionsofadevicefile. Toexecuteaspecifiedprogram. Rulefilesareprocessedinlexicalorder.
7
udevnamingcapabilities
Devicenamescanbedefined fromalabelorserialnumber, fromabusdevicenumber, fromalocationonthebustopology, fromakernelname, fromtheoutputofaprogram. Seehttp://www.reactivated.net/writing_udev_rules.html foraverycompletedescription.Seealsomanudev.
8
udevnamingruleexamples
#Namingtestingtheoutputofaprogram BUS=="scsi",PROGRAM="/sbin/scsi_id",RESULT=="OEM0815",NAME="disk1" #USBprintertobecalledlp_color BUS=="usb",SYSFS{serial}=="W09090207101241330",NAME="lp_color" #SCSIdiskwithaspecificvendorandmodelnumberwillbecalledboot BUS=="scsi",SYSFS{vendor}=="IBM",SYSFS{model}=="ST336",NAME="boot%n" #soundcardwithPCIbusid00:0b.0tobecalleddsp BUS=="pci",ID=="00:0b.0",NAME="dsp" #USBmouseatthirdportofthesecondhubtobecalledmouse1 BUS=="usb",PLACE=="2.3",NAME="mouse1" #ttyUSB1shouldalwaysbecalledpdawithtwoadditionalsymlinks KERNEL=="ttyUSB1",NAME="pda",SYMLINK="palmtophandheld" #multipleUSBwebcamswithsymlinkstobecalledwebcam0,webcam1,... BUS=="usb",SYSFS{model}=="XV3",NAME="video%n",SYMLINK="webcam%n"
9
udevpermissionruleexamples
Excerptsfrom/etc/udev/rules.d/40permissions.rules
#Blockdevices SUBSYSTEM!="block",GOTO="block_end" SYSFS{removable}!="1",GROUP="disk" SYSFS{removable}=="1",GROUP="floppy" BUS=="usb",GROUP="plugdev" BUS=="ieee1394",GROUP="plugdev" LABEL="block_end" #Otherdevices,byname KERNEL=="null",MODE="0666" KERNEL=="zero",MODE="0666" KERNEL=="full",MODE="0666"
10
Identifyingdevicedrivermodules
Kernel/modulecompiling
Eachdriverannounceswhichdeviceandvendor idsitsupports.Informationstoredinmodulefiles. Thedepmodacommandprocesses modulefilesandgenerates /lib/modules/<version>/modules.alias
Systemeverydaylife
Thedrivercore(usb,pci...)readsthedeviceid, vendoridandotherdeviceattributes.
Thekernelsendsaneventtoudevd,settingthe MODALIASenvironmentvariable,encodingthesedata.
Audeveventprocessruns modprobe$MODALIAS
modprobefindsthemoduletoload inthemodules.aliasfile.
11
Modulealiases
MODALIASenvironmentvariableexample(USBmouse): MODALIAS=usb:v046DpC03Ed2000dc00dsc00dp00ic03isc01ip02 Matchinglinein/lib/modules/<version>/modules.alias: aliasusb:v*p*d*dc*dsc*dp*ic03isc01ip02*usbmouse
12
udevmodproberuleexamples
Evenmoduleloadingisdonewithudev! Excerptsfrom/etc/udev/rules.d/90modprobe.rules
ACTION!="add",GOTO="modprobe_end" SUBSYSTEM!="ide",GOTO="ide_end" IMPORT{program}="ide_mediaexport$devpath" ENV{IDE_MEDIA}=="cdrom", RUN+="/sbin/modprobeQbaidecd" ENV{IDE_MEDIA}=="disk", RUN+="/sbin/modprobeQbaidedisk" ENV{IDE_MEDIA}=="floppy",RUN+="/sbin/modprobeQbaidefloppy" ENV{IDE_MEDIA}=="tape",RUN+="/sbin/modprobeQbaidetape" LABEL="ide_end" SUBSYSTEM=="input",PROGRAM="/sbin/grepmapudev",\ RUN+="/sbin/modprobeQba$result" #Loaddriversthatmatchkernelsuppliedalias ENV{MODALIAS}=="?*",RUN+="/sbin/modprobeQ$env{MODALIAS}"
13
Coldplugging
Issue:loosingalldeviceeventshappeningduringkernel initialization,becauseudevisnotreadyyet. Solution:afterstartingudevd,havethekernelemitueventsfor alldevicespresentin/sys. Thiscanbedonebytheudevtriggerutility. Strongbenefit:completelytransparentforuserspace. Legacyandremovabledeviceshandledandnamedinexactlythe sameway.
14
Debuggingeventsudevmonitor(1)
udevadmmonitorvisualizesthedrivercoreevents andtheudeveventprocesses. ExampleeventsequenceconnectingaUSBmouse:
UEVENT[1170452995.094476]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2 UEVENT[1170452995.094569]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UEVENT[1170452995.098337]add@/class/input/input28 UEVENT[1170452995.098618]add@/class/input/input28/mouse2 UEVENT[1170452995.098868]add@/class/input/input28/event4 UEVENT[1170452995.099110]add@/class/input/input28/ts2 UEVENT[1170452995.099353]add@/class/usb_device/usbdev4.30 UDEV[1170452995.165185]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2 UDEV[1170452995.274128]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UDEV[1170452995.375726]add@/class/usb_device/usbdev4.30 UDEV[1170452995.415638]add@/class/input/input28 UDEV[1170452995.504164]add@/class/input/input28/mouse2 UDEV[1170452995.525087]add@/class/input/input28/event4 UDEV[1170452995.568758]add@/class/input/input28/ts2
Itgivestimeinformationmeasuredinmicroseconds. Youcanmeasuretimeelapsedbetweentheuevent(UEVENTline),andthe completionofthecorrespondingudevprocess(matchingUDEVline).

15
Debuggingeventsudevmonitor(2)
udevadmmonitorenv showsthecompleteeventenvironmentforeachline.
UDEV[1170453642.595297]add@/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 UDEV_LOG=3 ACTION=add DEVPATH=/devices/pci0000:00/0000:00:1d.7/usb4/43/43.2/43.2:1.0 SUBSYSTEM=usb SEQNUM=3417 PHYSDEVBUS=usb DEVICE=/proc/bus/usb/004/031 PRODUCT=46d/c03d/2000 TYPE=0/0/0 INTERFACE=3/1/2 MODALIAS=usb:v046DpC03Dd2000dc00dsc00dp00ic03isc01ip02 UDEVD_EVENT=1
16
Miscudevutilities
udevinfo Letsusersquerytheudevdatabase. udevtest<sysfs_device_path> Simulatesaudevruntotesttheconfiguredrules.
17
Firmwarehotplugging
Alsoimplementedwithudev! Firmwaredataarekeptoutsidedevicedrivers Maynotbelegalorfreeenoughtodistribute Firmwareinkernelcodewouldoccupymemory permanently,evenifjustusedonce. Kernelconfiguration:needstobesetin CONFIG_FW_LOADER (DeviceDrivers>GenericDriverOptions>hotplug firmwareloadingsupport)
18
Firmwarehotpluggingimplementation
Kernelspace
Driver
callsrequest_firmware() Sleeps
Userspace
/sys/class/firmware/xxx/{loading,data} appear
firmwaresubsystemeventsenttoudev Calling/lib/udev/firmware_helper Kernel

Getreadytoloadfirmwaredata Growsabuffertoaccommodateincomingdata
/lib/udev/firmware_helper
echo1>/sys/class/firmware/xxx/loading catfw_image>/sys/class/firmware/xxx/data echo0>/sys/class/firmware/xxx/loading
Driver
wakesupafterrequest_firmware() Copiesthebuffertothehardware Callsrelease_firmware()
SeeDocumentation/firmware_class/foraniceoverview
19
udevfiles
/etc/udev/udev.conf udevconfigurationfile. Mainlyusedtoconfiguresyslogreportingpriorities. Examplesetting:udev_log="err" /lib/udev/rules.d/ Standardudeveventmatchingrules,installedbythedistribution. /etc/udev/rules.d/*.rules Local(custom)udeveventmatchingrules.Besttomodifythese. /lib/udev/devices/* static/devcontent(suchas/dev/console,/dev/null...). /lib/udev/* helperprogramscalledfromudevrules. /dev/* Createddevicefiles.
20
Kernelconfigurationforudev
Createdfor2.6.19 Caution:nodocumentationfound,andnottestedyetonaminimalisticsystem. Somesettingsmaystillbemissing. Subsystemsanddevicedrivers(USB,PCI,PCMCIA...)shouldbeaddedtoo! #Generalsetup CONFIG_HOTPLUG=y #Networking,networkingoptions CONFIG_NET=y CONFIG_UNIX=y CONFIG_NETFILTER_NETLINK=y CONFIG_NETFILTER_NETLINK_QUEUE=y #Pseudofilesystems CONFIG_PROC_FS=y CONFIG_SYSFS=y CONFIG_TMPFS=y CONFIG_RAMFS=y
Unixdomainsockets
Neededtomanage/dev
21
udevsummarytypicaloperation
Kerneldrivercore (usb,pci...) uevent
udevd
udeveventprocess Matcheseventtorules Creates/removes devicefiles /lib/udev/programsorothers Loadtherightmodule Notifyuserspace programs(GUI...)
22
udevresources
Homepage http://kernel.org/pub/linux/utils/kernel/hotplug/udev.html Sources http://kernel.org/pub/linux/utils/kernel/hotplug/ Theudevmanualpage: manudev
23
mdev,theudevforembeddedsystems
udevmightbetooheavyweightforsomeembeddedsystems, theudevddaemonstayinginthebackgroundwaitingforevents. BusyBoxprovidesasimpleralternativecalledmdev,availableby enablingtheMDEVconfigurationoption. mdev'susageisdocumentedindoc/mdev.txtintheBusyBox sourcecode. mdevisalsoabletoloadfirmwaretothekernellikeudev
24
mdevusage
Tousemdev,theprocandsysfsfilesystemsmustbemounted mdevmustbeenabledasthehotplugeventmanager
echo/sbin/mdev>/proc/sys/kernel/hotplug
Needtomount/devasatmpfs: mountttmpfsmdev/dev Tellmdevtocreatethe/deventriescorrespondingtothe devicesdetectedduringbootwhenmdevwasnotrunning:

mdevs
Thebehaviorisspecifiedbythe/etc/mdev.conf configurationfile,withthefollowingformat
<deviceregex><uid>:<gid><octalpermissions> [=path][@|$|*<command>]
Example
hd[az][09]*0:3660
25
Archiveyourlabdirectory
Cleanupfilesthatareeasyto retrieve,removedownloads. Generateanarchiveofyourlab directory.
1
Relateddocuments
Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

2
Lifeaftertraining
HerearethingswecoulddotosupportyouinyourembeddedLinux andkernelprojects: BSPdevelopmentforyourhardware (drivers,bootloader,toolchain) MaketheofficialLinuxsourcessupportyourhardware Systemdevelopmentandintegration Systemoptimization Huntingandfixingnastybugs Moretraining:seehttp://freeelectrons.com/training/.Your colleagueswhomissedthisclasscouldgotoourpublicsessions. Seehttp://freeelectrons.com/development andhttp://freeelectrons.com/services
3
Lastslides
Thankyou!
AndmaytheSourcebewithyou
4

Beagleboard

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Beagleboard

Încărcat de

Drepturi de autor:

Formate disponibile

FreeElectrons

Corrections,suggestions, contributionsandtranslationsarewelcome! Latestupdate:May31,2011

512MBRAM,512MBflash Ethernet,Wifi,Bluetooth RS232serial USBHost,USBOTG DVID,AudioInandOut microSD Expansionconnectors:UART,SPI,I2C,GPIO,keyboard,LCD Approximately130EUR http://www.igep.es

Introduction toembedded Linux

Copyright20092011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!

Thosefreedomsaregrantedforbothcommercialandnon commercialuse Theyimplytheavailabilityofsourcecode,softwarecanbe modifiedanddistributedtocustomers Goodmatchforembeddedsystems!

EmbeddedLinuxistheusage oftheLinuxkernelandvarious opensourcecomponentsin embeddedsystems

Manyopensourcecomponentsarewidelyused,onmillionsof systems Usuallyhigherqualitythanwhataninhousedevelopmentcan produce,orevenproprietaryvendors Ofcourse,notallopensourcecomponentsareofgoodquality, butmostofthewidelyusedonesare. Allowstodesignyoursystemwithhighqualitycomponents atthefoundations

Opensourcebeingfreelyavailable,itiseasytogetoneand evaluateit Allowstoeasilystudyseveraloptionswhilemakingachoice Mucheasierthanpurchasinganddemonstrationprocedures neededwithmostproprietaryproducts Allowstoeasilyexplorenewpossibilitiesandsolutions

Notnecessarilyinterestingtobetoorestrictiveontheamountof RAM/storage:havingflexibilityatthislevelallowstoreuseas manyexistingcomponentsaspossible.

StandardClibrary Linuxkernel Bootloader Hardware

EmbeddedLinux development environment

Copyright20092011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/embeddedlinuxintro Corrections,suggestions,contributionsandtranslationsarewelcome!

WehavechosenUbuntu,asitisa widelyusedandeasytousedesktop Linuxdistribution TheUbuntusetuponthetraininglaptops hasintentionallybeenleftuntouched afterthenormalinstallationprocess. LearningembeddedLinuxisalsoabout learningthetoolsneededonthe developmentworkstation!

OnanUbuntusystem,itisnotpossibletologinasroot,onlyas anormaluser. Thesystemhasbeenconfiguredsothattheuseraccount createdfirstisallowedtorunpriviledgedoperationsthrougha programcalledsudo.

Packagesarestoredinrepositories,usuallyonHTTPorFTP servers Oneshouldonlyusepackagesfromofficialrepositoriesofits distribution,unlessstrictlyrequired.

Theyareconnectedbyvariousmeans:almostalwaysaserial linefordebuggingpurposes,frequentlyanEthernetconnection, sometimesaJTAGforlowleveldebugging Host

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/toolchains Corrections,suggestions,contributionsandtranslationsarewelcome!

Kernel Kernelheaders CLibrary Applications

Therefore,compilingtheClibrary requireskernelheaders,andmany applicationsalsorequirethem.

Availablein<linux/...>and<asm/...>andafewother directoriescorrespondingtotheonesvisibleininclude/inthe kernelsources

SeveralClibrariesareavailable: glibc,uClibc,eglibc,dietlibc,newlib,etc. ThechoiceoftheClibrarymustbemadeat thetimeofthecrosscompilingtoolchain generation,astheGCCcompileriscompiled againstaspecificClibrary.

Cprogram Plainhelloworld (stripped) Busybox (stripped)

usedtobuildatoolchainthatrunsonyour workstationbutgeneratesbinariesforthe target Themostcommonsolutioninembedded

Forprocessorshavingafloatingpointunit,thetoolchainshould generatehardfloatcode,inordertousethefloatingpoint instructionsdirectly Forprocessorswithoutafloatingpointunit,twosolutions

Decisiontakenattoolchainconfigurationtime Alsopossibletoconfigurewhichfloatingpointunitshouldbe used

Thereisonesysrootforeachvariant:toolchainscanbemultilibif theyhaveseveralcopiesoftheClibraryfordifferent configurations(forexample:ARMv4T,ARMv5T,etc.)

Timetobuildyourtoolchain ConfigureCrosstoolNG Runittobuildyourowncross compilingtoolchain

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/bootloaders Corrections,suggestions,contributionsandtranslationsarewelcome!

Thebootloadercanthenofferallitsfeatures.Ittypically Kernel understandsfilesystemformatssothatthekernelfile from canbeloadeddirectlyfromanormalfilesystem. filesystem

Execution starts here

Triestofindavalidbootstrapimagefromvarious storagesources,andloaditintoSRAM(DRAMnot initializedyet).Sizelimitedto4KB.Nouserinteraction possibleinstandardbootmode. RunsfromSRAM.InitializestheDRAM,theNANDor SPIcontroller,andloadsthesecondarybootloaderinto RAMandstartsit.Nouserinteractionpossible.

RunsfromRAM.Initializessomeotherhardware devices(network,USB,etc.).Loadsthekernelimage fromstorageornetworktoRAMandstartsit.Shellwith commandsprovided. RunsfromRAM.Takesoverthesystemcompletely (bootloadersnolongerexists).

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/uboot Corrections,suggestions,contributionsandtranslationsarewelcome!

AssumingthatyourboardisalreadysupportedbyUBoot,there shouldbeonefilecorrespondingtoyourboard,forexample include/configs/igep0020.h ThisfilecanalsobeadjustedforaddorremovefeaturesfromU Boot

CPU configuration Memory configuration USB configuration Available commands andfeatures

MakesurethatthecrosscompilerisavailableinPATH CompileUBoot,byspecifyingthecrosscompilerprefix. Example,ifyourcrosscompilerexecutableisarmlinuxgcc: makeCROSS_COMPILE=armlinux Theresultisauboot.binfile,whichistheUBootimage

TheUBootshelloffersasetofcommands.Wewillstudy themostimportantones,seethedocumentationfora completereferenceorthehelpcommand.

NANDflash information UBoot information

Example setenvmmcboot'mmcinit0;iffatloadmmc0 80000000boot.ini;thensource;elseif fatloadmmc080000000uImage;thenrunmmc bootargs;bootm;fi;fi'

Timetostartthepracticallab! Communicatewiththeboard usingaserialconsole Configure,buildandinstallX LoaderandUBoot LearnUBootcommands SetupTFTPcommunication withtheboard

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelintro Corrections,suggestions,contributionsandtranslationsarewelcome!

LinusTorvaldshasbeenabletocreatealargeanddynamic developerandusercommunityaroundLinux. Nowadays,hundredsofpeoplecontributetoeachkernelrelease, individualsorcompaniesbigandsmall.

FortheLinuxkernel,thisbasicallyimpliesthat: WhenyoureceiveorbuyadevicewithLinuxonit, youshouldreceivetheLinuxsources,withtherightto study,modifyandredistributethem. WhenyouproduceLinuxbaseddevices,youmust releasethesourcestotherecipient,withthesamerights, withnorestriction..

Thisinterfaceisstableovertime:onlynewsystemcallscanbe addedbythekerneldevelopers ThissystemcallinterfaceiswrappedbytheClibrary,and userspaceapplicationsusuallynevermakeasystemcalldirectly butratherusethecorrespondingClibraryfunction

Note:inreality,manymoreminor versionsexistinsidethestableand developmentbranches

Copyright20042011,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:May31,2011, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/kernelusage Corrections,suggestions,contributionsandtranslationsarewelcome!

TomodifyakernelinaGNU/Linuxdistribution: theconfigurationfilesareusuallyreleasedin/boot/,together withkernelimages:/boot/config2.6.1711generic

Driveroptions CONFIG_JOLIET=y CONFIG_ZISOFS=y

# #DOS/FAT/NTFilesystems # #CONFIG_MSDOS_FSisnotset #CONFIG_VFAT_FSisnotset CONFIG_NTFS_FS=m #CONFIG_NTFS_DEBUGisnotset CONFIG_NTFS_RW=y

Bydefault,thekernelbuildsystemassumesthatthekernelis beingbuiltforthehostarchitecture,i.enativecompilation Thearchitectureisnotdefinedinsidetheconfiguration,butatan higherlevel Wewillseelaterhowtooverridethisbehaviour,toallowthe configurationofkernelsforadifferentarchitecture

I2C,SPI,1wire,supportforthepopularembeddedbuses Hardwaremonitoringsupport,infrastructureanddriversforthermal sensors

MMC/SD/SDIOsupport LEDsupport RealTimeClockdrivers Voltageandcurrentregulators Stagingdrivers,crappydriversbeingcleanedup

Runmakehelptofindifoneisavailableforyourplatform Toloadadefaultconfigurationfile,justrun makeacme_defconfig

Tocreateyourowndefaultconfigurationfile makesavedefconfig,tocreateaminimalconfigurationfile mvdefconfigarch/<arch>/myown_defconfig

makeinstallisrarelyusedinembeddeddevelopment,asthe kernelimageisasinglefile,easytohandle Itishoweverpossibletocustomizethemakeinstallbehaviour inarch/<arch>/boot/install.sh makemodules_installisusedeveninembedded development,asitinstallsmanymodulesanddescriptionfiles

Setupacrosscompiling environment ConfigurethekernelMakefile accordingly Crosscompilethekernelforanarm targetplatform Onthisplatform,interactwiththe bootloaderandbootyourkernel.

Astherootfilesystemisthefirstmountedfilesystem,itcannotbe mountedwiththenormalmountcommand Itismounteddirectlybythekernel,accordingtotheroot= kerneloption Whennorootfilesystemisavailable,thekernelpanics

Host NFS server /home/tux/rootfs/ /home/tux/rootfs/root/ /home/tux/rootfs/root/README /home/tux/rootfs/usr/ /home/tux/rootfs/bin/ /home/tux/rootfs/bin/ls

Kernel code and data

Root filesystem stored as a compressed cpio archive

Kernel image (uImage, zImage, bzImage)

Thekernelbuildprocesswillautomaticallytakethecontentsof theCONFIG_INITRAMFS_SOURCEoptionandintegratetheroot filesystemintothekernelimage