Through the Interface

September 25, 201 2

Creating a face-recognising security cam with a Raspberry Pi Part 3

In the last two posts in this series, we introduced the concept and architecture behind the Facecam, and looked at the desktop-based component to build our face recognition database (facedata.xml). In this post well take a look at the Raspberry Pi-resident face recognition engine. This component is implemented as a daemon (which is basically the Unix /Linux equiv alent of a Windows serv ice, for those unfamiliar with the term) that looks in an input folder for images to process and populates an output folder with the results of the face recognition process. In the nex t post in the series, well look at the implementation of a simple daemon to process the tex t files in this output folder, sending the tex t to an LED message-board. But back to today s topic the first thing we need to do is to install OpenCV on the Raspberry Pi, for which this post prov ed v ery helpful.
#I n s t a l lO p e n C Va n di t ss a m p l e s s u d op a c m a nSp y t h o n 2 n u m p yo p e n c vo p e n c v s a m p l e s #A d dt h ep iu s e rt ot h ev i d e og r o u p #( g o o df o rt e s t i n gw e b c a m r e l a t e ds a m p l e s ) s u d ou s e r m o daGv i d e op i

At this stage it should be possible for y ou to browse to and test the OpenCV samples, a step Im going to skip here. Nex t up, we can get the source for the daemon onto the local sy stem and build it. Now at some point Ill get around to creating a makefile for this, but as its only one .cpp file for now I hav ent bothered, although I hav e put it in a ZIP along with a couple of XML files it depends on.
s u d op a c m a nSu n z i pg c cp k g c o n f i g c d m k d i rf a c e r e c o g c df a c e r e c o g w g e t" h t t p : / / t h r o u g h t h e i n t e r f a c e . t y p e p a d . c o m / f i l e s / F a c R e c S r c . z i p " u n z i pF a c R e c S r c . z i p g + +` p k g c o n f i go p e n c vc f l a g sl i b s `F a c e R e c D a e m o n . c p pof a c e r e c d s u d oc pf a c e r e c d/ e t c / r c . d

It should now be possible to run the daemon by launching the facerecd command, but first we need to create our input and output directories:
c d m k d i r pf a c e s / i n m k d i rf a c e s / o u t m k d i rf a c e s / d e b u g f a c e r e c d

To av oid hav ing to launch this manually , we can edit the rc.conf file to hav e it launch on boot:
s u d on a n o/ e t c / r c . c o n f

Then if y ou scroll down to the last line in the file, y ou can make sure both motion and facerecd are there:
D A E M O N S = ( ! h w c l o c ks y s l o g n gn e t w o r ko p e n n t p d@ n e t f s@ c r o n d@ s s h d@ m o t i o nf a c e r e c d )

Y oull also need to edit the motion.conf file, to hook it up to the face detection component:
s u d on a n o/ e t c / m o t i o n / m o t i o n . c o n f

Y ou need to make sure motion is sav ing images, and that they get copied to the input folder. These are the settings I changed:
o u t p u tn o r m a l j p e g _ f i l e n a m e% C % v % q o n _ p i c t u r e _ s a v ec p% f/ h o m e / p i / f a c e s / i n

This breaks the uploader Py thon script we introduced way back when, but for now Im just fine with that (itll be a triv ial change in the code to strip of the frame number from the JPG filename). Lets now take a look at he C++ source for our daemon. To get started with writing the daemon, I took the boilerplate code from this article. I then took a big chunk of Sherv in Emamis original face recognition implementation, but stripped out a lot that wasnt needed (as were only reading from the database and dont need to train it, for instance). Most of this was using an earlier v ersion of the OpenCV API (which seems to be standard C functions), but I also ended up including some more recent C++ code OpenCV thankfully prov ides helpful functions to interoperate between its C and C++ APIs but that also means the code is less consistent than Id hav e liked. At some point Id like to come back and rework the C-based implementation to use C++ Im currently using a horrible mix of strings and char*, just for starters but I hav e to find the time (and right now this is working well enough for my purposes). Part of the issue stems from my decision right or wrong to code inside V isual Studio, transfer the .cpp across to the Pi v ia SFTP and then build locally using gcc. If this loop was streamlined by cross compiling directly on Windows or OSX, or ev en by editing the C++ code directly on the Pi then I might be more inclined to take the time to rework the implementation. Heres the C++ code from FaceRecDaemon.cpp :
# i n c l u d e< s y s / t y p e s . h > # i n c l u d e< s y s / s t a t . h > # i n c l u d e< s t d i o . h > # i n c l u d e< s t d l i b . h > # i n c l u d e< f c n t l . h > # i n c l u d e< e r r n o . h > # i n c l u d e< u n i s t d . h > # i n c l u d e< d i r e n t . h > # i n c l u d e< s t r i n g . h > # i n c l u d e< s t r i n g > # i n c l u d e< v e c t o r > # i n c l u d e< i o s t r e a m > # i n c l u d e< f s t r e a m > # i n c l u d e< a l g o r i t h m > # i n c l u d e< s y s l o g . h > # i n c l u d e" c v . h " # i n c l u d e" c v a u x . h " # i n c l u d e" h i g h g u i . h " # i n c l u d e" o p e n c v 2 / o p e n c v . h p p " # i f n d e fB O O L # d e f i n eB O O Lb o o l # e n d i f u s i n gn a m e s p a c es t d ; u s i n gn a m e s p a c ec v ; / /I n p u ta n do u t p u tf o l d e rl o c a t i o n s c o n s tc h a r*i n D i r=" / h o m e / p i / f a c e s / i n " ; c o n s tc h a r*o u t D i r=" / h o m e / p i / f a c e s / o u t " ; c o n s tc h a r*d e b u g D i r=" / h o m e / p i / f a c e s / d e b u g " ;

/ /H a a rC a s c a d ef i l e ,u s e df o rF a c eD e t e c t i o n c o n s tc h a r* f a c e C a s c a d e F i l e n a m e= " / h o m e / p i / f a c e r e c o g / h a a r c a s c a d e _ f r o n t a l f a c e _ a l t . x m l " ; c o n s tc h a r* l b l C a s c a d e F i l e n a m e= " / h o m e / p i / f a c e r e c o g / l b p c a s c a d e _ f r o n t a l f a c e . x m l " ; c o n s tc h a r* f a c e D a t a X m l=" / h o m e / p i / f a c e r e c o g / f a c e d a t a . x m l " ; / /S e tt o0i fy o ud o n tw a n ti m a g e so ft h eE i g e n v e c t o r s / /s a v e dt of i l e s( f o rd e b u g g i n g ) i n tS A V E _ E I G E N F A C E _ I M A G E S=1 ; / /Y o um i g h tg e tb e t t e rr e c o g n i t i o na c c u r a c yi fy o ue n a b l et h i s / / # d e f i n eU S E _ M A H A L A N O B I S _ D I S T A N C E / /G l o b a lv a r i a b l e s I p l I m a g e* *f a c e I m g A r r =0 ;/ /a r r a yo ff a c ei m a g e s C v M a t * p e r s o n N u m T r u t h M a t=0 ;/ /a r r a yo fp e r s o nn u m b e r s / /A r r a yo fp e r s o nn a m e s( i n d e x e db yt h ep e r s o nn u m b e r ) v e c t o r < s t r i n g >p e r s o n N a m e s ; / /D e f a u l td i m e n s i o n sf o rf a c e si nt h ef a c er e c o g n i t i o nd a t a b a s e i n tf a c e W i d t h=1 2 0 ; i n tf a c e H e i g h t=9 0 ; / /T h en u m b e ro fp e o p l ei nt h et r a i n i n gs e t i n tn P e r s o n s =0 ;

/ /T h en u m b e ro ft r a i n i n gi m a g e s i n tn T r a i n F a c e s / /T h en u m b e ro fe i g e n v a l u e s i n tn E i g e n s / /T h ea v e r a g ei m a g e I p l I m a g e*p A v g T r a i n I m g / /E i g e n v e c t o r s I p l I m a g e* *e i g e n V e c t A r r / /E i g e n v a l u e s C v M a t*e i g e n V a l M a t / /P r o j e c t e dt r a i n i n gf a c e s C v M a t*p r o j e c t e d T r a i n F a c e M a t=0 ; =0 ; =0 ; =0 ; =0 ; =0 ;

C v M e m S t o r a g e *s t o r a g e s t a t i cI p l I m a g e*s m a l l _ i m g d o u b l es c a l e=3 ;

=0 ; =0 ;

C a s c a d e C l a s s i f i e rf a c e C a s c a d e ; # d e f i n eP A D _ F A C E4 0 # d e f i n eP A D _ F A C E _ 28 0 / /F u n c t i o np r o t o t y p e s i n tg e t d i r ( s t r i n gd i r ,v e c t o r < s t r i n g >& f i l e s ) ; i n t l o a d T r a i n i n g D a t a ( C v M a t* *p T r a i n P e r s o n N u m M a t ) ; i n t f i n d N e a r e s t N e i g h b o r ( f l o a t*p r o j e c t e d T e s t F a c e ) ; i n tf i n d N e a r e s t N e i g h b o r ( f l o a t*p r o j e c t e d T e s t F a c e ,f l o a t* p C o n f i d e n c e ) ; b o o lr e c o g n i z e F r o m F i l e ( c o n s tc h a r* i n p u t F i l e , c o n s tc h a r* o u t p u t F i l e , C v H a a r C l a s s i f i e r C a s c a d e* c a s c a d e , f l o a t*p r o j e c t e d T e s t F a c e , C v M a t* t r a i n P e r s o n N u m M a t ) ; I p l I m a g e *r e s i z e I m a g e ( c o n s tI p l I m a g e* o r i g I m g ,i n tn e w W i d t h ,i n tn e w H e i g h t ) ; C v R e c tc r o p R e c t ( c o n s tC v R e c tr e c t ,i n tw ,i n th ) ; I p l I m a g e *c r o p I m a g e ( c o n s tI p l I m a g e* i m g ,c o n s tC v R e c tr e g i o n ) ; C v R e c td e t e c t F a c e I n I m a g e ( I p l I m a g e *i m g ,C a s c a d e C l a s s i f i e r& c a s c a d e ) ; v o i de x e c u t e ( ) ; / /G e tt h el i s to ff i l e si nad i r e c t o r y i n tg e t d i r ( s t r i n gd i r ,v e c t o r < s t r i n g >& f i l e s ) { D I R* d p ; s t r u c td i r e n t* d i r p ; i f ( ( d p=o p e n d i r ( d i r . c _ s t r ( ) ) )= =N U L L ) { c h a rm s g [ 2 0 0 ] ; s n p r i n t f ( m s g , s i z e o f ( m s g ) 1 , " E r r o r ( % d )o p e n i n g% s " , e r r n o , d i r . c _ s t r ( ) ) ; s y s l o g ( L O G _ I N F O ,m s g ) ; r e t u r ne r r n o ; } w h i l e( ( d i r p=r e a d d i r ( d p ) )! =N U L L ) { f i l e s . p u s h _ b a c k ( s t r i n g ( d i r p > d _ n a m e ) ) ; }

c l o s e d i r ( d p ) ; s o r t ( f i l e s . b e g i n ( ) ,f i l e s . e n d ( ) ) ; r e t u r n0 ; } / /O p e nt h et r a i n i n gd a t af r o mt h ef i l e' f a c e d a t a . x m l ' i n tl o a d T r a i n i n g D a t a ( C v M a t* *p T r a i n P e r s o n N u m M a t ) { / /C r e a t eaf i l e s t o r a g ei n t e r f a c e C v F i l e S t o r a g e *f i l e S t o r a g e= c v O p e n F i l e S t o r a g e ( f a c e D a t a X m l ,0 ,C V _ S T O R A G E _ R E A D ) ; i f( ! f i l e S t o r a g e ) { s y s l o g ( L O G _ I N F O ," U n a b l et oo p e nt r a i n i n gd a t a b a s e . " ) ; r e t u r n0 ; } / /L o a dt h en a m e s p e r s o n N a m e s . c l e a r ( ) ; / /M a k es u r ei ts t a r t sa se m p t y n P e r s o n s=c v R e a d I n t B y N a m e ( f i l e S t o r a g e ,0 ," n P e r s o n s " ,0 ) ; i f( n P e r s o n s= =0 ) { s y s l o g ( L O G _ I N F O ," N op e o p l ef o u n di nt r a i n i n gd a t a b a s e . " ) ; r e t u r n0 ; } / /L o a de a c hp e r s o n ' sn a m e f o r( i n ti = 0 ;i<n P e r s o n s ;i + + ) { s t r i n gs P e r s o n N a m e ; c h a rv a r n a m e [ 2 0 0 ] ; s n p r i n t f ( v a r n a m e ,s i z e o f ( v a r n a m e ) 1 ," p e r s o n N a m e _ % d " ,( i + 1 ) ) ; s P e r s o n N a m e=c v R e a d S t r i n g B y N a m e ( f i l e S t o r a g e ,0 ,v a r n a m e ) ; p e r s o n N a m e s . p u s h _ b a c k ( s P e r s o n N a m e ) ; } / /L o a dt h ed a t a n E i g e n s=c v R e a d I n t B y N a m e ( f i l e S t o r a g e ,0 ," n E i g e n s " ,0 ) ; n T r a i n F a c e s=c v R e a d I n t B y N a m e ( f i l e S t o r a g e ,0 ," n T r a i n F a c e s " ,0 ) ; * p T r a i n P e r s o n N u m M a t= ( C v M a t * ) c v R e a d B y N a m e ( f i l e S t o r a g e ,0 ," t r a i n P e r s o n N u m M a t " ,0 ) ; e i g e n V a l M a t= ( C v M a t * ) c v R e a d B y N a m e ( f i l e S t o r a g e ,0 ," e i g e n V a l M a t " ,0 ) ; p r o j e c t e d T r a i n F a c e M a t= ( C v M a t * ) c v R e a d B y N a m e ( f i l e S t o r a g e ,0 ," p r o j e c t e d T r a i n F a c e M a t " ,0 ) ; p A v g T r a i n I m g= ( I p l I m a g e * ) c v R e a d B y N a m e ( f i l e S t o r a g e ,0 ," a v g T r a i n I m g " ,0 ) ; e i g e n V e c t A r r= ( I p l I m a g e * * ) c v A l l o c ( n T r a i n F a c e s * s i z e o f ( I p l I m a g e* ) ) ;

f o r( i n ti = 0 ;i<n E i g e n s ;i + + ) { c h a rv a r n a m e [ 2 0 0 ] ; s n p r i n t f ( v a r n a m e ,s i z e o f ( v a r n a m e ) 1 ," e i g e n V e c t _ % d " ,i ) ; e i g e n V e c t A r r [ i ]= ( I p l I m a g e* ) c v R e a d B y N a m e ( f i l e S t o r a g e ,0 ,v a r n a m e ,0 ) ; } / /r e l e a s et h ef i l e s t o r a g ei n t e r f a c e c v R e l e a s e F i l e S t o r a g e ( & f i l e S t o r a g e ) ; s y s l o g ( L O G _ I N F O ," F a c er e c o g n i t i o nd a t a b a s el o a d e d . " ) ; r e t u r n1 ; } / /F i n dt h em o s tl i k e l yp e r s o nb a s e do nad e t e c t i o n . / /R e t u r n st h ei n d e x ,a n ds t o r e st h ec o n f i d e n c ev a l u e / /i n t op C o n f i d e n c e . i n tf i n d N e a r e s t N e i g h b o r ( f l o a t*p r o j e c t e d T e s t F a c e ,f l o a t* p C o n f i d e n c e ) { d o u b l el e a s t D i s t S q=D B L _ M A X ; i n ti N e a r e s t=0 ; f o r( i n ti T r a i n = 0 ;i T r a i n<n T r a i n F a c e s ;i T r a i n + + ) { d o u b l ed i s t S q=0 ; f o r( i n ti = 0 ;i<n E i g e n s ;i + + ) { f l o a td _ i= p r o j e c t e d T e s t F a c e [ i ]p r o j e c t e d T r a i n F a c e M a t > d a t a . f l [ i T r a i n*n E i g e n s+i ] ; # i f d e fU S E _ M A H A L A N O B I S _ D I S T A N C E / /M a h a l a n o b i sd i s t a n c e( m i g h tg i v eb e t t e rr e s u l t s / /t h a nE u c a l i d e a nd i s t a n c e ) d i s t S q+ =d _ i * d _ i/e i g e n V a l M a t > d a t a . f l [ i ] ; # e l s e / /E u c l i d e a nd i s t a n c e d i s t S q+ =d _ i * d _ i ; # e n d i f } i f( d i s t S q<l e a s t D i s t S q ) { l e a s t D i s t S q=d i s t S q ; i N e a r e s t=i T r a i n ; } } / /R e t u r nt h ec o n f i d e n c el e v e lb a s e do nt h eE u c l i d e a nd i s t a n c e , / /s ot h a ts i m i l a ri m a g e ss h o u l dg i v eac o n f i d e n c eb e t w e e n

/ /0 . 5t o1 . 0 ,a n dv e r yd i f f e r e n ti m a g e ss h o u l dg i v ea / /c o n f i d e n c eb e t w e e n0 . 0t o0 . 5 * p C o n f i d e n c e= 1 . 0 fs q r t ( l e a s t D i s t S q/( f l o a t ) ( n T r a i n F a c e s*n E i g e n s ) )/2 5 5 . 0 f ; / /R e t u r nt h ef o u n di n d e x r e t u r ni N e a r e s t ; } / /M a k es u r et h eg i v e nr e c t a n g l ei sc o m p l e t e l yw i t h i nt h eg i v e n / /i m a g ed i m e n s i o n s / /C r e a t e san e wi m a g ec o p yt h a ti so fad e s i r e ds i z e I p l I m a g e *r e s i z e I m a g e ( c o n s tI p l I m a g e* o r i g I m g ,i n tn e w W i d t h ,i n tn e w H e i g h t ) { i n to r i g W i d t h=0 ; i n to r i g H e i g h t=0 ; i f( o r i g I m g! =N U L L ) { o r i g W i d t h=o r i g I m g > w i d t h ; o r i g H e i g h t=o r i g I m g > h e i g h t ; } i f( o r i g I m g= =N U L L| | n e w W i d t h< =0| |n e w H e i g h t< =0| | o r i g W i d t h< =0| |o r i g H e i g h t< =0 ) { s y s l o g ( L O G _ I N F O ," E R R O Ri nr e s i z e I m a g e :B a dd e s i r e di m a g es i z e . " ) ; c l o s e l o g ( ) ; e x i t ( 1 ) ; } / /S c a l et h ei m a g et ot h en e wd i m e n s i o n s ,e v e ni ft h ea s p e c t / /r a t i ow i l lb ec h a n g e d I p l I m a g e* o u t I m g= c v C r e a t e I m a g e ( c v S i z e ( n e w W i d t h ,n e w H e i g h t ) , o r i g I m g > d e p t h , o r i g I m g > n C h a n n e l s ) ; i f( n e w W i d t h>o r i g I m g > w i d t h& &n e w H e i g h t>o r i g I m g > h e i g h t ) { / /M a k et h ei m a g el a r g e r c v R e s e t I m a g e R O I ( ( I p l I m a g e * ) o r i g I m g ) ;

/ /C V _ I N T E R _ C U B I Co rC V _ I N T E R _ L I N E A Ri sg o o df o re n l a r g i n g c v R e s i z e ( o r i g I m g ,o u t I m g ,C V _ I N T E R _ L I N E A R ) ; } e l s e { / /M a k et h ei m a g es m a l l e r c v R e s e t I m a g e R O I ( ( I p l I m a g e * ) o r i g I m g ) ; / /C V _ I N T E R _ A R E Ai sg o o df o rs h r i n k i n g / d e c i m a t i o n , / /b u tb a da te n l a r g i n g c v R e s i z e ( o r i g I m g ,o u t I m g ,C V _ I N T E R _ A R E A ) ; } r e t u r no u t I m g ; } C v R e c tc r o p R e c t ( c o n s tC v R e c tr e c t I n ,i n tw ,i n th ) { C v R e c tr o i=C v R e c t ( r e c t I n ) ; / /M a k es u r et h ed i s p l a y e di m a g ei sw i t h i nt h ev i e w i n gd i m e n s i o n s / /L i m i tt h eb o t t o m r i g h tf r o mp a s tt h ei m a g e i f( r o i . x+r o i . w i d t h>w ) r o i . w i d t h=w-r o i . x ; i f( r o i . y+r o i . h e i g h t>h ) r o i . h e i g h t=h-r o i . y ; / /L i m i tt h et o p l e f tf r o mb e f o r et h ei m a g e i f( r o i . x<0 ) r o i . x=0 ; i f( r o i . y<0 ) r o i . y=0 ; / /L i m i tt h et o p l e f tf r o ma f t e rt h ei m a g e i f( r o i . x>w 1 ) r o i . x=w 1 ; i f( r o i . y>h 1 ) r o i . y=h 1 ; / /L i m i tt h en e g a t i v es i z e s i f( r o i . w i d t h<0 ) r o i . w i d t h=0 ; i f( r o i . h e i g h t<0 ) r o i . h e i g h t=0 ; / /L i m i tt h el a r g es i z e s i f( r o i . w i d t h>w ) r o i . w i d t h=w-r o i . x ; i f( r o i . h e i g h t>h ) r o i . h e i g h t=h-r o i . y ;

r e t u r nr o i ; } / /R e t u r n san e wi m a g e ,ac r o p p e dv e r s i o no ft h eo r i g i n a l I p l I m a g e *c r o p I m a g e ( c o n s tI p l I m a g e* i m g ,c o n s tC v R e c tr e g i o n ) { C v S i z es i z e ; s i z e . h e i g h t=i m g > h e i g h t ; s i z e . w i d t h=i m g > w i d t h ; i f( i m g > d e p t h! =I P L _ D E P T H _ 8 U ) { s y s l o g ( L O G _ I N F O , " E R R O Ri nc r o p I m a g e :u n k n o w ni m a g ed e p t hg i v e ni nc r o p I m a g e ( ) . " ) ; c l o s e l o g ( ) ; e x i t ( 1 ) ; } / /F i r s tc r e a t ean e w( c o l o ro rg r e y s c a l e )I P LI m a g ea n dc o p y / /c o n t e n t so fi m gi n t oi t I p l I m a g e* i m a g e T m p= c v C r e a t e I m a g e ( s i z e ,I P L _ D E P T H _ 8 U ,i m g > n C h a n n e l s ) ; c v C o p y ( i m g ,i m a g e T m p ,N U L L ) ; / /C r e a t ean e wi m a g eo ft h ed e t e c t e dr e g i o n / /S e tr e g i o no fi n t e r e s tt ot h a ts u r r o u n d i n gt h ef a c e C v R e c tc h e c k e d=c r o p R e c t ( r e g i o n ,i m g > w i d t h ,i m g > h e i g h t ) ; c v S e t I m a g e R O I ( i m a g e T m p ,c h e c k e d ) ; / /C o p yr e g i o no fi n t e r e s t( i . e .f a c e )i n t oan e wi p l I m a g e / /( i m a g e R G B )a n dr e t u r ni t s i z e . w i d t h=c h e c k e d . w i d t h ; s i z e . h e i g h t=c h e c k e d . h e i g h t ; I p l I m a g e* i m a g e R G B= c v C r e a t e I m a g e ( s i z e ,I P L _ D E P T H _ 8 U ,i m g > n C h a n n e l s ) ; c v C o p y ( i m a g e T m p ,i m a g e R G B ,N U L L ) ; / /C o p yj u s tt h er e g i o n . c v R e l e a s e I m a g e ( & i m a g e T m p ) ; r e t u r ni m a g e R G B ; } / /P e r f o r mf a c ed e t e c t i o no nt h ei n p u ti m a g e ,u s i n gt h eg i v e nH a a r / /c a s c a d ec l a s s i f i e r .A s s u m e sg r e y s c a l ef o ri n p u ti m a g e s . / /R e t u r n sar e c t a n g l ef o rt h ed e t e c t e dr e g i o ni nt h eg i v e ni m a g e . C v R e c td e t e c t F a c e I n I m a g e ( I p l I m a g e *i m g ,C a s c a d e C l a s s i f i e r& c a s c a d e ) {

C v R e c tf o u n d _ f a c e ; i f( ! s m a l l _ i m g ) { s m a l l _ i m g= c v C r e a t e I m a g e ( c v S i z e ( c v R o u n d ( i m g > w i d t h/s c a l e ) , c v R o u n d ( i m g > h e i g h t/s c a l e ) ) , 8 ,1 ) ; } c v R e s i z e ( i m g ,s m a l l _ i m g ,C V _ I N T E R _ L I N E A R ) ; c v E q u a l i z e H i s t ( s m a l l _ i m g ,s m a l l _ i m g ) ; M a ti m g M a t ( s m a l l _ i m g ) ; / /D e t e c to b j e c t si nt h es m a l lg r a y s c a l ei m a g e . v e c t o r < R e c t >o b j e c t s ; c a s c a d e . d e t e c t M u l t i S c a l e ( i m g M a t , o b j e c t s , 1 . 1 f , 2 , C A S C A D E _ F I N D _ B I G G E S T _ O B J E C T , S i z e ( 2 0 ,2 0 ) ) ; i f( o b j e c t s . s i z e ( )>0 ) { / /F o u n da tl e a s to n ef a c e R e c tr=( R e c t ) o b j e c t s . a t ( 0 ) ; f o u n d _ f a c e . x=( i n t ) ( ( d o u b l e ) r . x*s c a l e ) ; f o u n d _ f a c e . y=( i n t ) ( ( d o u b l e ) r . y*s c a l e ) ; f o u n d _ f a c e . w i d t h=( i n t ) ( ( d o u b l e ) r . w i d t h*s c a l e ) ; f o u n d _ f a c e . h e i g h t=( i n t ) ( ( d o u b l e ) r . h e i g h t*s c a l e ) ; } e l s e { / /C o u l d n ' tf i n dt h ef a c e f o u n d _ f a c e=c v R e c t ( 1 , 1 , 1 , 1 ) ; } r e t u r nf o u n d _ f a c e ; } s t r i n gc o n v e r t ( c o n s ts t r i n g &l i n e ) { i n tl e n=l i n e . l e n g t h ( ) ; s t r i n gn L i n e=" " ; i f( l e n ) { n L i n e+ =l i n e [ 0 ] ;

f o r( i n ti=1 ;i<l e n ;+ + i ) { i f( i s u p p e r ( l i n e [ i ] ) ) n L i n e+ ='' ; n L i n e+ =l i n e [ i ] ; } } r e t u r nn L i n e ; } / /R e c o g n i z et h ep e r s o ni nt h es u p p l i e di m a g e b o o lr e c o g n i z e F r o m F i l e ( c o n s tc h a r* i n p u t F i l e , c o n s tc h a r* o u t p u t F i l e , c o n s tc h a r* d e b u g F i l e , C v H a a r C l a s s i f i e r C a s c a d e* c a s c a d e , f l o a t*p r o j e c t e d T e s t F a c e , C v M a t* t r a i n P e r s o n N u m M a t ) { / / s y s l o g ( L O G _ I N F O ," r e c o g n i z e F r o m F i l eb e g i n s " ) ; / /L o a dt h ei m a g ed i r e c t l ya sg r e y s c a l e I p l I m a g e* c h e c k I m g= c v L o a d I m a g e ( i n p u t F i l e ,C V _ L O A D _ I M A G E _ G R A Y S C A L E ) ; i f( ! c h e c k I m g ) { c h a rm s g [ 2 0 0 ] ; s n p r i n t f ( m s g , s i z e o f ( m s g ) 1 , " E R R O Ri nr e c o g n i z e F r o m F i l e ( ) :B a di n p u ti m a g ef i l e :% s " , i n p u t F i l e ) ; s y s l o g ( L O G _ I N F O ,m s g ) ; r e t u r nf a l s e ; } / /W e ' l lt r yt od e t e c tt h ef a c eu s i n gL B P C v R e c tf a c e R e c t=d e t e c t F a c e I n I m a g e ( c h e c k I m g ,f a c e C a s c a d e ) ; / /M a k es u r eav a l i df a c ew a sd e t e c t e d i f( f a c e R e c t . w i d t h>0 ) { / /G e tt h ed e t e c t e df a c ei m a g e I p l I m a g e* f a c e I m g=c r o p I m a g e ( c h e c k I m g ,f a c e R e c t ) ; / /M a k es u r et h ei m a g ei st h es a m ed i m e n s i o n sa st h e / /t r a i n i n gi m a g e s I p l I m a g e* s i z e d I m g= r e s i z e I m a g e ( f a c e I m g ,f a c e W i d t h ,f a c e H e i g h t ) ;

/ /G i v et h ei m a g eas t a n d a r db r i g h t n e s sa n dc o n t r a s t , / /i nc a s ei tw a st o od a r ko rl o wc o n t r a s t / /C r e a t ea ne m p t yg r e y s c a l ei m a g e I p l I m a g e* e q u a l i z e d I m g= c v C r e a t e I m a g e ( c v G e t S i z e ( s i z e d I m g ) ,8 ,1 ) ; c v E q u a l i z e H i s t ( s i z e d I m g ,e q u a l i z e d I m g ) ; i f( ! e q u a l i z e d I m g ) { s y s l o g ( L O G _ I N F O , " E R R O Ri nr e c o g n i z e F r o m F i l e ( ) :n oi n p u ti m a g e . " ) ; c l o s e l o g ( ) ; e x i t ( 1 ) ; } c h a ri m a g e F i l e [ 2 0 0 ] ; s t r c p y ( i m a g e F i l e ,d e b u g F i l e ) ; c h a r* d o t=s t r r c h r ( i m a g e F i l e ,' . ' ) ; * d o t=0 ; c v S a v e I m a g e ( i m a g e F i l e ,e q u a l i z e d I m g ) ; / /I ft h ef a c er e cd a t a b a s eh a sb e e nl o a d e d ,t h e nt r yt o / /r e c o g n i z et h ep e r s o nc u r r e n t l yd e t e c t e d i f( n E i g e n s>0 ) { / /P r o j e c tt h et e s ti m a g eo n t ot h eP C As u b s p a c e c v E i g e n D e c o m p o s i t e ( e q u a l i z e d I m g , n E i g e n s , e i g e n V e c t A r r , 0 ,0 , p A v g T r a i n I m g , p r o j e c t e d T e s t F a c e ) ; / /C h e c kw h i c hp e r s o ni ti sm o s tl i k e l yt ob e f l o a tc o n f i d e n c e ; i n ti N e a r e s t= f i n d N e a r e s t N e i g h b o r ( p r o j e c t e d T e s t F a c e ,& c o n f i d e n c e ) ; i n tn e a r e s t=t r a i n P e r s o n N u m M a t > d a t a . i [ i N e a r e s t ] ; / /W r i t er e s u l t st ot h eo u t p u tf i l e c h a rm s g [ 2 0 0 ] ; s n p r i n t f ( m s g , s i z e o f ( m s g ) 1 , " R e c o g n i s e d% sw i t hac o n f i d e n c eo f% f " , ( n e a r e s t>0?p e r s o n N a m e s [ n e a r e s t 1 ] . c _ s t r ( ):" n o b o d y " ) , c o n f i d e n c e ) ;

s y s l o g ( L O G _ I N F O ,m s g ) ; s t r i n gm e s s a g e= ( n e a r e s t>0? c o n v e r t ( p e r s o n N a m e s [ n e a r e s t 1 ] ): " U n r e c o g n i z e d " ) ; s t d : : o f s t r e a mo 1 ( o u t p u t F i l e ) ; o 1< <m e s s a g e . c _ s t r ( )< <s t d : : e n d l ; s t d : : o f s t r e a mo 2 ( d e b u g F i l e ) ; o 2< <m e s s a g e . c _ s t r ( )< <s t d : : e n d l ; } / /F r e et h er e s o u r c e su s e df o rt h i sf r a m e c v R e l e a s e I m a g e ( & f a c e I m g ) ; c v R e l e a s e I m a g e ( & s i z e d I m g ) ; c v R e l e a s e I m a g e ( & e q u a l i z e d I m g ) ; } c v R e l e a s e I m a g e ( & c h e c k I m g ) ; r e t u r nt r u e ; } i n tm a i n ( v o i d ) { / /O u rp r o c e s sI Da n dS e s s i o nI D p i d _ tp i d ,s i d ; / /F o r ko f ft h ep a r e n tp r o c e s s p i d=f o r k ( ) ; i f( p i d<0 ) { e x i t ( E X I T _ F A I L U R E ) ; } / /I fw eg o tag o o dP I D ,t h e nw ec a ne x i tt h ep a r e n tp r o c e s s i f( p i d>0 ) { e x i t ( E X I T _ S U C C E S S ) ; } / /C h a n g et h ef i l em o d em a s k u m a s k ( 0 ) ; / /O p e na n yl o g sh e r e o p e n l o g ( " f a c e r e c d " ,L O G _ P I D | L O G _ C O N S ,L O G _ U S E R ) ; / /C r e a t ean e wS I Df o rt h ec h i l dp r o c e s s s i d=s e t s i d ( ) ; i f( s i d<0 ) { / /L o gt h ef a i l u r e

s y s l o g ( L O G _ I N F O ," U n a b l et og e tS I D . " ) ; c l o s e l o g ( ) ; e x i t ( E X I T _ F A I L U R E ) ; } / /C h a n g et h ec u r r e n tw o r k i n gd i r e c t o r y i f( c h d i r ( " / " )<0 ) { / /L o gt h ef a i l u r e s y s l o g ( L O G _ I N F O ," U n a b l et oc h a n g ew o r k i n gd i r e c t o r y . " ) ; c l o s e l o g ( ) ; e x i t ( E X I T _ F A I L U R E ) ; } / /C l o s eo u tt h es t a n d a r df i l ed e s c r i p t o r s c l o s e ( S T D I N _ F I L E N O ) ; c l o s e ( S T D O U T _ F I L E N O ) ; c l o s e ( S T D E R R _ F I L E N O ) ; / /D a e m o n s p e c i f i ci n i t i a l i z a t i o ng o e sh e r e s t r u c ts t a ts t={ 0 } ; i f( s t a t ( i n D i r ,& s t )= =1 ) { m k d i r ( i n D i r ,0 7 0 0 ) ; } i f( s t a t ( o u t D i r ,& s t )= =1 ) { m k d i r ( o u t D i r ,0 7 0 0 ) ; } i f( s t a t ( d e b u g D i r ,& s t )= =1 ) { m k d i r ( d e b u g D i r ,0 7 0 0 ) ; } e x e c u t e ( ) ; e x i t ( E X I T _ S U C C E S S ) ; } v o i de x e c u t e ( ) { / /L o a dt h ep r e v i o u s l ys a v e dt r a i n i n gd a t a C v M a t* t r a i n P e r s o n N u m M a t=0 ; i f( l o a d T r a i n i n g D a t a ( & t r a i n P e r s o n N u m M a t ) ) { f a c e W i d t h=p A v g T r a i n I m g > w i d t h ; f a c e H e i g h t=p A v g T r a i n I m g > h e i g h t ; } e l s e { c l o s e l o g ( ) ; r e t u r n ; }

/ /P r o j e c tt h et e s ti m a g e so n t ot h eP C As u b s p a c e f l o a t* p r o j e c t e d T e s t F a c e= ( f l o a t* ) c v A l l o c ( n E i g e n s * s i z e o f ( f l o a t ) ) ; / /L o a dt h eH a a r C a s c a d ec l a s s i f i e rf o rf a c ed e t e c t i o n f a c e C a s c a d e . l o a d ( l b l C a s c a d e F i l e n a m e ) ; i f( f a c e C a s c a d e . e m p t y ( ) ) { s y s l o g ( L O G _ I N F O , " E R R O Ri nm a i nl o o p :C o u l dn o tl o a df a c ed e t e c t i o nc l a s s i f i e r . " ) ; c l o s e l o g ( ) ; e x i t ( 1 ) ; } C v H a a r C l a s s i f i e r C a s c a d e *c a s c a d e= ( C v H a a r C l a s s i f i e r C a s c a d e * ) c v L o a d ( f a c e C a s c a d e F i l e n a m e ,0 ,0 ,0 ) ; i f( ! c a s c a d e ) { s y s l o g ( L O G _ I N F O , " E R R O Ri nm a i nl o o p :C o u l dn o tl o a df a c ed e t e c t i o nc l a s s i f i e r . " ) ; c l o s e l o g ( ) ; e x i t ( 1 ) ; } v e c t o r < s t r i n g >f i l e s=v e c t o r < s t r i n g > ( ) ; c o n s tc h a r* c o n t e n t s=N U L L ; / *T h eB i gL o o p* / w h i l e( 1 ) { / /G e tt h ef i l e si no u r" i n "d i r e c t o r y f i l e s . c l e a r ( ) ; g e t d i r ( i n D i r ,f i l e s ) ; i f( ( i n t ) f i l e s . s i z e ( )> =3 ) c o n t e n t s=f i l e s [ 2 ] . c _ s t r ( ) ; e l s e c o n t e n t s=N U L L ; i f( c o n t e n t s! =N U L L ) { c h a ri n p u t [ 2 5 6 ] ; i n p u t [ 0 ]=0 ; s t r c a t ( i n p u t ,i n D i r ) ; s t r c a t ( i n p u t ," / " ) ; s t r c a t ( i n p u t ,c o n t e n t s ) ; c h a r* l a s t=s t r r c h r ( i n p u t ,' / ' ) ;

c h a ro u t p u t [ 2 5 6 ] ; o u t p u t [ 0 ]=0 ; s t r c a t ( o u t p u t ,o u t D i r ) ; s t r c a t ( o u t p u t ," / " ) ; s t r c a t ( o u t p u t ,l a s t+1 ) ; s t r c a t ( o u t p u t ," . t x t " ) ; c h a rd e b u g [ 2 5 6 ] ; d e b u g [ 0 ]=0 ; s t r c a t ( d e b u g ,d e b u g D i r ) ; s t r c a t ( d e b u g ," / " ) ; s t r c a t ( d e b u g ,l a s t+1 ) ; s t r c a t ( d e b u g ," . t x t " ) ; i f( r e c o g n i z e F r o m F i l e ( i n p u t , o u t p u t , d e b u g , c a s c a d e , p r o j e c t e d T e s t F a c e , t r a i n P e r s o n N u m M a t ) ) { r e m o v e ( i n p u t ) ; } } s l e e p ( 0 . 5 ) ;/ *w a i th a l fas e c o n d* / } c v R e l e a s e H a a r C l a s s i f i e r C a s c a d e ( & c a s c a d e ) ; i f( s t o r a g e ) { c v R e l e a s e M e m S t o r a g e ( & s t o r a g e ) ; } i f( c a s c a d e ) { c v R e l e a s e H a a r C l a s s i f i e r C a s c a d e ( & c a s c a d e ) ; } c v R e l e a s e I m a g e ( & s m a l l _ i m g ) ; c l o s e l o g ( ) ; }

The daemons job is to scan the incoming images sav ed by the motion detection component and copied into the input folder and find the largest contained face, should one ex ist. Any detected faces get ex tracted (and a copy gets sav ed to our debug folder, just so we can see how its working) and checked against the face recognition database (once again using the Eigenface algorithm). The results get stored to a tex t file in the output folder, to be display ed on the LED message-board. The most time-consuming part of this process by at least an order of magnitude is the initial face detection. I originally made use of Haar-like face detection, but ev en with some optimisations I borrowed from an iOS-based OpenCV implementation, I still found it far too slow on the Raspberry Pi (I as looking for <1 s results and was finding any thing from 5s to 90s, depending on the settings). Theres a lot that can be tweaked to control this such as increasing the minimum size of the object to be detected but despite v arious attempts, it still remained too slow.

At Sherv ins suggestion he ev en kindly shared some implementation details from his upcoming book, OpenCV 2 Hotshot: RAW I ended up making use of LBP face detection, instead. This is apparently only av ailable in the C++ API hence the mix ing of the C and C++ in today s implementation and the good news is that it certainly brought the cost of the detection down within acceptable limits. The bad news is that it generates lots of false positiv es, which means portions of the image often end up getting detected as faces and the Eigenface algorithm often seems to recognise these faces in the database with a depressingly high confidence lev el. Im sure theres lots that can be done to improv e the quality of the face detection, such as using a mask image. Heres a thought if Motion was able to sav e the information from its locate feature to be picked up by the facerecd, that would be ev en better: wed only hav e to search the area of motion in the image. Philipp Wagner kindly suggested adopting the approach shown in this tutorial, which may well help with performance and/or accuracy . I hav ent y et taken a v ery close look, as integrating this approach would probably lead to quite fundamental changes in the implementation (and Iv e just got too much else going on, at the moment). Ev en though the accuracy leav es a lot to be desired, the results are often v ery entertaining, but well see more about that, nex t time. And ev ery so often lightning strikes and the correct person is flagged. Posted at 07 :39 PM in Raspberry Pi | Permalink Com m ents

Gaston Nunez dijo... Hi Kean, Did Y ou see this?: http://www.raspberry pi.org/archiv es/2008, may be y ou can use turbo mode to speed up a little the process. Regards, Gaston Nunez Responder September 26, 201 2 at 08:31 PM

Kean Walmsley dijo en respuesta a Gaston Nunez... Hi Gaston, I did, thanks - I'v e already ov erclocked my other Pi (the one running Raspbian, as the raspi-config script makes it easier to do so). I hav en't y et decided whether I'll take the time to enable Turbo on my Arch Linux ARM dev ice. Regards, Kean Responder September 26, 201 2 at 1 0:38 PM

ali dijo...

Hi Kean, I tried to create a 64bit autocad .net app which tries to communicate with a 32 bit application in a separate process. So I am using WCF to manage the communication between apps. The serv ice host is the .net dll loaded inside autocad and the client is the other .net app(32bit). It seems that the client can not communicate with the serv er. Can any one guide me to figure out how WCF can be used inside autocad .net app? or may be any ex ample could be useful. Thanks, Ali Responder September 27 , 201 2 at 1 0:50 AM

Kean Walmsley dijo en respuesta a ali... Hi Ali, This question pretty much defines the term "off topic". :-) Please submit y our question v ia ADN or on the AutoCAD .NET Discussion Group. Regards, Kean Responder September 27 , 201 2 at 1 2:01 PM

AdamF dijo... Hi Kean, I'm following y our blog and hav e just tried to get the source for the daemon and build it but am hav ing no luck. I hav e done the wget fine and unzipped but when I run the g++ build I get this: [pi@alarmpi facerecog]$ gcc 'pkg-config opencv --cflags --libs' FaceRecDaemon.cpp -o facerecd gcc: error: pkg-config opencv --cflags --libs: No such file or directory I hav e run ev ery thing successfully before this point so know that I hav e all the respectiv e packages. Any hints? Thank y ou, Adam Responder October 21 , 201 2 at 1 0:1 1 PM

AdamF dijo en respuesta a AdamF... PS. That was actually me try ing gcc after failing with g++. I initially tried g++ as y ou suggested but had the ex act same error message. Responder October 21 , 201 2 at 1 0:55 PM

Kean Walmsley dijo en respuesta a AdamF... Hi Adam, First thing to try : make sure y ou're using backwards apostrophes, not normal ones. I hope this helps, Kean Responder October 22, 201 2 at 03:05 PM

AdamF dijo en respuesta a Kean Walmsley ... Aha, Thanks for that! That made it so I could compile the cpp file. Howev er when I ty pe facerecd in the facerecog folder I get bash: command not found :( Responder October 25, 201 2 at 1 1 :01 AM

Kean Walmsley dijo en respuesta a AdamF... Can y ou try ty ping this? ./facerecd Kean Responder October 25, 201 2 at 1 1 :1 0 AM

AdamF dijo... Got it! Would I need to hav e it like that in the DAEMONS = in the rc.conf? Thanks for y our help! Responder October 25, 201 2 at 01 :41 PM

Kean Walmsley dijo en respuesta a AdamF... If y ou copy the ex ecutabe across to /etc/rc.d then it shouldn't need any path qualifier... Kean Responder October 25, 201 2 at 01 :59 PM Comment below or sign in with Ty pePad Facebook Twitter and more...