OFF-LINE SIGNATURE VERIFICATION USING MULTIPLE NEURAL NETWORK CLASSIFICATION STRUCTURES

N . Papunzarkos, and H. Baltzakis

Elcctnc Circuits Analysis Laboratory, Dcpartmcnt of Elcctrical Xr Computer Engineering, Dcmoentus University of Thracc, 67 100 Xanthi, Grcccc

pai,amark(~vorcas.cc.dutli.gr
Abstract - This paper summarizes a research effort for an off-line signature recognition and verification system. The system uses three kinds of features extracted from the digital image of the signature: global features, grid information features and texture features. For each of them a special OCONs (One-Class-One-Network) classification structure has been implemented. In order the system to come to a decision, it uses the results from all the three neural network structures, combined with a simple Euclidean norm.

1. INTRODUCTION A signature recognition and verification system must provide two main tasks: to identify the owner of the signature and to decide whether the signature is genuine or forger. In general, we have two main kinds of handwritten Signature recognition systems: On-line recognition systems, where the computer is attached to special peripheral units capable of acquiring information about the way the human hand creates the pattern (velocities, pressures, etc.). Off-line recognition systems, where the only inforination the computer is acquired with, is usually the digitized image of the pattern. During the last several ycars, many scientists have tried to solve the problcm and several Signature Recognition Systems, both on-line and off-line, have been proposed [ 1-6.1. In this paper we propose an off-line signature recognition system, which is based on multiple neural network structure in combinations with tree powerfully feature sets. We use three different sets of features, each describing a different aspcct of the signature: global features, grid information features and texture leatures. Spccitically, for every class cach set of features is used by a separable neural network classilier. That is, each neural network specializes i n only one class of signatures (one signer) and only one set of features. This structure corresponds to a OCON (One-Class-One-Network) neural network classifier 17-81, 2. PRE-PROCESSING The preprocessing stage has four different parts: noise reduction, data area cropping, width normalization and skeletonization. I t is noted that the proposed method is applied to binary images only. Noise reduction: Before any further processing takes place, a noise reduction filter is applied to the binary scanned image. The goal is to eliininate single white pixels on black background and single black pixels on white background. In order to accomplish this, we ap a 3x3 mask to the image with a simple decision rule the number of 8-neighbors of a pixel that have the same

color with the central pixel is less than two, wc reverse the color of the central pixel Segmentation: For segmentation a projection technique is used. Specifically, vertical and horizontal projections are calculated and using of two threshold values we identify the area of the image that contains the signature (we discard the white space surrounding the signature) Width normalization: The image size is adjusted so that the width reaches a default value and the height to width ratio remains unchanged. Skeletonization: A simplified version of the skeletw nization technique described i n [9] is used.

Figure 1

The simplified algorithm consists of 3 simple steps:

1 , Mark all the points of the signature that are candidates

for removing (black points that have at least one white 8neighbour and at least two black 8-neighbohrs). 2. Examine one by one all of them, following the contour lines of the image, and remove these that their removal will not cause a break in the resulting pattern. 3. If at least one point was deleted go again to step 1 and repeat the process once more. Fig. 1 gives an example of this skeletonization technique. Skeletonization makes the extractcd features invariant to image characteristics like the qualities of the pen and paper the signer used and the digitizing method and quality.

3. FEATURE EXTRACTION The choice of the features that will be provided to the classifiers of the system is very important. I n this work we use three different sets of features. global features, grid information features and tcxture features.

0-7803-41 37-6/97/$10.000 1977 IEEE

DSP 97 - 727

Both global features and grid information features are classic in pattern recognition problems. Besides these, we also propose the use of co-occurrence matrices for different regions of the signature as a way of describing the texture of the signatures. 3.1 Set one: Global features 1. Signature height. The height of the signature, after width normalization, is another way of representing the height to width ratio. 2. Image area. The number of black pixels of the image. When it is about skeletonized images, i t represents a measure of the lines density in a signature image. 3. Pure width. The width of the image with horizontal blank spaces removed [2]. 4. Pure height. The height of the image with vertical blank spaces removed. 5 . Baseline shift. The defcrence between the vertical centers of gravity of the left and the right part of the image. It was taken as a measure for the orientation of the signature. 6. Vertical center of mass

EL - EP C L = I + y
i

(3)

with

EL =

[(Number of 8 - neighbors) - 21
All cross points

where CL: number of closed loops EL: number of extra departures EP; number of edge points The above method for defining the number of closed loops, works well only for compact signatures. When thc signatures are split to more than one different individual sections this derived number of closed loops is Palse. Despite this phenomenon, we assume that, i n every casc, the above number, even if it has no physical interpretation, is characteristic for each signature and describes the amount of complexity that the signature lines involve.

x=l

l=l

7. Horizontal center of mass

Figure 2

i=l

i=l

8. Maximum vertical projection

9. Maximum horizontal projection.

10. Vertical projection peaks. The number of local maximums of the vertical projection function 1 1 , Horizontal projection peaks. The number of local maximums of the liorizontal projection function 12. Global slant angle. The image was rotated from 30" to 40" with a step of 1". Each time the number of vertical 3-pixel connections was calculated. The angle that had the most vertical 3-pixel connections is the global slant angle [ 2 ] . 13. Local slant angle. The image is rotated in a similar way as above. For every angle of rotation, an array of the 70 inaxiinuin vertical projections is calculated and the suininary of all the elements of that array is stored. The angle that presents the maximum summary i s the local slant angle [ 2 ] . 14. Number of edge points. Edge point is a point that has only one 8-neighbor. 15. Number 01' cross points. Cross point i s a point that has at lcast 3-X neighbors. 16. Number of closed loops. The number of closed loops can be defined a s :

Some examples of corner, and edge points are shown in Fig. 2. 3.2 Set two: Grid iuformationfeatures The image is divided to 96 rectangular rcgions ( I 2x8) and for each region, the area is calculated. 3.3 Set three: Texture features To extract texture features we make use o F the cooccurrence matrices of the image. For binary iinagcs, they are 2x2 matrices describing the transition of black and white pixels for given directions and distances [ l o ] . The co-occurrence inatrix Pip.(, defined as: is
(4)
where: * aon s the number of times two white pixels appear i separated with distance d i n direction of p'. a , i s the number of tiincs il coinbination of a wliitc , arid a black piscl appears separated with distance tl it1 direction of @'. 0 al0is the same as a0, a,, the number of times two black pixels appear is separated with distance d i n direction of @. The image is divided to 6 rectangular regions ( 3 x 2 ) .

For each region the P , ~ , ,P,,5.,, P,,,, and P f inatriccs are calculated and the CL,,, and f i dements are used a s

,.

ii.,

features of the signature.

4. CLASSIFICATION Neural networks are among the most commonly used classifiers for pattern recognition problems. Despite thcir advantages, they suffer from sonic very sei jous

DSP 97 - 728

limitations that make their use, often, for certain kinds of problems, impossible. The first limitation is the size of the neural network. It is very difficult for many neural networks to get trained, and, as the amount of the ti-aining data (the number of 1iipu ts and tr ai n ing patterns) increases, this difficulty becomes a signilicant obstacle for the designer of a system to deal with. The second difficulty is that the geomctry and size of the network, thc training method and the training parameters, depend very much on the amount of the training data. For getting the most out of an neural network and achieving gotxi generalization it is an absolute need to know, at-design-timc, the number of classes the ncural network will have to deal with. When talking about a uscful signature recognition system, there is no way we can know thc number of signatures the neural network will be asked to deal with. In this work, we tried to solve these problems by using multiple fixed-size neural networks. We reduced the dimensionality of the input vector by dividing the features to thrce categories and implementing a different classillcation structure for each feature category, and we btabilized the number of training vectors by implementing a diffcrent ncural network for each class of signatures (each signcr). This approach offers another significant advantage: each time we want to add a set of signatures ( a new person) to the systems database, wc only have to train three new sinall neural networks (one Ibr each set of features) and not the entire neural network. 4.1 The structure of the classification W e use three neural network structum, one for each set of featurcs (Global fcatures, Grid infoi-ination features and Texture features). All fcatures are norinalized within the region [O,l]. In conjunction with the three network structures, a simple minimum Euclidean distance classifier for all the featurcs combined together is used
~~ ~~

4.2 The neural networks

Each network structure (Fig. 3: "1, "2, "3) consists of several OCON (One Class One Network) individual networks (Fig. 4). Each of these sub-networkb specializes in only one class of signatures (one person's signatures) and has only two output neurons. The subnetworks are trained in a way that they present only two possible combinations of the two output neurons. Output (1,O) indicates that the input is recognized. Output (0,I ) indicates that the input is not recognized. The training set lor each of the sub-networks consists of all the signatures of the specific owner that the network specializes for, and an number (30) of random countcrexamples among other owners.
~

112
t

IN

IM
Figure 4

Each neural network structure that declares success i n recognizing the input vector, produces a classification error which is the distance (Euclidean norm) i n its o ~ i t p ~ i t space between the ideal output of all it's sub-nctworks (e.g. [O,O,O,l,O,] and the actual output (e.g. [0.1,0.08,0.01,0.9S,0.03]). All tlie neural networks of tlic system are multi-layer perceptrons trained with the ALOPEX algorithm [ ] 2. 4.3 The overall decision If the number of the neural network classifiers that declared success is M (O<M<3), thcn each of thcm contributes to the final decision by:
Fh

Image after preprocessing

=(I-

J- loo % ,/Nunzher- (IJ-Classes M f 1

(5)

The Euclidean Norm contributes to the final decision by:

Thc systcm suimiarizcs thc contributions 0 1 tlic M+ I classificrs and g i w s tlic results in a fonrial llkc tlic
a b ovc Thodoros Kolokotronis Gcorgios Karaiskakis
DECISION

67 % 16%

Figure 3

Fig. 3 demonstrates tlic structure of the classiiier. The Iirst neural network box ( N N I ) has I6 inputs (global features), the second ("2) 96 inputs (12*8) and the 48 inputs (3*2'k4*2). The Euclidean Norm third ("3) Box has 160 inputs (15+96+48). Each box propagates its result t o the linal deciaion box where tlie linal decision is being madc.

5. HOW THE SYSTEM WORKS When the system is asked to decide whether iiii unknown signature image belongs to a person in thc database the following steps are followed The unknown image passes thought the pt-cprocessing and feature extraction stages. The three sets of features are applied to the input of all of the thrcc sets of sub-networks. The sub-networks are run Forward so that we get outputs for all of them.

DSP 97 - 729

For each of the three neural network structures, we search all its sub-network outputs trying to find the (1,O) combination indicating that the image is recognized. If exactly one (1,O) output i s found in a set of sub-networks, i t means that the specific sub-network structure has come to a positive decision about the owner. By the end of this stage we have, tlie most, three proposed owners. * We examine the Euclidean norm between the characteristics of the unknown image and the characteristics of all tlie images in the systems database (one by one). between the For example, the Euclidean norm (DN) Ieature vector of the unknown image X T and the feature vector of the Nth signature in the Database X, i s given from the following equation.

7. CONCLUSIONS

The main objective of this work was the use of multiple and multistage neural network structure Ibr offline signature verification. For each signer (for each class) a separable neural network is used trained with the ALOPEX algorithm. The entire neural network classifier has a parallel form and it is supplied by three different sets of features, each describing a different aspect of the signature: global features, grid information fealui-es and texture features. The proposed verification system was tested with many signatures and tlie verification results are very promised.

8. REFERENCES
Plainondon, A Coniparativc Analysis of Regional Con-clation, Dynamic Time Warping, a n d Skcletal Trec Matching J.or Signature Verification, IEEE Tran.mctio/i.r o i i Pattern Ariaijuis and Machine Iiitelligetice, vol. 12, no 7, J ~ l y 1990. Y. Qi aiid B. Hunt, Signaturc Vcrikication Using Global aiid Grid Features, Patterri R r c o , y i i t i o / i , vol. 27, n o 12, pp. 1621-1629, 1994. K. H a n and I. Scthi, "Handwritten Sigiiaturc Rctrieval and Identification, Pntterii Recognitio/i Letcers, vol. 17, pp. 83-90, 1996. 0. Ycdekcoglu, M. Akban, Y. Lirn, Off-Linc Signature Vcrificatioii with Tliickcncd t cinplatcs , C O M C O N 1995, Proc. q/C,O!\.I(CO.7 5, pp.131112, Rctliynno, Grcccc, 1995. J. Drougliard, R. Sabourin and M. Godbout, A Ncural Nctwork Approach t o Oft-Line Sigiiaturc Vcrificatioii Using Dircctional PDF, P a t t r r i i Recognition, vol. 29, 110. 3, pp. 415-424, 1996. L. Lcc, T. Bcrgcr a n d E. AvicLcr, Rcliablc 011lUic Signature Vcrification Systcms, IEEK Trailsactioris on Pattern Aria1,vsis rriitl Macl/i/ic Iritelligetiw, vol. 18, no. 6, 1111. 643-647, J~iiic 1996. A. Pandya and Robcrt B. Macy, Pattciii Rccogiiition with Neural Networks in C++, CRL Press aiitl IEEE Press, 1995. S. Hayhn, Neriral A7eri~vorlcs:d Co/,/pre/i,e/i.c Foiintlatiori , Ma cniilla n C ollcgc Publishing Coinpany, Ncw York, 1994. L. Lam a n d C. Sucii, A Dynamic Sliap: Prcscrvinp Tliinniiig Algorithm, ,Sig/iol lroce~~i/ig,.01. 22,1111. 199-208, 199 1. 1 -. [IO] S. Bow, Patterii Recognition niid I//ing~l Preprocmirig. Marcel Dckkcr Inc., 1992.

M.Parizcu arid R.

v I=I

() 7

The owner of the image t h a t presents the smallest value of the Euclidean norm i s the owner that is proposed by the fourth classifier. We estimate the results of all the four classifiers and we came to the final decision

6. TRAINING AND RESULTS In order to test the perforinance of the system, a master set of about four hundred (400) signature images was used. The signatures where taken itom 23 different persons ( 1 5-25 siynatures from each one).

w
Figure 5

For training the system we used a subset of about three hundred and fifty (3501, randomly selected from the master set. The performance of the system was checked by tlie use of the remaining subset of 50 signatures. The signers were asked to use as much variation as they could i n order to make the system more robust tc intra- person a1 varl at i ons . An examples of intra-personal variations are shown i n Fig. 5 and Fig. 6. All signatures in the testing set were identified correctly by the system.

Figure 6

DSP 97 - 730