Partial Matching Using Set Exclusion Criteria

Applied to livestock brand retrieval

Waldemar Villamayor-Venialbo, Horacio Legal-Ayala

Facultad Politcnica

Edson Justino, Jacques Facon

Programa de Ps-Graduao em Informtica

Universidad Nacional de Asuncin wvenialbo@pol.una.py, hlegal@pol.una.py

Pontifcia Universidade Catlica do Paran edson.justino@pucpr.br, facon@ppgia.pucpr.br

Partial Matching for livestock brand retrieval

Background
Inspiration The correct registration of cattle brands is a major issue in countries with old ranching tradition. The equitable fraud (for livestock theft) is, practically, institutionalized and systematized in these countries.

Brand inspector officers must prevent them ensuring that a particular brand cannot be converted in another brand by addition of extra marks.

Partial Matching for livestock brand retrieval

Background
How are the frauds done? The victims brand is used as a base for a new brand design. The new brand is registered at the livestock office entering in the legal system. The absent strokes are stamped on the stolen animals over the existing marking. The original owner can no longer prove his ownership on the stolen animals.
3

Partial Matching for livestock brand retrieval

Modifying an existing brand After the modifications the underlying brand will be hidden for an untrained eye, the common for brand inspectors officers who should deal with hundreds and, even, thousands brands.
Note that every new added stroke establish per se a new brand.

The JL Brand (original)

Transforming it into a H

Adding a F

Adding a hat
4

Partial Matching for livestock brand retrieval

Objective
The challenge To identify registered brands that could be converted in a candidate brand, or vice-versa. The method should be efficient enough to manage a large number of livestock brands.

Partial Matching for livestock brand retrieval

Objective
The contingencies
(For our particular case)

Brand registration is done by handwriting the brand on a graphics table, which implies low-noise gray-level images. Brands are drawn within a predefined bounding box.

All the strokes in the brand have the same width.

Similar brands may be drawn with slight differences in size, proportion, orientation, and slanting.
6

Partial Matching for livestock brand retrieval

Previous approaches
Two previous (unpublished) approaches were carried out by partner groups. Both techniques treat the brand image as a whole, without segmenting it.
1. The first method creates a set of meshes over the brand based on its bounding box at different orientations. Each mesh have a fixed number of cells, ensuring its invariance to changes in scale. Image features are then extracted from each cell and the sets of features of two images compared.

Partial Matching for livestock brand retrieval

1. (cont.) The method works properly when the modified brand contains internal add-ins and slight shape deformations. It fails when an added stroke changes the proportions of the bounding box, such as external add-ons.

Note that any of the added strokes (in black) from our previous example will make this method fail.

Partial Matching for livestock brand retrieval

Previous approaches
2. The other method operates similarly creating a set of meshes over the image at different orientations.
The mesh have a variable number of cells, and each cell in the mesh have a fixed size. Invariance in scale is lost. Finally, the sets of features of two images extracted from each cell of their meshes are compared. This approach overcomes the limitation of the first method, but fails if exists slight differences in the size of the two images.

Partial Matching for livestock brand retrieval

Our proposal
We present a solution based on a partial matching approach. Some key characteristic of the method is listed below:
The brands image is segmented to take apart its representative parts: the strokes forming it. Shape features are judiciously extracted from these components, and a descriptor set is formed with these local features. A set exclusion measure is formulated. This measure allows to determine if a registered brand is a subset or superset of another existing brand.
10

Partial Matching for livestock brand retrieval

The method
What follows is just an overview of the different stages of our approach. Note that the intermediate steps could be replaced by better and more robust techniques as they become available
Image preparation The experiments have been applied onto 8-bit gray-level images. Brand sketches digitized with the graphics table are noise free. No noise is required to be added for this application.

Images were binarized using Otsus threshold algorithm.

11

Partial Matching for livestock brand retrieval

The method
Image segmentation
The orientation space (OS) projection technique was chosen to decompose each stroke in the brand.

A. Each stroke is unfolded mapping the pixels into the OS according with the slope of the tangent line passing through it. B. Projecting back each continuous curve from the OS onto different image planes takes apart any overlapping strokes reconstructing the originals individually.

12

Partial Matching for livestock brand retrieval

The method
Feature extraction
The slight differences between the brand sketches are approximated by an unknown affine transformation.

So, for our shape features we choose the spatial multiscale affine invariants (SMA) for its robustness, low CPU time consumption and low complexity at the implementation time.
Local features are obtained taking account the relative spatial distribution of strokes taking by pairs, we consider each pair as a single brand component.
13

Partial Matching for livestock brand retrieval

The method
Partial matching classifier
The features extracted from each component are put together to form the brands descriptor set.
Considering as the set of pixels forming a single stroke, and the brand as a countable set of strokes, let represents the i-th stroke in that brand. The descriptor set of is defined as:

Here,

is the SMA feature extractor operator.

14

Partial Matching for livestock brand retrieval

The method
Partial matching classifier
Note that the same inclusion relations among brands holds for their descriptor sets. That is, , i.e., brand could be converted in by adding to the strokes in .
Set exclusion criteria One way to determine that a brand does not contain, nor it is contained in another brand, is establishing an exclusion measure . Such measure between two brands must yields 0 whenever a brand is a subset of the other. Other relations are shown below:

15

Partial Matching for livestock brand retrieval

The method
Partial matching classifier
Set exclusion measure Such exclusion measure is the workhorse of our partial matching approach. Let and be the number of elements in and , respectively, if we define our set exclusion measure by:

Where is some underlying norm in the descriptor space, e.g., the norm. is a threshold parameter determining the maximum allowed distance to consider similar corresponding components from and .

16

Partial Matching for livestock brand retrieval

The method
Partial matching classifier
Set exclusion measure
is a superset of , the shaded stroke is not present in ; is a subset of , since it lacks one stroke from ; has three strokes in common with , but the one it lacks from and the shaded stroke in makes it impossible the conversion between them; finally, have a completely distinct design from . According to the exclusion criterion both brands, and , must be rejected since ______ . At the same time, __ and may be accepted, because _______ and .
17

Partial Matching for livestock brand retrieval

The prototype program

The scheme of our livestock brand registration system and a screen capture of our implementation.

18

Partial Matching for livestock brand retrieval

Experimental results
Real and Synthesized Brands
A set of 315 randomly selected real cattle brand images was established to setup a basic database and carry out the testing. Three categories of test sets containing real and synthesized images have been prepared, totalizing near 4000 query brands. Synthesized images were created applying slight affine transformations and drawing additional strokes to the real original samples.
19

Partial Matching for livestock brand retrieval

Evaluation criteria
Precision (P) and recall (R) values were used as metrics to evaluate the intra-class and overall retrieval performance.

Performance of Partial Matching

The table summarizes the best intraclass retrieval performance of the prototype program. The column labeled S.E. list the amount of segmentation error, the P and R values are not affected by this error.
20

Partial Matching for livestock brand retrieval

Performance of Partial Matching

Overall retrieval performance of the proposed model for several values of . This includes the results of all test sets with the exception of the trivial cases. It can be seen that good performance (high rates of recall and precision) is accomplished for .

21

Partial Matching for livestock brand retrieval

Conclusion
We presented a partial matching algorithm based on set exclusion criteria. The experimental results have shown the accuracy of the methodology in cattle brand identification. We aim to enhance, in the future, the overall accuracy of the system, improving the stroke segmentation and feature extraction techniques with more powerful approaches as they become available.

22

Partial Matching for livestock brand retrieval

Thanks
23