Journal of Visual Languages and Computing 28 (2015) 134–146

Contents lists available at ScienceDirect

Journal of Visual Languages and Computing

journal homepage: www.elsevier.com/locate/jvlc

Individual strategies in the tasks of graphical retrieval

of technical drawings$
Denis R. Kasimov n, Aleksandr V. Kuchuganov, Valeriy N. Kuchuganov
Department of Automated Data Processing and Control Systems, Kalashnikov Izhevsk State Technical University,
426069 Studencheskaya 7, Izhevsk, Udmurt Republic, Russia

a r t i c l e in f o abstract

Article history: A designer's information need can have a lot of finer points not expressible completely
Received 11 November 2013 by textual attributes or global aggregated image features. The purpose of the study is to
Received in revised form develop a cognitive environment for content-based engineering drawing retrieval,
24 December 2014
allowing a user to concretize queries, to implement strategies that are most effective
Accepted 26 December 2014
for the current search task. We propose meaningful customizable graphic search patterns,
Available online 5 January 2015
various comparison modes and visualization of matched elements. The presented
experiments show that the approach yields good results in the sense of relevance and
is comfortable in terms of controllability. The possibility to customize search patterns can
Keywords:
Technical drawing result in improving retrieval precision by 10–15%. In practice, it can significantly increase
Content-based retrieval the degree of automation of the design-reuse process.
Matching graphs & 2015 Elsevier Ltd. All rights reserved.
Search strategy

1. Introduction addition to the high cost of manual labor in constructing and

When designing new products it is important to apply
accumulated knowledge such as existing components,
design solutions and technological processes. It is known
that design engineers spend approximately 25% of their
time searching for and using design information to com-
plete their work [1]. However, it is difficult to find a
required document among thousands of them considering
the graphic nature of technical documentation, which
includes drawings, 3D models and scanned documents
containing both text and graphics.
Currently the drawing retrieval problem is solved mainly
on the base of textual–numeric attributes, classification
codes and text descriptions created by people. Herewith, in
Abbreviations: FSLG, fuzzy spatial loaded graph; CSG, Constructive Solid
Geometry
☆
This paper has been recommended for acceptance by Shi Kho Chang.
n
Corresponding author. Tel.: þ7 341 277 6055x3257.
E-mail addresses: kasden@mail.ru (D.R. Kasimov),
Aleks_KAV@udm.ru (A.V. Kuchuganov),
kuchuganov@istu.ru (V.N. Kuchuganov).
maintaining an archive and incompleteness of part geometry
description, it is difficult to formulate queries, including
because of the influence of subjectivity, when humans
describe the same part differently.
These deficiencies are absent in methods that perform
drawing retrieval by a user-specified sample image. How-
ever, none of existing content-based drawing retrieval
systems has a sufficient level of transparency, control
and guidance, which, according to [2], are the main goals
any search engine has to adhere. The possible reason of
this is that in these approaches the representation of a
graphic object is weakly connected with its content.
In existing drawing retrieval methods a search dialog is
rather primitive, i.e. simply a sample image is specified,
and a retrieval system outputs results. Meanwhile, in
design activities various kinds of search tasks arise. The
most distinctive features of required drawings are different
in different situations: a projection view, a projection's
fragment, an external contour, standard mechanical features,
symmetry of a part, etc. In practice, it is necessary to consider
a part's feature (groove, pocket, hole, etc.) as a decisive
distinguishing characteristic in some situations and to ignore

http://dx.doi.org/10.1016/j.jvlc.2014.12.010
1045-926X/& 2015 Elsevier Ltd. All rights reserved.
 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
it in other cases, to modify the search prescription without
redrawing the sample image, to set exact requirements to
similarity of individual components, etc.
Search engines' users can be divided into three classes,
depending on how clear they understand their search
need [3]: (1) a browser has no clear end-goal, performs a
series of unrelated searches; (2) a surfer does exploratory
searches in the beginning, but in the process of working
his clarity of what he wants from the system is gradually
increased; (3) a searcher has a clear intent, performs well
related searches leading to an end-result. The same user in
different search sessions may relate to different classes,
and his search strategies vary considerably. Therefore, a
good retrieval system should provide an opportunity for
each class of users to build effective strategies, including
individual for each person.
The purpose of the study is to develop a retrieval
environment allowing a user to concretize queries, to
implement various strategies that are most effective for
the current search task. It is necessary to expand the
existing scenario of the drawing retrieval dialog.
The paper is organized as follows. In Section 2 related
works on content-based technical drawing retrieval are ana-
lyzed. In Section 3 we describe our approach for automatically
describing and matching drawing images, propose the retrie-
val environment's elements that are aimed to improve the
retrieval flexibility and relevance. In Section 4 the retrieval
system's prototype is described, experiments on evaluation of
the proposed retrieval environment are provided. In Section 5
we discuss obtained results. Finally, in Section 6 the conclu-
sion is made and directions for further work are listed.
2. Related work
There are many approaches to the content-based tech-
nical drawing retrieval. Extensive surveys are even
devoted to them [4,5].
In the area of graphical retrieval of drawings and in
pattern recognition in general, the approaches prevail in
which a global description is given to a graphic object, and
comparison of graphic objects consists in applying some
algebraic distance measure to their aggregated descriptions.
In the experimental system ShapeLab [6] two drawing
matching methods are implemented. In the first method,
drawings are represented as spherical harmonics; a fast
transformation is applied for their comparing. In the second
method, a drawing is represented as a 2D shape histogram, a
distribution of distances between pairs of randomly selected
points; for measuring similarity between two such histo-
grams the Minkowsky distance is used.
In the work [7] an approach for retrieving vector
drawings by a sketch drawn by a hand on a tablet device
is presented; earlier versions of this approach are consid-
ered in [8,9]. The content of drawings is decomposed into
topology (spatial arrangement of polygons in a drawing)
and geometry (shape of these polygons). Topological
information is organized in the form of a graph, in which
nodes represent the drawing's polygons, and edges corre-
spond to relationships between them of three types:
inclusion; adjacency; and spatial proximity. The topology
graph is then converted into multidimensional descriptors
135
on the base of computing its spectrum. To speed up
searching, descriptors are indexed on the base of the NB-
Tree. The retrieval process is divided into two steps: (1)
selecting drawings topologically similar to the query and
(2) computing geometry similarity between each candi-
date drawing and the query sketch.
The method described in [10] is based on the Hough
transform, which is applied for generation of a feature
vector. Not the whole cumulative matrix is used, which is
built when calculating the Hough transform, but only the
angular information. Comparison of images is performed
by calculating a distance between their feature vectors
according to a very simple formula.
The approach proposed in [11] represents drawings as
feature vectors of pixel blocks and calculates their simi-
larity by the linear weighted cosine similarity method.
In the method described in [12] an elementary descrip-
tor of a drawing image is a so-called local neighborhood
structure (for simplicity, also called as patch), which is
obtained by grouping lines/arcs around some reference
line. Matching is based on identification of dense regions
in a transformation parameter space (object center, scale,
rotation) using the mean shift method.
The system CADFind [13] indexes not only geometry of
the part in a drawing, but also text information associated
with it, including material, manufacturing process. Infor-
mation extracted from a drawing is represented in the
form of a so-called Group Technology (GT) code, a string of
textual–numeric symbols. Matching drawings is carried
out by fuzzy comparison of corresponding GT codes.
There are also approaches that use a structural descrip-
tion of a drawing's graphic object, but matching drawing
images has a mechanical bias; there is no meaningful
analysis of components and relations between them. In the
paper [14] a method for searching technical drawings on
the base of matching attributed graphs describing draw-
ings is proposed. Graph nodes correspond to primitives
(lines and curves), and edges describe spatial relations
between these primitives. To solve the graph matching
problem, the mean field theory is used.
In the paper [15] two alternative methods for searching
raster technical drawings in a patent database are considered:
by measuring the similarity of graphs and by comparing
histograms. Both methods involve transformation of the line
set of a drawing into directed graph of 6-nearest (by Euclidean
distance between line centers) neighbors, edges of which are
loaded by invariant attributes: relative angle and relative
position of two lines. These attributes are also used when
two-dimensional histograms are created. Matching graphs is
carried out with the use of a variant of the Hausdorff distance.
Comparison of histograms is performed by measuring the
Bhattacharya correlation.
The system that is proposed in [16] represents drawing
images in the form of hierarchical topology graphs and
performs their matching on the base of nested assignment
algorithm and the EMD distance.
The purpose of the work [17] was to carry out matching
parts' contours in such a way that the matching result
is adequate to engineering interpretation: determining
overall structural similarity with tolerance to local bending
and stretching. The two-level representation of shapes is
136 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
used: a set of feature points with their local geometry,
which represent high-level structure of a contour, and
groups of elements between feature points, which repre-
sent low-level geometric features of the contour. Connec-
tion between the levels is organized by means of a lookup
table. Feature points are determined among endpoints of a
contour's elements. Contour matching consists in finding
the optimal correspondence of feature points and groups
of elements between them.
The querying process in the existing approaches is
limited by specifying a sample image; searching is carried
out at the level of abstraction that was predefined by
developers and might not correspond to a user's need; at
that the user has no possibility to determine the appro-
priate set of search features. This eventually may adversely
affect search results.
In our view, the possibility to use careless sketches
as queries [6,7] is good only for simple parts. For creating an
image of a complex part a designer more likely will resort to a
CAD system in which will create a desired sketch. Further-
more, searching is often performed not at the beginning, but
during the process of developing a project, when there is
already a rough drawing of a designed part.
In the design field a search by an image-fragment is quite
important. This possibility is not provided at all in the
approaches [6,10,11,13,17]. In the works [7,16] it is restricted
by an image type – namely an image-fragment should be a
composition of contours. It is doubtful that the considered
approaches allow retrieving parts that have a given mechan-
ical feature (such as groove, keyseat, chamfer with a certain
angle, fillet of a specified radius, etc.). At the same time, this
search kind is demanded in developing technological process
of manufacturing a part, since exactly mechanical features
complicate the manufacturing process.
In our view, for a user it is easier to interact with such a
search system that provides him with information about
what, from its point of view, is the same and what is
different. Establishing correspondences between image
elements is implemented only in the approaches [16,17].
We share the opinion of [5] that applied to most of the
developed approaches high recognition quality is reported,
whereas they were evaluated at a quite limited amount of
drawings; their capabilities must be assessed in large-scale
experiments.
In this paper we present and evaluate a new approach
to the drawing retrieval problem, which is based on graph
matching. For the purpose of improving retrieval flexibility
and relevance, the dialog between a user and a search
system is carried out by means of meaningful customiz-
able search patterns of different abstraction level, with
help of visualization of matched elements, which is pro-
vided by our graph matching method.
3. Cognitive retrieval environment
The key conceptions underlying our approach to the
technical drawing retrieval problem are as follows:
 consideration of a graphic object as a multi-level system
like a linguistic one;
 usage of standard mechanical engineering elements
and qualitative characteristics when describing graphic
objects;
 automatic extraction of 3D constructive objects from a
drawing in order to describe it by them;
 representation of a graphic object at various levels of
abstraction;
 tolerance to affine as well as topological transforma-
tions of compared objects; and
 fixation and visualization of similarities and differences
between graphic objects.
The base model of representation of a graphic object is the
fuzzy spatial loaded graph [18,19] (FSLG) G¼ (V, AV, E, AE, AO),
where V is the set of nodes that describe components of the
object using the parameters AV, E is the set of edges that
represent spatial relations between the components using the
parameters AE, AO is global parameters of the object. One of
the components' parameter is similarity with typical config-
urations; this improves the representation's meaningfulness,
allows performing indirect, faster matching of components.
The parameters AV, AE, AO are represented as linguistic
variables [20]; this also allows increasing matching speed,
enhancing analysis and matching abilities, makes the repre-
sentation more resistant to deformations.
Four levels of the representation of graphic information
are given: (1) graph of singular points and segments, (2)
graph of contours, (3) graph of 3D objects, (4) graphs of
heterogeneous components. Unlike aggregated descriptors
used in many related works, the listed representations are
understandable by humans. The stratification into layers
allows fixing various, low- and high-level aspects of the
same drawing image.
Like in most related works, the drawing indexing stage
is provided in our approach, in which search patterns of
drawings are created and entered into a searchable data-
base by which searching is further performed.
Details of our models, methods and algorithms are
presented in [21–24].
3.1. Query drawing types
An existing detailed drawing, a CAD draft or a sketch
may serve as a sample image. A drawing format may be
both vector and raster. A raster drawing can be automati-
cally converted to a vector form by means of an automatic
vectorizer; our technology is presented in [25].
Due to the variety of the representation levels (see
Section 3.2), a designer may build a query on the base of a
sample image of any content: a projection view, an
external contour, a fragment of a projection, a standard
mechanical feature (see Fig. 1).
3.2. Search pattern kinds and their application areas
In the retrieval system drawing images are represented
as search patterns of the four types:
(1) Graph of singular points and segments describes a draw-
ing image at the level of chains of primitives. Graph nodes
 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
Fig. 1. Classification of query images by content.
correspond to singular connection points on the set of
graphic elements: junctions, angles, ends. Graph edges are
chains of elements between the singular points.
(2) Graph of contours considers a drawing image as a
composition of closed contours. Graph nodes repre-
sent external and internal contours of the drawing
image. Graph edges describe spatial relations between
the contours.
(3) Graph of 3D objects describes a part's drawing in
terms of primitive 3D bodies that form 3D model of
the part according to the Constructive Solid Geometry
(CSG). Graph nodes describe primitive 3D bodies, and
edges correspond to relations between them.
(4) Graph of heterogeneous components, with the purpose
to describe an image most meaningfully and completely,
decomposes a drawing image into objects of various
kind that are significant from designers' point of view
(contours, holes, grooves, bolts, nuts, chamfers, etc.).
Fig. 2 illustrates the proposed complex of search patterns.
Below the areas of the most optimal use of the search
patterns are specified:
(1.1.) Graph of singular points and segments of a graphic
object's external contour: search for rotary bodies by
137
a profile; retrieving drawings of parts, internal ele-
ments of which are insignificant; search for parts
having complex external shapes; selection of a sub-
set of drawings for the next search iteration.
(1.2.) Graph of singular points and segments of an entire
graphic object: search by a connected arbitrary
fragment of a projection; topological search (the
length of segments can vary greatly); search by a
standard mechanical feature such as groove, protu-
berance, bevel, etc.
(2.) Graph of contours of a graphic object: search by a
projection, taking into account its internal elements.
(3.) Graph of 3D objects of a part: precise search for
drawings; searching CSG 3D models without con-
verting them to 2D, using a drawing as a query.
(4.) Graphs of heterogeneous components such as
“Smoothed external contour & detail features”, “Con-
tours & projections of standard components”:
searching at the mixed perception level that
responds well to a search need.
Furthermore, two variants of the listed below search
patterns are provided: (1) strict, built by a query drawing
in its the original form, and (2) simplified (smoothed),
built by the query drawing cleaned of some of mechanical
features. The system recognizes mechanical features in the
drawings by means of a library of typical ones. A fragment
of the library is shown in Fig. 3. Two search pattern
variants of the same part are shown in Fig. 4. Simplified
search patterns capture only basic geometry of parts.
Therefore, when small details in a design are not impor-
tant or a preliminary exploration of a search space is
carried out, searching using simplified patterns is pre-
ferred since it is more resistant to local differences and
executes faster.
3.3. Search pattern customization
Due to the structural representation of drawings in the
system, a user may specify special requirements to search
results in a search pattern of a query drawing.
It is possible to specify requirements to the attributes
of individual components in the form of the degree of
similarity/dissimilarity with a query's attribute value or a
list/range of permissible values. Common variants of
requirements to an element's attribute value, regardless
of its type, are
(1) Approximately as in the query drawing (fuzzy similar-
ity, by default);
(2) Any (i.e. it does not matter); and
(3) Exactly as in the query drawing (crisp similarity).
Below specialized variants of requirements to attributes
are listed:
 segment form: straight, left smooth turn, right smooth
turn, left sharp turn, right sharp turn, winding;
 segment length: short, rather short, long, very long;
 angle between segments: very acute, acute, not very
acute, right, not very obtuse, obtuse, very obtuse;
138 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146

Fig. 2. Kinds of search patterns.

Fig. 4. Examples of the strict and automatically simplified search

patterns.

 contour size: small, rather small, big, very big; and

 permissible variation of a quantitative characteristic:
absent, small, rather small, big, very big, does not exceed
Fig. 3. Fragment of the library of typical mechanical features. the given value.

 contour shape: square, rectangle, rectangle with rounded From a user's point of view, the presence of certain
corners, rectangle with truncated corners (etc. according elements in target drawings is a decisive factor, the
to the library of typical contours); presence of other is preferable but is not necessary, while
 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
some elements should be absent at all. For this reason, the
possibility to specify obligatory and optional components
as well as elements that should not be in result drawings is
provided in all the search pattern types. If at least one of
obligatory components was not detected in a drawing or it
was found that the drawing contains at least one of
prohibited items, then the drawing is considered as not
corresponding to the query.
In the matching process the system considers various
variants of combination of segments through insignificant
angles. The most optimal variant of combination is the one
that does not lose details and leads to the best final result
of the matching according to the recursive preliminary
analysis [24]. The retrieval system automatically assigns
significant angles (depending on angle values and lengths
of connected segments). The user also has the possibility
to manually change the set of significant/insignificant
angles. The matching algorithm's ability to combine seg-
ments through insignificant angles allows identifying
similarities between objects that have considerable local
deviations from each other.
The evaluations of the impact of the described para-
meters of search patterns to the retrieval performance and
quality are depicted in Table 1.
Working with search patterns' parameters is carried
out mostly by mouse: hypergraphics, lists of qualitative
values to choose from. So, the system's interface is intui-
tive and easy to learn.
3.4. Developing and implementing a search strategy
Consider the work process of a user in the retrieval
system within a particular search session. Generally, before
the search, the user creates an initial query, based on his/
her experience and our recommendations on using the
system, and in the process of implementing his/her strat-
egy refines the query, including a return to early stages,
until a satisfactory result is obtained.
In Fig. 5 the scenario of the dialog on refining a query is
considered. The query refinement allows improving search
relevance and/or speed, narrowing or expanding a search.
As can be seen from the diagram, the proposed envir-
onment provides a user with many ways (strategies) of
solving a search task. The developed tools are aimed to
Table 1
The impact of some parameters to searching.
Parameter Change of the
parameter
The number of analyzed attributes Tightening
The attribute comparison mode
Individual requirements to attributes
The number of obligatory components Increasing
The number of significant angles Increasing
The maximum distance of combination of segments Increasing
through insignificant angles
The depth of the recursive preliminary analysis Increasing
139
allow a user to flexibly (depending on a task) customize a
search.
4. Experiments
4.1. Implementation – the system GrSearch
Our approach is implemented in the system for gra-
phical searching vector mechanical drawings, which is
called GrSearch. Fig. 6 shows its search dialog window,
in which a search pattern that will be used in the search is
selected and adjusted.
Fig. 7 provides examples of the visualization of
matched drawing elements by the system.
4.2. Refinement of a search
We evaluated how much search quality improves when
obvious concretization of a query is made. Experiments
were carried out on a test database containing 4000
drawings, in which almost 7000 projection views were
presented. Four search types were carried out:
 search by an external contour without query concreti-
zation,
 search by an external contour with query refinement,
 search by a graph of contours with default para-
meters, and
 search by a customized graph of contours.
Basing experiments on such search patterns as external
contours and graphs of contours is caused by the fact that
they are the most frequently used. The study of each
search type included execution of 50 queries; sample
images used in each search type were the same; CAD
drafts were used in the queries. The retrieval system was
run on the computer Pentium E6800 3.33 GHz, 2 GB DDR3.
Query refinement was directed towards both tighten-
ing and weakening requirements to search results.
Fig. 8 shows precision/recall graphs for performed
searches, which were built according to the 11-Point
Interpolated Average Precision method [26]. They show
that due to the refinement of queries the average retrieval
precision increased, especially at the beginning – by
10–15%.
Search The probability of missing The probability of obtaining
speed relevant documents irrelevant documents
Increases Increases Decreases
Is not Increases Decreases
changed
Increases Increases Decreases
Decreases Decreases Increases
Decreases Decreases Decreases
140 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
Fig. 5. Scenario of the dialog on refining a query.
In some cases there were no significant changes in
search results, while in many situations changes were very
noticeable and positive.
The average number of additional directives that were
made by a user in the query refinement process is 7.5 for
the searches by external contours and 5.2 for the searches
by graphs of contours; one directive is given by two mouse
clicks.
The degree of involvement of query refinement possi-
bilities in the searches can be seen in Table 2.
In 36% of the searches by external contours, irrelevant
search results were reduced by more than 2 times after the
query refinement. In 60% of the searches by graphs of
contours, irrelevant search results were reduced by more
than 10 times after the query concretization.
The average recall of the searches by external contours
without query refinement is 67.8%, and with the customiza-
tion is 74.9%. The average recall of the searches by graphs of
contours is 71.8% (without the query concretization) and
75.4% (with the query refinement). A relevant document was
considered as a positive detection if it was present on the
first result page that holds up to 75 drawings.
The average speed of the search by an external contour
without refinement is 44 s, and with the query refinement
is 36 s. The average speed of the search by a graph of
contours without the query refinement is 1 min 32 s, and
with the query refinement is 1 min 18 s.
Figs. 9 and 10 show results of some searches that were
conducted in the experiment. As we can see, the results
obtained by concretized queries better correspond to the
sample images.
The experiments also show that the system is well tuned
and provides quite adequate results by non-refined queries.
4.3. User study
In order to evaluate the system from the user perspec-
tive, we involve employees of design bureaus of a manu-
facturing enterprise. A total of 5 designers were involved.
The following tasks were assigned to the designers:
(1) Evaluate results of a search conducted by a query with
default parameters, on the scale of 0 to 10.
(2) Evaluate results of a search, conducted with the query
customization, on the scale of 0 to 10.
(3) Indicate, what features were used in the query refine-
ment process.
(4) Indicate, whether the search results are necessary for
carrying out the design/production engineering.
(5) Indicate, whether the search results are necessary
for studying the experience of the design/production
engineering.
In one experiment the initial search by an external
contour of a part gave a result set of 211 drawings, which
meet the expectations of a specialist (10 points). After
refinement of the query (the length and angle are exactly
the same as that of the sample), the number of results
reduced to 6. On the one hand, these results correspond
to the declared parameters. On the other hand, some
very similar drawings having other dimensions of the
 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146 141

Fig. 6. Search dialog window.

mentioned parameters are disappeared from the results.
As a result, the search with the query refinement was
evaluated by only 6 points. However, it was noted that, in
general, specification of parameters positively affects pre-
cision and speed of the search.
In other experiment the initial search by a graph of
contours (of a projection view of a part) gave a result set of
9 drawings, which was assessed by a specialist by 7 points.
After refining the query (the presence of a contour is
obligatory, a contour's shape is the same) the number of
results reduced to 4, and the retrieved results were
estimated by 10 points.
Among the query refinement features only the possi-
bility of excluding a contour from consideration was asse-
ssed as unclaimed. Regarding the usefulness of retrieved
results, the users’ opinion was unequivocally positive in
most cases.
The users made the following proposals for further
developing the system:
 To add possibility of searching a drawing by its verbal
description in order to simplify the querying process.
 To implement means of reducing a search space. For
example, knowing that there is one or another element
(a hole, a groove, etc.) in a drawing, it will be possible to
limit the retrieval by the presence of such an element.
 Creating a separate searchable database by each bureau
and using the system within a bureau.
 Increasing the search speed.
142 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
Fig. 7. Matched elements are shown in the pop-up window.
5. Discussion
In general, it can be judged that the proposed approach
has proven effective. The created content-based drawing
retrieval environment allows implementing a variety of
search strategies. The system retrieves different results by
the same sample image, depending on the settings of a
search pattern, which are specified during a dialog with
the system. The dialog occurs in a language familiar to a
user: via drawing, in engineering and commonly used
terms and concepts. This contributes to a more complete
mutual understanding between a human and a computer.
As we expected, query refining after first retrieval has led
to the increase in the retrieval precision. The labor intensity
of query refinement is insignificant. It may seem that the
query customization is an unnecessary possibility, especially
considering that retrieval systems usually arrange results in
the descending order of similarity degree. However:
 the simpler the query image, the larger the number of
results;
 viewing a large retrieval can be a tedious process;
 a partial loss of relevant documents is inevitable when
a search system takes into account those aspects that
are not included into a search need of a user.
Communication with the target audience confirmed that
the content-based drawing retrieval is demanded, but unfor-
tunately the designers that tested our system did not have an
experience of working with analogous software products.
Our system satisfied the users' expectations in the vast
majority of searches. In the users' opinion, the possibility to
perform different search types, to refine a query and to see in
search results elementwise correspondence of structures gives
more concrete decisions both in terms of continuation of
the search and subsequent use of information, for example,
the technological one, which in the database accompanies the
drawing found. An interesting fact is that the users expressed
the wish to formulate queries in a verbal form; meaningful-
ness of the representation of drawing images, which is
underlying in our system, brings us closer to solving this
problem.
Visualization of matched elements has proven to be a
rather important tool, which allows a user to determine in
what way it is necessary to modify his/her query. We think
that it will be necessary to provide users also with the
possibility to understand reasons according to which a certain
element of a query drawing was or was not matched with the
corresponding element of a retrieved drawing. In the future,
match marks can be used for solving design problems on the
base of search results, for example, to automatically adapt a
CNC program of a retrieved analog to a new designed part.
Experiments showed that for applying the system to
collections containing tens of thousands of drawings
additional research is needed. Now, even in the case of
several thousands of drawings, the system outputs results
far not at once. It is possible to remove this restriction by
 improving the matching method in such a way that
irrelevant documents are rejected as soon as possible;
 adapting the algorithm to parallel computations (on
GPU or in a distributed manner).
Results of the work [7] say that taking into account
spatial proximity has a little effect on the search quality.
Some our experiments were contrary to this statement.
Sometimes, especially for a process engineer, it is very
important, for example, where and how bores or other
mechanical features of a part are situated.
 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
Fig. 8. Precision/recall for the searches with and without query refinement.
The system can find similarities between images,
regardless that they differ significantly on the base of local
stretching and/or bending. In this respect we have moved
a little further than the work [17], in which the need for
such a search mode was raised for the first time. In our
approach it is possible to significantly narrow or expand
the boundaries of permissible differences between parts.
We believe that the possibilities of adjusting the pre-
cision of searching, performing the search at different
abstraction levels help a user to be sure that none of
relevant documents will be overlooked.
6. Conclusion
The proposed environment for graphical searching techni-
cal drawings has the following characteristics: various levels
of meaningful representation of graphic objects, the possibility
to concretize queries in qualitative concepts, various compar-
ison modes, and visualization of matched elements. Together,
143
this allows a user to implement effective individual strategies
of searching graphical design-engineering information.
We identify the following ways of improving the
retrieval environment:
 automatic synthesis of graphic search patterns by a
verbal description of a drawing;
 automatic generation of recommendations on search-
ing on the base of fuzzy rules built on graphic patterns
of common classes of mechanical parts;
 automatic tuning of search patterns on the base of
relevance feedback, component correspondence marks
and machine learning methods; and
 increasing the search speed.
We hope that in the near future complex retrieval
solutions like ours will enter designers' arsenal of usual
means. For example, it has already happened with the
content-based 3D model retrieval system IMShape [27]
which was incorporated in the regulations of a Russian
144 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146

Table 2
Refinement possibilities and their involvement in searches.

Search type Query refinement possibility Percentage of queries in which

the possibility was used (%)

By an external contour Specifying significant angles 62

Specifying requirement to the length of a segment 50
Specifying requirement to the form of a segment 30
Specifying requirement to an angle 22
Specifying obligatory segments 8

By a graph of contours Specifying obligatory contours 78

Specifying requirement to the size of a contour 42
Specifying requirement to the shape of a contour 40
Specifying contours that are excluded from the consideration 26
Specifying requirement to the position of a contour 10
Specifying requirement to the aspect ratio of a contour 10
Specifying requirement to the orientation of a contour 0
Specifying contours that should not be 0

Fig. 9. First six results of some searches by external contours.

 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
Fig. 10. First six results of some searches by graphs of contours.
enterprise as a mandatory preliminary design phase (see
[28]). The result of introduction of such solutions is the
[2]
significant increase in the degree of automation of the
design-reuse process, reduction of the labor intensity of
searching the documents containing graphic elements, [3]
and reduction of new product design time and costs.
[4]
Acknowledgments
[5]
This work was partially supported by the state assign- [6]
ment of the Ministry of Education and Science of the
Russian Federation on the subject of “Methodology and a [7]
system of retrieving, semantic referencing and extracting
knowledge from Russian- and English-language text and
[8]
graphic documents (in various information resources)”.
References [9]
[10]
[1] S. Allard, K.J. Levine, C. Tenopir, Design engineers and technical
professionals at work: observing information usage in the
145
workplace, J. Am. Soc. Inf. Sci. Technol. 60 (3) (2009) 443–454,
http://dx.doi.org/10.1002/asi.v60:3.
D. Tunkelang, Reconsidering relevance and embracing interaction,
Bull. Am. Soc. Inf. Sci. Technol. 36 (1) (2009) 20–23, http://dx.doi.
org/10.1002/bult.2009.1720360107.
R. Datta, D. Joshi, J. Li, J.Z. Wang, Image retrieval: ideas, influences,
and trends of the new age, ACM Comput. Surv. 40 (2) ;40(2008),
http://dx.doi.org/10.1145/1348246.1348248. (Article 5, 60 pp.).
N. Weber, A. Henrich, Retrieval of technical drawings in DXF format –
concepts and problems, in: LWA 2007 Workshop Proceedings, 2007,
pp. 213–220.
N. Bhatti, A. Hanbury, Image search in patents: a review, Int. J. Doc.
Anal. Recognit. 16 (4) (2013) 309–329.
J. Pu, K. Ramani, On visual similarity based 2D drawing retrieval,
Comput.-Aided Des. 38 (3) (2006) 249–259, http://dx.doi.org/
10.1016/j.cad.2005.10.009.
P. Sousa, M.J. Fonseca, Sketch-based retrieval of drawings using
spatial proximity, J. Vis. Lang. Comput. 21 (2) (2010) 69–80, http:
//dx.doi.org/10.1016/j.jvlc.2009.12.001.
M.J. Fonseca, J.A. Jorge, Towards content-based retrieval of technical
drawings through high-dimensional indexing, Comput. Graph. 27 (1)
(2003) 61–69, http://dx.doi.org/10.1016/S0097-8493(02)00244-3.
M.J. Fonseca, A. Ferreira, J.F. Jorge, Content-based retrieval of technical
drawings, Int. J. Comput. Appl. Technol. 23 (2) (2005) 86–100.
P. Fränti, A. Mednonogov, V. Kyrki, H. Kälviäinen, Content-based
matching of line-drawing images using the Hough transform, Int. J.
Doc. Anal. Recognit. 3 (3) (2000) 117–124.
146 D.R. Kasimov et al. / Journal of Visual Languages and Computing 28 (2015) 134–146
[11] L. Jiao, F. Huang, Z. Teng, An engineering drawings retrieval method
based on density feature and improved moment invariants, in:
ISIP'09 Proceedings, 2009, pp. 352–355.
[12] R. Liu, Y. Wang, T. Baba, D. Masumoto, Shape detection from line
drawings with local neighborhood structure, Pattern Recognit. 43 (5)
(2010) 1907–1916, http://dx.doi.org/10.1016/j.patcog.2009.11.022.
[13] D. Love, J. Barton, Aspects of design retrieval performance using
automatic GT coding of 2D engineering drawings, in: IDDME 2004
Proceedings, 2004, DOI: 10.1080/09537280500180873.
[14] R. Liu, T. Baba, D. Masumoto, Attributed graph matching based
engineering drawings retrieval, in: DAS 2004 Proceedings, LNCS
3163, 2004, pp. 378–388, DOI:10.1007/978-3-540-28640-0_36.
[15] B. Huet, N.J. Kern, G. Guarascio, B. Merialdo, Relational skeletons for
retrieval in patent drawings, in: ICIP 2001 Proceedings, vol. 2, 2001,
pp. 737–740, DOI:10.1109/ICIP.2001.958599.
[16] P. Wang, S. Jiang, An engineering drawing retrieval method based on
nested assignment algorithm, J. Comput. Inf. Syst. 9 (2) (2013)
713–720.
[17] S. Hou, K. Ramani, Structure-oriented contour representation and
matching for engineering shapes, Comput.-Aided Des. 40 (1) (2008)
94–108, http://dx.doi.org/10.1016/j.cad.2007.08.007.
[18] R. Krishnapuram, S. Medasani, S.-H. Jung, Y.-S. Choi,
R. Balasubramaniam, Content-based image retrieval based on a
fuzzy approach, IEEE Trans. Knowl. Data Eng. 16 (10) (2004)
1185–1199, http://dx.doi.org/10.1109/TKDE.2004.53.
[19] H. Aboulmagd, N. El-Gayar, H. Onsi, A new approach in content-
based image retrieval using fuzzy, Telecommun. Syst. 40 (1–2)
(2009) 55–66, http://dx.doi.org/10.1007/s11235-008-9142-9.
[20] L.A. Zadeh, The concept of a linguistic variable and its application to
approximate reasoning, Inf. Sci. 8 (3) (1975) 189–281.
[21] A.V. Kuchuganov, D.R. Kasimov, Graphical search of technical draw-
ings over data warehouses, Prikl. Inform. 2 (2012) 84–92. (in
Russian).
[22] D.R. Kasimov, A.V. Kuchuganov, A.E. Lopatkin, System of graphical
search of technical drawings, Intellekt. Sist. Proizv. 1 (2012) 152–157.
(in Russian).
[23] A.V. Kuchuganov, D.R. Kasimov, Multilevel cognitive analysis in
graphical retrieval of drawings, Pattern Recognit. Image Anal. 23
(4) (2013) 518–523.
[24] D.R. Kasimov, A.V. Kuchuganov, Initial level of perception in the task
of graphical search of technical drawings, in: OGRW-8-2011 Pro-
ceedings, 2011, pp. 103–106.
[25] A.V. Kuchuganov, P.P. Oskolkov, Automation of processing and
semantic coding of digital images, Vestn. Komp. Inf. Tekhnol. 1
(2013) 41–44. (in Russian).
[26] C. Manning, R. Prabhakar, H. Schütze, Introduction to Information
Retrieval, Cambridge University Press, New York, 2008(496 pp.).
[27] D. Kernitsky, IMShape – pick your model, SAPR Graf. 9 (2009) 4–6.
(in Russian).
[28] E.B. Baglayev, Unification – it's easy, 〈http://labktpp.ru/news2.html〉,
2013 (in Russian).