Parallel Computing 34 (2008) 727–734

Contents lists available at ScienceDirect

Parallel Computing
journal homepage: www.elsevier.com/locate/parco

A distributed genetic algorithm for restoration of vertical line scratches

Francesco Isgrò a,*, Domenico Tegolo b
a
Dipartimento di Scienze Fisiche, Università di Napoli Federico II, 80126 Napoli, Italy
b
Dipartimento di Matematica ed Applicazioni, Università di Palermo, Via Archirafi 34, 90123 Palermo, Italy

a r t i c l e i n f o a b s t r a c t

Article history: This paper reports a distributed algorithm for the restoration of still frames corrupted by
Available online 19 September 2008 vertical line scratches. The restoration is here approached as an optimisation problem,
and is solved using an ad-hoc Genetic Algorithm. The distributed algorithm is designed fol-
lowing a pipeline logical structure. The front end is a network of standard workstations
Keywords: with heterogeneous operating systems. The quality of image is appreciable and the compu-
Line scratch removal tational time is quite low with respect the sequential version.
Genetic algorithm
Ó 2008 Published by Elsevier B.V.
Optimisation problem
Parallel computing
Distributed system

1. Introduction

Line scratches in the photograms of old movies can be originated by a number of causes, and typically they are vertical
lines across the frame. From an image processing point of view we can define a vertical scratch as a few columns-wide por-
tion of the image where the intensity information is completely lost. Fig. 1 shows an example of image corrupted by a variety
of line scratches.
Over the last few years lot of work has been done for the digital restoration of old movies, due to the increasing compu-
tational power of microprocessors together with the low cost of computers and the very popular use of digital image pro-
cessing techniques. In particular a number of methods have been presented for the removal of line scratches.
In a single frame all scratches are not necessarily caused by similar reasons therefore, in order to cope with all of them,
complex models for the scratch are often used. Moreover a fully automatic restoration system is very hard to develop, be-
cause the correction of misalignment of brightness, detection and removal of shot [1] and scratch, noise reduction, etc., rep-
resent the main obstacles to solve the problem in the best way.
Our method intends to simplify the complexity of the model for the scratch removal and an implementation of it will be
able to restore only vertical line scratches in each frame without considering any knowledge on the other frames of the se-
quence. The main novelty of our restoration approach is that we recover the missing information inside a line scratch by
adopting a Genetic Algorithm approach. Our method can be briefly divided in two steps:

Interpolation: modifies the central column of the scratch by linear interpolation from its nearest neighbours outside the
scratch;
Genetic algorithm: restores the information assuming the nearest pixels around the scratch as the best chromosomes.

* Corresponding author.
E-mail addresses: francesco.isgro@unina.it (F. Isgrò), tegolo@math.unipa.it (D. Tegolo).

0167-8191/$ - see front matter Ó 2008 Published by Elsevier B.V.

doi:10.1016/j.parco.2008.09.005
728 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734

Fig. 1. Example of line scratch, it was extracted from http://www.mee.tcd.ie/ack/cd/lines/lines.htm.

The use of a GA comes about since line scratch restoration can be considered as an optimisation problem (assuming as
initial solution the one obtained by the interpolation step), and Ga’s are known as a robust heuristic method which is able
to find the optimal solution in the solution space of a given problem. The use of a GA for the optimisation, instead of faster
optimisation method is dictated by the fact that the function we minimise is not analytically expressed, but we can only give
an optimality condition (which can be interpreted as a convergence condition).
In this paper, we discuss a distributed implementation of this image restoration algorithm. The paradigm we follow for
the parallelisation of the genetic algorithm is very close to the classical coarse grained. Although the current implementation
is on a small size distributed system (only 4 stations) we obtain an average speed up with respect to the sequential version of
the algorithm of about 2.5.
The paper is structured as follows. Next section reviews some previous work on scratch restoration. Section 3 explains our
genetic algorithm for scratch restoration, and in Section 4 we introduce the distributed version. Details of the current dis-
tributed implementation and experimental results are given in Section 5. Section 6 is left to the final remarks.

2. Previous work on scratch restoration

Images, and in particular movies, are nowadays the most used media for delivering information and emotions. In the last
20 years movie technologies have achieved a number of results in terms of image quality, and old corrupted films can prove
unpleasant to see to the average spectator.
Digital image processing can give good tools to retrieve and restore some visual defects in each frame of old films. Some
companies have realised manual or semiautomatic system for digital film restoration [2], other present systems that are fully
automatic [3], but only for a particular class of defects. The state of art is therefore still far from the realisation of a fully auto-
mated system capable of coping with any kind of corrupted frame.
Algorithms for detecting scratches in an image were presented by Buisson et al. in [4,5], they combine two detectors:
morphological transformation [6] and dynamic detector (Kalman Filter). Morphological filters [7] and motion estimation
techniques, to track scratch on the frame, gave an appreciate method to sniff out line scratches. The line scratches were re-
stored by a simple polynomial interpolation, based on Fourier series.
A number of methods have been presented for the removal of line scratches. For instance [8] used information in the same
portion of the image coming from temporarily close frames in the sequence (Block-Matching). Other authors recover the
missing information studying the edge around the scratch [9,10], or produce estimation of motion prediction [11]. The res-
toration of line scratches using an articulate mathematical model is proposed by Kokaram [12], and a family of 1D linear
model along the row of image has been proposed by [13]. A parallel video restoration system [14], based on a 3D auto-
regressive algorithm to reduce noise from the sequences, has been developed at the Nanyang Technological University
[15], in order to restore old motion picture [16], this system is able to suppress mixed noise and recover lost signal.
In spite of the corruption present in the images, the human viewer can still interpret the scene, and therefore reconstruct
the information missing from the image. Following this observation some image restoration methods are based on scene
interpretation. Kokaram in [17] proposed a Stochastic model as 2D auto-regressive for the basis of image interpretation
and interpolation; Morris et al. [11] use Markov chain Monte Carlo methods to detect and remove line scratches. Another
approach, which is the one followed by the distributed algorithm presented in this paper, tackles the image restoration prob-
lem as an optimisation problem [18]. In this method the cost function used attempts to reduce the gap between line
scratches and rest of image.

3. Genetic algorithm for scratch removal

Genetic algorithms (GA) are considered as a general optimisation algorithm, and they also are known as evolutionary
algorithms since their first introduction by Holland [19] and their first applications in 1980s [20]. They are based on the evo-
lution of a particular species (biological or not) and are inspired by the Darwinian theory. In accordance to a specific problem,
a binary representation allows to represent the elements of a population in the digital world, therefore they can be manip-
ulated and elaborated to extract the best solution.
 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734 729

Table 1
Simple model of genetic algorithm

Phase Corresponding steps in our algorithm

1 Create initial population 1
2 Mating of chromosomes 2,3
3 Select new population 4,5
4 Check convergence otherwise goto 2 6

Scratch removal can be interpreted as an optimisation problem, and in this way it is not necessary to give any interpre-
tation of the image [18]. This characteristic allows to assign to each pixel in the scratch a grey level value that satisfies a cost
function based on colour information. The cost function we adopt here is designed to minimise the difference between the
scratch and the not corrupted part of the image around the scratch.
Let B be a scratch interval in the image of width w, i.e., the set of image columns B ¼ ðci0 ;    ; ci0 þw1 Þ, and let A and C be
the two intervals on both sides of the scratch, i.e., A ¼ ðci0 w ;    ; ci0 1 Þ and C ¼ ðci0 þw ;    ; ci0 þ2w1 Þ. For each row i we define
the following variables: rA is the standard deviation of the pixels in the interval A; rC , the standard deviation of the pixel in
the interval C; rB the standard deviation in the interval inside the scratch. We indicate we lB , lA and lC , the mean values of
the same intervals. Given these definitions we say that a set of grey levels can be assigned to the pixels of the scratch B on the
line i if the following conditions are satisfied
minðrA ; rC Þ < rB < maxðrA ; rC Þ
minðlA ; lC Þ < lB < maxðlA ; lC Þ
ð1Þ
jlB;t  lB;t1 j < e
jrB;t  rB;t1 j < e;
where rB;t and lB;t indicate the values of lB and rB at the iteration t, and e is an opportune threshold.
The optimal values for the pixels on the scratch are determined by a GA, which is started after the pixels in the central
column (i.e., the column obtained adding to i0 the result of the integer part of ðw  1Þ=2) of the scratch have been given a
colour by linear interpolation from the pixels of the two intervals A and C defined above, all the other pixels in the scratch
are assumed having intensity value equal to zero. In Table 1 it is given a simple model of a GA.
The starting population of the GA concerns the arbitrary number of initial chromosomes. Our method creates the initial
chromosomes with an interpolation on the nearest pixels on the row: two-dimensional mask centred in the central column
of the scratch and including a set of neighbour pixels outside of the scratch. The sequential version of our algorithm performs
the restoration row by row independently. Let us assume we are restoring line i, the GA can be sketched as follows:

(1) select a starting population of chromosomes, where a chromosome for us is the 8-bit grey level of each pixel; the pop-
ulation we select is composed by all the pixels in a mask centred in the central column of the scratch, and including
the two intervals A and C. The number of rows included in the mask is not fixed, and it is left as a parameter defined by
the user;
(2) perform the crossover operation between random couples of adjacent chromosomes, i.e., each pixel is mated with one
of its 8-connected neighbours included in the population; the crossover is performed swapping the low order parts in
the binary representation of the grey level value. For instance, given the two chromosomes (0000 0101) and (0010
1000), the two new chromosomes are (0000 1000) and (0010 0101);
(3) perform a bit-mutation with probability c to change one or more bit randomly;
(4) compute the fitness function; for each chromosome (even the parents) h the fitness value is given by the function
FðhÞ ¼j p  k j ð2Þ
where p is the pixel value of the centre of the mask;
(5) for each pixel we have now three candidates (the parent and the two new chromosomes), we select only one of them
for the new population as the one having the highest score with the fitting function; after this stage the size of the
population is again the original one;
(6) if the convergence criteria given in condition (1) is satisfied (or the maximum number of iterations is reached) exit else
goto (2).

4. Distributed algorithm

Over the last decade some work has been done in the field of parallel genetic algorithms, see [21] for a review of relevant
work. The reason for the success of this field is that Ga’s can be very time consuming when applied to large and hard prob-
lems. Different classes of algorithms have been identified: global parallelisation, coarse grained, fine grained, and a hybrid
methodology which includes algorithms combining in some way the characteristics of the previous three classes.
730 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734

As explained in Section 3 the serial algorithm works on a single scratch by evolving different populations, as it restores a
single scratch on a line by line basis, i.e., the resulting value on image line i of a scratch is the result of the evolution of an
independent population pi . This leads to a natural way to parallelise the algorithm, by distributing the various populations on
different processing elements (ideally one for each population). A second level of parallelism is introduced as we can paral-
lelise the restoration of all the scratches in the image. If we consider the classification of Ga’s reported above we can say that
our algorithm falls in the coarse grained class, since we partition the problem in different sub-populations. The main differ-
ence with the standard coarse grained algorithm is that in our case no migration (chromosome moving from a population to
the other in order to improve evolution) is included. Reason for this is that our serial algorithm was working naturally on
independent populations.
Fig. 2 shows the distributed architecture on which genetic restoration algorithm will be running. Starting from the input
image and the position of its scratches, the network returns as output the restored input image.
From the architectural point of view that structure can be seen as a vector of pipes. Each pipe works on each scratch in the
following manner:

(1) Initial Population module (IPM): this module defines the starting population. This task administers a random partition
of the populations and distributes them among the processing units dedicated to the next module. The cardinality of
the partition is given by the number of workstations present on the network.
(2) Mating and reproduction module (MRM): this module is composed by two tasks:
 Crossover module (CM): first of all this module identifies the number of workstations able to compute crossover
operation. Those workstations receive equitable partition of population and then perform the crossover. It is worth
to mention that, if we define as CP the Cardinality of Population, then the maximum number of workstation on
which IPM can distribute those partitions is CP
2
.
 Bit-mutation module (BM): it performs a random bit-wise mutation on the elements of a random subset of the
whole population. The process extracts a subset from whole population and on integer representation of each ele-
ment does a mutation on a random bit.
(3) Fitting function module: this module calculates the fitness function (2). In this case the p chromosome is given by ele-
ment sites on the cross between central column of the i scratch and the central row of the partition population, h and k
represent other chromosomes of the mask. In the following example (Fig. 3) the scratch is 5 columns wide (those filled
with ‘‘?”s, i.e., columns from 7 to 11). The pth chromosome is in this example the pixel of coordinates (9,3) and the
dimension of the mask is 5  5.

Scratch 1 Scratch M

Initial ........... Initial Initial ........... Initial

Population Population Population Population

Crossover Crossover ........ Crossover Crossover

........ ........ ........
........ ........ ........
Bit Mutation ........ Bit Mutation ........ Bit Mutation ........ Bit Mutation

MRM MRM MRM MRM

Fitness Fitness Fitness Fitness

Fig. 2. Logical structure of the distributed system.

Fig. 3. Example of selection of scratch.

 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734 731

(4) Converge checking: this module evaluate the convergence (1), it returns the value 1 or 0 if the convergence criteria is
satisfied or not [18]. A check-sum element, linking all workstations, identifies an interactive check to restart the pro-
cess if this component is false (0). In other words the process will be run until check-sum is false. If the number of
iterations is larger than a given threshold the algorithm is stopped.

5. Experimental results

In this section we report the current implementation of the distributed algorithm, and results of the restoration of cor-
rupted images.
Since the algorithm works by starting a genetic algorithm for each scan-line of the scratch, it is clear that the algorithm
has a very high degree of parallelism, as it starts a large number of independent processes that can run in parallel. For in-
stance in an image of 288 rows and 6 scratch lines, there are 1728 independent processes that can potentially run in parallel.
Our distributed system consisted of four networked PCs (Pentium III 733 MHz), and we did not implement a full paral-
lelisation because of clear computational constraints. We then decided to map a single scratch on each available PC (if more
scratches than PCs then the algorithm waits for the first PC available). One of the PCs must of course act also as master PC in
order to take care of the distribution of the scratches over the distributed system.
Inside each PC we have a second level of parallelism associating a process to the restoration of each row (inside the
scratch) and we run a small group of them (10 processes) in parallel. Every time a process terminates, a new one replaces
it until the whole scratch has been restored. As for the first level of parallelism, inside each PC we need a master process to
start the various sub-processes. In terms of speed-up, in this way there is not too much increase with respect the time for the
restoration of a single scratch, which is quite similar to the one of the serial version (as the process are run in time sharing on
the same processors); but there is of course an improvement in the time spent for the restoration of the whole image, as four
scratches are restored in parallel on different processors. This implementation was done for testing the validity of the dis-
tributed approach, more than providing a fully distributed version of the algorithm.
In this section we report the results obtained on three different frames. For copyright reasons we cannot show results on
real old frames from corrupted movies, but we show results on three old photographs, digitised into grey level images of size
463  288 to which we added artificial Gaussian scratches, i.e., the level of data corruption goes like a Gaussian centred in
the centre of the scratch. However we must notice that our approach does not make any Gaussian assumption on the level of
data corruption, as it assumes that there is no information at inside the scratch. In more practical terms we do assume that
inside the scratch all the pixel intensities are set to zero. The average width of the scratches introduced in the images is 4
columns.
For each one of the experiments we report the total computational time for the sequential algorithm and for the distrib-
uted version. For each scratch we also report the minimum, maximum and mean number of iterations before converging

Fig. 4. Results of the restoration algorithm. (a) Original image, (b) Artificially corrupted image, (c) Restored image, (d) Details of the restored parts.
732 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734

among all the populations, and the number of populations (i.e., scan-lines inside the scratch) for which the GA did not con-
verge after the maximum number of iterations. For these last cases we perform the restoration by simple linear interpola-
tion. For this experiment we set the maximum number of iterations for the genetic algorithm to 400,000, and the threshold e
used in the condition (1) to 0.0001. The probability c associated to the bit-mutation is set here to 0.01. The population asso-
ciated to a scan-line is selected considering the mask including the two lines above and the two lines below the scan-line.
The first example, Fig. 4, reports the computational time for the sequential algorithm of 1574.86 s, and for the distributed
version 565.7 s, for a speed up of 2.7. The results on the iteration are reported in the table below. As shown in the image the
result of the restoration is satisfactory (see Table 2).
For the second example, Fig. 5, we obtained a computational time of 2549.45 s, and of 753.43 s for the distributed version,
with a speed up of 3.3. The results on the iteration are given in Table 3. Again the quality of the restoration is satisfactory.

Table 2
Report on the number of iterations for each scratch for the image in Fig. 4

Scratch Min # iter. Mean # iter. Max # iter. # non converging pop.
1 3 10,832 369,063 48
2 6 16,000 361,873 67
3 3 10,765 393,891 60
4 2 25,079 383,287 58

Fig. 5. Results of the restoration algorithm. (a) Original image, (b) Artificially corrupted image, (c) Restored image, (d) Details of the restored parts.

Table 3
Report on the number of iterations for each scratch for the image in Fig. 5

Scratch Min # iter. Mean # iter. Max # iter. # non converging pop.
1 3 22,192 394,200 79
2 5 33,395 346,307 117
3 4 41,474 397,994 106
4 2 19,869 330,402 69
 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734 733

Fig. 6. Results of the restoration algorithm. (a) Original image, (b) Artificially corrupted image, (c) Restored image, (d) Details of the restored parts.

Table 4
Report on the number of iterations for each scratch for the image in Fig. 6

Scratch Min # iter. Mean # iter. Max # iter. # non converging pop.
1 3 31,641 390,529 67
2 5 25,831 372,262 51
3 4 26,778 341,799 73
4 2 26,759 386,415 74
5 8 32,643 378,097 148
6 6 23,175 358,109 78

It must be noted the large number of lines for which convergence is not reached, due the fact that the threshold value for
condition (1) is too small for this case.
The last image, Fig. 6, is a case where there are more scratches droppedj.kastelein@elsevier.com than available PC’s in the
distributed system. In this case the time spent by the sequential version is 3916.49 s, whereas the distributed version took
1809.04 s, with a speed up of 2.1. The quality of the restoration in this case is not particularly good, as you can see some salt
and pepper effect in the restoration of the 4th scratch. This result is probably due to the fact that the two areas around the
scratch have quite different distributions of the grey levels. The report on the number of iterations is given in Table 4.

6. Conclusions

The implementation of a method for scratch restoration on a distributed system has been presented in this paper. The
restoration is treated in this work as an optimisation problem, and it is solved by means of an ad hoc genetic algorithm. This
avoids the need of any image interpretation, differently from other common restoration techniques.
Further work needs to be done to improve the fitness function in order to refine the image restoration and to discriminate
chromosomes associated with different classes of scratches; moreover studies will be proceeding to evaluate new topologies
of distributed systems for specific data.
The experimental results reported in this paper show that the algorithm returns reasonably good results in terms of
images restoration and that the speed up obtained by the use of the distributed system is appreciable. The speed up would
be more evident on an on-line system restoring whole sequences.
734 F. Isgrò, D. Tegolo / Parallel Computing 34 (2008) 727–734

In terms of comparison with other restoration methods we do not think that the quality returned by our algorithm is
superior, and moreover our algorithm is very slow, especially in its sequential version. A possible way to obtain a faster algo-
rithm that we are planning to try is to consider a single population for the whole scratch, instead that an independent pop-
ulation for each scan-line. This would have the double advantage of a faster algorithm (only one large population needs to
evolve), and it would introduce a larger genetic variety. We are also aware that we could enrich the genetic variety of the
population by using a less strict crossover rule, than the one adopted now: for instance making the amounts of bits used
from each relative a random quantity, instead than the simple 4 + 4 rule used now. From the point of view of the distributed
genetic algorithm such an approach would fit well in a classical coarse grained approach, by dividing the large population is
small sub-populations evolving in parallel, and introducing a certain level of migration among the sub-populations.

Acknowledgement

The authors would like to thank the anonymous referees for their detailed comments and suggestions made toward the
improvement of this paper. This work makes use of results produced by the PI2S2 Project managed by the Consorzio
COMETA, a project co-funded by the Italian Ministry of University and Research (MIUR) within the Programme Operativo
Nazionale ‘‘Ricerca Scientifica, Sviluppo Tecnologico, Alta Formazione” (PON 2000–2006). More information is avaliable at
http://www.pi2s2.it and http://www.consorzio-cometa.it.

References

[1] A. Machı`, M. Tripiciano, Video shot detection and characterization in semi-automatic digital video restoration, in: Proceedings of ICPR2000, vol. I, 2000,
pp. 855–859.
[2] <http://www.vcpc.univie.ac.at/activities/projects/frame/general.html>, Tech. rep.
[3] P. Shallauer, A. Pinz, W. Haas, Automatic restoration algorithms for 35 mm film, Journal of Computer Vision Research 1 (3) (1999) 60–85.
[4] O. Buisson, B. Besserer, S. Boukir, F. Helt, Deterioration detection for digital film restoration, in: Proceedings of Computer Vision and Pattern
Recognition, 1997, pp. 78–84.
[5] L. Joyeux, O. Buisson, B. Besserer, S. Boukir, Detection and removal of line scratch in motion picture films, in: Proceedings of CVPR99, vol. I, 1999, pp.
548–553.
[6] D. Ferrandiere, Restauration automatique de films anciens, PhD thesis, Ecole National Superieure des Mines de Patis, 11/1997.
[7] V. Di Gesù, Artificial vision and soft computing, in: Fundamenta Informaticae, vol. 39, 1999, pp. 101–119.
[8] O. Hadar, M. Robbins, Y. Novogrozky, D. Kaplan, Image motion restoration from a sequence of images, Optical Engineering 35 (10) (1996) 2898–2904.
[9] J. Canny, A computational approach to edge detection, IEEE Transactions on PAMI 8 (1986) 679–689.
[10] N.-D. Kim, S. Udpa, Non linear operation for edge detection and linear scratch removal, in: IEEE International conference on Systems, Man, and
Cybernetics, vol. 5, 1998, pp. 4401–4404.
[11] R.D. Morris, W.J. Fitzgerald, A.C. Kokaram, A sampling based approach to line scratch removal from motion picture frames, in: Proceedings of ICIP’96,
vol. 1, 1996, pp. 801–804.
[12] A. Kokaram, Motion Picture Restoration, Springer, 1998.
[13] L. Maddalena, Efficient methods for scratch removal in image sequences, in: IEEE Proceedings of 11th International Conference on Image Analysis and
Processing, 2001, pp. 547–552.
[14] M. Ceccarelli, A. Petrosino, High performance motion analysis for video restoration, in: IEEE Proceedings of 14th International Conference on Digital
Signal Processing, vol. 2, 2002, pp. 689–692.
[15] W. Goh, M. Chong, S. Kalra, D. Krishnan, Bi-directional 3d auto-regressive model approach to motion picture restoration, in: Proceedings of IEEE
ICASSP-96, vol. 4, 1996, pp. 2275–2278.
[16] S.J. Doo, M.G. Kang, Generalized adaptive spatio-temporal auto-regressive model for video sequence restoration, in: Proceedings of IEEE ICIP 99, vol. 1,
1999, pp. 185–188.
[17] A. Kokaram, Removal of line artifacts for digital dissemination of archived film and video, in: Proceedings of International Conference on Multimedia
Computing and Systems, vol. 2, 1999, pp. 245–249.
[18] F. Isgrò, D. Tegolo, Scratch detection and removal from static images using simple statistics and genetic algorithms, in: Proceedings of IEEE ICIP’01, vol.
1, 2001, pp. 265–268.
[19] J. Holland, Adaptation in Natural and Artificial Systems, The University of Michigan Press, Ann Arbor, 1975.
[20] D. Goldberg, Genetic Algorithms in search, Optimization and Machine Learning, Addison-Wesley, 1989.
[21] E. Cantù-Paz, A survey of parallel genetic algorithms, Tech. rep., Illinois Genetic Algorithms Laboratory (T.R.97007/1997).