Documente Academic
Documente Profesional
Documente Cultură
art ic l e i nf o a b s t r a c t
Article history: To overcome the shortcomings in the traditional white blood cells (WBCs) identification methods based
Received 25 March 2013 on the color or gray images captured by light microscopy, a microscopy hyperspectral imaging system
Received in revised form was used to analyze the blood smears. The system was developed by coupling an acousto-optic tunable
7 May 2013
filter (AOTF) adapter to a microscopy and driven by a SPF Model AOTF controller, which can capture
Accepted 17 May 2013
hyperspectral images from 550 nm to 1000 nm with the spectral resolution 2–5 nm. Moreover,
Available online 26 June 2013
a combined spatial-spectral algorithm is proposed to segment the nuclei and cytoplasm of WBCs from
Keywords: the microscopy hyperspectral images. The proposed algorithm is based on the pixel-wise improved
Microscopy hyperspectral imaging spectral angle mapper (ISAM) segmentation, followed by the majority voting within the active contour
Blood cells
model regions. Experimental results show that the accuracy of the proposed algorithm is 91.06% (nuclei)
Image segmentation
and 85.59% (cytoplasm), respectively, which is higher than that of the spectral information divergence
(SID) algorithm because the new method can jointly use both the spectral and spatial information of
blood cells.
& 2013 Elsevier Ltd. All rights reserved.
0030-3992/$ - see front matter & 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.optlastec.2013.05.022
226 Q. Li et al. / Optics & Laser Technology 54 (2013) 225–231
information of blood cells. Second, due to uneven staining, color Then, an automatic spatial-spectral segmentation algorithm based
mixing often occurs which leads to the smooth variations of the on the microscopy hyperspectral images of blood smears is
average luminance in some regions of the image. In addition, the presented and used for WBCs segmentation. Unlike those tradi-
microscopy imaging conditions, smear thickness, and background tional light microscopy based segmentation methods, the hyper-
illumination may result in biases and changes in shape, color, and spectral based algorithm can segment the WBCs using both the
scale. All these factors influenced the accuracy of the automatic spatial and spectral information of blood smears. The experiment
segmentation results and lead to the algorithms complicated. results show that the proposed method is effective for WBCs
Therefore, there are still some difficulties and challenging pro- segmentation.
blems to segment WBCs accurately based on the traditional
microscopic images.
The microscopy hyperspectral imaging technology may offer a 2. Materials and methods
solution for these problems. Hyperspectral imaging, also known as
imaging spectroscopy or spectral imaging, is a technology that can 2.1. Blood smear preparation
provide a digital image with far more spectral (color) information
for each pixel than traditional color cameras. The concept of The samples used in this study are peripheral blood smears (or
hyperspectral imaging was originally defined by Goetz in the late peripheral blood films) which are glass microscope slides coated
1980s and the hyperspectral imaging system was originally devel- on one side with thin layer of fresh venous blood. After collecting
oped for the remote sensing of the earth [12]. The advantage of the blood specimen, discard the first few drops of blood and then
hyperspectral imaging technique is that it can acquire a reflectance make the smears as follow: place a drop of blood, about 2 mm in
spectrum for each pixel in the image, which can be used to classify diameter approximately an inch from the frosted area of the slide;
the surface cover materials that can hardly be identified by lightly balance another slide (spreader slide) on the fingertips and
traditional gray or color imaging methods. In recent years, this place the spreader slide at a 301 angle in front of the drop of blood;
technology has been extended from the remote sensing field to the pull the spreader slide back toward the blood droplet and blood
life science field. Different kinds of microscopy hyperspectral spreads along the edge of the spreader slide; quickly push the
imaging system have been developed and used for biomedical spreader across the slide with smooth action; allow the blood
analysis of various biological tissues. For example, Morris et al. smears to air-dry completely and finally dyed with Giemsa [24].
have developed imaging spectrometers for fluorescence and Those well-made peripheral smears which are thick at the frosted
Raman microscopic analysis on rat brainstem [13]. Shonat et al. end and become progressively thinner toward the opposite end
used this technology to generate in vivo hemoglobin saturation are selected and stored for future use.
(SO2) and oxygen tension (PO2) maps in the cerebral cortex of mice
[14]. Some more recent studies have developed various micro-
scopy hyperspectral imaging systems based on prism grating 2.2. Microscopy hyperspectral imaging system and image acquisition
prism (PGP), acousto-optic tunable filters (AOTF), liquid crystal
tunable filter (LCTF), and variable interference filter (VIF) for The microscopy hyperspectral imaging system used in this
different biomedical applications [15,16]. All these studies shown paper consists of a microscope (Nikon 80i, Nikon Co., Ltd., Japan),
that the microscopy hyperspectral imaging technology can obtain an AOTF adapter (CVA200-0.55-1.0-L, Brimrose, USA), a SPF
both images (structural information) and spectra (biochemical Model AOTF controller (VFI-138.5-93-SPS-A-C2, Brimrose, USA),
information) of biological tissues which have the significant a 1/1.8 in. high-density cooled charge coupled device detector
advantages in the area of life science. However, only a few detailed (CCD, DS-2MBWc, Nikon, Japan), a data collection and control
investigations on automated WBCs segmentation method based module (Camera Control Unit DS-U2, Nikon, Japan), and a personal
microscopy hyperspectral imaging technology have been reported. computer (A8800t, Lenovo, China). The front interface of the AOTF
In the past decades, researchers have proposed different adapter was coupled to the microscope with an F-mount adapter
segmentation algorithms to identify targets from the hyperspec- and the back end was coupled to the CCD with C-mount. The light
tral images. Kruse et al. proposed a spectral angle mapper (SAM) source used in the microscope is a 120 W halogen lamp (Epi-
algorithm to detect spectrally active targets in Airborne Visible illumination, Eclipse 80i, Nikon). The AOTF adapter driven by the
Infrared Imaging Spectrometer (AVIRIS) data [17]. This is one of the SPF Model AOTF controller provides narrow bandwidth, rapid
pixel based methods which have been commonly used in hyper- wavelength selection, and intensity control. The minimum wave-
spectral images segmentation. Then, some algorithms were pro- length selection sweep interval of the AOTF adapter is 20ns,
posed to segment and classify objects from the multispectral and which makes it possible for the system to capture the microscopy
hyperspectral images, such as the modified phase correlation hyperspectral images quickly. The designed wavelength range of
(MPC) [18], the active contours and graph cuts [19], the Markov the system is from 550 nm to 1000 nm and the spectral resolution
random field (MRF) model-based algorithm [20], and the support is 2–5 nm (2 nm at 543 nm; 5 nm at 792 nm).
vector machines (SVM) [21]. Moreover, some spatial-spectral The software running on the personal computer can control the
methodologies which consider spatial information along with AOTF adapter to filter the light passed through the microscope and
spectral information to improve the segmentation accuracy of a record the image data on the CCD detector. In this way, the
hyperspectral image also have been investigated recently [22,23]. microscopy hyperspectral imaging system can capture hundreds
These algorithms have been used to analyze the remote sensing of spectral bands covering the narrow spectral features of the
images acquired in fields under the vegetation scene, the urban captured blood smears with high accuracy. As shown in Fig. 1,
environment, and the AVIRIS Indian Pines data set. However, little the microscopy hyperspectral data can be visualized as a three
studies have been reported on the automated white blood cells dimensional cube because of its intrinsic structure, where the cube
segmentation using both the spatial and spectral information face is a function of the spatial coordinates and the depth is a
extracted from the microscopy hyperspectral images. function of wavelength [25]. From the figure it can be seen that the
In this study, an AOTF based microscopy hyperspectral imaging microscopy hyperspectral images of blood smears contain both
(MHSI) system was developed and used to capture hyperspectral spectral and spatial information of blood cells, which makes it
images of blood smears. A preprocessing method was used to possible to improve the accuracy of the automated WBCs segmen-
remove the influence of noises and artifacts computationally. tation with some new hyperspectral based algorithms.
Q. Li et al. / Optics & Laser Technology 54 (2013) 225–231 227
images with a threshold value of SA. However, this method often quantities of representative recognition results from the experts
leads to some segmentation errors when the wavelengths shifting [32]. After the active contour models and the ISAM process, we use
several bands with the influence of noise. To overcome this the majority voting rule to fuse the segmentations by weighting
disadvantage, we use the improved spectral angle mapper (ISAM) each candidate result equally and assigning to each pixel the label
algorithm in this paper. The ISAM algorithm calculates the spectral that most segmentation results agree on. Therefore, the WBCs can
angle (SA) between the reference spectrum and the test spectrum be segmented accurately by using both the spatial and spectral
with shift forward and backward one or two bands according to information contained in the microscopy hyperspectral images.
the following equation:
0 1
N 3. Results
B ∑ t iþj r i C
!! ! ! B C
SAð T ; R Þ ¼ MaxðSAj ð T j ; R ÞÞ ¼ MaxB
B arccos s i¼1
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ffi C
C ð5Þ
@ N N A 3.1. Microscopy hyperspectral images of blood smears
∑ t 2iþj ∑ r 2i
i¼1 i¼1
In this study, microscopy hyperspectral images of blood smears
where j denotes the band number offset (j¼ ¼ −2, −1, 0, 1, and 2), were captured using the home made microscopy hyperspectral
SAj denotes the spectral angle between the reference spectrum imaging system with 100 immersion oil (numerical aperture
and the test spectrum shifted j bands forward or backward. Then 1.30) objective lens. For each blood smear, 80 single bands cover
the maximum one can be considered as the SA between the the wavelength ranges from 550 nm to 1000 nm with the average
reference spectrum and the test spectrum and compared with the band width 5.6 nm were captured and stored in band sequential
threshold. Therefore, the ISAM algorithm can get more accurate format (BSQ). As each single band image contains 1024 1024
results than the SAM algorithm as it can eliminate the influence of 12 bit pixels, one scene of microscopy hyperspectral images of
wavelengths shift. blood smear consists of approximately 160 Mb data. For the
By combining the active contour models and the ISAM, we can convenience of processing, we resized the 160 Mb data cube
get the spatial-spectral algorithm to segment the WBCs from the to four sub-cubes with the size of 512 512 80 ( 12 bit) for
microscopy hyperspectral images of blood smears. Fig. 2 shows further analysis. A representative subtotal of microscopy hyper-
the general flow-chart of the proposed segmentation algorithm. spectral images captured at various wavelengths is shown in Fig. 3.
At the input we have the N-band microscopy hyperspectral data From these single band images it can be seen that there are many
cube of blood smears. After preprocessing, the PCA transformation differences among different parts of the blood cells in different
is performed on the data and the first principal component single band images. These microscopy hyperspectral images can
is selected for the active contour model analysis. Then, the ISAM provide both spatial and spectral information of blood cells which
is performed on the microscopy hyperspectral images to segment is very useful for automatic WBCs segmentation.
them into regions. Consequently, these independent candidate
segmentations results should be somehow combined into a single 3.2. Preprocessing result
final segmentation. Among all the combination methods, majority
vote (also named majority rule, decision fusion or label voting) is After image acquisition, the microscopy hyperspectral images
by far the simplest for implementation. This method does not can be calibrated and normalized by the methods proposed in
need to assume prior knowledge of the behavior of the individual previous section. Fig. 4 shows some single band images extracted
segmentation, and it also does not require training on large from different scenes of microscopy hyperspectral images before
and after preprocessing. From the figure it can be seen that the
unwanted effects (noises and artifacts) contained in the original
images have been removed by the preprocessing procedure.
Therefore, the relative transmittance instead of the absolute
intensity data can be got for WBCs segmentation.
Fig. 3. Single band images selected from microscopy hyperspectral data of a blood smear.
Fig. 4. Preprocessing results, (a)–(c) single band images before preprocessing, and (d)–(f) single band images after preprocessing.
230 Q. Li et al. / Optics & Laser Technology 54 (2013) 225–231
Fig. 5. WBCs segmentation results. The first column: single band images at wavelength 626 nm; the second column: segmentation results of SID algorithm; the third
column: segmentation results of the proposed algorithm; and the last column: ground truth labeled manually.
626 nm, the segmentation results of the SID algorithm and Table 1
the proposed algorithm, and the manually labeled images. From Comparisons of performance of the SID and the proposed algorithms.
the figure it can be seen that the spatial-spectral algorithm can
Measure SID (%) Proposed method (%)
get better results than the SID algorithm as both the spatial
and spectral information has been used to process each Nuclei Cytoplasm Nuclei Cytoplasm
pixel.
In order to compare the effectiveness of the proposed algo- NU 10.15 10.55 6.15 8.56
rithm and the SID algorithm quantitatively, the segmentation NO 4.77 12.46 2.79 5.85
Avg 85.08 76.99 91.06 85.59
results for each method were compared with the manually
extracted segmentation results (the ground truth) and the error
ratio estimated using the following equations [33]:
N U ¼ CardðM−ðM∩AÞÞ=SM ð6Þ
segmented region where a number closer to 1 represents a lower
error and higher accuracy, while a number closer to 0 represents a
N O ¼ CardðA−ðM∩AÞÞ=SA ð7Þ
higher error and lower reliability. Table 1 lists the performance
evaluation results of the SID and the proposed spatial-spectral
Avg ¼ ð1−ðN U ⊕N O ÞÞ ð8Þ
algorithm using Eqs. (6)–(8). From the table it can be seen that the
where M denotes the set of pixels in the manually segmented proposed method has a lower over- and under-segmentation error
region, A denotes the set of pixels in the automatically extracted ratio and a higher average accuracy Avg. Therefore, the proposed
region when using the SID algorithm or the proposed algorithm, segmentation method produced a better performance than the
SM and SA represent the total number of pixels in M and A, comparative algorithm as regards nuclei and cytoplasm. One of the
respectively, NU and NO denote the inaccuracy of the under reasons for the good performance of the proposed methods in
segmentation ratio and over segmentation ratio, respectively, the both NU and NO was that false positive regions were effectively
symbol Card(M) denotes the cardinality of set M, Avg represents removed by using both the spatial and spectral information of the
the accuracy between the ground truth and the automated microcopy hyperspectral images.
Q. Li et al. / Optics & Laser Technology 54 (2013) 225–231 231
4. Conclusion [6] Park J, Keller JM. Snakes on the watershed,. IEEE Transactions on Pattern
Analysis and Machine Intelligence 2001;23:1201–5.
[7] Yang L, Meer P, Foran DJ. Unsupervised segmentation based on robust
WBCs are the principal ingredient in the human immune estimation and color active contour models. IEEE Transactions on Information
system. The morphometry and number of WBCs can be used as Technology in Biomedicine 2005;9:475–86.
a reference to identify the diseases because they vary with respect [8] Jiang K, Liao QM, Xiong Y. A novel white blood cell segmentation scheme
based on feature space clustering. Soft Computing 2006;10:12–9.
to the age for each person [2]. However, manually and semi- [9] Ko BC, Gim J-W, Nam J-Y. Automatic white blood cell segmentation using
automated methods are time consuming works for biologists. stepwise merging rules and gradient vector flow snake. Micron 2011;42:
Therefore, some automated WBCs identification and segmentation 695–705.
[10] Adollah R, Mashor MY, Rosline H, Harun NH. Multilevel thresholding as a
algorithms have been proposed to assist biologists. Most of these
simple segmentation technique in acute leukemia images. Journal of Medical
methods are based on the color or gray images of blood smears Imaging and Health Informatics 2012;2:285–8.
captured by light microscopy, which lead to these algorithms [11] Khashman A. Investigation of different neural models for blood cell type
complicated because of the limited information this kind of images identification. Neural Computing & Applications 2012;21:1177–83.
[12] Goetz AFH, Vane G, Solomon JE, Rock BN. Imaging spectrometry for earth
can provide. This paper presents a homemade microscopy hyer- remote sensing. Science 1985;228:1147–53.
spectral imaging system to capture the blood smears and intro- [13] Morris HR, Hoyt CC, Treado PJ. Imaging spectrometers for fluorescence and
duces a preprocessing method to remove some effects such as the raman microscopy - acoustooptic and liquid-crystal tunable filters. Applied
Spectroscopy 1994;48:857–66.
inhomogeneous spectral response, noises and artifacts of the [14] Shonat RD, Wachman ES, Niu WH, Koretsky AP, Farkas DL. Near-simultaneous
system. Then a spatial-spectral scheme is proposed for segmenting hemoglobin saturation and oxygen tension maps in mouse brain using an
the nuclei and cytoplasm of WBCs. The new method is based on AOTF microscope. Biophysical Journal 1997;73:1223–31.
[15] Li Q, Wang Y, Zhang J, Xu G, Xue Y. Quantitative analysis of protective effect of
the pixel-wise ISAM segmentation algorithm, followed by majority erythropoietin on diabetic retinal cells using molecular hyperspectral imaging
voting within the ACM regions. Unlike those pixel-wise segmenta- technology. IEEE Transactions on Biomedical Engineering 2010;57:1699–706.
tion methods process each pixel independently according to [16] Uhr JW, Huebschman ML, Frenkel EP, Lane NL, Ashfaq R, Liu HY, Rana DR,
Cheng L, Lin AT, Hughes GA, Zhang XJJ, Garner HR. Molecular profiling of
spectra without considering information about spatial structures,
individual tumor cells by hyperspectral microscopic imaging. Translational
the proposed method can improve the segmentation perfor- Research 2012;159:366–75.
mances by the incorporation of the spatial information into [17] Kruse, F.A., Lefkoff, A.B., and Boardman, J.W., The spectral image processing
the segmentation process. The experimental results show that system (SIPS)-software for integrated analysis of AVIRIS data, presented at the
Summaries of the 4th Annual JPL Airborne Geoscience Workshop, Pasadena,
the combined spectral-spatial segmentation method can use both 1992.
the structure (spatial) and biochemical (spectral) information [18] Ertuerk A, Ertuerk S. Unsupervised segmentation of hyperspectral images
of the blood smears to segment the WBCs, which makes the using modified phase correlation. IEEE Geoscience and Remote Sensing Letters
2006;3:527–31.
segmentation algorithm more accurate than the RGB color based [19] De La Vega SH, Manian V. Object segmentation in hyperspectral images using
methods. active contours and graph cuts. International Journal of Remote Sensing
As a preliminary research, we just presented the automated 2012;33:1246–63.
[20] Sarkar A, Vulimiri A, Paul S, Iqbal J, Banerjee A, Chatterjee R, Ray SS.
WBCs segmentation algorithm based on the microscopy hyper-
Unsupervised and supervised classification of hyperspectral imaging data
spectral imaging technology and listed the corresponding results using projection pursuit and Markov random field segmentation. International
to verify the possibility of this method. In the future study, we will Journal of Remote Sensing 2012;33:5799–818.
propose some new algorithms to perform the detailed analysis on [21] Mountrakis G, Im J, Ogole C. Support vector machines in remote sensing:
a review. ISPRS Journal of Photogrammetry and Remote Sensing 2011;66:
the blood smears, such as the classification of different kinds of 247–59.
blood cells, the cell counting, the morphological parameters calcu- [22] Gorretta N, Rabatel G, Fiorio C, Lelong C, Roger JM. An iterative hyperspectral
lation, and the degree of oxygen saturation retrieval, which may be image segmentation method using a cross analysis of spectral and spatial
information. Chemometrics and Intelligent Laboratory Systems 2012;117:
useful for a better interpretation of further studies involving the 213–23.
structural and biochemical changes and experimental models of [23] Li J, Bioucas-Dias JM, Plaza A. Spectral-Spatial hyperspectral image segmenta-
diseases. tion using subspace multinomial logistic regression and Markov random
fields. IEEE Transactions on Geoscience and Remote Sensing 2012;50:809–23.
[24] Corrons JLV, Albarede S, Flandrin G, Heller S, Horvath K, Houwen B, Nordin G,
Sarkani E, Skitek M, Van Blerk M, Libeer JC, Eqalm. Guidelines for blood smear
Acknowledgment preparation and staining procedure for setting up an external quality assess-
ment scheme for blood smear interpretation. Part I: control material. Clinical
Chemistry and Laboratory Medicine 2004;42:922–6.
This work is supported in part by the National Natural Science
[25] Rajpoot, K.M. and Rajpoot, N.M., Wavelet based segmentation of hyperspectral
Foundation of China (Grant no. 61177011, 61240006), the project colon tissue imagery, Proceedings INMIC 2003. 7th International Multi-Topic
supported by the Shanghai Committee of Science and Technology, Conference (IEEE Cat. no.03EX757), pp. 38–43|x+497, 2004.
China (Grant no. 11JC1403800), and the State Key Development [26] Wachman ES, Niu WH, Farkas DL. Imaging acousto-optic tunable filter with
0.35-micrometer spatial resolution. Applied Optics 1996;35:5220–6.
Program for Basic Research of China (Grant no. 2011CB932903). [27] Ceelie H, Dinkelaar RB, van Gelder W. Examination of peripheral blood films
using automated microscopy; evaluation of Diffmaster Octavia and Cellavision
References DM96. Journal of Clinical Pathology 2007;60:72–9.
[28] Guo N, Zeng L, Wu Q. A method based on multispectral imaging technique for
white blood cell segmentation. Computers in Biology and Medicine 2007;37:
[1] Khashman A. IBCIS: intelligent blood cell identification system. Progress in 70–6.
Natural Science 2008;18:1309–14. [29] Guan Y, Li Q, Wang Y, Liu H, Zhu Z. Pathological leucocyte segmentation
[2] Huang D-C, Hung K-D, Chan Y-K. A computer assisted method for leukocyte algorithm based on hyperspectral imaging technique. Optical Engineering
nucleus segmentation and recognition in blood smear images. Journal of 2012;51.
Systems and Software 2012;85:2104–18. [30] Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. International
[3] Dorini LB, Minetto R, Leite NJ. Semiautomatic white blood cell segmentation Journal of Computer Vision 1987;1:321–31.
based on multiscale analysis. Biomedical and Health Informatics, IEEE Journal [31] Klemencic A, Kovacic S, Pernus F. Automated segmentation of muscle fiber
of 2013;17:250–6. images using active contour models. Cytometry 1998;32:317–26.
[4] Froom P, Havis R, Barak M. The rate of manual peripheral blood smear reviews [32] Lam L, Suen CY. Application of majority voting to pattern recognition: an
in outpatients,. Clinical Chemistry and Laboratory Medicine 2009;47:1401–5. analysis of its behavior and performance. IEEE Transactions on Systems Man
[5] Harms H, Aus HM, Haucke M, Gunzer U. Segmentation of stained blood cell and Cybernetics, Part A: Systems and Humans 1997;27:553–68.
images measured at high scanning density with high magnification and high [33] Ko B, Seo M, Nam JY. Microscopic cell nuclei segmentation based on adaptive
numerical aperture optics. Cytometry 1986;7:522–31. attention window. Journal of Digital Imaging 2009;22:259–74.