Image processing with scikit-image and Dash

Emmanuelle Gouillart
plotly and scikit-image team

@EGouillart
Image processing for science and business
What is scikit-image?
An open-source (BSD)

generic image processing library

for the Python language

(and NumPy data arrays)
What is scikit-image?
An open-source (BSD)

generic image processing library

for the Python language

(and NumPy data arrays)

for 2D & 3D images

scientific images rather than
Instagram Filters

simple API & gentle learning curve

Image processing for science
Manipulations of images to
I Transform them for other purposes: color balance,
enhancement, filtering, inpainting, ...
I Extract information from them: classification, measurements...
a) flltering b) feature c) segmentation d) measures
extraction

non-local means denoising blob detection super-pixels skeleton & local diameter

total variation denoising

ridge detection particle properties
segmentation from markers
What scikit-image is not
No deep learning

mostly because of architecture

(GPU) constraints
But a great tool for
I pre-processing: intensity and
size normalization, data
augmentation, ...
I post-processing: instance
segmentation, measures...
What scikit-image is not
No deep learning

No bleeding-edge algorithm

mostly because of architecture

(GPU) constraints
But a great tool for
mature well-established
I pre-processing: intensity and
algorithms, limit API size
size normalization, data
augmentation, ...
I post-processing: instance
segmentation, measures...
The Scientific Python ecosystem

Development
image file(s) environment

array 1 75 ... 32

...
10 3 ... 2

machine learning

8 23 ... 36 12 43 ... 6 1 41 ... 98

...
...

...

1 13 ... 0 66 ... 97 1 38 75 ... 6

...

8 5 ... 73 4 10 ... 55
...

...

78 98 ... 9 10 3 ... 9

image properties
object properties
visualization
First steps with scikit-image

>>> from skimage import io , filters , measure

>>> im = io . imread ( ’ image_file . png ’)
>>> im . shape # 2 - D array
(512 , 512)
>>> threshold = filters . threshold_otsu ( im ) # float
>>> im_binary = im < threshold # numpy mask
>>> labels = measure . label ( im_binary )
>>> type ( im ) , type ( labels ) # images are numpy arrays
( < type ’ numpy . ndarray ’ >, < type ’ numpy . ndarray ’ >)
>>> labels [0 , 0] # pixels are array elements
0
x
I Images are NumPy arrays
I Pixels are array elements
I Simple API relying on functions
new = function(image)
Also works with 3-D images
Most functions work for
I Optional parameters: keyword
2D & 3D arguments
Manipulating images as numerical (numpy) arrays

I Pixels are arrays elements

import numpy as np
image = np . ones ((5 , 5) )
image [0 , 0] = 0
image [2 , :] = 0
x
Manipulating images as numerical (numpy) arrays

I Pixels are arrays elements

import numpy as np
image = np . ones ((5 , 5) )
image [0 , 0] = 0
image [2 , :] = 0
x

>>> coffee.shape
(400, 600, 3)
>>> red channel = coffee[...,
0]
>>> image 3d = np.ones((100,
100, 100))
API of scikit-image
module skimage

submodule filters restoration segmentation ...

function denoise_bilateral

variables input output

array (array)
+ optional
parameters
Consistent names
Consistent names

filters . median ( image , selem=None , ←-

out=None , . . . )
morphology . binar y_erosio n ( image , ←-
selem=None , out=None )
restoration . d e n o i s e _ b il a t e r a l ( ←-
image , win_size=None , . . . )
Consistent names

filters . median ( image , selem=None , ←-

out=None , . . . )
morphology . binar y_erosio n ( image , ←-
selem=None , out=None )
restoration . d e n o i s e _ b il a t e r a l ( ←-
image , win_size=None , . . . )
Combining scikit-image and scikit-learn
Feature extraction followed by classification

I Daisy features (skimage)

I Random Forest classifier (sklearn)
I Back to image processing: Gaussian filtering, thresholding and
mathematical morphology (skimage).
Datasheet
Package statistics
I http://scikit-image.org/
I Release 0.15 (1 - 2 release per year)
I > 200 contributors, 5-10 maintainers
I 20000 unique visitors / month Among 1000 best ranked
packages on PyPi
Documentation at a glance: galleries of examples

Powered by sphinx-gallery
Getting started: finding documentation
Auto documenting your API with links to examples
Auto documenting your API with links to examples
Picture denoising
Picture denoising
Denoising tomography images

In-situ imaging of phase separation

in silicate melts From basic (generic) to advanced
Wanted: gallery examples using (specific) filters
scientific datasets
Denoising tomography images

Histogram of pixel values

From basic (generic) to advanced

(specific) filters
bilateral = restoration.denoise_bilateral(dat)
bilateral = restoration.denoise_bilateral(dat, sigma_range←-
=2.5, sigma_spatial=2)
tv = restoration.denoise_tv_chambolle(dat, weight=0.5)
Massive data processing and parallelization

Competitive environment: some other tools use

GPUs, Spark, etc. scikit-image uses NumPy!
Roadmap: experiment with numba, Pythran
Divide into blocks: use util.view as blocks to
iterate conveniently over blocks
Parallel processing: use joblib.Parallel or
dask (da.map overlap)
Better integration desirable (border effects)
experimental apply parallel function
Experimental chunking and parallelization
Keeping interaction easy for large data
from joblib i m p o r t Memory
memory = Memory ( cachedir= ’ . / c a c h e d i r ’ , verbose =0)

@memory . cache
d e f mem_label ( x ) :
r e t u r n measure . label ( x )

@memory . cache
d e f mem_threshold_otsu ( x ) :
r e t u r n filters . thres hold_ots u ( x )

[...]
val = m e m _ t h r e s h o l d _ o t s u ( dat )
objects = dat > val

median_dat = m e m _ m e d i a n _ f i l t e r ( dat , 3 )
val2 = m e m _ t h r e s h o l d _ o t s u ( median_dat [ objects ] )
liquid = median_dat > val2
s e g m e n t a t i o n _ r e s u l t = np . copy ( objects ) . astype ( np . uint8 )
s e g m e n t a t i o n _ r e s u l t [ liquid ] = 2

aggregates = m e m _ b i n a r y _ f i l l _ h o l e s ( objects )
cores = m e m _ b i n a r y _ e r o s i o n ( aggregates , np . ones ( ( 1 0 , 1 0 , ←-
10) ) )
More interaction for faster discovery: widgets
Interaction with images: annotations
More interaction for faster discovery: web applications made easy

Package: http://github.com/plotly/dash-canvas
pip install dash-canvas
Gallery: https://dash-canvas.plotly.host/
Based on react-sketch, fabricJS and Dash.
https://dash-canvas.plotly.host/segmentation/
Available components
dash html components
HTML elements html.H5

dash core components

reactive components
dcc.Slider, dcc.Dropdown

Charts (plotly and others)

Interactive DataTable

Specialized libraries: daq, dash-bio

Any ReactJS component wrapped w/ Dash

dash-canvas: drawing and transformation of annotations

Modular tools for annotations

and selections
DashCanvas (
i d= ’ c a n v a s −c o l o r ’ ,
lineWidth =5 ,
filename=” image . j p g ” ,
hide_buttons =[ ’ zoom ’ , ←-
’ pan ’ ] ,
)

Functions to transform annota-

tions (eg to Numpy arrays used
by skimage)
from dash_canvas i m p o r t ←-
utils
Gallery of examples

https://dash-canvas.plotly.host/
Roadmap of dash-canvas
Richer interaction with images

Annotations added by callbacks

e.g. load annotation geometry

Annotations triggering callbacks

(click, release, hover...

3-D images and timeseries

itk-snap

Richer annotations &

visualizations

More examples for the gallery

Roadmap of dash-canvas
Richer interaction with images

Annotations added by callbacks

e.g. load annotation geometry

Annotations triggering callbacks

(click, release, hover...

3-D images and timeseries

itk-snap

Richer annotations &

visualizations

More examples for the gallery

Thank you!
Feedback very welcome
Be in touch @EGouillart