Interpolation of RGB components

in Bayer CFA images

Demosaicing of Bayer-sampled
color images
Problem: Most digital color cameras,
capture only one color component at
each spatial location. The remaining
components must be reconstructed by
interpolation from the captured samples.
Objective: Develop algorithms to
interpolate each color plane (called
demosaicing) with best quality
reconstruction, and with minimal
computational complexity.

Construction of color image from

color planes
+

Lighthouse
original

Lighthouse
red original

Lighthouse
green original

Lighthouse
blue original

Formation of Color planes

Lighthouse
red subsampled

Lighthouse
green subsampled

Lighthouse
blue subsampled

Lighthouse
Bayer CFA image

Color plane interpolation

Green channel: bilinear interpolation

GA
GL

GR

GI

GB
GI =

1
(GL + GR + GB + G A )
4

Color plane interpolation

Red channel: bilinear interpolation

RNW
RSW

RC =

RNE

RC
RS

RSE

1
( RNW + RNE + RSW + RSE )
4

RS =

1
(RSW + RSE )
2

Lighthouse
red interpolated

Lighthouse
green interpolated

Lighthouse
blue interpolated

Lighthouse
Interpolated
color image

Lighthouse
original

Can we do better?
Color planes have severe aliasing.
Better interpolation of the individual
planes has little effect.

10

Lighthouse
red interpolated
with bilinear
interpolator

Lighthouse
red interpolated
with bicubic
interpolator

11

Can we do better?
Color planes have severe aliasing.
Better interpolation of the individual
planes has little effect.
We could optically prefilter the image
(blur it) so that aliasing is less
severe.

Lighthouse
red interpolated
with bilinear
interpolator

12

Lighthouse
prefiltered red
interpolated
with bilinear interpolator

Lighthouse
Interpolated
color image

13

Lighthouse
Prefiltered
Interpolated
color image

Lighthouse
original

14

Demosaicing Approaches
Non-Adaptive Single-Channel Interpolation:
Interpolate each color channel separately using
a standard technique, such as nearest-neighbor
interpolation, bilinear interpolation, etc.
Edge-Directed Interpolation: Estimate potential
edges, avoid interpolating across the edges.
Edge-directed interpolation
1.Calculate horizontal gradient H = |G1 G2|
2.Calculate vertical gradient V = |G3 G4|
3.If H > V,
Gx = (G3 + G4)/2
Else if H < V,
Gx = (G1 + G2)/2
Else
Gx = (G1 + G2 + G3 + G4)/4

3
1

Demosaicing Approaches
Edge-Directed Interpolation: Based on the
assumption that color channels have similar
texture, various edge detectors can be used.
Edge-directed interpolation

1
2
3

5
8
9

1.
2.
3.

Calculate horizontal gradient H = | (R3 + R7)/2 R5 |

Calculate vertical gradient V = | (R1 + R9)/2 R5 |
If H > V,
G5 = (G2 + G8)/2
Else if H < V,
G5 = (G4 + G6)/2
Else
G5 = (G2 + G8 + G4 + G6)/4

15

Demosaicing Approaches
Constant-Hue-Based Interpolation: Hue does not
change abruptly within a small neighborhood.
Interpolate green channel first.
Interpolate hue (defined as either color differences or color
ratios).
Estimate the missing (red/blue) from the interpolated hue.

Red

Green

Interpolate

Interpolate
d Red

Interpolate

Demosaicing Approaches
Edge-Directed Interpolation of Hue: It is a
combination of edge-directed interpolation and
constant-hue-based interpolation. Hue is
interpolated as in constant-hue-based
interpolation approach, but this time, hue is
interpolated based on the edge directions (as in
the edge-directed interpolation algorithm).

16

Demosaicing Approaches
Using Laplacian For Enhancement: Use the
second-order gradients of red/blue channels to
enhance green channel.
1
2
3

5
8
9

1.
2.
3.

Calculate horizontal gradient H = |G4 G6| + |R5 R3 + R5 R7|

Calculate vertical gradient V = |G2 G8| + |R5 R1 + R5 R9|
If H > V,
G5 = (G2 + G8)/2 + (R5 R1 + R5 R9)/4
Else if H < V,
G5 = (G4 + G6)/2 + (R5 R3 + R5 R7)/4
Else
G5 = (G2 + G8 + G4 + G6)/4 + (R5 R1 + R5 R9 + R5 R3 + R5 R7)/8

Demosaicing CFAs
bilinear interpolation
Interpolation of green pixels
G8 = (G3+G7+G9+G13) / 4
Interpolation of red/blue pixels :
* Interpolation of a red/blue pixel at a green position:
B7 = (B6+B8) / 2
R7 = (R2+R12) / 2
* Interpolation of a red/blue pixel at a blue/red position:
R8 = (R2+R4+R12+R14) / 4
B12 = (B6+B8+B16+B18) / 4

17

Demosaicing CFAs
Constant hue-based
interpolation (Cok)
Hue:
Interpolate G first

Demosaicing CFAs
Median-based interpolation
(Freeman)
1. Linear interpolation
2. Median filter on color
differences

18

Demosaicing CFAs
Gradient-based interpolation
(LaRoche-Prescott)
1. Interpolation on G

Demosaicing CFAs
Gradient-based interpolation
(LaRoche-Prescott)
2. Interpolation of color
differences

19

Demosaicing CFAs

bilinear

Cok

Freeman

LaRoche

Demosaicking CFAs

Generally, Freemans is the best, especially for natural images.

20

A Valuable Resource:
A Study of Spatial Color
Interpolation Algorithms for
Single-Detector Digital Cameras
http://scien.stanford.edu/class/psych22
1/projects/99/tingchen/
URL on Resources page

This week in Lab

Implement Bilinear
Interpolation
Implement one other
Non-Adaptive
Algorithm
Implement one
Adaptive Algorithm

21