PCA BASED IMAGE FUSION

By Guided By Ms.Arathi, Anila Mohan Asst.Professor, Muhammed Aslam. N Dept ECE, Pamela M.E.S.C.E Mariyadas Paulose Pratheek.T.K

Overview
Introduction Types of input images for fusion

Multifocus images PAN-Multispectral images MRI-CT images IR-Visual images

Principal Component Analysis Background Mathematics Supporting PCA Steps in PCA based image fusion Conclusion References

Introduction
PCA stands for Principal Component Analysis

Using this technique, we are going to fuse multi-

focus images to get an image of better visual quality. Often due to variations in the screen depth , it is not possible to capture an image where the entire scene appears sharp. Only scene areas that are at the focus plane appears sharp. Areas in front or behind the focus plane appears blurred

Types of input images for the fusion process

Multifocus images
PAN Multispectral images MRI-CT images IR-Visual images

Multi-Focus Images

Multi-Focus Images

Panchromatic Image
It is a 2D light intensity function, f(x,y) where x

and y are spacial coordinates and value of f at (x,y) is proportional to the brightness of the scene at that point In panchromatic image , only the total intensity of radiation falling on each pixel is recorded

Panchromatic Image

Multi Spectral Image

Each component of f(x,y) indicates the brightness

of the scene at point (x,y) at the corresponding spectral band A Multi spectral image is one that captures image data at specific frequencies across the electromagnetic spectrum These type of images are captured in Remote Sensing Radiometers
So from all the explanations , we can represent an image as a matrix

Multi spectral Image

Medical Images CT & MRI & Fused Image

Satellite Images captured in different sub-bands and the Fused Image

Visual & IR and Fused Images

Existing Image Fusion Techniques

Like multi-focus image fusion, there is multi-

sensor image fusion which is the process of combining relevant information from two or more images into a single image Image fusion methods 1.High pass filtering techniques 2.IHS transform based image fusion 3.PCA based image fusion 4.Wavelet transform image fusion 5.Curvelet transform based image fusion This presentation illustrates PCA based image fusion

Background Mathematics
Before getting into PCA we have to get some

elementary background mathematical skills

Standard Deviation
To explain this we need a data set

X=[1 2 3 4 5] , let it be a set. Here X1=1,X 2=2, etc

There are number of things that can be

calculated about a data set Mean elements

n= no of

Background Mathematics
Consider two sets given below

A=[0 8 12 20] B=[8 9 11 12] Both have the same mean(10) . But the difference between these sets is that the spread of data Standard Deviation of a data set is a measure of how spread out the data is Standard Deviation is the average distance from the mean of the data set to a point Standard Deviation is given by

Background Mathematics
Standard Deviation of A=8.3266, Standard

Deviation of B=1.8257. As expected Standard Deviation of A is greater.

Variance
It is the another measure of the spread of data in

a data set and is given by

Background Mathematics
Covariance Standard Deviation and variance only operate on 1Dimentional set Covariance is always measured between 2Dimentions It gives how much the dimensions vary from the mean with respect to each other and it is given by

Background Mathematics
If covariance is positive, it means both the

dimensions increases together If covariance is negetive , it means one dimension increases and other decreases If covariance is zero , it means two dimensions are independent of each other

Covariance Matrix
Covariance matrix (C) for a set of data with n

dimensions, is a (n x n) matrix If n=2 , then C= cov(x,x) cov(x,y) cov(y,x) cov(y,y)

Background Mathematics
Matrix Algebra Consider the example for matrix multiplication given below

Background Mathematics
In the first example , the resulting vector is not an

integer multiple of the original vector, but in the second example, the resulting vector is 4 times the original vector 3 2 =Eigenvector

1 3

=Non Eigenvector

And the matrix is known as transformation matrix

Background Mathematics
Eigenvector can only be found for square

matrices And , given an (n x n) matrix that does have eigenvectors, then there are n of them

Principal Component Analysis

There are several techniques used for image

fusion, among these techniques Principal Component Analysis is one of the easiest and simplest technique. Principal Component Analysis (PCA) is employed to transform original image to its eigenspace. By retaining the principal components with influencing eigenvalues, PCA keeps the key features in the original image and reduces noise level.

Principal Component Analysis

The central idea of Principal Component

Analysis(PCA) is to reduce the dimensionality of a data set consisting of large no of interrelated variables , while retaining as much as possible of the variations present in the data set This is achieved by transforming into a new set of random variables (Principal Components), which are uncorrelated and which are ordered so that the first few retains most of the variations present in all of the original variables Let X be the data set having p no of random variables X={(X1,Y1,Z1,..) , (X2,Y2,Z2) . . . . .} then by PCA we get dimensionally reduced

Principal Component Analysis

Principal Component Analysis

These Principal Components are the

eigenvectors of covariance matrix of X. Then for X we will get a (p x p) covariance matrix and corresponding p no of eigenvectors From these p no of eigenvectors we select the first few retains most of the variations present in all of the original variables These few is selected such that those having random values of maximum value

PCA based image fusion

There are several steps while fusing multi-focus

images. They are given below Let I1 and I2 be the two multi-focus images I1 and I2 have same size. Let those be (n x n) matrices 2 Step 1: Consider a empty (n x 2)matrix A. Load all elements of matrix I1 into the first column of A and load all elements of matrix I2 into the second column of A . Now we got a 2D matrix A (data). Step 2: Then we create a matrix B such that, B=A x A Now we got a (2 x 2) matrix B Step 3: Now we will find the eigenvalues and corresponding eigenvectors of B

PCA based image fusion

Step 4: Largest eigenvalue is found Corresponding eigenvector for larger eigenvalue is found This eigen vector is normalised and let its coefficient be a and b Step 5: Fused image=(a x I1) +(b x I2)

MatLab Program
I1=double(rgb2gray(imread('Time1.bmp'))); I2=double(rgb2gray(imread('Time2.bmp'))); A=[ ]; A=I1(:);

b=I2(:);
A=[A,b]; B=A'*A; [v,d]=eig(B) if(d(1,1)>d(2,2))

a=v(:,1)
else a=v(:,2) end n1=abs(a(1,1)) n2=abs(a(2,1)) fusedimage = (n1*I1) + (n2*I2); imshow(uint8(I1));figure, imshow(uint8(I2)); figure,imshow(uint8(fusedimage));

Result

Fused Image

Simulink Block Diagram

Conclusion
PCA based fusion aims at combining two images

to get a single image which has the good information from both the images. It is the simplest fusion technique, where the data is transformed to another set of co-ordinates, which better represents the data. It calculates a weight for the input images based on their eigen values and eigen vectors, with which each image is multiplied and added to get the fused image. It is not the best method of image fusion. Transform domain methods have shown to give much more promising results.

Questions?? ??

END
THANK YOU