CH01 Introduction (also see Machine Vision PPT slides) CH02 IP Fundamentals Check this link: http://www.gnu.org/software/octave/download.html 1.

Sensing and bio-inspiration 2. What is an image 3. Colors and color depth: binary, intensity, indexed, RGB, etc. 4. Image coordinates 5. reading images a. im0c = imread('house01.jpg'); b. explore im0c buffers c. [r c] = size( im0c ); d. ndims( im0c ); e. imshow( im0c ); f. im0g = rgb2gray( im0g ); g. imshow( im0g ) % 256 intensity levels by default h. imshow( im0g, 32 ) % 32 intensity levels i. imshow( im0g, [32 128] )% min/max intensity range j. imshow( im0g, [] ) % min/max intensity range = min/max(im0g) k. pixval % command after imshow l. figure and subplot % for multiple image display m. imview n. imfinfo o. imwrite( im0g, 'filename.format', 'quality', 0~100); 6. Image Basic Types and conversions a. Colored: im0c b. Intensity: im0g c. Pseudo color: im0p = im0g 60; imshow(im0i, jet) d. Indexed: im0i = rgb2ind( im0c, jet(256) ); e. Binary: im0b = im2bw( im0g, 0.5); 7. Image plotting (2D and 3D) a. Plot(im0g(:,1)) % plotting a column b. plot(im0g(1,:)) % plotting a row c. surf( double( im0g ) ); d. try a face image e. im1g = rand(240, 320); f. im1c(:,:,1) = rand(240, 320); 2, and 3 8. Setting and getting sub-images a. x = im0g(20:100, 40:200); b. x(1:20, 1:20) = 0; c. try it with colored images: im0c 9. Images in a program a. A simple program that reads rgb image converts it to intensity and shows it b. May be another example 10. HW Assignment (2): Write a MatLab program that a. Accepts an rgb image file name, colormap idx, threshold, save filename

b. Design your own color map of 256 colors and make it available as one of the options for users to select. c. Verifies the type of the image is rgb (3 buffs) d. Converts img into intensity, indexed (according to passed idx), binary (according to passed threshold) e. Writes your initials (two letters) in red pixels on the rgb version at the upper left corner with a 10% scale of the image. f. In the intensity img Find all pixels within the range [0-50] or [200255] and turn them off. g. Displays all 4 images versions on one figure h. Turn in a hard copy of your code and screen shots of your output. i. Grad students only: one page paper on imaging technologies. To be presented next class.

CH03 Intensity Transformation and Spatial Filtering 1. Intensity Transformation Functions a. Imadjust(im0g, [li hi], [lo ho], gamma);

by default gamma=1;

iout iin eiin

x = 0:.01:1; y = x .* exp( a * x ); plot(x, y); experiment with a=0, <0, >0.

Imadjust(im0g, [0 1], [1 0]); inversion or complement equivalent to imcomplement(im0g); It can be applied when a range of intensities (could represent an object) is desired to be highlighted: Imadjust(im0g, [0.5 0.75], [0 1]). Imadjust(im0g, [], [], gamma); adjusts the brightness lower or higher. Experiment with gamma = .01, .1, 1, 10; plot linear data function Demo some image examples

b. Logarithmic and Contrast-Stretching Transformation 0.1 * log( double(im0g) + 1); the double is to insure a real img datatype

iout c log(1 iin )

The shape is similar to gamma < 0; plot linear data function Demo some image examples c. Contrast Stretching function

x = 0:.01:1;m = .5; E=5; y = 1./(1+(m./(double(x)+eps)).^E); plot(x,y)

iout

1 1 (m / iin ) E

, m: see figure, E: slope coefficient

Plot linear data function Demo some image examples

2. Histogram Processing What histogram and image histogram? Demo some examples linear and image. x = rand(100,1); bar( hist(x, 10)); % nBins = 10. read an image to intensity imhist( im0g ); Question: how do we relate histogram info to image scene. Can save hist and plot it any way we like: h = imhist( im0g ); area(h); Histogram Equalization The idea of histogram equalization is as follows: Compute h = imhist(im0g), normalize h as hn = h ./ numel(im0g), compute the cumulative sum of hn as cumsum(hn) and use the resulting function as the intensity transformation function for im0g. Demo and plot the trans function. The procedure is implemented in a native function called histeq an example: im1g=.25*im0g;im1h=histeq(im1g,256);subplot(2,2,1);imshow(im1g);subplot (2,2,2);imshow(im1h); subplot(2,2,3);imhist(im1g);subplot(2,2,4);imhist(im1h); Histogram Matching

The function histeq supports another method for equalization. We can specify a histogram (PDF not CDF) that the histeq will follow to enforce its distribution on the passed image. Use the function: histeqMatch: discuss the function Example: h = histeqMatch( [.25 77 11; .25 128 23;.50 200 5], 256);area(h) 3. Spatial Filtering Noise: uniform PDF: imnoise(im0g,'speckle',.1 ) % .1=variance Binary: imnoise(im0g,'salt & pepper', .1 ) % .1 = noise density Gaussian: imnoise(im0g,'gaussian',.1, .1 ) % mean, variance Convolution: linear and non-linear

Convert image data to normalized double: im0g = double(im0g) ./ double(max(im0g(:))); Generate a kernel: k = ones(3) .* 1/9; Convolute an image with k: conv2( im0g , k ); The size of the kernel is arbitrary, typically 3x3 fspecial: filter special k = fspecial('filterType'); imfilter(im0g, k) 'average' averaging filter, [r c] or size 'disk' circular averaging filter, radius 'gaussian' Gaussian lowpass filter, [r c] or size 'log' logarithmic filter, [r c] or size 'unsharp' unsharp contrast enhancement filter, alpha (0~1) medfilt2(im0g, [r c]) surf: some examples on the kernel shapes

4. HW Assignment (3) Write a matlab program that: 1) Accepts an rgb image 2) Prompts the user to add noise using type, density, mean and variance. 3) Add noise to an intensity (gray-level) copy of the image 4) Filters the image using 4 different types of image filters 5) Allows the users to enter their customized kernel 6) Displays rgb img, intensity img, all filtered imgs in one figure 7) Label the different images appropriately on the figure 8) Graduate students only: write a matlab function that builds customized Gaussian mixture PDF as explained in image equalization via histogram matching section.

CH04 Intro to MatLab GUIDE & Image Arithmetic Logic Operations (ALO) 1. MatLab GUIDE Blank gui Panel: property inspector, set units to pixels Axes: units = pixels Create 4 axes 240x320, in1, in2, out1, out2. Menu editor: tools->menu editor, File->Load Image: callback, hObject, handles? Global variables as a single structure Load image callback: function Load_Image_Callback(hObject, eventdata, handles) global gv; [fname pname] = uigetfile('*.*'); gv.im1 = imread( strcat(pname,fname) ); axes(handles.axes1); im = image(gv.im1); set(im, 'HitTest', 'off'); set(handles.axes1, 'XTick', [], 'YTick', []); Main Menu item: color transformation->rgb2gray global gv; gv.im2 = rgb2gray(gv.im1); axes(handles.axes2); colormap( gray(256) ); im = image(gv.im2); set(im, 'HitTest', 'off'); set(handles.axes2, 'XTick', [], 'YTick', []); inputdlg: hdrs = {'x-val', 'y-val', 'z-val'}; ttl = 'example of message box'; numl = 1; def = {'11', '22', '33'}; op.Resize = 'ON'; usrin = inputdlg(hdrs, ttl, numl, def, op); if ~isempty(usrin)

x = str2num(usrin{1}); y = str2num(usrin{2}); z = str2num(usrin{3}); disp((x+y+z)/3.0); else disp('canceled ...'); end x= 11, y=99; msg = sprintf('x=%.3f, y=%d', x,y) standard icon: msgbox(msg,'this is a test msgbox','warn'); 'icon' is 'none', 'error', 'help', 'warn', or 'custom'. The default is 'none'. Custom icon: x = imread(imgfilename.jpg); msgbox(msg,'this is a custom msgbox','custom', x, jet(256)); Main Menu item: color maps->sub items: all maps Popup-menu implementation for color maps and edit1 (or inputdlg) text for map size global gv; contents = get(hObject,'String'); op = contents{get(hObject,'Value')}; p0 = str2num( get(handles.colorMap_edit1, 'String') ); switch op case 'gray' gv.colorMap = gray(p0); case 'jet' gv.colorMap = jet(p0); need a function to update the four images. Call it imgupdate. This function should redraw all images with new color map. Colormap command should be in the imageupdate after the image() or imshow() command. 2. 3. 4. 5. 6. 7. Threshold: slider Noise: popup-menu. See the noise option above. Binary Arithmetic & Logical Operations (ALO) Gray scale ALO: im2double(gray) to be able to max-normalize ALO results RGB ALO: im2double(rgb) to be able to max-normalize ALO results Masking: bin-bin, bin-rgb (loading a 3D bin as a mask), bin-rgb (run time conversion of bin mask to 3D) 8. Main menu item: image ALO, implement all for gray and binary. 9. Button group with radio buttons: see example in test.gif and .m

5. HW Assignment(4)& Mid-Term 1)Implement all image processing tools demonstrated above in this class in MatLab GUIDE. Your interface should include at least the following features: 1- load/save images 2- Image type transformation: gray, indexed, colormaps 3- Thresholding 4- Noise manipulation: uniform, Gaussian, salt&pepper 5- Filtering: mean, median, Gaussian

6- Image ALO 2) Graduate students only: None. % a demo menu program function op = -1; while op ~= 0 disp(' [0] Quit'); disp(' [1] load image'); [] = test00()

op = input(' enter your option '); switch op case 0 return; case 1 imshow(rand(100), jet(256)); end end ALO Handler contents = get(hObject,'String'); op = contents{get(hObject,'Value')}; switch op case 'a+b' gv.im{3} = imadd( gv.im{2}, gv.im{4}); case 'a-b' gv.im{3} = imsubtract( gv.im{2}, gv.im{4}); % works as long as the result is positive unless intended otherwise. use imabsdiff below case '|a-b|' gv.im{3} = imabsdiff( gv.im{2}, gv.im{4}); case 'axb' gv.im{3} = immultiply(gv.im{2}, gv.im{4}); case 'a/b' gv.im{3} = imdivide( gv.im{2}, gv.im{4} + eps); % epsilon to avoid devide by zero case 'a&b' gv.im{3} = double( gv.im{2} & gv.im{4} ); % make all logical results doubles case 'a|b' gv.im{3} = double(gv.im{2} | gv.im{4}); case '^' gv.im{3} = double( xor(gv.im{2}, gv.im{4}) ); case '!a' gv.im{3} = imcomplement( gv.im{2} ); case '!b' gv.im{3} = imcomplement( gv.im{4} );

case 'fxa' f = str2num( gv.im{3} = f case 'fxb' f = str2num( gv.im{3} = f case 'a/max' % nothing to end

get(handles.arithmetic_edit1, 'String') ); * gv.im{2}; get(handles.arithmetic_edit1, 'String') ); * gv.im{4}; be done, see the call for normImg below

gv.im{3} = normImg(gv.im{3}); updateImgs();

CH05 IMAGE SEGMENTATION 1. What is image segmentation? Partitioning/Clustering the image space into a finite number of homogeneous regions that share identical or similar features based on criteria including: intensity, color and texture. Segmentation aims at reducing dimensionality of the original image and mapping its space to a significantly smaller space to facilitate further advanced image analysis and understanding. 2. Segmentation Methods Thresholding: given a known intensity region, we can devise a band-pass filter that can turn off the entire image but the target region. >> example: see image file obj05.jpg and try to segment the blue regions. >> extracting original object by masking Thresholding method is limited due to the loss of information in color depth. Furthermore, the output is binary regions. Histogram: binary low-pass, binary high-pass, binary band-pass Color space quantization: (gray and rgb) Reducing the intensity/color depth. This method will collapse intervals of intensity/color to the quantized limits and similar colors are merged. % --- Executes on selection change in colorSpace_listbox11. function colorSpace_listbox11_Callback(hObject, eventdata, handles) % hObject handle to colorSpace_listbox11 (see GCBO) % eventdata reserved - to be defined in a future version of MATLAB % handles structure with handles and user data (see GUIDATA) % Hints: contents = get(hObject,'String') returns colorSpace_listbox11 contents as cell array % contents{get(hObject,'Value')} returns selected item from colorSpace_listbox11 global gv; contents = get(hObject,'String'); op = contents{get(hObject,'Value')};

p0 = str2num( get( handles.colorSpace_edit1, 'String' ) ); switch op case 'RGB_Q' gv.im{2} = case 'Gray_Q' gv.im{3} = gv.im{3} = case 'Gray' gv.im{2} = case 'HSV' gv.im{2} = case 'Ind' gv.im{2} = case 'ntsc' gv.im{2} = case 'YcBCr' gv.im{2} = end

im2double(grayslice(gv.im{1}, p0)); im2double(grayslice(gv.im{2}, p0)); normImg(gv.im{3}); rgb2gray( gv.im{1} ); rgb2hsv( gv.im{1} ); rgb2ind( gv.im{1} ); rgb2ntsc( gv.im{1} ); rgb2ycbcr( gv.im{1} );

gv.im{2} = normImg(gv.im{2}); updateImgs(); Color Filters: RGB vs. HSI(V,L)

>> Demo color selector with RGB vs. HIS. Set I to white and then black, see how RGB is color blind when in this case when you move the cursor over the color space vs. HSI model behavior. >> Demo HSV-based filter component

Skin Segmentation and masking nametorgb( colorname ); rgb2name( [ r g b ] ); text2speech tts( hello);

contents = get(hObject,'String'); op = contents{get(hObject,'Value')}; switch op case 'name2rgb' % name2rgb name = get(handles.colorname_edit1, 'String'); idx = find( strncmpi( name, gv.nameList, length(name))); rgb = gv.rgbList(idx, :); set(handles.colorname_edit1, 'String', num2str([rgb(1) rgb(3)])); set( handles.colorNamePnl, 'BackgroundColor', rgb(1,:)); case 'rgb2name' % rgb2name rgb = str2num( get(handles.colorname_edit1, 'String') ); d = dist(rgb, gv.rgbList'); idx = find(d==min(d));

rgb(2)

name = gv.nameList{ idx(1) }; set(handles.colorname_edit1, 'String', name); set( handles.colorNamePnl, 'BackgroundColor', rgb(1,:)); end

Clustering: kmeans (3D vs. 5D) >> function and demo % % % % % % by: Aaron Rababaah kmeans an image based on 5D (R,G,B,x,y) x = (double) rgb image k = number of clusters org = map to original colors [boolean] is5d = (boolean) include spatial coords to rgb = 5 dim

function [xo] = im5Dkmeans(x,k,is5d,org) [h w] = size(x(:,:,1)); if is5d x = im25d(x); x = reshape(x,h*w,5); else x = reshape(x,h*w,3); end [idx cen] = kmeans(x,k,'distance','sqEuclidean','Replicates',3); if org == 0 xo = reshape(idx,h,w); else L = length(idx); r = zeros(L,1); g = zeros(L,1); b = zeros(L,1); for i=1:k ids = find(idx==i); r(ids) = cen(i,1); g(ids) = cen(i,2); b(ids) = cen(i,3); end r = reshape(r,h,w); g = reshape(g,h,w); b = reshape(b,h,w); xo(:,:,1) = r; xo(:,:,2) = g; xo(:,:,3) = b; end

HW Assignment(5) 1. Implement Band-pass threshold, Color quantization and HSV-based segmentation as demonstrated in class in your GUI 2. (Grads only): find a research paper that addresses any segmentation related subjects, read it and report a half-page summary on it. Be ready to share your thoughts with the class.

CH06 REGION PROCESSING Image Morphology Connected Components Analysis (CCA)

After the foreground image is computed, the connected component analysis (CCA) is applied to

I tF ( x, y ) for foreground segmentation into separate regions of interest each of which represents
a TOI Ti. The goal of CCA is to label each physically connected pixels as a single region with one unique label. So other higher processing algorithms can readily identify the separated regions by their unique labels. The algorithm of CCA is illustrated in an example in figure 2.18, and the pseudo-code is given below Note that: the Neighbor pixels are shown in the figure below:

i-1,j-1 i-1,j i-1,j+1

I,j-1

I,j

Red: the current pixel, yellow: the neighboring pixels considered for labels. These neighbours are already processed if scanning is done leftright and topbottom

ALGORITHM CCA( I t ( x, y ) ) { // 1st pass For each pixel in I t ( x, y ) { If I t ( x, y ) 0

F

{ Nk = getNeighbors(x,y); If ALL Nk = 0

I tF ( x, y ) = newlabel;
Else {

I tF ( x, y ) = minlabel( Nk );
Store equivalentLabels in Nk; } } } // 2nd pass For each pixel in I t ( x, y ) { If I t ( x, y ) 0
F F

I tF ( x, y ) = minLabel(equivalentLabels);
} } }

Figure 2.18 Left: the foreground image I t ( x, y ) after background subtraction. Right: after CCA is applied, I t ( x, y ) is segmented into N connected components. 1) Build the props string: gv.propsOps = { 'Area' 'BoundingBox' 'Image' 'PixelList' 'PixelIdxList' 'Eccentricity'};
F

'Centroid' 'EquivDiameter' 'Solidity' 'Extent' 'Orientation'

'Extrema' ... 'SubarrayIdx' ... 'MajorAxisLength' ... 'MinorAxisLength' ... 'FilledArea' ...

2) Call bwlabeln: case 'Do CCA' [gv.im{3} gv.ncca] = bwlabeln( gv.im{2} ); gv.im{3} = normImg( gv.im{3}); % if you decide to norm img 3) Compute and save props case 'Props' gv.propsMat = []; gv.props = regionprops( gv.ncca * gv.im{3}, gv.propsOps multiplying by ncca since the regprop function requires an int img gv.propsMat(1,:) = [gv.props.Area]; gv.propsMat(2,:) = [gv.props.EquivDiameter]; gv.propsMat(3,:) = [gv.props.Solidity]; gv.propsMat(4,:) = [gv.props.MajorAxisLength]; gv.propsMat(5,:) = [gv.props.Extent]; gv.propsMat(6,:) = [gv.props.MinorAxisLength]; gv.propsMat(7,:) = [gv.props.Orientation]; gv.propsMat(8,:) = [gv.props.FilledArea]; gv.propsMat(9,:) = [gv.props.Eccentricity]; for i=1:gv.ncca % needed for filtering gv.props(i).id = i; gv.props(i).alive = 1; % all objs are initially are alive end

);

4) Compute boundaries and shape signatures case 'Boundary pol_Sig' % polar coords boundary signature (1D array) gv.propsSig = {gv.props.PixelList}; % shape signature for i=1:gv.ncca gv.propsSig{i} = signature(gv.propsSig{i}); end

case 'Plot Sig' figure; [h w] = screen2wins( gv.ncca ); for i=1:gv.ncca

subplot(h,w,i); plot(gv.propsSig{i}); % p1 = Obj ID (1-ncca) title(['Shape Signature of Object: ' num2str(i) ] ); end

%---------------------------------------------------% this function is needed to compute the proper subplots % splits the screen into subwins according to the the ratio of the monitor % H = 4/3 W % By: Aaron Rababaah function [h w] = screen2wins( n ) h = ceil(sqrt(n * 3/4)); w = ceil( n / h ); 5) Draw IDs case 'Draw ID' axes(gv.ax{3}); for i=1:gv.ncca p = gv.props(i).BoundingBox; text(p(1)+p(3)/2, p(2)+p(4)/2, num2str(i), 'Color', 'white'); end 6) Prints case 'Print_Obj' % all props per obj for i=1:gv.ncca disp( ['-------------( Object ' num2str(i) ' )-------------']); disp( gv.props(i) ); end case 'Print_Prop' % all objs per prop disp( ['-------------( gv.propsMat(1,:)' disp( ['-------------( gv.propsMat(2,:)' disp( ['-------------( gv.propsMat(3,:)' disp( ['-------------( gv.propsMat(4,:)' disp( ['-------------( gv.propsMat(5,:)' disp( ['-------------( gv.propsMat(6,:)' disp( ['-------------( gv.propsMat(7,:)' disp( ['-------------( gv.propsMat(8,:)' disp( ['-------------( gv.propsMat(9,:)' 7) Prop filters Area )-------------']); EquivDiameter )-------------']); Solidity )-------------']); MajorAxisLength )-------------']); Extent )-------------']); MinorAxisLength )-------------']); Orientation )-------------']); FilledArea )-------------']); Eccentricity )-------------']);

function [] = cca_filter(op, handles, p1, p2) global gv; z = []; switch op case 'Area_f' z = and ([gv.props.Area] >= p1, [gv.props.Area] <= p2); case 'Centroid_f' z = and ([gv.props.Centroid] >= p1, [gv.props.Centroid] <= p2); case 'MajorAx_f' z = and ([gv.props.MajorAxisLength] >= p1, [gv.props.MajorAxisLength] <= p2); case 'MinorAx_f' z = and ([gv.props.MinorAxisLength] >= p1, [gv.props.MinorAxisLength] <= p2); case 'Eccent_f' z = and ([gv.props.Eccentricity] >= p1, [gv.props.Eccentricity] <= p2); case 'Orient_f' z = and ([gv.props.Orientation] >= p1, [gv.props.Orientation] <= p2); case 'EqvDiam_f' z = and ([gv.props.EquivDiameter] >= p1, [gv.props.EquivDiameter] <= p2); case 'Solidity_f' z = and ([gv.props.Solidity] >= p1, [gv.props.Solidity] <= p2); case 'Extent_f' z = and ([gv.props.Extent] >= p1, [gv.props.Extent] <= p2); end

for i=1:gv.ncca if z(i) gv.props(i).alive = 1; x = gv.im{3}; x(gv.props(i).PixelIdxList) = gv.props(i).id / gv.ncca; gv.im{3} = x; else gv.props(i).alive = 0; x = gv.im{3}; x(gv.props(i).PixelIdxList) = 0; gv.im{3} = x; end end

Region Properties and Filtering Perimeter and Shape Signature Analysis >> Cartesian Boundary >> Polar Boundary

>> Chain coding Signal Filtering Signal Resampling and Amplitude Normalization

HW Assignment(6) 1. Implement morphological operators {dilate, erode, open, close, perimeter}, CCA, region properties. Integrate all to your GUI. 2. Extra credit: implement shape signature. 3. (Grads only):

fill,

CH07 OBJECT REPRESENTATION, CHARACTERIZATION, FEATURE EXTRACTION Region Properties (check cca) Boundary Signature Chain Coding

Chain code: 0010300111212123233323 function [cc] = chaincode(b,unwrap) % Freeman Chain Code % % Description: Give Freeman chain code 8-connected representation of a % boundary % Author.....: Alessandro Mannini <mannini@esod.it> % Date.......: 2010, september % % usage: % -------------------------------------------------------% [cc] = chaincode(b,u) % % INPUT: % -------------------------------------------------------% b - boundary as np-by-2 array;

% np is the number of pixels and each element is a pair (y,x) of % pixel coordinates % unwrap - (optional, default=false) unwrap code; % if enable phase inversions are eliminated % % % OUTPUT: % -------------------------------------------------------% cc is structure with the following fields: % % cc.code - 8-connected Freeman chain code as 1-by-np array (or % 1-by-(np-1) if the boundary isn't close) % cc.x0,cc.y0 - respectively the abscissa and ordinate of start point % cc.ucode - unwrapped 8-connected Freeman chain code (if required) % % % % used direction-to-code convention is: % % % % % % and in terms of deltax,deltay if current: % -------------------------% | deltax | deltay | code | % |------------------------| % | 0 | +1 | 2 | % | 0 | -1 | 6 | % | -1 | +1 | 3 | % | -1 | -1 | 5 | % | +1 | +1 | 1 | % | +1 | -1 | 7 | % | -1 | 0 | 4 | % | +1 | 0 | 0 | % -------------------------%

3 \ 4 -/ 5 next

2 1 | / P -- 0 | \ 6 7 compared to the

pixel

% check input arguments if nargin>2 error('Too many arguments'); elseif nargin==0 error('Too few arguments'); elseif nargin==1 unwrap=false; end % compute dx,dy by a circular shift on coords arrays by 1 element sb=circshift(b,[-1 0]); delta=sb-b; % check if boundary is close, if not cut last element if abs(delta(end,1))>1 || abs(delta(end,2))>1 delta=delta(1:(end-1),:); end

% check if boundary is 8-connected n8c=find(abs(delta(:,1))>1 | abs(delta(:,2))>1); if size(n8c,1)>0 s=''; for i=1:size(n8c,1) s=[s sprintf(' idx -> %d \n',n8c(i))]; end error('Curve isn''t 8-connected in elements: \n%s',s); end % convert dy,dx pairs to scalar indexes thinking to them (+1) as base-3 numbers % according to: idx=3*(dy+1)+(dx+1)=3dy+dx+4 (adding 1 to have idx starting % from 1) % Then use a mapping array cm % -------------------------------------% | deltax | deltay | code | (base-3)+1 | % |-------------------------------------| % | 0 | +1 | 2 | 8 | % | 0 | -1 | 6 | 2 | % | -1 | +1 | 3 | 7 | % | -1 | -1 | 5 | 1 | % | +1 | +1 | 1 | 9 | % | +1 | -1 | 7 | 3 | % | -1 | 0 | 4 | 4 | % | +1 | 0 | 0 | 6 | % --------------------------------------idx=3*delta(:,1)+delta(:,2)+5; cm([1 2 3 4 6 7 8 9])=[5 6 7 4 0 3 2 1]; % finally the chain code array and the starting point cc.x0=b(1,2); cc.y0=b(1,1); cc.code=(cm(idx))'; % If unwrapping is required, use the following algorithm % % if a(k), k=1..n is the original code and u(k) the unwrapped: % % - u(1)=a(1) % - u(k)=g(k), % g(k) in Z | (g(k)-a(k)) mod 8=0 and |g(k)-u(k-1)| is minimized % if (unwrap) a=cc.code; u(1)=a(1); la=size(a,1); for i=2:la n=round((u(i-1)-a(i))/8); u(i)=a(i)+n*8; end cc.ucode=u'; end

end Central Moments

Invariant Moments

function phi = invmoments(F) %INVMOMENTS Compute invariant moments of image. % PHI = INVMOMENTS(F) computes the moment invariants of the image % F. PHI is a seven-element row vector containing the moment % invariants as defined in equations (11.3-17) through (11.3-23) of % Gonzalez and Woods, Digital Image Processing, 2nd Ed. % % F must be a 2-D, real, nonsparse, numeric or logical matrix. % % % Copyright 2002-2004 R. C. Gonzalez, R. E. Woods, & S. L. Eddins Digital Image Processing Using MATLAB, Prentice-Hall, 2004 $Revision: 1.5 $ $Date: 2003/11/21 14:39:19 $

if (ndims(F) ~= 2) | issparse(F) | ~isreal(F) | ~(isnumeric(F) | ... islogical(F)) error(['F must be a 2-D, real, nonsparse, numeric or logical ' ... 'matrix.']); end F = double(F); phi = compute_phi(compute_eta(compute_m(F))); %-------------------------------------------------------------------% function m = compute_m(F) [M, N] = size(F); [x, y] = meshgrid(1:N, 1:M); % % x y F Turn x, y, and F into column vectors to make the summations a bit easier to compute in the following. = x(:); = y(:); = F(:);

% DIP equation (11.3-12) m.m00 = sum(F);

% Protect against divide-by-zero warnings. if (m.m00 == 0) m.m00 = eps; end % The other central moments: m.m10 = sum(x .* F); m.m01 = sum(y .* F); m.m11 = sum(x .* y .* F); m.m20 = sum(x.^2 .* F); m.m02 = sum(y.^2 .* F); m.m30 = sum(x.^3 .* F); m.m03 = sum(y.^3 .* F); m.m12 = sum(x .* y.^2 .* F); m.m21 = sum(x.^2 .* y .* F); %-------------------------------------------------------------------% function e = compute_eta(m) % DIP equations (11.3-14) through (11.3-16). xbar = m.m10 / m.m00; ybar = m.m01 / m.m00; e.eta11 e.eta20 e.eta02 e.eta30 e.eta03 e.eta21 (m.m11 - ybar*m.m10) / m.m00^2; (m.m20 - xbar*m.m10) / m.m00^2; (m.m02 - ybar*m.m01) / m.m00^2; (m.m30 - 3 * xbar * m.m20 + 2 * xbar^2 * (m.m03 - 3 * ybar * m.m02 + 2 * ybar^2 * (m.m21 - 2 * xbar * m.m11 - ybar * m.m20 2 * xbar^2 * m.m01) / m.m00^2.5; e.eta12 = (m.m12 - 2 * ybar * m.m11 - xbar * m.m02 2 * ybar^2 * m.m10) / m.m00^2.5; = = = = = =

m.m10) / m.m00^2.5; m.m01) / m.m00^2.5; + ... + ...

%-------------------------------------------------------------------% function phi = compute_phi(e) % DIP equations (11.3-17) through (11.3-23). phi(1) phi(2) phi(3) phi(4) phi(5) e.eta20 + e.eta02; (e.eta20 - e.eta02)^2 + 4*e.eta11^2; (e.eta30 - 3*e.eta12)^2 + (3*e.eta21 - e.eta03)^2; (e.eta30 + e.eta12)^2 + (e.eta21 + e.eta03)^2; (e.eta30 - 3*e.eta12) * (e.eta30 + e.eta12) * ... ( (e.eta30 + e.eta12)^2 - 3*(e.eta21 + e.eta03)^2 ) + ... (3*e.eta21 - e.eta03) * (e.eta21 + e.eta03) * ... ( 3*(e.eta30 + e.eta12)^2 - (e.eta21 + e.eta03)^2 ); phi(6) = (e.eta20 - e.eta02) * ( (e.eta30 + e.eta12)^2 - ... (e.eta21 + e.eta03)^2 ) + ... 4 * e.eta11 * (e.eta30 + e.eta12) * (e.eta21 + e.eta03); phi(7) = (3*e.eta21 - e.eta03) * (e.eta30 + e.eta12) * ... ( (e.eta30 + e.eta12)^2 - 3*(e.eta21 + e.eta03)^2 ) + ... (3*e.eta12 - e.eta30) * (e.eta21 + e.eta03) * ... ( 3*(e.eta30 + e.eta12)^2 - (e.eta21 + e.eta03)^2 ); Intensity Histogram features 3D Color Histogram features = = = = =

case 'RGB Hists' x = gv.im{1}; r = x(:,:,1); g = x(:,:,2); b = x(:,:,3); for i=1:gv.ncca [h(:,1) x] = imhist(r(gv.props(i).PixelIdxList)); [h(:,2) x] = imhist(g(gv.props(i).PixelIdxList)); [h(:,3) x] = imhist(b(gv.props(i).PixelIdxList)); z{i} = h; end gv.rgbHist = z; % assumes RGB hists are computed 1st case 'Gray Hist Sig' gv.sigLbl = 'Gray Histogram'; for i=1:gv.ncca h = gv.rgbHist{i}; % convert RGB hits to one gray hist by left multiply by % transformation vector of rgb2gray coefficients gv.grayHist{i} = [0.299 0.587 0.114]* (h'); % observe the transpose gv.propsSig{i} = gv.grayHist{i}; end

Edge Features and Hough Transform o Edge detectors o Hough o Hough histogram signature

Example:

.:

Texture Analysis: o Spatial texture features o Spectral texture features (FFT-based)

HW Assignment(7) 1. Implement Invariant moments, RGB histograms, and Intensity histogram features in your program. 2. (Grads only): Find one other method for object representation, understand the algorithm, implement and demonstrate it with images of your choice. (be ready to present it in class).

CH08 OBJECT/PATTERN RECOGNITION AND CLASSIFICATION Template Matching Via Minimum Euclidean Distance (dist function) Euclidian Distance

Given a vector space of nFeatures x nVectors 1 column = 1 vector x = rand(3, 5) 5 vectors 3 features each 0.5807 0.8303 0.1417 0.9907 0.1817 0.3634 0.3991 0.1431 0.7346 0.4318 0.2480 0.2992 0.5010 0.6888 0.7990

>> v = rand(3,1) v = 0.0462 0.8506 0.9934 >> d = dist(x', v) d = 1.0057 1.3177 0.8319 0.9969 0.5204 >> m = min(d) m = 0.5204 >> id = find( d == m ) id = 5 ; observe x transpose ()

Correlation-based Classification

Uncorrelated data v0 = rand(100,1); v1 = rand(100,1); scatter(v0, v1) corr(v0,v1) = 0.0607

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Completely correlated data v0 = rand(100,1); v1 = 3.5 * v0; scatter(v0, v1) corr(v0,v1)= 1.0000
3.5

2.5

1.5

0.5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Partially correlated v1 = (2 .* v0) .^ 2 + 2.5 * v0 - 17 + 7*rand(100,1); corr(v0,v1) = 0.5899 scatter(v0, v1);

-4

-6

-8

-10

-12

-14

-16

-18

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Using the same example before (in Euclidean) >> c = corr(x, v) c = -0.2934 -0.9373 0.2161 -0.9157 0.9728 >> find( c == max(c) ) ; a short cut to compute the index also, ; observe max not min as opposed to Euclidian ; distance ; observe no x transpose ()

ans = 5

K-Means clustering and classification

>> x = rand(3, 100); >> [id c] = kmeans(x', 3); >> c c = 0.2995 0.4400 0.7598 0.4659 0.3241 0.7430 0.1925 0.6824 0.5419

% observe the x transpose % centers vectors

id: the ids of the 100 vectors of the training set. Test k-means classification of an unseen vector: >> v = rand(3,1) v = 0.6385 0.6984 0.0981 >> d = dist(c', v) d = 0.7872 0.7653 0.6294 >> find( d == min(d) ) ans = 3

Neural Networks

Figure C.1 The Perceptron, the functional unit in Artificial Neural Networks Input signal (feature vector), weights, bias & bias weight (threshold), aggregation, activation function and output signal

Figure C.2 Input feature spaces that demonstrate, Left: linearly separable, right: linearly inseparable vector space

Figure C.3 Multi-layer Perceptron Typical Architecture

Using the same example (training set and vector) of the kmeans: (1) Training the network

>> [net g] = nntrain(x, id', 5, 10000, 1);

TRAINGD, Epoch 0/10000, MSE 1.89081/1e-006, Gradient 3.87868/1e-030 TRAINGD, Epoch 1000/10000, MSE 0.0917575/1e-006, Gradient 0.0282827/1e-030

TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD, TRAINGD,

Epoch 2000/10000, MSE 0.067669/1e-006, Gradient 0.0173372/1e-030 Epoch 3000/10000, MSE 0.0575123/1e-006, Gradient 0.0112892/1e-030 Epoch 4000/10000, MSE 0.0532136/1e-006, Gradient 0.00748766/1e-030 Epoch 5000/10000, MSE 0.0511766/1e-006, Gradient 0.00544244/1e-030 Epoch 6000/10000, MSE 0.0499952/1e-006, Gradient 0.00437156/1e-030 Epoch 7000/10000, MSE 0.0491734/1e-006, Gradient 0.00378341/1e-030 Epoch 8000/10000, MSE 0.0485274/1e-006, Gradient 0.00342915/1e-030 Epoch 9000/10000, MSE 0.0479811/1e-006, Gradient 0.00319522/1e-030 Epoch 10000/10000, MSE 0.0474979/1e-006, Gradient 0.00302987/1e-030 Maximum epoch reached, performance goal was not met.

(2)

Testing the network with an unseen vector

>> nnTest(net, v) ans = 3

Fuzzy Logic

Figure E.1 Concept of Input space to output space mapping process

Figure E.2 Classical Crisp Set Vs. Fuzzy Set

(1)

Building the FIS (fuzzy inference System) Add, Edit variables (input and output) Define in/out var names (Temp, Humidity, Pressure, FanSpeed, etc.), MFS (Tri, Trapezoid, Gauss, Sigmoid, etc.)

(2)

Write fuzzy rules

(3)

View and test rules

(4)

View Surface three (2 in with 1 out) variables at a time 3D limitation

(5) (6)

Save model to *.fis tempcontrol.fis Load model to workspace >> tcontrol = readfis('tempcontrol.fis') tcontrol = name: type: andMethod: orMethod: defuzzMethod: impMethod: aggMethod: input: output: rule: 'temcontrol' 'mamdani' 'min' 'max' 'centroid' 'min' 'max' [1x2 struct] [1x1 struct] [1x9 struct]

(7)

Use/test the model (the fis object) >> evalfis( [.5 .5], tcontrol)

ans = 0.5000 >> evalfis( [.75 .1], tcontrol) ans = 0.4188

Probabilistic Classification:

a = rand(8, 5); % generate class a training samples row-vectors b = rand(8, 5); % generate class b training samples c = rand(8, 5); % generate class c training samples ma = [mean(a);mean(b);mean(c)]'; % building the class means matrix row-vector ca(:,:,1) = cov(a); % building the class covars matrix [5x5x3] ca(:,:,2) = cov(b); ca(:,:,3) = cov(c); x = rand(4, 5); % generate testing data set row-vectors (4 vectors)

p = [0.3 0.5 0.2];% probability of each class sum = 1 %if p is not passed, the classes are assumed equally likely id = bayesgauss(x, ca, ma, p); % id = the classifications of the 4 testing vectors HW Assignment(8) 1. Implement at least two of the Pattern Recognition methods. 2. Graduate Students only: Implement Template Matching, Neural Networks and Probabilistic-based Classification.

CH09 VIDEO PROCESSING Image acquisition Image grabber Real time (on-line) processing Off-line Processing: processing a directory of images Applications: motion tracking, surveillance, dynamic sign language, etc.

HW Assignment(9) Given the image sequence (compressed file on blackboard) write a motion tracking algorithm to track the moving objects in the scene. Using the CCA algorithm, compute the following: 1. Bounding box 2. Center of the bottom edge of the bounding box 3. The size 4. Velocity 5. Classify its posture, standing, squatting and crawling (Graduate Students only)