Media Processing in Processing

Media Processing in Processing
Collection Editor: Davide Rocchesso
Collection Editor: Davide Rocchesso Authors: Richard Baraniuk Pietro Polotti Ricardo Radaelli-Sanchez Davide Rocchesso
Online: <http://cnx.org/content/col10268/1.12/ >
CONNEXIONS Rice University, Houston, Texas
PHHV hvide ohesso

This selection and arrangement of content is licensed under the Creative Commons Attribution License: http://creativecommons.org/licenses/by/2.0/
Table of Contents
1 Programming in Processing 2 Media Representation in Processing 3 Graphic Composition in Processing 4 Signal Processing in Processing: Sampling and Quantization 5 Signal Processing in Processing: Convolution and Filtering 6 Discrete-Time Convolution 7 Signal Processing in Processing: Elementary Filters 8 Textures in Processing 9 Signal Processing in Processing: Miscellanea Glossary Bibliography Index Attributions
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF I F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IU F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F QI F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F RU F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F SU F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F FF F F F F F F F F F F F F F F F F F TI F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F TW F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F FF F F F F F F F F F F F F F F F F F VQ F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F WI F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IIH F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F III F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F IIP F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F FIIR
iv
Chapter 1
Programming in Processing
1
1.1 Introduction
his introdution is sed on hniel hi'mn9s tutoril 2 F Processing is lnguge nd development environment oriented towrd interaction design F sn the ourse Media Processing in Processing (MPP)D proessing is one of the min instruments used to introdue some fundmentls in sound nd imge proessingF roessing is n extension of tv tht supports mny tv strutures with simpli(ed syntxF roessing n e used in three
note:
Programming Modes Basic -
equene of ommnds for simple drwing y grphi primitivesF ! pplet withE out nose3
size(256,256); background(0); stroke(255); ellipseMode(CORNER); ellipse(72,100,110,130); triangle(88,100,168,100,128,50); stroke(140); strokeWeight(4); line(96,150,112,150); line(150,150,166,150); line(120,200,136,200);
Table 1.1
Intermediate -
roedurl progrmming !
1 This content is available online at <http://cnx.org/content/m12968/1.8/>. 2 http://www.shiman.net/itp/classes/ppaint/ 3 http://cnx.org/content/m12968/latest/pinocchiononose.html
P pplet with nose4
CHAPTER 1.
PROGRAMMING IN PROCESSING
void setup() { size(256,256); background(0); } void draw() { stroke(255); strokeWeight(1); ellipseMode(CORNER); ellipse(72,100,110,130); triangle(88,100,168,100,128,50); stroke(140); beginShape(TRIANGLES); vertex(114, 180); vertex(mouseX, mouseY); vertex(140, 180); endShape(); strokeWeight(4); line(96,150,112,150); line(150,150,166,150); line(120,200,136,200); }
Table 1.2
Complex -
yjetEyriented rogrmming @tvA !
4 http://cnx.org/content/m12968/latest/pinocchionose.html
Q pplet with olE ored nose5

Puppet pinocchio; void setup() { size(256,256); background(0); color tempcolor = color(255,0,0); pinocchio = new Puppet(tempcolor); } void draw() { background(0); pinocchio.draw(); } class Puppet { color colore; Puppet(color c_) { colore = c_; } void draw () { stroke(255); strokeWeight(1); ellipseMode(CORNER); ellipse(72,100,110,130); stroke(colore); beginShape(TRIANGLES); vertex(114, 180); vertex(mouseX, mouseY); vertex(140, 180); endShape(); strokeWeight(4); line(96,150,112,150); line(150,150,166,150); } }
Table 1.3
File
he roessing progrms n e onverted into tv ppletsF sn order to do thtD one just goes to the menu nd hooses ExportF es resultD (ve (les will e reted nd put in n applet folderX index.html E html ode to visulize the pplet lename.jar E the ompiled ppletD inluding ll dt @imgesD soundsD etFA lename.pde E the roessing soure ode lename.java E the tv ode emedding the roessing soure ode loading.gif E n imge to e displyed while the pplet is eing lodedF woreoverD y mens of Export Application it is possile to generte n exeutle pplition for vinuxD wyD or indows pltformsF
5 http://cnx.org/content/m12968/latest/pinocchioclassy.html
CHAPTER 1.
1.2 Data Types

1.2.1 Variables
e vrile is pointer to memory lotionD nd it n refer either to primitive vlues @intD floatD eFA or to ojets nd rrys @tles of primitiveEtype elementsAF he opertion of ssignment b = a produes he opy of the ontent of a into bD if the vriles refer to primitive typesF he retion of new referene @pointerA to the sme ojet or rryD if the vriles refer to ojets or rrysF note: o hve ler understnding of omputer siene terms suh s those tht followD we reommend looking t ikipedi6
Denition 1.1: scope
within progrmD it is region where vrile n e essed nd its vlue modi(ed

Denition 1.2: global scope
de(ned outside the methods setup@A nd drw@AD the vrile is visile nd usle nywhere in the progrm
Denition 1.3: local scope
de(ned within ode lok or funtionD the vrile tkes vlues tht re lol to the lok or funtionD nd ny vlues tken y glol vrile hving the sme nme re ignoredF
Example 1.1: Array declaration and allocation
int[] arrayDiInteri = new int[10];
1.3 Programming Structures

1.3.1 Conditional Instructions
ifX
if (i == NMAX) { println("finished"); } else { i++; }
1.3.2 Iterations
whileX
int i = 0; //integer counter while (i < 10) { //write numbers between 0 and 9
6 http://wikipedia.org/
S
} println("i = "+ i); i++;
forX
for (int i = 0; i < 10; i++) { //write numbers between 0 and 9 println("i = "+ i); }
Example 1.2: Initializing a table of random numbers
int MAX = 10; float[] tabella = new float[MAX]; for (int i = 0; i < MAX; i++) tabella[i] = random(1); //random numbers between 0 and 1 println(tabella.length + " elements:"); println(tabella);
1.4 Functions
puntions llow modulr pproh to progrmmingF sn roessingD in the intermediate progrmming modeD we n de(ne funtions other thn setup() nd draw()D usle from within setup() nd draw()F
Example 1.3: Example of function
int raddoppia(int i) { return 2*i; }
e funtion is hrterized y the entities @with referene to the exmple @ixmple IFQX ixmple of funtionAA X return type @intA nme @raddoppiaA prmeters @iA ody @return 2*iA
CHAPTER 1.
1.5 Objects and Classes

e lss is de(ned y set of dt nd funtionsF en ojet is n instne of lssF ie versD lss is the strt desription of set of ojetsF 7 note: por n introdution to the onepts of ojet nd lss see yjets nd glsses F
Example 1.4: Example of class
Dot myDot; void setup() { size(300,20); colorMode(RGB,255,255,255,100); color tempcolor = color(255,0,0); myDot = new Dot(tempcolor,0); } void draw() { background(0); myDot.draw(10); } class Dot { color colore; int posizione; //****CONSTRUCTOR*****// Dot(color c_, int xp) { colore = c_; posizione = xp; } void draw (int ypos) { rectMode(CENTER); fill(colore); rect(posizione,ypos,20,10); }
e lss is hrterized y the following entities @with referene to the exmple @ixmple IFRX ixmple of lssAA X nme @DotA dt @colore, posizioneA onstrutor @Dot()A funtions @or methodsD draw()A
7 "Objects
and Classes" <http://cnx.org/content/m11708/latest/>
U en ojet @instne of lssA is delred in the sme wy s we delre vrileD ut we hve to llote spe for it @s we did for the rrysA y mens of its onstrutor @with referene to the exmple @ixmple IFRX ixmple of lssAAF helrtionX @Dot myDot;A ellotionX @myDot = new Dot(tempcolor,0)A seX @myDot.draw(10);A 8 note: por quik introdution to the tv syntx see tv yntx rimer
Exercise 1.1
ith the following draw() method we wnt to pint the window kground with gry whose intensity depends on the horizontl position of the mouse pointerF
(Solution on p. 11.)
void draw() { background((mouseX/100)*255); }
roweverD the ode does not do wht it is expeted to doF hyc

Exercise 1.2
ht does the following ode frgment print outc
int[] a = new int[10]; a[7] = 7; int[] b = a; println(b[7]); b[7] = 8; println(a[7]); int c = 7; int d = c; println(d); d = 8; println(c);
Exercise 1.3
he following sketh genertes set of IHH moving irles nd drws ll hords linking the interE setion points of ll ouples of interseting irlesF
/* Structure 3 A surface filled with one hundred medium to small sized circles. Each circle has a different size and direction, but moves at the same slow rate. Display: A. The instantaneous intersections of the circles B. The aggregate intersections of the circles
8 "Java
Syntax Primer" <http://cnx.org/content/m11791/latest/>
CHAPTER 1.
Implemented by Casey Reas <http://groupc.net> 8 March 2004 Processing v.68 <http://processing.org> modified by Pietro Polotti 28 March, 2006 Processing v.107 <http://processing.org> */ int numCircle = 100; Circle[] circles = new Circle[numCircle]; void setup() { size(800, 600); frameRate(50); for(int i=0; i<numCircle; i++) { circles[i] = new Circle(random(width), (float)height/(float)numCircle * i, int(random(2, 6))*10, random(-0.25, 0.25), random(-0.25, 0.25), i); } ellipseMode(CENTER_RADIUS); background(255); } void draw() { background(255); stroke(0); for(int i=0; i<numCircle; i++) { circles[i].update(); } for(int i=0; i<numCircle; i++) { circles[i].move(); } for(int i=0; i<numCircle; i++) { circles[i].makepoint(); } noFill();
class Circle {
W
float x, y, r, r2, sp, ysp; int id; Circle( float px, float py, float pr, float psp, float pysp, int pid ) { x = px; y = py; r = pr; r2 = r*r; id = pid; sp = psp; ysp = pysp; } void update() { for(int i=0; i<numCircle; i++) { if(i != id) { intersect( this, circles[i] ); } } } void makepoint() { stroke(0); point(x, y); } void move() { x += sp; y += ysp; if(sp > 0) { if(x > width+r) { x = -r; } } else { if(x < -r) { x = width+r; } } if(ysp > 0) { if(y > height+r) { y = -r; } } else { if(y < -r) { y = height+r; } } }
void intersect( Circle cA, Circle cB )
IH
{ float float float float dx = cA.x - cB.x; dy = cA.y - cB.y; d2 = dx*dx + dy*dy; d = sqrt( d2 );
CHAPTER 1.
if ( d>cA.r+cB.r || d<abs(cA.r-cB.r) ) { return; // no solution } // // calculate the two intersections between the two circles cA and cB, // whose coordinates are (paX, paY) and (pbX, pbY), respectively. //
stroke(255-dist(paX, paY, pbX, pbY)*4); line(paX, paY, pbX, pbY);
IF gomplete the missing prt tht is expeted to ompute the intersetions of the irlesD in suh wy to drw the hords linking the intersetion pointsF st is possile to use the omputtion of intersetion oordintes in dEho referene system @ Circle-Circle Intersection AD then onverting the result into the roessing window oordinte systemF PF wke the hords timeEvrile y giving di'erent speeds to di'erent irlesF
Exercise 1.4
wke the sketh of ixerise IFQ intertiveF por exmpleD mke the irle displement dependent on the horizontl position of the mouseF
II
Solutions to Exercises in Chapter 1

he vrile mouseX is of int typeD nd the division it is sujet to is of the integer typeF st is neessry to perform type casting from int to float y mens of the instrution (float)mouseXF
Solution to Exercise 1.2 (p. 7) Solution to Exercise 1.1 (p. 7)
7 8 7 7
Solution to Exercise 1.3 (p. 7)
/* Structure 3 A surface filled with one hundred medium to small sized circles. Each circle has a different size and direction, but moves at the same slow rate. Display: A. The instantaneous intersections of the circles B. The aggregate intersections of the circles Implemented by Casey Reas <http://groupc.net> 8 March 2004 Processing v.68 <http://processing.org> modified by Pietro Polotti 28 March, 2006 Processing v.107 <http://processing.org> */ int numCircle = 100; Circle[] circles = new Circle[numCircle]; void setup() { size(800, 600); frameRate(50); for(int i=0; i<numCircle; i++) { circles[i] = new Circle(random(width), (float)height/(float)numCircle * i, int(random(2, 6))*10, random(-0.25, 0.25), random(-0.25, 0.25), i); } ellipseMode(CENTER_RADIUS);
IP
} background(255);
CHAPTER 1.
void draw() { background(255); stroke(0); for(int i=0; i<numCircle; i++) { circles[i].update(); } for(int i=0; i<numCircle; i++) { circles[i].move(); } for(int i=0; i<numCircle; i++) { circles[i].makepoint(); } noFill();
class Circle { float x, y, r, r2, sp, ysp; int id; Circle( float px, float py, float pr, float psp, float pysp, int pid ) { x = px; y = py; r = pr; r2 = r*r; id = pid; sp = psp; ysp = pysp; } void update() { for(int i=0; i<numCircle; i++) { if(i != id) { intersect( this, circles[i] ); } } } void makepoint() { stroke(0); point(x, y); }
IQ
void move() { x += sp; y += ysp; if(sp > 0) { if(x > width+r) { x = -r; } } else { if(x < -r) { x = width+r; } } if(ysp > 0) { if(y > height+r) { y = -r; } } else { if(y < -r) { y = height+r; } } }
void intersect( Circle cA, Circle cB ) { float dx = cA.x - cB.x; float dy = cA.y - cB.y; float d2 = dx*dx + dy*dy; float d = sqrt( d2 ); if ( d>cA.r+cB.r || d<abs(cA.r-cB.r) ) { return; // no solution } float float float float float float float float a = (cA.r2 - cB.r2 + d2) / (2*d); h = sqrt( cA.r2 - a*a ); x2 = cA.x + a*(cB.x - cA.x)/d; y2 = cA.y + a*(cB.y - cA.y)/d; paX paY pbX pbY = = = = x2 y2 x2 y2 + + h*(cB.y h*(cB.x h*(cB.y h*(cB.x cA.y)/d; cA.x)/d; cA.y)/d; cA.x)/d;
IR
CHAPTER 1.
/* Structure 3 A surface filled with one hundred medium to small sized circles. Each circle has a different size and direction, but moves at the same slow rate. Display: A. The instantaneous intersections of the circles B. The aggregate intersections of the circles Implemented by Casey Reas <http://groupc.net> 8 March 2004 Processing v.68 <http://processing.org> modified by Pietro Polotti 28 March, 2006 Processing v.107 <http://processing.org> */ int numCircle = 100; Circle[] circles = new Circle[numCircle]; void setup() { size(800, 600); frameRate(50); for(int i=0; i<numCircle; i++) { circles[i] = new Circle(random(width), (float)height/(float)numCircle * i, int(random(2, 6))*10, random(-0.25, 0.25), random(-0.25, 0.25), i); } ellipseMode(CENTER_RADIUS); background(255); } void draw() { background(255); stroke(0); if(mousePressed){ for(int i=0; i<numCircle; i++) { circles[i].sp = mouseX*random(-5, 5)/width;
IS
} } for(int i=0; i<numCircle; i++) { circles[i].update(); } for(int i=0; i<numCircle; i++) { circles[i].move(); } for(int i=0; i<numCircle; i++) { circles[i].makepoint(); } noFill();
class Circle { float x, y, r, r2, sp, ysp; int id; Circle( float px, float py, float pr, float psp, float pysp, int pid ) { x = px; y = py; r = pr; r2 = r*r; id = pid; sp = psp; ysp = pysp; } void update() { for(int i=0; i<numCircle; i++) { if(i != id) { intersect( this, circles[i] ); } } } void makepoint() { stroke(0); point(x, y); } void move() { x += sp; y += ysp; if(sp > 0) { if(x > width+r) { x = -r; }
IT
} else { if(x < -r) { x = width+r; } } if(ysp > 0) { if(y > height+r) { y = -r; } } else { if(y < -r) { y = height+r; } }
CHAPTER 1.
void intersect( Circle cA, Circle cB ) { float dx = cA.x - cB.x; float dy = cA.y - cB.y; float d2 = dx*dx + dy*dy; float d = sqrt( d2 ); if ( d>cA.r+cB.r || d<abs(cA.r-cB.r) ) { return; // no solution } float float float float float float float float a = (cA.r2 - cB.r2 + d2) / (2*d); h = sqrt( cA.r2 - a*a ); x2 = cA.x + a*(cB.x - cA.x)/d; y2 = cA.y + a*(cB.y - cA.y)/d; paX paY pbX pbY = = = = x2 y2 x2 y2 + + h*(cB.y h*(cB.x h*(cB.y h*(cB.x cA.y)/d; cA.x)/d; cA.y)/d; cA.x)/d;
Chapter 2
Media Representation in Processing
1
2.1 Visual Elements

2.1.1 Coordinates
sn roessingD the representtion of grphi ojets is sed on rtesin Qh oordinte systemD s displyed in pigure PFI @goordinte systemAF
Coordinate system
Figure 2.1:
3D coordinate system used in Processing
Ph imges re proessed y ting on the E plneD thus ssuming tht the oordinte is zeroF he funtion size() de(nes the disply window size nd the rendering engine tht will e used to pint onto the windowF he defult engine is teePhD the Ph grphi tv liryF sf one wnts to progrm in QhD he
1 This
content is available online at <http://cnx.org/content/m12983/1.10/>.
IU
IV
CHAPTER 2.
MEDIA REPRESENTATION IN PROCESSING
must hoose either the Qh @roessing QhA rendering engineD espeilly suited for weEoriented grphisD or yixqvD whih delegtes mny typil Qh opertions to the grphi ord thus freeing the g from mny omputtionsF
2.1.2 Images
sn roessingD n imge n e ssigned to n ojet of the lss PImageF he funtion loadImage("myImage") tkes (le @gif or jpgA myImageD ontining the pixel oding of n imgeD nd gives k the ontent of suh imgeD whih n e ssigned to vrile of type PImageF he (le myImage must e loded in the data folder of the diretory hving the sme nme s the roessing sketh we re working tF note: hen the New ommnd is exeutedD proessing opens up folder nmed sketch_??????? within the Processing diretoryD orresponding to the nme ssigned ye the system to the newly reted (leF uh folder is essile from the roessing menu item Sketch/Add FileF he lss PImage gives essD y the (elds width nd heightD to the width nd height of the loded imgeF he imge ontent is essed vi the pixels[] (eldF
Example 2.1: Loading and visualizing an image
size(400,300); PImage b; b = loadImage("gondoliers.jpg"); println("width=" + b.width + " height=" + b.height); image(b, 0, 0, 400, 300); // position (0,0); width=400; height=300; image(b, 20, 10, 100, 80); // position (20,10); width=100; height=80;
2.1.3 Colors
ine our olor reeptors @conesAD eh tuned to wvelength regionD re of three kindsD olor models re lwys referred to threeEdimensionl speF sn dditive olor modelsD eh of three xes orrespond to se olorD nd y mixing three olored light ems one n otin ll olors within gamut volume in the spe de(ned y the three xesF he three se olors n e hosen ritrrily orD more oftenD sed on the pplition domin @eFgFD olor of three phosphors or lser emsAF sn printing proessesD sutrtive olor models re usedD where the strting point is the white surfe nd primry ink olors re used to sutrt olor from whiteF 2 note: quide to olor modelsX httpXGGenFwikipediForgGwikiGolorspe sn proessing color is primitive type used to speify olorsF st is relized y QPEit numerD where the (rst yte spei(es the lph vlueD nd the other ytes speify triple either in the qf or in the rf modelF he hoie of one model or the other is mde y the colorMode() funtionF ith three ytesD numer of 256 256 256 = 16777216 re representleF
2.1.3.1 The RGB model
golors re represented y triple of numersD eh giving the intensity of the primry olors edD qreenD nd flueF ih numer n e n unsigned integerD thus tking vlues etween H nd PSSD or e expressed s )oting point numer etween 0.0 nd 1.0F ith even lrger )exiilityD the modelD typeD nd rnge of olors n e set with the funtion colorMode()F he qf model is dditiveF
2 http://en.wikipedia.org/wiki/color_space
IW
2.1.3.2 HSB Model
golors re represented y triple of numersD the (rst numer giving the rueD the seond giving turtionD nd the third giving the frightnessF note: yften the model is lled rD where stnds for lueF he hue tkes vlues in degrees etween 0 @redA nd 360D eing the vrious hues rrnged long irumferE ene nd eing red positioned t 0 F turtion nd rightness vry etween 0 nd 100F he sturtion is the degree of purity of olorF sf pure olor is dded with white light its degree of purity dereses until the olor eventully sits on gry sle when sturtion is zeroF sn physil termsD the rightness is proportionl to the signl power spetrumF sntuitivelyD the rightness is inresed when the light intensity inresesF he threeEdimensionl rf spe is well represented y ylinderD with the hue @nominl sleA rrnged long the irumfereneD the sturtion @rtio sleA rrnged long the rdiusD nd the rightness @intervl sleA rrnged long the longitudinl xisF elterntivelyD suh threeEdimensionl spe n e ollpsed into two dimensionsD s in the color chooser of the imgeEproessing progrm qimp3 D displyed in pigure PFP @qimp olor hooserAF elong the irumfereneD the three primry olors @redD greenD nd lueA re visileD 120 prt from eh otherD seprted from the seondry olors @mgentD ynD yellowAF ih seondry olor is omplementry to the primry olor in front of it in the irumfereneF por instneD if we tke the green omponent out of white lightD we otin mgent lightF he tringle insried in the irumferene hs vertex pointing to seleted hueF he opposite side ontins the gry sleD thus representing olors with null sturtion nd vrile rightnessF qoing from the referene vertex to the opposite side we hve grdul derese in sturtionF
3 http://www.gimp.org
PH
CHAPTER 2.
Gimp color chooser
Figure 2.2:
Color chooser of the software Gimp
2.1.3.3 Alpha channel
st is yte used to lend nd interpolte etween imgesD for exmple to render trnsprenyF st n e otinedD from vrile of type colorD with the method alpha()F he lph hnnel n e mnipulted with the method blend() of the lss PImageF
Example 2.2: Loading and visualizing an image with transparency
PI
size(400,300); PImage b = loadImage("gondoliers.jpg"); PImage a = loadImage("gondoliers.jpg"); float ramp = 0; for (int j = 0; j < b.height; j++) for (int i = 0; i < b.width; i++) { b.set(i, j, b.get(i,j) + color(0,0,0, 255 - (int)((1-ramp)*255)) ); ramp = ramp + 1/(float)(b.width * b.height); } a.blend(b, 0, 0, b.width, b.height, 80, 10, 450, 250, BLEND); image(a, 0, 0, 400, 300);
Table 2.1
sn roessingD it is possile to ssign olor to vrile of type color y mens of the funtion color()D nd the model n e previously set with colorMode()F he funtions red()D green()D blue()D hue()D saturation()D nd brightness() llow to move from one model to the otherF
colorMode(RGB); color c1 = color(102, 30,29); colorMode(HSB); color c2 = color(hue(c1), saturation(c1), brightness(c1)); colorMode(RGB); color c3 = color(red(c2), green(c2), blue(c2)); // the variables c1, c2, and c3 contain the coding of the same color
2.1.3.4 Tinging an image
en imge n e tinged with olor nd its trnspreny n e set y ssigning given vlue to the lph hnnelF por this purposeD the funtion tint() n e usedF por exmpleD lue tone n e ssigned to the inlid imge of ixmple PFI @voding nd visulizing n imgeA y just preeding the seond image() ommnd with tint(0, 153, 204, 126) F
2.1.4 Translations, Rotations, and Scale Transformations
Representing Points and Vectors
sn omputer grphisD points nd vetors re represented with the

Denition 2.1: homogeneous coordinates
qudruples of numersD where the (rst triple is to e red in the EE speD while the fourth numer indites vector if it tkes vlue HD or point if it tkes vlue IF e trnsltion is otined y ddingD in homogeneous oordintesD vetor to pointD nd the reE sult is pointF elterntively we see trnsltion s mtrixEvetor produt @see wtrix erithE
PP
1 0 0
CHAPTER 2.
tx
meti AD where the mtrix is

4
en the ngle ntiElokwise rottion y

cos () (sin ()) sin () cos () 0 0 0 0 0 0 0 1 0 0 F 0 1
0 1 0 0 0 1 0 0 0
y ty F D nd the vetor is the one representing the point z tz 1 1 round the xis z @rollAD is otined y the rottion mtrix
ottions round the xes x @pitchA nd y @yawA re relized y mens of
rottion mtries of the sme kindD nd rottion round n ritrry xis n e otined y omposition @left multiplyA of elementry rottions round eh of the min xesF
Translations
sn two dimensionsD the funtion rotate() is used to rotte ojets in the imge windowF his is otined y @leftA multiplying the oordintes of eh pixel of the ojet y rottion mtrixF ottions re lwys spei(ed round the top left orner of the window @ [0, 0] oordinteAF rnsltions n e used to move the rottion xis to other pointsF ottion ngles re spei(ed in rdinsF ell tht 2rd = 360 F por exmpleD insert the rottion rotate(PI/3) efore the seond image() ommnd in ixmple PFI @voding nd visulizing n imgeAF sn three dimensionsD we n use elementry rottions round the oordinte xes rotateX()D rotateY()D e rotateZ()F
Scale Transformations
Rotations
he funtion translate() moves n ojet in the imge windowF st tkes two or three prmetersD eing the displements long the diretions xD y @nd zAD respetivelyF
he funtion scale() llows to expnd or ontrt n ojet y multiplition of its point oordintes y onstntF hen it is invoked with two or three prmetersD di'erent slings n e pplied to the three xesF
2.1.5 Typographic Elements
ivery tool or lnguge for medi mnipultion gives the opportunity to work with written words nd with their fundmentl visul elementsX typogrphi hrtersF he spet of type hs two min omponentsX font nd sizeF roessing hs the lss PFont nd the methods loadFont() @to lod font nd ssign it to n ojet of the PFont lssA nd textFont() @to tivte font with spei( sizeAF sn order to lod fontD this hs to e preEloded into the diretory data of the urrent skethF he tool Create FontD essile from the Tools menu in roessingD retes the itmps of the hrters tht the progrmmer intends to useF he (le with the itmps is put in the data diretoryF efter these preliminry opertionsD the font n e used to write some textD using the funtion text()F ith this funtionD string of hrters n e put in the Ph or Qh speD possily inserting it within retngulr oxF he lignment of hrters in the ox is governed y the funtion textAlign()F sn the defult on(gurtionD the written text n e sptilly trnsformed like ny other ojetF he olor of hrters n e set with the usul fill()D like for ny other grphi ojetF
Example 2.3: Overlapped text
4 "Matrix
Arithmetic" <http://cnx.org/content/m10090/latest/>
PQ
PFont fonte; /*The font have been previously created in the data folder*/ fonte = loadFont("HoeflerText-Black-48.vlw"); textFont(fonte, 12); fill(10, 20, 250, 80); textAlign(RIGHT); text("pippo pippo non lo sa", 10, 14, 35, 70); textFont(fonte, 24); fill(200, 0, 0, 100); text("ppnls", 25, 5, 50, 90);
Table 2.2
roessing llows tight ontrol of the sptil ouption of hrters nd of the distne etween onE tiguous hrters @see pigure PFQ @ypefe metrisAAF he funtion textWidth() omputes the horizontl extension of hrter or stringF st n e usedD together with the ext oordintes pssed to text()D to ontrol the kerning nd the tracking etween hrtersF he textSize() llows to rede(ne the size of hrtersF he textLeading() reEde(nes the distne in pixels etween djent text linesF his distne is mesured etween the baselines of the strings of hrtersF vetters suh s 4p4 or 4q4 extend elow the seline for numer of pixels tht n e otined with the textDescent()F snstedD the textAscent() gives k the mximum extension ove the seline @typillyD the height of the letter 4d4AF
Typeface metrics
Figure 2.3:
Typeface metrics
PR
CHAPTER 2.
2.2 Auditory Elements

2.2.1 Sounds
ntill version et IIPD roessing gve the possiility to progrm severl udio funtionlities y mens of some ore primitivesF sn those older versions only two si primitives re ville to plyk nd lod .wav (lesF sn more reent versionsD roessing delegte sound mngement nd proessing funtionlities to externl lirries5 F he most used lirries re iss6 D winim7 nd oni8 F es in the se of imgesD in order to proess nd plyk sounds the soure (les hve to e stored in the data folder of the urrent skethF he lirry oni 9 is the most omplex oneF ith its funtionsD one n do sample playbackD reltime pourierEsed spetrl nlysisD .wav (le svingF sn order to use the oni lirryD the progrmmer hs to downlod the .zip (le from oni10 F yne deompressedD the diretory Sonia_?_? hs to e opied into the diretory Processing/librariesF pinllyD the ommnd import hs to e inserted into the ode y seleting it from the menu item Sketch / Import Library / Sonia_?_?F note: sn order to run the pplets produed with oni from we rowserD the hil furk9s tyn plugin hs to e downloded nd instlled from the site httpXGGwwwFsoftsynthFomGjsynGpluginsG11 F he lirry winim12 D sed on tv ound13 D is more userEfriendlyD wellEdoumented nd reommendedD if one wnts to work with sounds employing highElevel primitivesD without deling with lowElevel numeril detils nd u'er mngementF
2.2.2 Timbre
sn this setionD we (rst use then nlyze n pplition for the explortion of timresD similr in oneption to the golor ghooser of pigure PFP @qimp olor hooserAD nd here lled ound ghooserF por the momentD let us think out sound timre in nlogy with olor in imgesF por exmpleD the vrious instruments of the orhestr hve di'erent nd hrterizing timres @olorsAF vter onD we will de(ne the physil nd pereptul spets of timre more urtelyF sn the ound ghooser ppletD four sounds with di'erent timres n e plyed y liking onto one of the mrked rdiiF ih rdius orresponds to musil instrument @timreGolorAF fy hnging position long the rdius it is possile to her how the rightness is hngedF wore preiselyD s long s we proeed towrd the entreD the sounds gets poorerF vet us nlyze the roessing ode tht implements the ound ghooser in its slient spetsF he Sonia.start(this) ommnd is neessry to tivte the oni udio engineF he line Sample mySample1 delres vrile imed t ontining udio smplesF everl methods n e pplied to suh vrileF emong theseD the play methods plys the sound smple kF sn the draw() ode setion the grphi spet of the pplet is de(nedF pinllyD y the funtion mouseReleased()D we detet when the mouse is relesed fter eing pressedD nd where it hs een relesedF et this point sequeneo of if onditions (nds wht instrumentGtimre hs een seleted ording to the liking pointF woreoverD within the funtion mouseReleased() the funtion filtra(float[] DATAF, float[] DATA, float RO, float WC) is invokedF his funtionD whih is implemented in the lst segment of the ode listingD performs sound (lteringF wore preiselyD it is lowEpss (lterD thus (lter tht leves the low frequenies unltered nd redues the intensity of the high frequeniesF eording to the rdil position of the mouse likD the (ltering e'et hngesD eing more drmti @tht is the sound eomes drkerA s the mouse is relesed loser nd
5 http://processing.org/reference/libraries/index.html 6 http://www.tree-axis.com/Ess/ 7 http://code.compartmental.net/tools/minim/ 8 http://sonia.pitaru.com/ 9 http://sonia.pitaru.com/ 10 http://sonia.pitaru.com/ 11 http://www.softsynth.com/jsyn/plugins/ 12 http://code.compartmental.net/tools/minim/ 13 http://java.sun.com/j2se/1.5.0/docs/guide/sound/programmer_guide/contents.html
PS loser to the entreF e lighter reliztion of the ound ghooser y mens of the lirry winim is proposed in prolem ixerise PFRF
PT rumpet14 yoe15 iolin16 plute17 eppletX hoosing timre nd ontrolling its rightness 18
CHAPTER 2.
import pitaru.sonia_v2_9.*; Sample mySample1, mySample2, mySample3, mySample4; Sample mySample1F, mySample2F, mySample3F, mySample4F; float[] data1, data2, data3, data4; float[] data1F, data2F, data3F, data4F; int sr = 11025; // sampling rate
void setup() { size(200, 200); colorMode(HSB, 360, height, height); Sonia.start(this); mySample1 = new Sample("flauto.aif"); mySample2 = new Sample("oboe.wav"); mySample3 = new Sample("tromba.wav"); mySample4 = new Sample("violino.wav"); mySample1F = new Sample("flauto.aif"); // ... OMISSIS ... data1 = new float[mySample1.getNumFrames()]; // creates new arrays the length of the sample // for the original sound // ... OMISSIS ... data1F = new float[mySample1.getNumFrames()]; // creates new arrays the length of the sample // for the filtered sound // ... OMISSIS ... mySample1.read(data1); // ... OMISSIS ... } void draw() { // ... OMISSIS ... } void mouseReleased() { float ro; float roLin; float wc; // FLAUTO
PU
Table 2.3
he ontent of PImage ojet is essile through its pixels[] (eldF he pixelsD orresponding to rowEyErow reding of the imgeD re ontined in this rry of size width*heightF wodify the ode in ixmple PFP @voding nd visulizing n imge with trnsprenyA to use the (eld pixels[] insted of the method get()F he (nl outome should remin the smeF
Exercise 2.2 Exercise 2.3
Exercise 2.1
gomplete the ode reported in le PFQ to otin the omplete ound ghooser ppletF
(Solution on p. 28.) (Solution on p. 28.)
edd some olor to the rdii of the ound ghooserD y repling the line instrutions with rect instrutions nd oloring the rs with rightness tht inreses goint from the entre to the peripheryF
Exercise 2.4
rodue new version of the ound ghooser of prolem ixerise PFP employing the lirry winimF xote the gined ompt form nd simpliity of the odeF
PV
CHAPTER 2.

he invotion b.set() should e repled y
b.set(i,j,b.pixels[j*b.width+i]+ color(0,0,0, 255 - (int)((1-ramp)*255)) );
Solution to Exercise 2.2 (p. 27) 19 Solution to Exercise 2.3 (p. 27) Solution to Exercise 2.4 (p. 27)
roessing soure odeF
epplet with roessing soure odeF 20

import ddf.minim.*; import ddf.minim.effects.*; Minim minim; AudioPlayer mySample1, mySample2, mySample3, mySample4; LowPassSP lpf1, lpf2, lpf3, lpf4; float cutoff1, cutoff2, cutoff3, cutoff4; void setup() { size(200, 200); colorMode(HSB, 360, height, height); minim = new Minim(this); mySample1 = minim.loadFile("flauto.aif"); mySample2 = minim.loadFile("oboe.wav"); mySample3 = minim.loadFile("tromba.wav"); mySample4 = minim.loadFile("violino.wav"); lpf1 = new LowPassSP(4000, lpf2 = new LowPassSP(4000, lpf3 = new LowPassSP(4000, lpf4 = new LowPassSP(4000, mySample1.addEffect(lpf1); mySample2.addEffect(lpf2); mySample3.addEffect(lpf3); mySample4.addEffect(lpf4); } void draw() { stroke(255); strokeWeight(1); fill(0, 88, 88);
19 See 20 See
mySample1.sampleRate()); mySample2.sampleRate()); mySample3.sampleRate()); mySample4.sampleRate());
the le at <http://cnx.org/content/m12983/latest/./sound_chooser.pde> the le at <http://cnx.org/content/m12983/latest/./sound_chooser_color.pde>
PW
ellipseMode(CORNER); ellipse(50,50,100,100); beginShape(LINES); vertex(50, 100); vertex(90, 100); vertex(110, 100); vertex(150, 100); vertex(100, 50); vertex(100, 90); vertex(100, 110); vertex(100, 150); endShape();
void mouseReleased() { // FLUTE if ((mouseX > 95) && (mouseX < 105)&& (mouseY > 50)&& (mouseY < 90)) { cutoff1 = map(mouseY, 50, 90, 1000, 30); lpf1.setFreq(cutoff1); println(mouseY + " + " +cutoff1); mySample1.rewind(); mySample1.play(); } // OBOE if ((mouseX > 110) && (mouseX < 149)&& (mouseY > 95)&& (mouseY < 105)) { cutoff2 = map(mouseX, 110, 149, 30, 1000); lpf2.setFreq(cutoff2); println(mouseX + " + " +cutoff2); mySample2.rewind(); mySample2.play(); } // TRUMPET if ((mouseX > 95) && (mouseX < 105)&& (mouseY > 110)&& (mouseY < 149)) { cutoff3 = map(mouseY, 110, 149, 30, 1000); lpf3.setFreq(cutoff3); println(mouseY + " + " +cutoff3); mySample3.rewind(); mySample3.play(); } // VIOLIN if ((mouseX > 50) && (mouseX < 90)&& (mouseY > 95)&& (mouseY < 105)) { cutoff4 = map(mouseX, 50, 90, 1000, 30);
QH
CHAPTER 2.
lpf4.setFreq(cutoff4); println(mouseX + " + " +cutoff4); mySample4.rewind(); mySample4.play();
// safely stop the Minim engine upon shutdown. public void stop(){ mySample1.close(); mySample2.close(); mySample3.close(); mySample4.close(); minim.stop(); super.stop(); }
Chapter 3
Graphic Composition in Processing
1
3.1 Graphic primitives

sn roessingD we n rrnge pointsD linesD surfesD nd volumes @ojets in HD ID PD nd Q dimensionsD respetivelyA in Qh spe orD where this mkes sense in the Ph spe of the imge windowF he numer of prmeters of the ojet primitives will determine if these ojets hve to e positioned in the E or in the EE speF
3.1.1 0D
Points
he point() is the simplest of the grphi primitivesF hen invoked with two oordinte prmetersD it positions point in the E speF hen invoked with three oordinte prmetersD it positions point in the EE speF e pointD in geometri senseD does not hve dimensionD ut it n e ssigned n ouption in pixels nd olorD y the funtions strokeWeight() nd stroke()D respetivelyF por exmpleD the simple roessing sketh
stroke(180,0,0); strokeWeight(10); point(60,60);
drws dot in the imge windowF e set of points n e grouped into single ojet @loud of pointsA y the delimiters beginShape(POINTS) nd endShape()F fetween them eh point hs to e spei(ed with the vertex() funtionF rnsformE tions of rottionD trnsltionD nd sling do not pply to the inside of omposite ojets desried with beginShape() nd endShape()D ut they n preede the de(nition of omposite ojet nd pply to the wholeF
3.1.2 1D
Straight Lines Collections of points
he line() drws line segment etween two points in the plne or the Qh speD with width nd olor tht n e set with strokeWeight() nd stroke()D respetivelyF
1 This
QI
QP
Collections of segments
CHAPTER 3.
GRAPHIC COMPOSITION IN PROCESSING
e set of segments n e de(nedD s we sw for pointsD y the delimiters beginShape() nd endShape()F fetween themD verties re listed y lls to the funtion vertex()F sing the invotion beginShape(LINES) the verties re tken in ouplesD eh identifying segmentF ith the rgumentEfree invotion beginShape() the vertiesD tken one fter the otherD de(ne polygonl lineF ith the losure endShape(CLOSE) the line is losed on itself y linking the (rst nd lst vertiesF he olor of suh polygon n e set y using the fill() funtion orD onverselyD left equl to the kground olor with the noFill()F
Curves
he funtion curve()D when lled with eight prmetersD drws urve on the imge plneD with initil nd (nl points determinedD respetivelyD y the seond nd third ouple of oordintes pssed s rgumentsF he (rst nd lst ouple of oordintes de(ne two ontrol points for the urveD whih is n interpolting splineD thus pssing for ll four pointsF sn roessingD howeverD only the urve segment etween the intermedite points is visulizedF ieewiseEpolynomil urveD with polynomils onneted with ontinuity t the knots 2 note: ee sntrodution to plines ndD for n introdution to the spei( kind of splines @gtmullE omA used in roessingD the term spline in ikipediF sn order to hve n ritrry numer of ontrol points we must use the funtion curveVertex() to speify eh point in the lok delimited y beginShape() nd endShape()F es opposed to the curve()D in the bezier() funtion ll the two ontrol points spei(ed y the four middle prmeters re not points touhed y the urveF hey only serve to de(ne the shpe of the approximating Bzier curveD whih hs the following interesting propertiesX it is entirely ontined in the convex hull de(ned y the extreml points nd the ontrol pointsY trnsformtions of trnsltionD rottionD or slingD ppied to the extreml nd ontrol points determine similr trnsformtion of the urveF es we n see y running the ode
stroke(255, 0, 0); line(93, 40, 10, 10); line(90, 90, 15, 80); stroke(0, 0, 0); noFill(); bezier(93, 40, 10, 10, 90, 90, 15, 80);
Denition 3.1: Spline
the ontrol points ly on the tngent pssing y the extreml pointsF sn order to hve n ritrry numer of ontrol points one must use the bezierVertex() to speify eh point within lok delimited y beginShape() nd endShape()F sn this wyD n ritrrily involute urve n e tred in the Qh speF sn PhD the funtion bezierVertex() hs six prmeters tht orrespond to the oordintes of two ontrol points nd one nhor pointF he (rst invotion of bezierVertex() hs to e preeded y ll to vertex() whih (xes the (rst nhor point of the urveF here re other methods tht llow to red the oordintes or the slope of the tngent to n ritrry point of fzier or spline urveF uh point n e spei(ed y prmeter t tht n go from 0 @(rst extremeA to 1 @seond extremeAF st is lso possile to set the preision of pproximting or interpolting urves in QhF por detils see the roessing referene mnul 3 F he roessing sketh in tle @le QFIA shows the di'erene etween the spline interpolting urve nd the fzier urveF note: ee the term Bzier curve in ikipediF
2 "Introduction to Splines" <http://cnx.org/content/m11153/latest/> 3 http://www.processing.org/reference/index_ext.html
QQ pplet tht ompres the fzier urve @redA nd the interpoltE ing spline @lkA 4
void setup() { c1x = 120; c1y = 110; c2x = 50; c2y = 70; background(200); stroke(0,0,0); size(200, 200); } int D1, D2; int X, Y; int c1x, c1y, c2x, c2y; void draw() { if (mousePressed == true) { X = mouseX; Y = mouseY; // selection of the point that is modified D1 = (X - c1x)*(X - c1x) + (Y - c1y)*(Y - c1y); D2 = (X - c2x)*(X - c2x) + (Y - c2y)*(Y - c2y); if (D1 < D2) { c1x = X; c1y = Y; } else { c2x = X; c2y = Y; } } background(200); stroke(0,0,0); strokeWeight(1); noFill(); beginShape(); curveVertex(10, curveVertex(10, curveVertex(c2x, curveVertex(c1x, curveVertex(190, curveVertex(190, endShape();
10); 10); c2y); c1y); 190); 190);
stroke(255,30,0); bezier(10,10,c2x,c2y,c1x,c1y,190,190); strokeWeight(4); point(c1x,c1y); point(c2x,c2y);
QR
CHAPTER 3.
Table 3.1
3.1.3 2D
yjets in two or three dimensions tke olor tht n e determined y the illumintionD s explined in etion QFQ @vightingAD or estlished y the method fill()D whih lso gives the possiility to set the degree of trnsprenyF
note:
Triangles
he tringle is the fundmentl onstrution element for Qh grphisF sn ftD y juxtposition of tringles one n pproximte ny ontinuous surfeF sn roessingD howeverD the tringles re spei(ed in Ph y the primitive triangle()D whose six prmeters orrespond to the oordintes of the verties in the imge windowF iven though eh tringle is de(ned in PhD it n e rotted nd trnslted in the Qh speD s it hppens in the roessing sketh
void setup(){ size(200, 200, P3D); fill(210, 20, 20); } float angle = 0; void draw(){ background(200); // clear image stroke(0,0,0); angle += 0.005; rotateX(angle); triangle(10,10,30,70,80,80); }
Collections of triangles
beginShape() nd endShape()F fetween themD the verties of the tringles re listed y lls to the funtion vertex()F fy the invotion beginShape(TRIANGLES) the verties re tken in triplesD eh de(ning tringleD while the invotion beginShape(TRIANGLE_STRIP) tkes the verties one fter the other to de(ne strip md of tringulr fetsF sf the vertex() hs three rgumentsD the verties re loted in the Qh
e set of tringles n e de(nedD similrly to wht we did for points nd segmentsD y the delimiters
spe nd the orresponding tringles identify plnr surfes in speF

Quadrilaterals
etngles re de(nedD in roessingD y the funtion rect() of four prmetersD where the (rst ouple spei(esD y defultD the position in the the Ph plne of the topEleft ornerD nd the third nd fourth prmeters speify the width nd heightD respetivelyF he mening of the (rst ouple of prmeters n e hnged with the funtion rectMode()X rectMode(CORNER) gives the defult positioningY rectMode(CENTER) gives the positioning of the enter of the retngle t the spei(ed pointY with the rectMode(CORNERS) the four prmeters re interpreted s the oordintes of the topEleft nd ottomEright vertiesD respetivelyF e generi qudrilterl is de(ned y the oordintes of its four vertiesD pssed s prmeters to the funtion quad()F st is importnt to notie tht in QhD while tringle stys plnr in ny seD qudruple of points does not neessrily ly on plneF ieversD the qudrilterls tht re de(ned y Qh rotoEtrnsltions of qudruples of Ph vertiesD remin plnrF roessing llows only eight prmeters to e pssed to quad()D thus foring the de(nition of qudrilterl s qudruple of verties in PhF
Collections of quadrilaterals
e set of qudrilterls n e de(nedD similrly to wht we sw for tringlesD y the delimiters beginShape() nd endShape()F fetween themD verties re listed y lls to the funtion vertex()F fy using the invoE tion beginShape(QUADS) the verties re tken in qudruplesD eh identifying qudrilterlD while the
QS invotion beginShape(QUAD_STRIP) tkes the verties one fter the other to de(ne strip md of qudriE lterl fetsF sf the vertex() hve three prmetersD the plnrity of the resulting fes is not ensuredD nd the resulting rendering n e misledingF por instneD y running the ode
size(200,200,P3D); lights(); beginShape(QUADS); vertex(20,31, 33); vertex(80, 40, 38); vertex(75, 88, 50); vertex(49, 85, 74); endShape();
we relize tht the qudrilterl is rendered s the juxtposition of two tringles elonging to di'erent plnesF e generi polygon is de(ned s set of vertiesD nd it hs surfe tht n e oloredF sn roessing the verties re listed within ouple beginShape(POLYGON); E endShape(); etullyD the polygon hs to e intended in generlized senseD s it is possile to use the bezierVertex() nd curveVertex() to speify urved pro(lesF por instneD the reder my try to drw the moonX
fill(246, 168, 20); beginShape(POLYGON); vertex(30, 20); bezierVertex(80, 10, 80, 75, 30, 75); bezierVertex(50, 70, 60, 25, 30, 20); endShape();
Polygons
he funtion ellipse() drws n ellipse in the Ph plneF sts four prmeters re interpretedD s in the se of rect()D s position followed y width nd heightF he position n e set in di'erent wys ording to the ellipseMode()D whose prmeter n tke vlues CORNER, CORNERS, CENTER, CENTER_RADIUSF he (rst ouple of these possile vlues hve to e referred to the retngle tht is irumsried to the ellipseF
3.1.4 3D
Ellipses
roessing o'ers very limited repertoire of QhEojet primitivesD essentilly only lls nd oxesF
Boxes
he funtion box() produes ue when invoked with single prmeter @edgeAD prllelepiped when invoked with three prmeters @widthD heightD depthAF
Balls
he funtion sphere() produesD y n pproximting polyhedronD sphere whose rdius is spei(ed s prmeterFhe funtion sphereDetail() n e used to speify the numer of verties of the polyhedron tht pproximtes the idel sphereF
3.2 The stack of transformations

e rottion or trnsltion n e imgined s opertions tht rotte or trnslte the grtesin referene systemF sn other termsD fter rotate() or translate() the following positioning opertions of the ojets will hve new oordinte systemF hen vrious ojets re positioned in di'erent wys in speD
QT
CHAPTER 3.
it is useful to keep tre of the oordinte systems tht re setD one fter the otherF he dt struture tht is suited for ontining suh systems is the stk of trnsformtions @matrix stackAF ith the funtion pushMatrix() the urrent oordinte system is put on top of the stkF yn the other hndD to revert to the oordinte system efore the lst trnsformtionD we hve to ll popMatrix()F etullyD the stk ontins the 0ne trnsformtion mtriesD ording to wht is ditted y OpenGL nd desried in etion QFS @ills of ypenqvAF
Example 3.1
sn this exmple two ojets re positioned in the Qh speX plnr squre nd ueF he (rst sves the oordinte system onto the stkD then some trnsformtions re ppliedD nd (nlly the squre is drwnF o go k to the previous oordinte system nd pply new trnsformtions to position the ueD we pply popMatrix()F issentillyD the pushMatrix() nd popMatrix() determine the sope for the geometri positioning of n ojetF
pushMatrix() float angle; void setup(){ size(100, 100, P3D); int angle = 0; } void draw(){ background(200); angle += 0.003; pushMatrix(); translate(25,50); rotateZ(angle); rotateY(angle); rectMode(CENTER); rect(0,0,20,20); popMatrix(); translate(75,50,-25); rotateX(angle); box(20); }
3.3 Lighting
he roessing lighting model ehoes the model used in OpenGLD tht is the Phong shadingF uh model is not physilly justi(edD ut it is prtiulrly e0ientF ypenqv onsiders s illuminted eh polygon whose norml forms n ute ngle with the diretion of inoming lightF his hppens regrdless of ny msking ojetsF hereforeD shdows re not stF ypenqv is sid to use lol illumintion modelD sine multiple re)etions mong surfes nd st shdows re not utomtilly renderedF en environmentl light is villeD whih is not oming from ny prtiulr diretionD nd whose olor is spei(ed y the prmeters of the tivtion ll ambientLight()F e diretionl light soure is set with the directionalLight()D whose prmeters speify olor nd inoming diretionF he method lights() tivtes defult omintion of gry mient light nd diretionl lightD the ltter lso gryD oming from the frontl diretionF st is possile to set point light soure in given point of spe y the ll pointLight()F pinllyD the method spotLight() tivtes light em whih n e ontrolled in its olorD
QU positionD diretionD pertureD nd onentrtion round the xisF he exponent e tunes the fllo' round the xisX cose () @QFIA hen hitting plnr surfeD diretionl light produes re)eted light long severl diretionsD dependE ing on the surfe propertiesF sn the se of perfetlyEdi'usive @or LambertianA surfeD the light rdites evenly from the surfe long ll diretionsD with n intensity tht is lrger for inident diretions loser to the surfe normlF ie versD if the surfe is perfetly re)etingD light is only re)eted long the diretion tht is speulrly symmetri @out the surfe normlA to the inident diretionF sn ypenqvD to hve some )exiility in de(ning the illumintionD eh soure hs the three illumintion omponentsX mientD di'useD nd speulrF hese three omponents re seprtely de(ned nd intert with the respetive omponents tht de(ne the surfe properties of ojetsF he olors de(ned in the methods directionalLight()D pointLight()D nd spotLight() de(ne the vmertin omponent of illumintionF he lightSpecular() spei(es the olor of the omponent of inoming light tht is sujet to speulr re)etionF sn roessingD the properties of surfes re ontrolled y the methods ambient() @ting on the mE ient omponent of inoming lightsA nd specular() @ting on the speulr omponentAF he funtion shininess() ontrols the onentrtion of the speulrlyEre)eted emD y oe0ient tht ts similrly to the exponent of @QFIAF he represented ojets n lso e onsidered s soures of lightD nd they n e ssigned n emission light y the emmissive() llF roweverD the soures de(ned in this wy do not illuminte the other ojets on the seneF sn ypenqv the pointD spotD nd mient lights re ttenuted with inresing distneD ording to the model 1 @QFPA attenuation = a + bd + cd2 he method ligthFalloff() llows to speify the prmeters aD bD nd cF
Example 3.2
rereD ue nd QUAD_STRIP re positioned in spe nd illuminted y rotting soureF woreoverD soft (xed light is setF xotie the sene of shdows nd the pprent plnrity of surfes in the QUAD_STRIPF
float r; float lightX, lightY, lightZ; void setup() { size(400, 400, P3D); r = 0; ambient(180, 90, 0); specular(0, 0, 240); lightSpecular(200, 200, 200); shininess(5); } void draw() { lightX = 100*sin(r/3) + width/2; lightY = 100*cos(r/3) + height/2; lightZ = 100*cos(r); background(0,0,0); noStroke(); ambientLight(153, 102, 0); lightSpecular(0, 100, 200);
QV
CHAPTER 3.
pointLight(100, 180, 180, lightX, lightY, lightZ); pushMatrix(); translate(lightX, lightY, lightZ); emissive(100, 180, 180); sphere(4); //Put a little sphere where the light is emissive(0,0,0); popMatrix(); pushMatrix(); translate(width/2, height/2, 0); rotateX(PI/4); rotateY(PI/4); box(100); popMatrix(); pushMatrix(); translate(width/4, height/2, 0); beginShape(QUAD_STRIP); vertex(10,13,8); vertex(13,90,13); vertex(65,76,44); vertex(95,106,44); vertex(97,20,70); vertex(109,70,80); endShape(); popMatrix(); r+=0.05;
3.4 Projections
3.4.1 Perspective projections
e perspetive projetion is de(ned y center of projection nd plane of projectionF he projector rays onnet the points in the sene with the enter of projetionD thus highlighting the orresponding points in the plne of projetionF he pigure QFI shows setion where the plne of projetion produes stright line whose siss is dD nd the enter of projetion is in the originF
QW
Figure 3.1
fy similrity of two tringles it is esy to relize tht the point hving ordinte y gets projeted onto the plne in the point hving ordinte yp = yd z F
x
sn generlD the projetion of point hving homogeneous
the z xis nd interseting suh xis in position d is otinedD in homogeneous oordintesD y multiplition
1 0 with the mtrix 0 0 0 1 0 0 0 0 1
1 d
y oordintes onto z 1 x y z
z d xd z yd z
plne orthogonl to
0 F 0 0
he projeted point eomes
D whih n e normlized
y multiplition of ll its element y
d z
F es resultD we otin
d 1
3.4.2 Parallel views
rllel views re otined y tking the enter of projetion k to in(nity @AF sn this wyD the projetor rys re ll prllelF
RH
3.4.2.1 Orthographic projection
CHAPTER 3.
he orthogrphi projetion produes lss of prllel views y sting projetion rys orthogonl to the plne of projetionF sf suh plne is positioned orthogonlly to the z xis nd pssing y the originD the projetion mtrix turns out to e prtiolrly simpleX
1 0 0 0 0 1 0 0 0 0 0 0 0 0 F 0 1
emong orthogrphi projetionsD
the axonometric projections re sed on the possiility to mesure the ojet long three orthogonl xesD nd on the orienttion of the plne of projetion with respet to these xesF sn prtiulrD in the isometri projetion5 the projetions of the xes form ngles of 120 F he isometri projetion hs the property tht equl segments on the three xes remin equl when they re projeted onto the plneF sn order to otin the isometri projetion of n ojet whose min xes re prllel to the oordinte xesD we 1 n (rst rotte the ojet y 45 out the y xisD nd then rotte y arctan = 35.264 out the x 2 xisF
3.4.2.2 Oblique projection
e n tlk out olique projetion every time the projetor rys re olique @nonEorthogonlA to the projetion plneF sn order to devite the projetor rys from the norml diretion y the ngles nd we must use projetion mtrix
1 0 0 1 0 0 0 0 (tan ()) 0 0 0 (tan ()) 0 0 1
3.4.3 Casting shadows
es we hve seenD roessing hs lol illumintion modelD thus eing impossile to st shdows diretlyF roweverD y mnipulting the 0ne trnsformtion mtries we n st shdows onto plnesF he method is lled ashing in the eyeD thus mening tht the optil enter of the sene is moved to the point where the light soure is positionedD nd then perspetive trnsformtion is mdeD with plne of projetion tht oinides with the plne where we wnt to st the shdow onF
Example 3.3
he following progrm projets on the )oor the shdow produed y light soure positioned on the y xisF he result is shown in pigure QFP @gsting shdowA
5 http://en.wikipedia.org/wiki/Isometric_projection
RI
Casting a shadow
Figure 3.2
size(200, 200, P3D); float centro = 100; float yp = 70; //floor (plane of projection) distance from center float yl = 40; //height of light (center of projection) from center translate(centro, centro, 0); //center the world on the cube noFill(); box(yp*2); //draw of the room pushMatrix(); fill(250); noStroke(); translate(0, -yl, 0); // move the virtual light bulb higher sphere(4); //draw of the light bulb stroke(10); popMatrix(); pushMatrix(); //draw of the wireframe cube noFill(); rotateY(PI/4); rotateX(PI/3); box(20); popMatrix(); // SHADOW PROJECTION BY COMPOSITION // OF THREE TRANSFORMATIONS (the first one in // the code is the last one to be applied)
RP
CHAPTER 3.
translate(0, -yl, 0); // shift of the light source and the floor back // to their place (see the translation below) applyMatrix(1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1/(yp+yl), 0, 0); // projection on the floor // moved down by yl
translate(0, yl, 0); // shift of the light source to center // and of the floor down by yl pushMatrix(); // draw of the cube that generate the shadow fill(120, 50); // by means of the above transformations noStroke(); rotateY(PI/4); rotateX(PI/3); box(20); popMatrix();
3.5 Pills of OpenGL

OpenGL is set of funtions tht llow the progrmmer to ess the grphi systemF ehnilly spekingD it is n Application Programming Interface (API)F sts min sope is the grphi rendering of sene populted y Qh ojets nd lightsD from given viewpointF es fr s the progrmmer is onernedD ypenqv llows to desrie geometri ojets nd some of their propertiesD nd to deide how suh ojets hve to e illuminted nd seenF es fr s the implementtion is onernedD ypenqv is sed on the graphic pipeline D mde of modules s reported in pigure QFQ @he ypenqv pipelineAF en exellent ook on intertive grphis in ypenqv ws written y Angel IF The OpenGL pipeline
Figure 3.3
sn roessing @nd in ypenqvAD the progrmmer spei(es the ojets y mens of world oordintes @standard coordinatesAF he model-view matrix is the trnsformtion mtrix used to go from stnE drd oordintes to spe ssoited with the merF his llows to hnge the mer viewpoint nd orienttion dynmillyF sn ypenqv this is done with the funtion gluLookAt()D whih is reprodued in roessing y the camera()F he (rst triple of prmeters identi(es the positionD in world oordintesD of the optil enter of the mer @eye pointAF he seond triple of prmeters identi(es point where the mer is looking t @center of the sceneAF he third triple of oordintes identi(es vetor imed t speifying the viewing vertilF por exmpleD the progrm
RQ
void setup() { size(100, 100, P3D); noFill(); frameRate(20); } void draw() { background(204); camera(70.0, 35.0, 120.0, 50.0, 50.0, 0.0, (float)mouseX /width, (float)mouseY /height, 0.0); translate(50, 50, 0); rotateX(-PI/6); rotateY(PI/3); box(45); }
drws the wireframe of ue nd enles the dynmi rottion of the merF he projection matrix is responsile for the projetion on the viewing windowD nd this projetion n e either prllel @orthogrphiA or perspetiveF he orthogrphi projetion n e tivted with the ll ortho()F he perspetive projetion is the defult oneD ut it n e expliitly tivted with the ll perspective()F rtiulr projetionsD suh s the olique onesD n e otined y distortion of ojets y pplition of the applyMatrix()F here is lso the texture matrixD ut textures re treted in nother moduleF por eh type of mtrixD ypenqv keeps stkD the urrent mtrix eing on topF he stk dt struture llows to sve the stte @y the pushMatrix()A efore performing new trnsformtionsD or to remove the urrent stte nd tivte previous sttes @y the popMatrix()AF his is re)eted in the roessing opertions desried in etion QFP @he stk of trnsformtionsAF sn ypenqvD the trnsformtions re pplied ording to the sequene IF ush on the stkY PF epply ll desired trnsformtions y multiplying y the stkEtop mtrixY QF hrw the ojet @'eted y trnsformtionsAY RF op from the stkF e viewport is retngulr re of the disply windowF o go from the perspetive projetion plne to the viewport two steps re tkenX @iA trnsformtion into P x P window entered in the origin @ normalized device coordinates A @iiA mpping the normlized window onto the viewportF sing the normlized devie oordintesD the clipping opertionD tht is the elimintion of ojets or prts of ojets tht re not visile through the windowD eomes trivilF screenX()D screenY()D nd screenZ() gives the E oordintes produed y the viewport trnsformtion nd y the previous opertors in the hin of pigure QFQ @he ypenqv pipelineAF he viewing frustum is the solid ngle tht enompsses the perspetive projetionD s shown in pigE ure QFR @he viewing frustumAF he ojets @or their prtsA elonging to the viewing volume re visulizedD the remining prts re sujet to lippingF sn roessing @nd in ypenqvA the frustum n e de(ned y positioning the six plnes tht de(ne it @frustum()AD or y spei(tion of the vertil ngleD theD aspect ratioD nd the positions of the front nd k plnes @perspective()AF yne my sk how the system reE moves the hidden fesD iFeFD those fes tht re msked y other fes in the viewing volumeF ypenqv uses the z-buer lgorithmD whih is supported y the grphi elertorsF he ord memory stores Ph memory re @the zEu'erA orresponding to the pixels of the viewing windowD nd ontining depth vluesF fefore polygon gets projeted on the viewing window the ord heks if the pixels 'eted y
RR
CHAPTER 3.
suh polygon hve depth vlue smller thn the polygon eing drwnF sf this is the seD it mens tht there is n ojet tht msks the polygonF
The viewing frustum
Figure 3.4
ophistited geometri trnsformtions re possile y diret mnipultion of the projetion nd modelE view mtriesF his is possileD in roessingD strting from the unit mtrixD loded with resetMatrix()D nd proeeding y mtrix multiplies done with the applyMatrix()F
Exercise 3.1
un nd nlyze the roessing ode

size(200, 200, P3D); println("Default matrix:"); printMatrix(); noFill(); ortho(-width/2, width/2, -height/2, height/2, -100, 100); translate(100, 100, 0); println("After translation:"); printMatrix(); rotateX(atan(1/sqrt(2)));
RS
println("After about-X rotation:"); printMatrix(); rotateY(PI/4); println("After about-Y rotation:"); printMatrix(); box(100);
ht is visulized nd wht it the kind of projetion usedc row do you interpret the mtries printed out on the onsolec gn one invert the order of rottionsc
Exercise 3.2
rite roessing progrm tht performs the olique projetion of ueF
(Solution on p. 46.) (Solution on p. 46.)
isulize ue tht projets its shdow on the )oorD ssuming tht the light soure is t in(nite distne @s it is the seD in prtieD for the sunAF
Exercise 3.3
RT
CHAPTER 3.

he wirefrme of ue is visulized in isometri projetionF he ltter three mtries representD one fter the otherD the three opertions of trnsltion @to enter the ue to the windowAD rottion out the x xisD nd rottion out the y xisF e sequene of two rottions orrespond to the produt of two rottion mtriesD nd the outome is not order independent @produt is not ommuttiveAF he produt of two rottion mtries Rx Ry orrespond to performing the rottion out y (rstD nd then the rottion out xF por exmpleX
size(200, 200, P3D); float theta = PI/6; float phi = PI/12; noFill(); ortho(-width/2, width/2, -height/2, height/2, -100, 100); translate(100, 100, 0); applyMatrix(1, 0, - tan(theta), 0, 0, 1, - tan(phi), 0, 0, 0, 0, 0, 0, 0, 0, 1); box(100);
e do it similrly to ixmple QFQD ut the trnsformtion is orthogrphiX

size(200, 200, P3D); noFill(); translate(100, 100, 0); pushMatrix(); rotateY(PI/4); rotateX(PI/3); box(30); popMatrix(); translate(0, 60, 0); //cast a shadow from infinity (sun) applyMatrix(1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1); fill(150); pushMatrix(); noStroke(); rotateY(PI/4); rotateX(PI/3); box(30); popMatrix();
Chapter 4
Signal Processing in Processing: Sampling and Quantization
1
4.1 Sampling
foth sounds nd imges n e onsidered s signlsD in one or two dimensionsD respetivelyF ound n e desried s )utution of the ousti pressure in timeD while imges re sptil distriutions of vlues of luminne or olorD the ltter eing desried in its qf or rf omponentsF eny signlD in order to e proessed y numeril omputing deviesD hve to e redued to sequene of disrete samplesD nd eh smple must e represented using (nite numer of itsF he (rst opertion is lled samplingD nd the seond opertion is lled quantization of the domin of rel numersF
4.1.1 1-D: Sounds
mpling isD for oneEdimensionl signlsD the opertion tht trnsforms ontinuousEtime signl @suh sD for instneD the ir pressure )utution t the entrne of the er nlA into disreteEtime signlD tht is sequene of numersF he disreteEtime signl gives the vlues of the ontinuousEtime signl red t 1 intervls of T seondsF he reiprol of the smpling intervl is lled sampling rate Fs = T F sn this module we do not explin the theory of smplingD ut we rther desrie its mnifesttionsF por more extensive yet essile tretmentD we point to the Introduction to Sound Processing PF por our purposesD the proess of smpling IEh signl n e redued to three fts nd theoremF Fact 1 E he pourier rnsform2 of disreteEtime signl is funtion @lled spectrumA of the ontinuous vrile D nd it is periodi with period 2F qiven vlue of D the pourier trnsform gives k omplex numer tht n e interpreted s mgnitude nd phse @trnsltion in timeA of the sinusoidl omponent t tht frequenyF Fact 2 E mpling the ontinuousEtime signl x (t) with intervl T we get the disreteEtime signl x (n) = x (nT )D whih is funtion of the disrete vrile nF Fact 3 E mpling ontinuousEtime signl with smpling rte Fs produes disreteEtime signl whose frequeny spetrum is the periodi replition of the originl signlD nd the replition period is Fs F he pourier vrile for funtions of disrete vrile is onverted into the frequeny vrile f @in rertzA y mens of f = 2T F he pigure RFI @prequeny spetrum of smpled signlA shows n exmple of frequeny spetrum of signl smpled with smpling rte Fs F sn the exmpleD the ontinuousEtime signl hd ll nd only the frequeny omponents etween Fb nd Fb F he replis of the originl spetrum re sometimes lled imagesF
1 This content is available online at <http://cnx.org/content/m13045/1.5/>. 2 "Derivation of the Fourier Transform" <http://cnx.org/content/m0046/latest/>
RU
RV
CHAPTER 4.
SIGNAL PROCESSING IN PROCESSING: SAMPLING AND QUANTIZATION
Frequency spectrum of a sampled signal
Figure 4.1
qiven the fts @pF RUAD we n hve n intuitive understnding of the mpling heoremD historilly ttriuted to the sientists xyquist nd hnnonF Theorem 4.1: mpling heorem e ontinuousEtime signl x (t)D whose spetrl ontent is limited to frequenies smller thn Fb @iFeFD it is ndElimited to Fb A n e reovered from its smpled version x (n) if the smpling rte is lrger thn twie the ndwidth @iFeFD if Fs > 2Fb A he reonstrution n only our y mens of (lter tht nels out ll spetrl imges exept for the one diretly oming from the originl ontinuousEtime signlF sn other wordsD the neled imges re those hving frequeny omponents higher thn the Nyquist frequency de(ned s F2 F he ondition required y the smpling theorem @heorem RFID mpling heoremD pF RVA is equivlent to sying tht no overlps etween spetrl imges re llowedF sf suh superimpositions were presentD it wouldn9t e possile to design (lter tht elimintes the opies of the originl spetrumF sn se of overlppingD (lter tht elimintes ll frequeny omponents higher thn the xyquist frequeny would produe signl tht is 'eted y aliasingF he onept of lising is well illustrted in the elising epplet3 D where ontinuousEtime sinusoid is sujet to smplingF sf the frequeny of the sinusoid is too high s ompred to the smpling rteD we see tht the the wveform tht is reonstruted from smples is not the originl sinusoidD s it hs muh lower frequenyF e ll hve fmilirity with lising s it shows up in moving imgesD for instne when the wgon wheels in western movies strt spinning kwrdF sn tht seD the smpling rte is given y the frame rateD or numer of pitures per seondD nd hs to e relted with the spinning veloity of the wheelsF his is one of severl stroosopi 4 phenomenF sn the se of soundD in order to eome wre of the onsequenes of the 2 periodiity of disreteEtime signl spetr @see pigure RFI @prequeny spetrum of smpled signlAA nd of violtions of the ondition of the smpling theoremD we exmine simple seF vet us onsider sound tht is generted y sum of sinusoids tht re hrmonis @iFeFD integer multiplesA of fundmentlF he spetrum of suh sound
s
3 "Aliasing Applet" <http://cnx.org/content/m11448/latest/> 4 http://www.michaelbach.de/ot/mot_strob/
RW would disply peks orresponding to the fundmentl frequeny nd to its integer multiplesF tust to give onrete exmpleD imgine working t the smpling rte of 44100 rz nd summing 10 sinusoidsF prom the smpling theorem we know thtD in our seD we n represent without lising ll frequeny omponents up to 22050 rzF oD in order to void lisingD the fundmentl frequeny should e lower thn 2205 rzF he roessing @C oniA ode reported in tle le RFI implements genertor of sounds formed y 10 hrmoni sinusoidsF o produe suh sounds it is neessry to lik on point of the disply windowF he x oordinte would vry with the fundmentl frequenyD nd the window will show the spetrl peks 1 of orresponding to the generted hrmonisF hen we lik on point whose x oordinte is lrger thn 10 the window widthD we still see ten spetrl peksF ytherwiseD we violte the smpling theorem nd lising will enter our representtionF
SH elising testX epplet to experiene the e'et of lising on sounds otined y summtion of IH sinusoids in hrmoni rtio 5
CHAPTER 4.
import pitaru.sonia_v2_9.*; int L = 1024; // FFT length (positive frequency) int L2 = int(L*4); // table length int sr = 44100; int H = 10; //number of harmonics int ampiezza; float[] Data; float freq = 10.00; // fundamental frequency [Hz] Sample mySample; float oneCycle = TWO_PI/L; void setup() { size(1024,200); frameRate(1); colorMode(HSB, 360, height, height); Sonia.start(this); mySample = new Sample(L2); // empty sample with L2 frames. Data = new float[L2]; // array with as many frames as samples. //populate the array with sample data, i.e. a sine wave for(int k = 1; k <=H; k++){ // Additive Synthesis for(int i = 0; i < L2; i++){ Data[i] = Data[i] + sin(i*oneCycle*freq*k)/10; } } mySample.write(Data); // write from 'Data' array into sample mySample.repeat(); // loop the sample */ } void draw() { background(0,20,0); strokeWeight(0); stroke(0,230,0); mySample.getSpectrum(L); // FFT stroke(255); //println(); for ( int i = 1; i < L; i++){ ampiezza = int(mySample.spectrum[i]*L); line(i, height, i, height - 2*ampiezza/(0.003*i)); } } void mouseReleased() { freq = (float)(mouseX/2); // to change the fundamental println(freq + " Hz"); for (int i = 0; i < L2; i++) Data[i] = 0; for(int k = 1; k <=H; k++){ // Additive Synthesis for(int i = 0; i < L2; i++){ Data[i] = Data[i] + sin(i*oneCycle*freq*k)/20; } } mySample.write(Data); // write from 'Data' array into sample mySample.getSpectrum(L); // initial steps mouseReleased();
SI
Table 4.1
4.1.2 2-D: Images
vet us ssume we hve ontinuous distriutionD on plneD of vlues of luminne orD more simply sttedD n imgeF sn order to proess it using omputer we hve to redue it to sequene of numers y mens of smplingF here re severl wys to smple n imgeD or red its vlues of luminne t disrete pointsF he simplest wy is to use regulr gridD with sptil steps X e Y F imilrly to wht we did for soundsD we de(ne 1 1 nd FY = Y F es in the oneEdimensionl seD lso for twoEdimensionl the sptil smpling rtes FX = X signlsD or imgesD smpling n e desried y three fts nd theoremF Fact 1 E he pourier rnsform of disreteEspe signl is funtion @lled spectrumA of two ontinuous vriles X nd Y D nd it is periodi in two dimensions with periods 2F qiven ouple of vlues X nd Y D the pourier trnsform gives k omplex numer tht n e interpreted s mgnitude nd phse @trnsltion in speA of the sinusoidl omponent t suh sptil frequeniesF Fact 2 E mpling the ontinuousEspe signl s (x, y ) with the regulr grid of steps X D Y D gives disreteEspe signl s (m, n) = s (mX, nY )D whih is funtion of the disrete vriles m nd nF Fact 3 E mpling ontinuousEspe signl with sptil frequenies FX nd FY gives disreteEspe signl whose spetrum is the periodi replition long the grid of steps FX nd FY of the originl signl spetrumF he pourier vriles X nd Y orrespond to the frequenies @in yles per meterA represented y the vriles fX = 2 X nd fY = 2Y F he pigure RFP @petrum of smpled imgeA shows n exmple of spetrum of twoEdimensionl smpled signlF hereD the ontinuousEspe signl hd ll nd only the frequeny omponents inluded in the entrl hexgonF he hexgonl shpe of the spetrl support @region of nonEnull spetrl energyA is merely illustrtiveF he replis of the originl spetrum re often lled spetrl imagesF
X Y
5 http://cnx.org/content/m13045/latest/./aliasingDP.html
SP
CHAPTER 4.
Spectrum of a sampled image
Figure 4.2
qiven the ove fts @pF SIAD we n hve n intuitive understnding of the mpling heoremF Theorem 4.2: mpling heorem @in PhA e ontinuousEspe signl s (x, y)D whose spetrl ontent is limited to sptil frequenies elonging to the retngle hving semiEedges FbX nd FbY @iFeFD ndlimitedA n e reovered from its smpled version s (m, n) if the sptil smpling rtes re lrger thn twie the respetive ndwidths @iFeFD if FX > 2FbX nd FY > 2FbY A sn prtieD the sptil smpling step n not e lrger thn the semiEperiod of the (nest sptil frequeny @or the (nest detilA tht is represented in the imgeF he reonstrution n only e done through (lter tht elimintes ll the spetrl imges ut the one oming diretly from the originl ontinuousEspe signlF sn other wordsD the (lter will ut ll imges whose frequeny omponents re higher thn the Nyquist frequency de(ned s F2 nd F2 long the two xesF he ondition required y the smpling theorem @heorem RFPD mpling heorem @in PhAD pF SPA is equivlent to requiring tht there re no overlps etween spetrl imgesF sf there were suh overlpsD it wouldn9t e possile to eliminte the opies of the originl signl spetrum y mens of (lteringF sn se of overlppingD (lter utting ll frequeny omponents higher thn the xyquist frequeny would give k signl tht is 'eted y lisingF e note how lising n e produed y downEsmpling @or deimtingA smpled imgeF trting from disreteEspe imgeD we n selet only suset of smples rrnged in regulr gridF his will determine the periodi repetition of the spetrl imgesD tht will end up overlppingF sn order to explore the onepts of smplingD downEsmplingD nd lisingD run the pplet drwing ellipses 6 F ith the keyord rrow you n doule or hlve the horizontl nd vertil smpling stepsF e simple introdution to the (rst elements of imge proessing is found in higitl smge roessing fsis7 F
X Y
6 http://cnx.org/content/m13045/latest/resampling_ellipse.html 7 "Digital Image Processing Basics" <http://cnx.org/content/m10973/latest/>
SQ
4.2 Quantization
ith the djetive 4digitl4 we indite those systems tht work on signls tht re represented y numersD with the @(niteA preision tht omputing systems llowF p to now we hve onsidered disreteEtime nd disreteEspe signls s if they were olletions of in(niteEpreision numersD or rel numersF nfortuntelyD omputers only llow to represent (nite susets of rtionl numersF his mens tht our signls re sujet to quntiztionF por our purposesD the most interesting quntiztion is the liner oneD whih is usully ourring in the proess of onversion of n nlog signl into the digitl dominF sf the memory word dedited to storing numer is mde of b itsD then the rnge of suh numer is disretized into 2b quntiztion levelsF eny vlue tht is found etween two quntiztion levels n e pproximted y truntion or rounding to the losest vlueF he pigure RFQ @mpling nd quntiztion of n nlog signlA shows n exmple of quntiztion with representtion on 3 its in two9s omplement8 F
Sampling and quantization of an analog signal
Figure 4.3
he pproximtion introdued y quntiztion mnifests itself s noiseD lled quantization noiseF yftenD for the nlysis of soundEproessing iruitsD suh noise is ssumed to e white nd deEorrelted with the signlD ut in relity it is pereptully tied to the signl itselfD in suh n extent tht quntiztion n e pereived s n e'etF o hve visul nd intuitive explortion of the phenomenon of quntiztionD onsider the pplet 9 tht llows to vry etween 1 nd 8 the numer of its dedited to the representtion of eh of the qf hnnels representing olorF he sme numer of its is dedited to the representtion of n udio signl oupled to the imgeF he visul e'et tht is otined y reduing the numer of its is similr to solarizationF
Exercise 4.1
ixtend the ode of the pplet le RFI to dd some intertion feturesX
8 "Two's Complement and Fractional Arithmetic for 16-bit Processors" 9 http://cnx.org/content/m13045/latest/quantagondole.html
<http://cnx.org/content/m10808/latest/>
SR

CHAPTER 4.
wke the fundmentl frequeny of the utomtillyEgenerted sound hnging rndomly t eh frme @see random()10 AF wke the frmerte @nd the metronome for generting notesA dependent on the horizontl position of the mouse @mouseXAF wke the numer of hrmonis of the sound @iFeFD its rightnessA dependent on the vertil position of the mouse @mouseYAF int the window kground in suh wy tht moving from left to right the tint hnges from lue to redF sn this wyD lue olor will orrespond to slow tempoD nd red olor to fst tempoF wke the olor sturtion of the kground dependent on the vertil position of the mouseF sn this wy sound with few hrmonis @low rightnessA will orrespond to gryish olorD while sound rih of hrmonis @high rightnessA will orrespond to sturted olorF edd ontrol to stop the omputtion nd disply of the spetrum nd rete n e'et of imge freezingD while sound ontinues to e generted @for instne y keeping the mouse utton pressedAF edd ontrol to nel the dependene of tempoD rightnessD nd olor sturtion on mouse position @for instne y pressing keyAF edd ontrol thtD in se of imge freezing @mouse utton pressedAD will stop the genertion of new notes while 4freezing4 the urrent noteF pinllyD dd ontrol thtD in se of imge freezingD will stop the sound genertion nd mke the pplition silentF
10 http://www.processing.org/reference/random_.html
SS

he proposed extensions re implemented in the roessing ode11 F
11 http://cnx.org/content/m13045/latest/./aliasingFermoSemplice.pde
ST
CHAPTER 4.
Chapter 5
Signal Processing in Processing: Convolution and Filtering
1
5.1 Systems
por our purposesD system is ny proessing element thtD given s input sequene of smples x (n)D produes s output sequene of smples y (n)F sf the smples re oming from temporl series we tlk out discrete-time systemsF sn this module we will not e onerned with ontinuousEtime proessingD even though the priniples here desried n e generlized to funtions of ontinuous vrileF snstedD the sequene of numer n ome from the smpling of n imgeD nd in this se it will e pproprite to tlk of discrete-space systems nd use two indees m nd n if smpling is done y retngulr grid of rows nd olumnsF sn this module we re only deling with linear systemsD thus mening tht the following priniple holdsX sf y1 nd y2 re the responses to the input sequenes x1 nd x2 then the input a1 x1 + a2 x2 produes the response a1 y1 + a2 y2 enother importnt onept is time @nd speA invrineF e system is timeEinvrint if time shift of D smples in the input results in the sme time shift in the outputD iFeFD x (n D) produes y (n D)F gses of nonEinvrine re found whenever the system hnges its hrteristis in time @or speAD for exmple s n e'et of humn ontrolF hose systems where the smpling rte t the input is di'erent thn the one t the output re lso nonEinvrintF por instneD deimtors re timeEvrint systemsF e series onnetion of linear time-invariant @vsA loks is itself liner nd timeEinvrint systemD nd the order of loks n e hnged without 'eting the inputEoutput ehviorF vs systems n e thoroughly desried y the response they give to unitEmgnitude impulseF
Denition 5.3: The impulse in discrete time (space) Denition 5.2: Time invariance Denition 5.1: Superposition principle
is the signl with vlue1 t the instnt zero @in the point with oordintes [0, 0]AD nd 0 in ny other instnt @pointAF
1 This
SU
SV
CHAPTER 5.
SIGNAL PROCESSING IN PROCESSING: CONVOLUTION AND FILTERING
5.2 Impulse response and convolution

e ll h the output signl of vs system whose input is just n impulseF uh output signl is lled impulse responseF ine ny disreteEtime @EspeA signl n e thought of s weighted sum of trnsE lted impulsesD eh smple tht shows up to the input activates n impulse response whose mplitude is determined y the vlue of the smple itselfF woreoverD sine the impulse responses re tivted t distne of one smpling step from eh other nd re extended over severl smplesD the e'et of eh input smple is distriuted over timeD on numer of ontiguous smples of the output signlF feing the system liner nd timeEinvrintD the suessive impulse responses sum their e'etsF sn other wordsD the system hs memory of the pst smplesD previously given s input to the systemD nd it uses suh memory to in)uene the presentF o hve physil nlogyD we n think of regulr strokes of snre drumF he response to eh stroke is distriuted in time nd overlps with the responses to the following strokesF gonsider the signl x tht is zero everywhere ut t the instnts 1D 0D nd 1 where it hs vlues 1D 0.5D nd 0.25D respetivelyF et every instnt nD x (n) n e expressed s 1 (n + 1) + 0.5 (n) + 0.25 (n 1)F fy linerityD the output n e otined y omposition of refully trnslted nd weighted impulse responsesX y (n) = 1h (n + 1) + 0.5h (n) + 0.25h (n 1)F
Denition 5.4: Convolution of two signals Example 5.1
o generlize the exmple ixmple SFI we n de(ne the opertion of convolutionF

y (n) = h x (n) =
m=
h and x
he opertion of onvolution n e fully understood y the expliit onstrution of some exmples of onvolution produtF he module hisreteEime gonvolution @ghpter TA gives the grphi onstrution of n exmples nd it o'ers pointers to other exmplesF
5.2.1 Properties
(x (m) h (n m))
he properties of the onvolution opertion re well illustrted in the module roperties of gonvolution2 F he most interesting of suh properties is the extensionX sf x (n) is extended over M1 smplesD nd h (n) is extended over M2 smplesD then the onvolution produt y (n) is extended over M1 + M2 1 smplesF hereforeD the signl convolution product is longer thn oth the input signl nd the impulse responseF enother interesting property is the ommuttivity of the onvolution produtD suh tht the input signl nd the impulse response n hnge their roles without 'eting the output signlF
Property 5.1:
5.3 Frequency response and ltering

he pourier rnsform3 of the impulse response is lled Frequency Response nd it is represented with H ( )F he pourier trnsform of the system output is otined y multiplition of the pourier trnsform of the input with the frequeny responseD iFeFD Y () = H () X ()F he frequeny response shpesD in multiplitive fshionD the inputEsignl spetrum orD in other wordsD it performs some ltering y emphsizing some frequeny omponents nd ttenuting some othersF e (ltering n lso operte on the phses of the spetrl omponentsD y delying them of di'erent mountsF piltering n e performed in the time domain @or spe dominAD y the opertion of onvolutionD or in the frequency domain y multiplition of the frequeny responseF
2 "Properties of Convolution" <http://cnx.org/content/m10088/latest/> 3 "Derivation of the Fourier Transform" <http://cnx.org/content/m0046/latest/>
SW
Exercise 5.1
ke the impulse response tht is zero everywhere ut t the instnts 1D 0D nd 1 where it hs vlues 1D 0.5D nd 0.25D respetivelyF ede(ne the (ltering opertion filtra() of the ound ghooser4 presented in the module wedi epresenttion in roessing @ghpter PAF sn this se (ltering is operted in the time domin y onvolutionF
5.3.1 Causality
he notion of uslity is quite intuitive nd it orresponds to the experiene of stimulting system nd getting k response only t future time instntsF por disreteEtime vs systemsD this hppens when the impulse response is zero for negtive @disreteA time instntsF gusl vs systems n produe with no ppreile dely n output signl sample-by-sampleD euse the onvolution opertor ts only on present nd pst vlues of the input signlF sn ixerise SFI the impulse response is not uslD ut this is not prolem euse the whole input signl is lredy villeD nd the (lter n proess the whole lok of smplesF
5.4 2D Filtering
he notions of impulse responseD onvolutionD frequeny responseD nd (ltering nturlly extend from Ih to PhD thus giving the fundmentl onepts of imge proessingF
Denition 5.5: Convolution of two 2D signals (images)
sf x is the imge tht we re onsideringD it is esy to relize tht onvolution is performed y multiE plition nd trnsltion in spe of convolution mask or kernel h @it is the impulse response of the proessing systemAF es in the Ih se (ltering ould e interpreted s omintion of ontiguous smples @where the extension of suh luster depends on the extension of the (lter impulse responseA tht is repeted in timeD smple y smpleF oD in Ph spe (ltering n e interpreted s omintion of ontiguous smples @pixelsA in lusterD whose extension is given y the onvolution mskF he soElled memory of IEh systems eomes in PEh sort of distance eectF es in the Ih seD the pourier trnsform of the impulse response is lled Frequency response nd it is indited y H (X , Y )F he pourier trnsform of the system output is otined y pourierEtrnsforming the input nd multiplying the result y the frequeny responseF Y (X , Y ) = H (X , Y ) X (X , Y )F
Exercise 5.2
y (m, n) = h x (m, n) =
k=
l=
(x (k, l) h (m k, n l))
gonsider the roessing ode of the lurring exmple5 nd (nd the lines tht implement the onvolution opertionF
4 "Rappresentazione di Media in Processing" <http://cnx.org/content/m12664/latest/#sound_chooser> 5 http://processing.org/learning/topics/blur.html
TH
CHAPTER 5.
SIGNAL PROCESSING IN PROCESSING: CONVOLUTION AND FILTERING

void filtra(float[] DATAF, float[] DATA, float WC, float RO) { //WC and R0 are useless, here kept only to avoid rewriting other //parts of code for(int i = 2; i < DATA.length-1; i++){ DATAF[i] = DATA[i+1] + 0.5*DATA[i] + 0.25*DATA[i-1]; } }
for(int y=0; y<height; y++) { for(int x=0; x<width/2; x++) { float sum = 0; for(int k=-n2; k<=n2; k++) { for(int j=-m2; j<=m2; j++) { // Reflect x-j to not exceed array boundary int xp = x-j; int yp = y-k; //... omissis ... //auxiliary code to deal with image boundaries sum = sum + kernel[j+m2][k+n2] * red(get(xp, yp)); } } output[x][y] = int(sum); } }
Chapter 6
Discrete-Time Convolution
1
6.1 Overview
gonvolution is onept tht extends to ll systems tht re oth linear and time-invariant2 @LTIAF he ide of discrete-time convolution is extly the sme s tht of ontinuousEtime onvolution3 F por this resonD it my e useful to look t oth versions to help your understnding of this extremely importnt oneptF ell tht onvolution is very powerful tool in determining system9s output from knowledge of n ritrry input nd the system9s impulse responseF st will lso e helpful to see onvolution grphilly with your own eyes nd to ply round with it someD so experiment with the pplets4 ville on the internetF hese resoures will o'er di'erent pprohes to this ruil oneptF
6.2 Convolution Sum

es mentioned oveD the onvolution sum provides oniseD mthemtil wy to express the output of n vs system sed on n ritrry disreteEtime input signl nd the system9s responseF he convolution sum is expressed s y [n] = (x [k ] h [n k ]) @TFIA
k=
es with ontinuousEtimeD onvolution is represented y the symol BD nd n e written s @TFPA fy mking simple hnge of vriles into the onvolution sumD k = n kD we n esily show tht onvolution is commutativeX x [n] h [n] = h [n] x [n] @TFQA 5 por more informtion on the hrteristis of onvolutionD red out the roperties of gonvolution F
y [n] = x [n] h [n]
6.3 Derivation
e know tht ny disreteEtime signl n e represented y summtion of sled nd shifted disreteEtime impulsesF ine we re ssuming the system to e liner nd timeEinvrintD it would seem to reson tht
1 This content is available online at <http://cnx.org/content/m10087/2.18/>. 2 "System Classications and Properties" <http://cnx.org/content/m10084/latest/> 3 "Continuous-Time Convolution" <http://cnx.org/content/m10085/latest/> 4 http://www.jhu.edu/signals 5 "Properties of Convolution" <http://cnx.org/content/m10088/latest/>
TI
TP
CHAPTER 6.
DISCRETE-TIME CONVOLUTION
n input signl omprised of the sum of sled nd shifted impulses would give rise to n output omprised of sum of sled nd shifted impulse responsesF his is extly wht ours in convolutionF felow we present more rigorous nd mthemtil look t the derivtionX vetting H e h vs systemD we strt with the following eqution nd work our wy down the onvolution sum3
y [n] = H [x [n]] = H = = =
k= k= k= k=
(x [k ] [n k ])
(H [x [k ] [n k ]]) (x [k ] H [ [n k ]]) (x [k ] h [n k ])
@TFRA
vet us tke quik look t the steps tken in the ove derivtionF efter our initil equtionD we using the h sifting property6 to rewrite the funtionD x [n]D s sum of the funtion times the unit impulseF xextD we n move round the H opertor nd the summtion euse H [] is linerD h systemF feuse of this linerity nd the ft tht x [k] is onstntD we n pull the previous mentioned onstnt out nd simply multiply it y H []F pinllyD we use the ft tht H [] is time invrint in order to reh our (nl stte E the onvolution sum3 e quik grphil exmple my help in demonstrting why onvolution worksF
Figure 6.1:
A single impulse input yields the system's impulse response.
6 "The
Impulse Function": Section The Sifting Property of the Impulse <http://cnx.org/content/m10059/latest/#sifting>
TQ
Figure 6.2:
linearity.
A scaled impulse input yields a scaled response, due to the scaling property of the system's
We now use the time-invariance property of the system to show that a delayed input results in an output of the same shape, only delayed by the same amount as the input.
Figure 6.3:
TR
CHAPTER 6.
We now use the additivity portion of the linearity property of the system to complete the picture. Since any discrete-time signal is just a sum of scaled and shifted discrete-time impulses, we can nd the output from knowing the input and the impulse response.
Figure 6.4:
6.4 Convolution Through Time (A Graphical Approach)

sn this setion we will develop seond grphil interprettion of disreteEtime onvolutionF e will egin this y writing the onvolution sum llowing x to e uslD lengthEm signl nd h to e uslD lengthEkD vs systemF his gives us the (nite summtionD
m1
y [n] =
l=0
(x [l] h [n l])
@TFSA
xotie tht for ny given n we hve sum of the produts of xl nd timeEdelyed hl F his is to sy tht we multiply the terms of x y the terms of timeEreversed h nd dd them upF qoing k to the previous exmpleX
TS
Figure 6.5:
This is the end result that we are looking to nd.
Figure 6.6:
Here we reverse the impulse response, h , and begin its traverse at time 0.
TT
CHAPTER 6.
We continue the traverse. See that at time 1 , we are multiplying two elements of the input signal by two elements of the impulse response.
Figure 6.7:
Figure 6.8
TU
Figure 6.9:
If we follow this through to one more step, n = 4, then we can see that we produce the same output as we saw in the initial example.
ht we re doing in the ove demonstrtion is reversing the impulse response in time nd 4wlking it ross4 the input signlF glerlyD this yields the sme result s slingD shifting nd summing impulse responsesF his pproh of timeEreversingD nd sliding ross is ommon pproh to presenting onvolutionD sine it demonstrtes how onvolution uilds up n output through timeF
TV
CHAPTER 6.
Chapter 7
Signal Processing in Processing: Elementary Filters
1
7.1 FIR lters

he Finite Impulse Response @psA (lters re ll those (lters hrterised y n impulse response with (nite numer of smplesF hey re relized y the opertion of onvolution2 F por eh smple of the onvolution produt weighted sum of (nite numer of input smples is omputedF
7.1.1 Averaging lter
he simplest non trivil ps (lter is the (lter tht omputes the running verge of two ontiguous smplesD nd the orresponding onvolution n e expressed s y (n) = 0.5x (n) + 0.5x (n 1) @UFIA F he impulse response hs vlues 0.5 t instnts 0 nd 1D nd zero nywhere elseF sf we put sinusoidl signl into the (lterD the output will still e sinusoidl signl sled in mplitude nd delyed in phse ording to the frequeny response 3 D whih is
H ( ) = cos (i e 2) 2
@UFPA
nd its mgnitude nd phse re represented in pigure UFI @wgnitude nd phse response for the verging (lterAF
all'impulso e convoluzione <http://cnx.org/content/m12809/latest/#convolution> 3 "Signal Processing in Processing: Convoluzione e Filtraggio" <http://cnx.org/content/m12809/latest/#freqrespp>
1 This content is available online at <http://cnx.org/content/m13047/1.4/>. 2 "Signal Processing in Processing: Convoluzione e Filtraggio": Section Risposta
TW
UH
CHAPTER 7.
SIGNAL PROCESSING IN PROCESSING: ELEMENTARY FILTERS
Magnitude and phase response for the averaging lter
Figure 7.1
he one just presented is (rstEorder @or lenght 2A (lterD euse it uses only one smple of the pst sequene of inputF prom pigure UFI @wgnitude nd phse response for the verging (lterA we see tht the frequeny response is of the low-pass kindD euse the high frequenies re ttenuted s ompred to the low frequeniesF ettenuting high frequenies mens smoothing the rpid signl vritionsF sf one wnts steeper frequeny response from n ps (lterD the order must e inresed orD in other wordsD more smples of the input signl hve to e proessed y the onvolution opertor to give one smple of outputF
7.1.2 Symmetric second-order FIR lter
e symmetri seondEorder ps (lter hs n impulse response whose form is [a0 , a1 , a0 ]D nd the frequeny response turns out to e H () = (a1 + 2a0 cos ()) e(i) F gonvolution n e expressed s y (n) = a0 x (n) + a1 x (n 1) + a0 x (n 2) @UFQA sn the speil se where a0 = 0.17654 nd a1 = 0.64693 the frequeny response @mgnitude nd phseA is represented in pigure UFP @wgnitude nd phse response of seondEorder ps (lter AF
UI
Magnitude and phase response of a second-order FIR lter
Figure 7.2
7.1.3 High-pass lters
qiven the simple lowEpss (lters tht we hve just seenD it is su0ient to hnge sign to oe0ient to otin high-pass kind of responseD iFeF to emphsize high frequenies s ompred to low frequeniesF por exmpleD the pigure UFQ @prequeny response @mgnitudeA of (rstE @leftA nd seondE @rightA order highEpss ps (lterFA displys the mgnitude of the frequeny responses of highEpss ps (lters of the (rst nd seond orderD whose impulse responses reD respetively [0.5, 0.5] nd [0.17654, 0.64693, 0.17654]F
Frequency response (magnitude) of rst- (left) and second- (right) order high-pass FIR lter.
(a)
Figure 7.3
(b)
UP
CHAPTER 7.
o emphsize high frequenies mens to mke rpid vritions of signl more evidentD eing those vriE tions time trnsients in the se of soundsD or ontours in the se of imgesF
7.1.4 FIR lters in 2D
sn PhD the impulse response of n ps (lter is onvolution msk with (nite numer of elementsD iFeF mtrixF sn prtiulrD the averaging lter n e representedD for exmpleD y the onvolution mtrix
1 9
1 1 1
1 1 1
1 F 1
Example 7.1: Noise cleaning
he lowEpss (lters @ndD in prtiulrD the smoothing (ltersA perform some sort of smoothing of the input signlD in the sense tht the resulting signl hs smoother designD where rupt disontinuities re less evidentF his n serve the purpose of reduing the pereptul e'et of noises dded to udio signls or imgesF por exmpleD the ode reported elow @pF UPA lods n imgeD it orrupts with white noiseD nd then it (lters hlf of it with n verging (lterD thus otining pigure UFR @moothingAF
Smoothing
Figure 7.4
// smoothed_glass // smoothing filter, adapted from REAS: // http://processing.org/learning/topics/blur.html size(210, 170); PImage a; // Declare variable "a" of type PImage a = loadImage("vetro.jpg"); // Load the images into the program image(a, 0, 0); // Displays the image from point (0,0) // corrupt the central strip of the image with random noise
UQ
float noiseAmp = 0.2; loadPixels(); for(int i=0; i<height; i++) { for(int j=width/4; j<width*3/4; j++) { int rdm = constrain((int)(noiseAmp*random(-255, 255) + red(pixels[i*width + j])), 0, 255); pixels[i*width + j] = color(rdm, rdm, rdm); } } updatePixels(); int n2 = 3/2; int m2 = 3/2; float val = 1.0/9.0; int[][] output = new int[width][height]; float[][] kernel = { {val, val, val}, {val, val, val}, {val, val, val} }; // Convolve the image for(int y=0; y<height; y++) { for(int x=0; x<width/2; x++) { float sum = 0; for(int k=-n2; k<=n2; k++) { for(int j=-m2; j<=m2; j++) { // Reflect x-j to not exceed array boundary int xp = x-j; int yp = y-k; if (xp < 0) { xp = xp + width; } else if (x-j >= width) { xp = xp - width; } // Reflect y-k to not exceed array boundary if (yp < 0) { yp = yp + height; } else if (yp >= height) { yp = yp - height; } sum = sum + kernel[j+m2][k+n2] * red(get(xp, yp)); } } output[x][y] = int(sum); } } // Display the result of the convolution // by copying new data into the pixel buffer loadPixels(); for(int i=0; i<height; i++) { for(int j=0; j<width/2; j++) {
UR
} } updatePixels();
CHAPTER 7.
pixels[i*width + j] = color(output[j][i], output[j][i], output[j][i]);
por the purpose of smoothingD it is ommon to rete onvolution msk y reding the vlues of qussin ell in two vrilesF e property of gussin funtions is tht their pourier trnsform is itself gussinF hereforeD impulse response nd frequeny response hve the sme shpeF roweverD the trnsform of thin ell is lrge ellD nd vie versF he lrger the ellD the more evident the smoothing e'et will eD with onsequentil loss of detilsF sn visul termsD gussin (lter produes n e'et n similr to tht of oplesent glss superimposed over the imgeF en exmple of qussin ell is
1 4 1 273 7 4 1 4 7 4 1 16 26 16 4 26 41 26 7 4 26 7 F 16 4 4 1 16
gonverselyD if the purpose is to mke the ontours nd slient trts of n imge more evident @edge crispening or shrpeningAD we hve to perform highEpss (lteringF imilrly to wht we sw in etion UFIFQ @righEpss (ltersA this n e done with onvolution mtrix whose entrl vlue hs opposite sign s ompred to surrounding vluesF por instneD the onvolution mtrix e'et of pigure UFS @idge rispeningAF
Edge crispening
1 1 1
1 9
1 1
produes the
Figure 7.5
US
7.1.4.1 Non-linear ltering: median lter
e (lter whose onvolution msk is signlEdependent looses its hrteristis of linerityF wedin (lters use the msk to selet set of pixels of the input imgesD nd reple the entrl pixel of the msk with the medin vlue of the seleted setF qiven set of N @oddA numersD the medin element of the set is 1 1 smller elements from N2 lrger elementsF e typil medin (lter msk is the one tht seprtes N2 rossEshpedF por exmpleD 3 3 msk n overD when pplied to ertin imge pointD the pixels with vlues
x 7 x 4 9 99 12 D x x
thus repling the vlue 99 with the men vlue 9F

ewrite the (ltering opertion filtra() of the ound ghooser @le PFQA presented in the module wedi epresenttion in roessing @ghpter PA in suh wy tht it implements the ps (lter whose frequeny response is represented in pigure UFP @wgnitude nd phse response of seondE order ps (lter AF ht hppens if the (lter is pplied more thn onec
Exercise 7.2
Exercise 7.1
gonsidered the roessing ode of the lurring exmple4 ontined in the roessing exmples5 D modify it so tht it performs qussin (lteringF
Exercise 7.3
wodify the ode of ixmple UFI @xoise leningA so tht the e'ets of the verging (lter @pF UPA
1 10
msk nd the
msk is inresed furtherc henD try to implement the medin (lter with 3 3 rossEshped mskF
1 1 1
1 2 1
1 re ompredF 1
ht hppens if the entrl vlue of the onvolution
7.2 IIR Filters

he (ltering opertion represented y @UFIA is prtiulr se of dierence equationD where smple of output is only funtion of the input smplesF wore generllyD it is possile to onstrut recursive di'erene equtionsD where ny smple of output is funtion of one or more other output smplesF y (n) = 0.5y (n 1) + 0.5x (n) @UFRA llows to ompute @usllyA eh smple of the output y only knowing the output t the previous instnt nd the input t the sme instntF st is esy to relize tht y feeding the system represented y @UFRA with n impulseD we otin the in(niteElength sequene y = [0.5, 0.25, 0.125, 0.0625, ...]F por this purposeD (lters of this kind re lled Innite Impulse Response @ssA (ltersF he order of n ss (lter is equl to the numer of pst output smples tht it hs to store for proessingD s ditted y the di'erene equtionF hereforeD the (lter of @UFRA is (rstEorder (lterF por given (lter orderD ss (lters llow frequeny responses tht re steeper thn those of ps (ltersD ut phse distortions re lwys introdued y ss (ltersF sn other wordsD the di'erent spetrl omponents re delyed y di'erent time mountsF por exmpleD pigure UFT @wgnitude nd phse response of the ss (rstEorder (lterA shows the mgnitude nd phse responses for the (rstEorder ss (lter represented y the di'erene eqution @UFRAF glled a the oe0ient tht weights the dependene on the output previous vlue @0.5 in the spei( @UFRAAD the impulse response tkes the form h (n) = an F he more a is loser to 1D the more sustined is the impulse response in timeD nd the frequeny response inreses its steepnessD thus eoming emphsizing its lowEpss hrterF yviouslyD vlues of a lrger thn 1 gives divergent impulse response ndD thereforeD n unstable ehvior of the (lterF
4 http://processing.org/learning/topics/blur.html 5 http://processing.org/learning/topics/
UT
CHAPTER 7.
Magnitude and phase response of the IIR rst-order lter
Figure 7.6
ss (lters re widely used for oneEdimensionl signlsD like udio signlsD espeilly for relEtime smpleE yEsmple proessingF ie versD it doesn9t mke muh sense to extend reursive proessing onto two dimensionsF hereforeD in imge proessing ps (lters re mostly usedF
7.2.1 Resonant lter
sn the udio (eldD seondEorder ss (lters re prtiulrly importntD euse they relize n elementry resontorF qiven the di'erene eqution y (n) = a1 y (n 1) + a2 y (n 2) + b0 x (n) @UFSA one n verify tht it produes the frequeny response of pigure UFU @wgnitude nd phse response of the seondEorder ss (lter AF he oe0ients tht gives dependene on the pst n e expressed s a1 = 2rcos (0 ) nd a2 = r2 D where 0 is the frequeny of the resonne pek nd r gives peks tht gets nrrower when pprohing 1F
UU
Magnitude and phase response of the second-order IIR lter
Figure 7.7
erify tht the (ltering opertion filtra() of the ound ghooser @le PFQA presented in module wedi epresenttion in roessing @ghpter PA implements n ss resonnt (lterF ht is the reltion etween r nd the mouse position long the smll rsc
Exercise 7.4
UV
CHAPTER 7.

//filtra = new function void filtra(float[] DATAF, float[] DATA, float a0, float a1) { for(int i = 3; i < DATA.length; i++){ DATAF[i] = a0*DATA[i]+a1*DATA[i-1]+a0*DATA[i-2];//Symmetric FIR filter of the second order }
fy writing for loop tht repets the (ltering opertion ertin numer of timesD one n verify tht the e'et of (ltering is emphsizedF his intuitive result is due to the ft thtD s fr s the signl is onernedD going through m (lters of order N @in our se N = 2 A is equivlent to going through single (lter of order mN
// smoothing Gaussian filter, adapted from REAS: // http://processing.org/learning/topics/blur.html size(200, 200); PImage a; // Declare variable "a" of type PImage a = loadImage("vetro.jpg"); // Load the images into the program image(a, 0, 0); // Displays the image from point (0,0) int n2 = 5/2; int m2 = 5/2; int[][] output = new int[width][height]; float[][] kernel = { {1, 4, 7, 4, 1}, {4, 16, 26, 16, 4}, {7, 26, 41, 26, 7}, {4, 16, 26, 16, 4}, {1, 4, 7, 4, 1} }; for (int i=0; i<5; i++) for (int j=0; j< 5; j++) kernel[i][j] = kernel[i][j]/273; // Convolve the image for(int y=0; y<height; y++) { for(int x=0; x<width/2; x++) { float sum = 0; for(int k=-n2; k<=n2; k++) { for(int j=-m2; j<=m2; j++) { // Reflect x-j to not exceed array boundary int xp = x-j;
UW
int yp = y-k; if (xp < 0) { xp = xp + width; } else if (x-j >= width) { xp = xp - width; } // Reflect y-k to not exceed array boundary if (yp < 0) { yp = yp + height; } else if (yp >= height) { yp = yp - height; } sum = sum + kernel[j+m2][k+n2] * red(get(xp, yp));
} } output[x][y] = int(sum);
// Display the result of the convolution // by copying new data into the pixel buffer loadPixels(); for(int i=0; i<height; i++) { for(int j=0; j<width/2; j++) { pixels[i*width + j] = color(output[j][i], output[j][i], output[j][i]); } } updatePixels();
wedin (lterX
// smoothed_glass // smoothing filter, adapted from REAS: // http://www.processing.org/learning/examples/blur.html size(210, 170); PImage a; // Declare variable "a" of type PImage a = loadImage("vetro.jpg"); // Load the images into the program image(a, 0, 0); // Displays the image from point (0,0) // corrupt the central strip of the image with random noise float noiseAmp = 0.1; loadPixels(); for(int i=0; i<height; i++) { for(int j=width/4; j<width*3/4; j++) { int rdm = constrain((int)(noiseAmp*random(-255, 255) + red(pixels[i*width + j])), 0, 255); pixels[i*width + j] = color(rdm, rdm, rdm);
VH
} } updatePixels();
CHAPTER 7.
int[][] output = new int[width][height]; int[] sortedValues = {0, 0, 0, 0, 0}; int grayVal; // Convolve the image for(int y=0; y<height; y++) { for(int x=0; x<width/2; x++) { int indSort = 0; for(int k=-1; k<=1; k++) { for(int j=-1; j<=1; j++) { // Reflect x-j to not exceed array boundary int xp = x-j; int yp = y-k; if (xp < 0) { xp = xp + width; } else if (x-j >= width) { xp = xp - width; } // Reflect y-k to not exceed array boundary if (yp < 0) { yp = yp + height; } else if (yp >= height) { yp = yp - height; } if ((((k != j) && (k != (-j))) ) || (k == 0)) { //cross selection grayVal = (int)red(get(xp, yp)); indSort = 0; while (grayVal < sortedValues[indSort]) {indSort++; } for (int i=4; i>indSort; i--) sortedValues[i] = sortedValues[i-1]; sortedValues[indSort] = grayVal; } } } output[x][y] = int(sortedValues[2]); for (int i=0; i< 5; i++) sortedValues[i] = 0; } } // Display the result of the convolution // by copying new data into the pixel buffer loadPixels(); for(int i=0; i<height; i++) { for(int j=0; j<width/2; j++) { pixels[i*width + j] = color(output[j][i], output[j][i], output[j][i]); } }
VI
updatePixels();
VP
CHAPTER 7.
Chapter 8
Textures in Processing
1
8.1 Color Interpolation

es seen in qrphi gomposition in roessing2 D one n otin surfes s olletions of polygons y mens of the de(nition of vertex within the ouple beginShape() E endShape()F st is possile to ssign olor to one or more vertiesD in order to mke the olor vritions ontinuous @gradientAF por exmpleD you n try to run the ode
size(200,200,P3D); beginShape(TRIANGLE_STRIP); fill(240, 0, 0); vertex(20,31, 33); fill(240, 150, 0); vertex(80, 40, 38); fill(250, 250, 0); vertex(75, 88, 50); vertex(49, 85, 74); endShape();
in order to otin ontinuous nune from red to yellow in the strip of two tringlesF
8.1.1 Bilinear Interpolation
he grphil system performs n interpoltion of olor vlues ssigned to the vertiesF his type of bilinear interpolation is de(ned in the following wyX por eh polygon of the olletion por eh side of the polygon one ssigns to eh point on the segment the olor otined y mens of liner interpoltion of the olors of the verties i e j tht de(ne the polygonX Cij () = e scan line sns the polygon @orD etterD its projetion on the imge windowA interseting t eh step two sides in two points l ed r whose olors hve lredy een identi(ed s Cl e Cr F sn eh point of the sn line the olor is determined y liner interpoltion Clr ( ) = (1 ) Cl + Cr e signi(tive exmple of interpoltion of olors ssoited to the verties of ue n e found in exmples of roessing3 D in the ode qf gue4 F

1 This content is available online at <http://cnx.org/content/m13048/1.3/>. 2 "Composizione Graca in Processing" <http://cnx.org/content/m12665/latest/> 3 http://processing.org/learning/3d/ 4 http://processing.org/learning/3d/rgbcube.html
(1 ) Ci + Cj
VQ
VR
CHAPTER 8.
TEXTURES IN PROCESSING
8.2 Texture
hen modeling omplex sene y mens of omposition of simple grphil elements one nnot go eyond ertin threshold of omplexityF vet us think out the exmple of modeliztion of nturl seneD where one hs to represent eh single vegetl elementD inluding the grss of medowF st is unoneivle to do this mnullyF st would e possile to set nd ontrol the grss elements y mens of some lgorithmsF his is n pproh tkenD for exmpleD in rendering the hir nd skin of hrters of the most sophistited nimtion movies @see for exmpleD the snrediles5 AF ytherwiseD espeilly in se of intertive grphisD one hs to resort to using texturesF sn other wordsD one employs imges tht represent the visul texture of the surfes nd mp them on the polygons tht model the ojets of the seneF sn order to hve qulittive rendering of the surfes it is neessry to limit the detil level to frgments not smller thn one pixel ndD thusD the texture mapping is inserted in the rendering hin t the rastering level of the grphi primitivesD iFeF where one psses from Qh geometri desription to the illumintion of the pixels on the displyF st is t this level tht the removl of the hidden surfes tkes pleD sine we re interested only in the visile frgmentsF sn roessingD texture is de(ned within lok beginShape() E endShape() y mens of the funtion texture() tht hs s unique prmeter vrile of type PImageF he following lls to vertex() n ontinD s lst ouple of prmetersD the point of the texture orresponding to the vertexF sn ftD eh texture imge is prmeterized y mens of two vriles u nd vD tht n e referred diretly to the line nd olumn of texel @pixel of textureA orD lterntivelyD normlized etween 0 nd 1D in suh wy tht one n ignore the dimension s well s the width nd height of the texture itselfF he mening of the prmeters u nd v is estlished y the ommnd textureMode() with prmeter IMAGE or NORMALIZEDF sn the ode tht follows the imge representing roken glss 6 is employed s texture nd followed y olor interpoltion nd the defult illumintionF he shading of the surfesD produed y mens of the illumintion nd the olorsD is modulted in multiplitive wy y the olors of the textureF
size(400,400,P3D); PImage a = loadImage("vetro.jpg"); lights(); textureMode(NORMALIZED); beginShape(TRIANGLE_STRIP); texture(a); fill(240, 0, 0); vertex(40,61, 63, 0, 0); fill(240, 150, 0); vertex(340, 80, 76, 0, 1); fill(250, 250, 0); vertex(150, 176, 100, 1, 1); vertex(110, 170, 180, 1, 0); endShape();
Example 8.1
8.3 Texture mapping

st is evident tht the mpping opertions from texture imge to n ojet surfeD of ritrry shpeD implies some form of interpoltionF imilrly to wht hppens for olorsD only the verties tht delimit the surfe re mpped onto ext points of the texture imgeF ht hppens for the internl points hs to e estlished in some wyF etullyD roessing nd ypenqv ehve ording to wht illustrted in
5 http://www.computerarts.co.uk/in_depth/features/inside_the_incredibles 6 See the le at <http://cnx.org/content/m13048/latest/.>
VS etion VFIFI @filiner snterpoltion AD iFeF y iliner interpoltionX (rst liner interpoltion over eh oundry segment is sded y liner interpoltion on sn lineF sf u nd v exeed the limits of the texture imgeD the system @roessingA n ssume tht this is repeted periodilly nd (x it to the vlues t the orderF e prolem tht ours is tht pixel on disply does not neessrly orrespond extly to texelF yne n mp more thn one texel on pixel orD vieversD texel n e mpped on severl pixelsF he (rst se orresponds to downsmpling thtD s seen in mpling nd untiztion7 D n produe lisingF he e'et of lising n e ttenuted y mens of low pss (ltering of the texture imgeF he seond se orresponds to upsmplingD tht in the frequeny domin n e interpreted s inresing the distne etween spetrl imgesF
8.4 Texture Generation

extures re not neessrely imported from imgesD ut they n lso e generted in n lgorithmi fshionF his is prtiulrly reommended when one wnts to generte regulr or pseudoErndom ptternsF por exmpleD the pttern of hessEord n e generted y mens of the ode
PImage textureImg = loadImage("vetro.jpg"); // dummy image colorMode(RGB,1); int int int for biro = 0; bbiro = 0; scacco = 5; (int i=0; i<textureImg.width; i+=scacco) { bbiro = (bbiro + 1)%2; biro = bbiro; for (int j=0; j<textureImg.height; j+=scacco) { for (int r=0; r<scacco; r++) for (int s=0; s<scacco; s++) textureImg.set(i+r,j+s, color(biro)); biro = (biro + 1)%2;
} } image(textureImg, 0, 0);
he use of the funtion rndomD omined with (lters of vrious typeD llows wide )exiility in the prodution of texturesF por exmpleD the pttern represented in pigure VFI @elgorithmillyEgenerted ptE ternA ws otined from modi(tion of the ode generting the hessEordF sn prtiulrD we dded the line scacco=floor(2+random(5)); within the outer forD nd pplied n verging (lterF
7 "Signal
Processing in Processing: Campionamento e Quantizzazione" <http://cnx.org/content/m12751/latest/>
VT
CHAPTER 8.
Algorithmically-generated pattern
Figure 8.1
row ould one modify the ode ixmple VFI in order to mke the reks in the glss more evidentc
he exerise onsists in modifying the ode of the genertor of the hessEord in etion VFR @exture qenertionA in order to generte the texture pigure VFI @elgorithmillyEgenerted ptE ternAF
Exercise 8.3
his exerise onsists in running nd nlyzing the following odeF ry then to vry the dimensions of the smll squres nd the (ltering typeF
size(200, 100, P3D); PImage textureImg = loadImage("vetro.jpg"); // dummy image colorMode(RGB,1); int int int for biro = 0; bbiro = 0; scacco = 5; (int i=0; i<textureImg.width; i+=scacco) { // scacco=floor(2+random(5)); bbiro = (bbiro + 1)%2; biro = bbiro; for (int j=0; j<textureImg.height; j+=scacco) { for (int r=0; r<scacco; r++) for (int s=0; s<scacco; s++) textureImg.set(i+r,j+s, color(biro)); biro = (biro + 1)%2;
} } image(textureImg, 0, 0); textureMode(NORMALIZED); beginShape(QUADS); texture(textureImg); vertex(20, 20, 0, 0);
VU
vertex(80, 25, 0, 0.5); vertex(90, 90, 0.5, 0.5); vertex(20, 80, 0.5, 0); endShape(); // ------ filtering ------PImage tImg = loadImage("vetro.jpg"); // dummy image float val = 1.0/9.0; float[][] kernel = { {val, val, val}, {val, val, val}, {val, val, val} }; int n2 = 1; int m2 = 1; colorMode(RGB,255); // Convolve the image for(int y=0; y<textureImg.height; y++) { for(int x=0; x<textureImg.width/2; x++) { float sum = 0; for(int k=-n2; k<=n2; k++) { for(int j=-m2; j<=m2; j++) { // Reflect x-j to not exceed array boundary int xp = x-j; int yp = y-k; if (xp < 0) { xp = xp + textureImg.width; } else if (x-j >= textureImg.width) { xp = xp - textureImg.width; } // Reflect y-k to not exceed array boundary if (yp < 0) { yp = yp + textureImg.height; } else if (yp >= textureImg.height) { yp = yp - textureImg.height; } sum = sum + kernel[j+m2][k+n2] * red(textureImg.get(xp, yp)); } } tImg.set(x,y, color(int(sum))); } } translate(100, 0); beginShape(QUADS); texture(tImg); vertex(20, 20, 0, 0); vertex(80, 25, 0, 0.5); vertex(90, 90, 0.5, 0.5); vertex(20, 80, 0.5, 0); endShape();
VV
CHAPTER 8.
VW

st is su0ient to onsider only piee of the textureD with lls of the type vertex(150,
176, 0.3, 0.3);
WH
CHAPTER 8.
Chapter 9
Signal Processing in Processing: Miscellanea
1
9.1 Economic Color Representations

sn wedi epresenttion in roessing2 we sw how one devotes 8 its to eh hnnel orresponding to primry olorF sf we dd to these the lph hnnelD the totl numer of its per pixel eomes 32F e do not lwys hve the possiility to use suh ig mount of memory for olorsF hereforeD one hs to dopt vrious strtegies in order to redue the numer of its per pixelF
9.1.1 Palette
e (rst solution omes from the oservtion tht usully in n imgeD not ll of the 224 representle olors re present t the sme timeF upposing tht the numer of olors neessry for n ordinry imge is not greter thn 256D one n think out memorizing the odes of the olors in tle @paletteAD whose elements re essile y mens of n index of only 8 itsF husD the imge will require memory spe of 8 its per pixel plus the spe neessry for the pletteF por exmples nd further explntions see olor depth3 in ikipediF
9.1.2 Dithering
elterntivelyD in order to hve low numer of its per pixelD one n pply digitl proessing tehnique lled ditheringF he ide is tht of otining mixture of olors in pereptul wyD exploiting the proximity of smll points of di'erent olorF en exhustive presenttion of the phenomenon n e found t the voie dithering4 of ikipediF
9.1.3 Floyd-Steinberg's Dithering
he ploydEteinerg9s lgorithm is one of the most populr tehniques for the distriution of olored pixels in order to otin dithering e'etF he ide is to minimize the visul rtifts y proessing the errorE di'usionF he lgorithm n e resumed s followsX hile proeeding top down nd from left to right for eh onsidered pixelD lulte the di'erene etween the gol olor nd the losest representle olor @errorA
1 This content is available online at <http://cnx.org/content/m13085/1.5/>. 2 "Rappresentazione di Media in Processing" <http://cnx.org/content/m12664/latest/> 3 http://en.wikipedia.org/wiki/Color_depth 4 http://en.wikipedia.org/wiki/Dithering
WI
WP
CHAPTER 9.
SIGNAL PROCESSING IN PROCESSING: MISCELLANEA
spred the error on the ontiguous pixels ording to the msk

7 16
0 X 5
1 16
of the error to the pixel on the right of the onsidered oneD dd of the error to the pixel ottom left with respet to the onsidered oneD nd so onF fy mens of this lgorithm it is possile to reprodue n imge with di'erent gry levels y mens of devie le to de(ne only white nd lk pointsF he msk of the ploydEteinerg9s lgorithm ws hosen 1 produes hessord lyout ptternF in wy tht uniform distriution of gry intensities 2
3 16
0 3
7 F 1
ht isD dd
Exercise 9.1
fy mens of roessingD implement progrm to proess the (le len5 D 4dithered4 lk nd white version of the fmous ven6 imgeF he imgeD treted only in the left hlfD should result similr to tht of pigure WFI
5 See the le at <http://cnx.org/content/m13085/latest/lena.jpg> 6 http://en.wikipedia.org/wiki/Lenna
WQ
Figure 9.1
9.2 Economic Sound Representations

sn se of udio signlsD the use of dithering ims t reduing the pereptul e'et of the error produed y the hnges in the quntiztion resolutionD tht one typilly performs when reording nd proessing udio signlsF por exmpleD when reoding musiD more thn ITEits of quntiztion re usully employedF purthermoreD mthemtil opertions pplied to the signl @sD for instneD simple dynmil vritionsA require n inresing of the it depth tht is of the numer of itsF es soon s one rehes the (nl produtD the udio ghD the numer of quntiztion its hs to e redued to ITF sn eh of these onseutives proesses of reEquntiztion one introdues n errorD tht dds upF sn se of redution of the numer
WR
CHAPTER 9.
of itsD it is possile to trunte the vlues @tht is the digits fter the deiml point re negleted nd set to zeroA or round them @tht is the deiml numer is pproximted to the losest integerAF sn oth ses one introdues n errorF sn prtiulrD when one onsiders signls with well de(ned pitch @s in the se of musil signlsAD the error eomes periodiElikeF prom the exmple of the voie dithering7 of ikipediD the reson of this dditionl pseudoEperiodi noise @iFeF hrmoniElikeA eomes lerF his distortion orresponds to uzzElike sound tht 4follows4 the pith of the quntized soundF he whole result is quite nnoying from pereptul point of viewF sn se of udio signlsD thusD dithering hs the funtion of trnsforming this uzzElike sound in kE ground noise similr to rustlingD less nnoying from listening point of viewF sn pigure WFP n exmple of some periods of the wveform of lrinet quntized with IT its is reportedF he result of redution of the numer of its to V is represented in pigure WFQF st is lerly visile how the redution of the quntiztion levels produes series of lines with onstnt mplitudeF he pplition of dithering genertes further trnsformtion thtD how one n see in pigure WFR 4reks4 the onstnt lines y mens of the introdution of white noiseF he pigure WFSD pigure WFT nd pigure WFU represent the pourier trnsforms of the sounds of pigure WFPD pigure WFQ nd pigure WFRD respetivelyF sn the frequey representtionD it is lso visile how the hnge to quntiztion t V its introdues improper hrmonis @pigure WFTA not present in the sound t IT it @pigure WFSAF hese hrmonis re neled y the e'et of dithering @pigure WFUAF
Figure 9.2
7 http://en.wikipedia.org/wiki/Dithering
WS
Figure 9.3
WT
CHAPTER 9.
Figure 9.4
WU
Figure 9.5
WV
CHAPTER 9.
Figure 9.6
WW
Figure 9.7
here exist lso methods tht exploit pereptul ftors s the ft tht our er is more sensitive in the entrl region of the udio nd nd less sensitive in the higher regionF his llows one to mke the e'et of reEquntiztion less udileF his is the se of the noise shaping tehniquesF his method onsists in 4modeling4 the quntiztion noiseF pigure WFV represents the sound of lrinet reEquntized with V itsF e dithering nd noiseEshping were pplied to the soundF he resultD pprently destrutive from the point of view of the wveformD orresponds to soundD whose spetrum is loser to tht of the originl sound t IT itsD exepted for onsiderle inresing of energy in the very high frequeny region @pigure WFWAF his high frequeny noise produes this 4ru1ed4 wveformD iFeF high energy fst mplitude vritionsF his noise is nywy not udileF husD t listening testD the (nl result is etter thn the previous oneF
IHH
CHAPTER 9.
Figure 9.8
IHI
Figure 9.9
st is possile to think out the noise shping s the udio ounterprt of the ploydEteinerg9s lgorithm for grphisF sn the udio se the error propgtion ours in the timeEdomin insted of the speEdominF he most simple version of noise shping n e otined y mens of the de(nition of the quntiztion error e (n) = y (n) Q (y (n)) @WFIA where y (n) = x (n) + e (n 1) @WFPA nd x is the nonEquntized signlF purther detils out noise shping n e found t ikipediD noise shping8 F ht presented ove n e tested in the sound exmples lrinet9 D lrinet t V its10 D lrinet t V its with dithering11 nd lrinet t V it with noise shping12 tht ontin the lrinet sounds represented in pigure WFPD pigure WFQD pigure WFRD e pigure WFVD respetivelyF
8 http://en.wikipedia.org/wiki/Noise_shaping 9 See the le at <http://cnx.org/content/m13085/latest/./clarinetto.wav> 10 See the le at <http://cnx.org/content/m13085/latest/./clarinetto8.wav> 11 See the le at <http://cnx.org/content/m13085/latest/./clarinetto8dith.wav> 12 See the le at <http://cnx.org/content/m13085/latest/./clarinetto8dithNs.wav>
IHP
CHAPTER 9.
9.3 Histogram-Based Processing.

Denition 9.1: Image Histograms
qrphi representtion y mens of vertil rsD where eh r represents the numer of pixels present in the imge for given intensity of the gry sle @or olor hnnelAF ikipedi de(nitionF13 emong the exmples in roessing14 D one (nds the ode ristogrm15 tht overlp n imge to its own histogrmF he histogrm o'ers syntheti representtion of imgesD in whih one looses the informtion onerning the pixel positions nd onsiders only the hromti spetsF his provides informtion out the Tonal Gamma of n imge @gry intensity tht re presentA nd out the Dynamics @extension of the onl qmmAF he imge of hess ordD for exmpleD hs onl qmm tht inludes only two gry levels @lk nd whiteA ut it hs mximl dynmis @sine white nd lk re the two extremity of the representle gry levelsAF he histogrm is the strting point for vrious proessing e'ets iming t lning or ltering the hromti ontents of n imgeF sn generlD the question is uilding mp go = f (gi ) for the gry levels @or olorEhnnel levelsA tht n e pplied to eh pixelF he histogrm n drive the de(nition of this mpF
9.3.1 Translation and Expansion of an Histogram
sf the mp is of the kind go = gi + k the histogrm is trnslted in the sense of higher or lower rightnessD ording to the sign of kF yn the other sideD if the mp is of the kind go = kgi the histogrm will e expnded or ompressedD for vlues of k smller or greter thn 1D respetivelyF he contrast stretching is one of the opertions of this kind of liner sling tht tries to extend the dynmi rnge of n imgeF he intervl y mens of whih one performs the sling is set on the sis of the histogrmD negletingD for exmpleD the tils of the distriution orresponding to 10% of the drkest nd rightest pixelsF
9.3.2 Non Linear Scaling
wore in generlD the mp go = f (gi ) n e non linerD nd this llows greter )exiility in the mnipultion of the histogrmF e useful instrument is the one tht llows to mnipulte intertively the sling mp nd to see the results on the imge ndGor on the histogrm in rel timeF he instrument Color Tools/Curves of the imge proessing softwre qimp16 does thisD using n interpolting splineF sn roessing it is possile to uild similr instrumentD s reported in ixmple WFIF
Example 9.1
epplet tht llows to pply non liner sling to the gry levels nd to nlyze the e'et y mens of n histogrmF 17
9.3.3 Equalization of an Histogram
he non liner sling is the tool to equalize the histogrmD tht is to shpe it in desirle wyF en imge hs lned tonl gmmD if ll of the gry levels re represented nd if the distriution is pproximtely uniformF sn other wordsD one ims t )t histogrmF ithout entering too muh into the mthemtil detilsD one n sy tht the non liner mp to e used for the equliztion is otined from the cumulated
13 http://en.wikipedia.org/wiki/Color_histogram 14 http://www.processing.org/learning/topics/ 15 http://www.processing.org/learning/topics/histogram.html 16 http://www.gimp.org 17 See the le at <http://cnx.org/content/m13085/latest/histogram_t.html>
IHQ
distribution
of the histogrm of the imge f (gi ) = g k=0 (h (k ))D where h (k ) is the frequenyD properly sled y mens of normliztion onstntD with whih the kEth gry level ppers F
i
Exercise 9.2
wodify the roessing ode of ixmple WFI in order to dd the opertion of equliztion of the histogrmF
9.4 Segmentation and Contour Extraction

9.4.1 Contours
he ojets tht populte sene represented in n imge re usully detetle y mens of their ontoursX hredElike pro(les tht orrespond to fst vrition of olor or of gry intensityF he ontour extrtion is one of the typil opertion of the imge proessingF prom the point of view of the wthemtil enlysisD fst vrition of funtion orresponds to high vlue of the derivative funtionF sn the frme of higitl ignl roessing @hAD the derivtive n e pproximted y mens of di'erene opertionD iFeF y mens of (lterF he (lters tht let fst vritions through nd eliminte slow vritions re of highEpss typeF st is not surprisingD thusD tht for ontour extrtionD one uses onvolution msks similr to tht seen in ilementry pilters18 employed for edge rispeningF wore in detilD one n sy tht in Ph one looks for points with mximum mplitude of the gradientD iFeF pointsD where the Laplacian of the imge tht is its seond sptil derivtive is neessrily zeroF he vplin n e pproximted @in the disrete speA y sine the result is too sensitive to noise nd to smll detilsF st isD thusD useful to omine the vplin msk with tht of lowEpss (lterF he omintion of vplin nd qussin @LoGA (lter produesD in the se 5 y5D the msk
0 0 1 0 0 0 1 16 2 1 0 0 1 2 1 0 2 1 2 1 0 0 1 0 0 0 mens of onvolution msk 1 0 1 4 1 1 F 0 0
he diret pplition of the vplin is often not stisfyingD
sn the softwre qimpD ontour enhner is
villeD sed on the di'erene etween two qussin (ltersD orresponding to two qussin urves with di'erent mplitudeF his produes n inhiition e'et outside the prinipl ontoursD in wy similr to wht hppens in our pereptul systemF
9.4.2 Regions
sn mny pplitions it is neessry to isolte the vrious ojets tht populte seneD strting from their representtionD s pixel olletions of n imgeF por exmpleD it ould e interesting to isolte n ojet in foreground from the backgroundF sn this seD one tlks out segmenttion or extrtion of regionsF he most simple wy for isolting di'erent regions is tht of doing it on olor sisD or on gryEintensity sisF elso in this seD the opertion n e driven y the histogrm tht n e helpful in order to estlish gry thresholdF ell the drker pixels will e mpped to lkD while ll the lighter ones will e mpped to whiteF por exmpleD if the histogrm presents two evident mximD it is possile to ssign the region of one mximum to the foreground @sine it isD for exmpleD lighterA nd the region orresponding to the other mximum to the kground @sine it isD for exmpleD drkerAF he threshold will e hosen in etween the two mximF ometimes it is neessry to estlish multipliity of thresholdsD in order to isolte di'erent
18 "Signal
Processing in Processing: Filtri Elementari" <http://cnx.org/content/m12827/latest/#crispp>
IHR
CHAPTER 9.
regions y mens of di'erent gry levelsF por olor imgesD the thresholds n e di'erent for di'erent qf hnnelsF
epply the vplin (lter nd the voq (lter to the imge of venF
how tht the extrtion of kground (gure y mens of threshold n e implemented y mens of non liner mp go = f (gi )F ht should e the form of this mpc
Exercise 9.5
imploy progrm for imge proessing @exF qimp19 A in order to isolte @y mens of thresholdingA the reks in the imge of the roken glss20 F
9.5 Audio Dynamic Compression

por udio signlsD similrly to wht seen for imgesD one onsiders the prolem of reduing the dt neessry to represent soundD while preserving n eptle qulity of the signl from pereptul point of viewF st is not esy to de(neD wht is ment y 4eptle qulity4D when one perform dt redution orD etterD dt ompressionF sn generl the qulittive evlution prmeters of the udio ompression stndrds re sttistilD sed on the results of listening tests mde on groups of listenersD representing wide gmm of usersF he udio ompression stndrds re usully founded on the optimiztion of the dynmis of the signlD tht is on the optimiztion of the numer of its employed for the quntiztionF e well known exmple of ompression stndrd is tht of mpQD in whih one exploits psyhoousti phenomenD s the ft tht louder sounds msk @mke inudileA softer soundsF sn the reprodution of digitlized soundsD the thing tht one wnts to msk is the quntiztion noiseF sn other wordsD if the sound hs wide dynmis @it is loudA one n dopt greter quntiztion stepD sine the louder quntiztion noise produed y the rougher sudivision of the quntiztion levels is nywy msked y the reprodued soundF till simplifying things in drsti wyD one ould sy tht mpQ vries the step of quntiztion ording to the dynmis of the sound nd in di'erent wy in di'erent nds of frequenyF sn other wordsD the signl is divided into mny frequeny nds @in similr wy s the iqulizer of riE( system doesA nd eh nd is quntized seprtelyF his llows redution of even PH times of the numer of its with respet to (xed IT it dynmisF enother ompression tehnique is provided y the muElw @ElwAF his stndrd is used minly in udio systems for digitl ommunition in xorth emeri nd tpnF sn this seD the min ide is to modify the dynmi rnge of the nlogil udio signl efore the quntiztionF fehind these ompression tehniquesD there is one more psyhoousti phenomenon tht is the ft tht our pereption of the intensity is not liner ut logrithmiElikeF sn other wordsD our pereption ehves pproximtely ording to wht shown in pigure WFIHF
19 http://www.gimp.org 20 See the le at <http://cnx.org/content/m12837/latest/vetro.jpg>
IHS
Figure 9.10
ht the muElw tully performs is redution of the dynmi rnge of the signl y mens of n mplitude reEsling ording to the mp desried in pigure WFIIF st is visile how the e'et is tht of mplifying the smll mplitudesD reduing the rnge of mplitude vlues of the signl @in the sense of ig mplitudesA ndD s onsequeneD inresing the reltionship @the mplitude di'ereneA etween the sound nd the quntiztion noiseF efterwrdsD liner quntiztion of the non linerly distorted signl is performedF es one wnts to ply k the digitl signlD this is (rst onverted into n nlogil signl nd then trnsformed y mens of urve performing n inverse mplitude distortion with respet to tht of pigure WFIIF he glol result is equivlent to non liner quntiztion of soundD where the quntiztion step is igger @rougherA for igger mplitudes nd smller @more detiledA for smller mplitudesF his orrespondsD t lest from qulittive point of viewD to the wy of funtioning of our pereptul systemF e re more sensitive to the intensity di'erenes in se of soft sounds nd less sensitive in se of loud nd very loud soundsF he eElwD dopted in the digitl systems in iuropeD is very similr to the muElwF
IHT
CHAPTER 9.
Figure 9.11
IHU

size(300, 420); PImage a; // Declare variable "a" of type PImage a = loadImage("lena.jpg"); // Load the images into the program image(a, 0, 0); // Displays the image from point (0,0) int[][] output = new int[width][height]; for (int i=0; i<width; i++) for (int j=0; j<height; j++) { output[i][j] = (int)red(get(i, j)); } int grayVal; float errore; float val = 1.0/16.0; float[][] kernel = { {0, 0, 0}, {0, -1, 7*val}, {3*val, 5*val, val }}; for(int y=0; y<height; y++) { for(int x=0; x<width; x++) { grayVal = output[x][y];// (int)red(get(x, y)); if (grayVal<128) errore=grayVal; else errore=grayVal-256; for(int k=-1; k<=1; k++) { for(int j=-1; j<=0 /*1*/; j++) { // Reflect x-j to not exceed array boundary int xp = x-j; int yp = y-k; if (xp < 0) { xp = xp + width; } else if (x-j >= width) { xp = xp - width; } // Reflect y-k to not exceed array boundary if (yp < 0) { yp = yp + height; } else if (yp >= height) { yp = yp - height; } output[xp][yp] = (int)(output[xp][yp] + errore * kernel[-j+1][-k+1]); } } } } for(int i=0; i<height; i++)
IHV
CHAPTER 9.
for(int j=0; j<width; j++) if (output[j][i] < 128) output[j][i] = 0; else output[j][i] = 255; // Display the result of dithering on half image loadPixels(); for(int i=0; i<height; i++) { for(int j=0; j<width/2; j++) { pixels[i*width + j] = color(output[j][i], output[j][i], output[j][i]); } } updatePixels();
int grayValues = 256; int[] hist = new int[grayValues]; int[] histc = new int[grayValues]; PImage a; void setup() { background(255); stroke(0,0,0); size(300, 420); colorMode(RGB, width); framerate(5); a = loadImage("lena.jpg"); image(a, 0, 0); } void draw() { // calculate the histogram for (int i=0; i<width; i++) { for (int j=0; j<height; j++) { int c = constrain(int(red(get(i,j))), 0, grayValues-1); hist[c]++; } } // Find the largest value in the histogram float maxval = 0; for (int i=0; i<grayValues; i++) { if(hist[i] > maxval) { maxval = hist[i]; } } // Accumulate the histogram
IHW
histc[0] = hist[0]; for (int i=1; i<grayValues; i++) { histc[i] = histc[i-1] + hist[i]; } // Normalize the histogram to values between 0 and "height" for (int i=0; i<grayValues; i++) { hist[i] = int(hist[i]/maxval * height); } if (mousePressed == true) { //equalization for (int i=1; i<grayValues; i++) { println(float(histc[i])/histc[grayValues-1]*256); } loadPixels(); println("click"); for (int i=0; i<width; i++) for (int j=0; j<height; j++) { //normalized cumulated histogram mapping pixels[i+j*width] = color( int( float(histc[constrain(int(red(a.get(i,j))), 0, grayValues-1)])/ histc[grayValues-1]*256)); } updatePixels(); } // Draw half of the histogram stroke(50, 250, 0); strokeWeight(2); for (int i=0; i<grayValues; i++) { line(i, height, i, height-hist[i]); }
st is step mpD with trnsition from 0 to 255 set in orrespondene of the hosen thresholdF
IIH
GLOSSARY
Glossary
C Convolution of two 2D signals (images)
y (m, n) = h x (m, n) = y (n) = h x (n) =

G global scope
k=
l=
(x (k, l) h (m k, n l))
Convolution of two signals h and x

m=
(x (m) h (n m))
de(ned outside the methods setup@A nd drw@AD the vrile is visile nd usle nywhere in the progrm qudruples of numersD where the (rst triple is to e red in the EE speD while the fourth numer indites vector if it tkes vlue HD or point if it tkes vlue IF
H homogeneous coordinates
Image Histograms
qrphi representtion y mens of vertil rsD where eh r represents the numer of pixels present in the imge for given intensity of the gry sle @or olor hnnelAF ikipedi de(nitionF21
L local scope
de(ned within ode lok or funtionD the vrile tkes vlues tht re lol to the lok or funtionD nd ny vlues tken y glol vrile hving the sme nme re ignoredF
scope
within progrmD it is region where vrile n e essed nd its vlue modi(ed ieewiseEpolynomil urveD with polynomils onneted with ontinuity t the knots 22 ndD for n introdution to the spei( kind of note: ee sntrodution to plines splines @gtmullEomA used in roessingD the term spline in ikipediF sf y1 nd y2 re the responses to the input sequenes x1 nd x2 then the input a1 x1 + a2 x2 produes the response a1 y1 + a2 y2
Spline
Superposition principle
T The impulse in discrete time (space)
is the signl with vlue1 t the instnt zero @in the point with oordintes [0, 0]AD nd 0 in ny other instnt @pointAF e system is timeEinvrint if time shift of D smples in the input results in the sme time shift in the outputD iFeFD x (n D) produes y (n D)F
Time invariance
21 http://en.wikipedia.org/wiki/Color_histogram 22 http://cnx.org/content/m12986/latest/
Bibliography
I idwrd engelF Interactive Computer Graphics: A Top-Down Approach With OPENGL primer package2nd EditionF rentieErllD snFD PHHIF P hvide ohessoF Introduction to Sound ProcessingF wondo istremoD PHHQF httpXGGwwwFmondoE estremoFomF
III
IIP
INDEX
Index of Keywords and Terms

re listed y the setion with tht keyword @pge numers re in prenthesesAF ueywords do not neessrily pper in the text of the pgeF hey re merely ssoited with tht setionF Ex. pplesD IFI @IA Terms re referened y the pge they pper onF Ex. pplesD I
Keywords
2 3 A
Ph grphisD Q@QIA Qh grphisD Q@QIA lisingD RV epplition rogrmming snterfe @esAD RP pproximting fzier urveD QP spet rtioD RQ udio ditheringD W@WIA verging (lterD UP xonometri projetionsD RH kgroundD IHQ filiner interpoltionD V@VQAD VQ fzier urveD QP enter of projetionD QV enter of the seneD RP lippingD RQ olor ditheringD W@WIA olor modelsD P@IUA olor pletteD W@WIA ommuttiveD TI ontrst strethingD IHP onvex hullD QP gonvolutionD S@SUAD SVD T@TIAD TP onvolution mskD SW gonvolution of two Ph signls @imgesAD SW gonvolution of two signls h nd xD SV onvolution sumD TI oordinte systemD P@IUA umulted distriutionD IHP urvesD Q@QIA derivtiveD IHQ di'erene equtionD US disrete timeD T@TIA disreteEspe systemsD SU disreteEtime onvolutionD TI disreteEtime systemsD SU ditheringD WI hD T@TIA hynmisD IHP
G H
K L
edge rispeningD UR edge detetionD W@WIA equlizeD IHP eye pointD RP pilteringD S@SUAD SV pinite smpulse esponseD TW ps (ltersD U@TWA (rstE nd seondEorder (lters for sounds nd imgesD U@TWA )shing in the eyeD RH fontD PP foregroundD IHQ frme rteD RV prequeny esponseD SVD SW frustumD RQ glol sopeD R grdientD VQD IHQ highEpssD UI histogrmD W@WIA homogeneous oordintesD PI ss (ltersD U@TWA smge ristogrmsD IHP imge proessingD I@IA imge representtionD P@IUA impulse responseD SVD T@TIA sn(nite smpulse esponseD US intertion designD I kernelD SW kerningD PQ vmertinD QU vplinD IHQ liner nd timeEinvrintD TI liner systemsD SU liner timeEinvrintD SU lol sopeD R voqD IHQ lowEpssD UH
INDEX
IIQ shdingD VR shrpening (ltersD U@TWA signlsD T@TIA signls nd systemsD T@TIA smoothingD UP smoothing (ltersD U@TWA solriztionD SQ sound proessingD I@IA sound representtionD P@IUA spetrumD RUD SI plineD QPD IIH uperposition prinipleD SU ystemsD S@SUA T texelD VR texture mppingD V@VQAD VR texturesD VR he impulse in disrete time @speAD SU thresholdD IHQ ime invrineD SU onl qmmD IHP trkingD PQ trnsltion nd rottionD P@IUA typeD PP type stingD II U unstleD US W wirefrmeD RQ Z zEu'erD RQ girleEgirle sntersetionD IH
vsD TI M wedi roessing in roessing @wAD I N noise shpingD WW xyquist frequenyD RVD SP O ojetEoriented progrmmingD I@IA ypenqvD QTD QTD RP orderD US P pletteD WI hong shdingD QT pithD WR plne of projetionD QV proedurl progrmmingD I@IA roessingD I proessing lnguge nd environmentD I@IA projetor rysD QV Q quntiztionD RU quntiztion noiseD SQ untiztion of sounds nd imgesD R@RUA R rsteringD VR reursiveD US rendering engineD IU S smpleEyEsmpleD SW smplingD RU mpling of Ih nd Ph signlsD R@RUA smpling rteD RU sn lineD VQ sopeD R
IIR
ATTRIBUTIONS
Attributions
golletionX Media Processing in Processing idited yX hvide ohesso vX httpXGGnxForgGontentGolIHPTVGIFIPG vienseX httpXGGretiveommonsForgGliensesGyGPFHG woduleX 4rogrmming in roessing4 fyX hvide ohesso vX httpXGGnxForgGontentGmIPWTVGIFVG gesX IEIT gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX rogrmming in roessing fyX hvide ohesso vX httpXGGnxForgGontentGmIPTIRGIFVG woduleX 4wedi epresenttion in roessing4 fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPWVQGIFIHG gesX IUEQH gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX wedi epresenttion in roessing fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPTTRGIFVG woduleX 4qrphi gomposition in roessing4 fyX hvide ohesso vX httpXGGnxForgGontentGmIPWVTGIFWG gesX QIERT gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX qrphi gomposition in roessing fyX hvide ohesso vX httpXGGnxForgGontentGmIPTTSGIFUG woduleX 4ignl roessing in roessingX mpling nd untiztion4 fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIQHRSGIFSG gesX RUESS gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX ignl roessing in roessingX mpling nd untiztion fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPUSIGIFUG
ATTRIBUTIONS
IIS
woduleX 4ignl roessing in roessingX gonvolution nd piltering4 fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIQHRTGIFQG gesX SUETH gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX ignl roessing in roessingX gonvolution nd piltering fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPVHWGIFPG woduleX 4hisreteEime gonvolution4 fyX irdo delliEnhezD ihrd frniuk vX httpXGGnxForgGontentGmIHHVUGPFIVG gesX TIETU gopyrightX irdo delliEnhezD ihrd frniuk vienseX httpXGGretiveommonsForgGliensesGyGIFH woduleX 4ignl roessing in roessingX ilementry pilters4 fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIQHRUGIFRG gesX TWEVI gopyrightX hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX ignl roessing in roessingX ilementry pilters fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPVPUGIFPG woduleX 4extures in roessing4 fyX ietro olottiD hvide ohesso vX httpXGGnxForgGontentGmIQHRVGIFQG gesX VQEVW gopyrightX ietro olottiD hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX extures in roessing fyX hvide ohessoD ietro olotti vX httpXGGnxForgGontentGmIPVQUGIFIG woduleX 4ignl roessing in roessingX wisellne4 fyX ietro olottiD hvide ohesso vX httpXGGnxForgGontentGmIQHVSGIFSG gesX WIEIHW gopyrightX ietro olottiD hvide ohesso vienseX httpXGGretiveommonsForgGliensesGyGPFHG fsed onX ignl roessing in roessingX wisellne fyX hvide ohesso vX httpXGGnxForgGontentGmIPVQVGIFSG
e ourse on fundmentls of imgeGsound proessing nd grphi progrmming explined y mens of the free lnguge nd environment roessingF
About Connexions
ine IWWWD gonnexions hs een pioneering glol system where nyone n rete ourse mterils nd mke them fully essile nd esily reusle free of hrgeF e re eEsed uthoringD tehing nd lerning environment open to nyone interested in edutionD inluding studentsD tehersD professors nd lifelong lernersF e onnet ides nd filitte edutionl ommunitiesF gonnexions9s modulrD intertive ourses re in use worldwide y universitiesD ommunity ollegesD uEIP shoolsD distne lernersD nd lifelong lernersF gonnexions mterils re in mny lngugesD inluding inglishD pnishD ghineseD tpneseD stlinD ietnmeseD prenhD ortugueseD nd hiF gonnexions is prt of n exiting new informtion distriution system tht llows for Print on Demand BooksF gonnexions hs prtnered with innovtive onEdemnd pulisher yy to elerte the delivery of printed ourse mterils nd textooks into lssrooms worldwide t lower pries thn trditionl demi pulishersF

Media Processing in Processing

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Media Processing in Processing

Încărcat de

Drepturi de autor:

Formate disponibile

Media Processing in Processing

Collection Editor: Davide Rocchesso

Media Processing in Processing

Online: <http://cnx.org/content/col10268/1.12/ >

CONNEXIONS Rice University, Houston, Texas

PHHV hvide ohesso

Programming Modes Basic -

1 This content is available online at <http://cnx.org/content/m12968/1.8/>. 2 http://www.shiman.net/itp/classes/ppaint/ 3 http://cnx.org/content/m12968/latest/pinocchiononose.html

P pplet with nose4

yjetEyriented rogrmming @tvA !

Q pplet with olE ored nose5

1.2 Data Types

within progrmD it is region where vrile n e essed nd its vlue modi(ed

int[] arrayDiInteri = new int[10];

1.3 Programming Structures

Example 1.2: Initializing a table of random numbers

int raddoppia(int i) { return 2*i; }

1.5 Objects and Classes

and Classes" <http://cnx.org/content/m11708/latest/>

void draw() { background((mouseX/100)*255); }

roweverD the ode does not do wht it is expeted to doF hyc

ht does the following ode frgment print outc

Syntax Primer" <http://cnx.org/content/m11791/latest/>

void intersect( Circle cA, Circle cB )

stroke(255-dist(paX, paY, pbX, pbY)*4); line(paX, paY, pbX, pbY);

Solutions to Exercises in Chapter 1

stroke(255-dist(paX, paY, pbX, pbY)*4); line(paX, paY, pbX, pbY);

stroke(255-dist(paX, paY, pbX, pbY)*4); line(paX, paY, pbX, pbY);

2.1 Visual Elements

3D coordinate system used in Processing

content is available online at <http://cnx.org/content/m12983/1.10/>.

MEDIA REPRESENTATION IN PROCESSING

MEDIA REPRESENTATION IN PROCESSING

Gimp color chooser

Color chooser of the software Gimp

2.1.3.3 Alpha channel

sn omputer grphisD points nd vetors re represented with the

MEDIA REPRESENTATION IN PROCESSING

meti AD where the mtrix is

en the ngle ntiElokwise rottion y

ottions round the xes x @pitchA nd y @yawA re relized y mens of

MEDIA REPRESENTATION IN PROCESSING

2.2 Auditory Elements

MEDIA REPRESENTATION IN PROCESSING

(Solution on p. 28.) (Solution on p. 28.)

MEDIA REPRESENTATION IN PROCESSING

Solutions to Exercises in Chapter 2

roessing soure odeF

epplet with roessing soure odeF 20

mySample1.sampleRate()); mySample2.sampleRate()); mySample3.sampleRate()); mySample4.sampleRate());

the le at <http://cnx.org/content/m12983/latest/./sound_chooser.pde> the le at <http://cnx.org/content/m12983/latest/./sound_chooser_color.pde>

MEDIA REPRESENTATION IN PROCESSING

lpf4.setFreq(cutoff4); println(mouseX + " + " +cutoff4); mySample4.rewind(); mySample4.play();

3.1 Graphic primitives

content is available online at <http://cnx.org/content/m12986/1.9/>.

GRAPHIC COMPOSITION IN PROCESSING

10); 10); c2y); c1y); 190); 190);

stroke(255,30,0); bezier(10,10,c2x,c2y,c1x,c1y,190,190); strokeWeight(4); point(c1x,c1y); point(c2x,c2y);

GRAPHIC COMPOSITION IN PROCESSING

spe nd the orresponding tringles identify plnr surfes in speF

3.2 The stack of transformations

GRAPHIC COMPOSITION IN PROCESSING

GRAPHIC COMPOSITION IN PROCESSING

sn generlD the projetion of point hving homogeneous

1 This content is available online at <http://cnx.org/content/m12968/1.8/>. 2 http://www.shiman.net/itp/classes/ppaint/ 3 http://cnx.org/content/m12968/latest/pinocchiononose.html

the le at <http://cnx.org/content/m12983/latest/./sound_chooser.pde> the le at <http://cnx.org/content/m12983/latest/./sound_chooser_color.pde>

5.3 Frequency response and ltering

This is the end result that we are looking to nd.

7.1 FIR lters

Magnitude and phase response for the averaging lter

7.1.3 High-pass lters

Magnitude and phase response of the IIR rst-order lter