Visual Sensors and Algorithms

Ivan Petrovic

To look up
QR Decomposition

Blur Lens

Radial Image Distortion

Sobel Operator

Canny Edge Detector

Image Plane (tied into Focal Distance for cameras)

Jacobian Transposition

Actual Notes
𝑥 𝑥
Apparently equation (*) = 𝑢 = 𝑥1 , 𝑣 = 𝑥2.
3 3

Fun fact: Colour images are represented by a vector of values for each pixel~

Describe an object by a histogram of colours (defined by a specific colour combination). Image is

represented by a vector in 𝑛-dimensional vector space (ℎ1 , ℎ2 , … , ℎ𝑛 ) (ℎ𝑖 is the number of pixels
having colour 𝑖).

Similarity of image 𝐻 obtained via histogram comparison.

A motion of a group of pixels corresponds to a motion of an object in space – optical flow.

If the intensity change is smooth we can expand the image intensity function to a Taylor series.
𝑓(𝑥 + 𝑑𝑥, 𝑦 + 𝑑𝑦, 𝑡 + 𝑑𝑡) = 𝑓(𝑥, 𝑦, 𝑡). <-Brightness constancy constraint; we assume the
brightness is constant and hence that it doesn’t modify colour values.
𝛿𝑓 𝛿𝑓 𝑑𝑥 𝛿𝑓 𝑑𝑦
Eventually through partial derivatives, you get − 𝛿𝑡 = 𝑥 𝑑𝑡
+ 𝑑𝑦 𝑑𝑡 (*). This is the optical flow
equation.
𝑑𝑥 𝑑𝑦
This gives constraints on the velocities (𝑢, 𝑣) = ( 𝑑𝑡 , 𝑑𝑡 ) dependent on the local derivations of the
𝑑𝑓 𝑑𝑓 𝑑𝑓
image intensity function: (𝐸𝑥 , 𝐸𝑦 , 𝐸𝑡 ) = (𝑑𝑥 , 𝑑𝑦 , 𝑑𝑡 ).
Equation (*) is not sufficient to compute optical flow (𝑢, 𝑣), but only the linear constraint (𝐸𝑥 𝑢 +
𝐸𝑦 𝑣 + 𝐸𝑡 = 0).

We need an additional equation, i.e. an additional constraint.

Small aperture problem: If you have lines going North-South of infinite length and a small aperture
to view them from, you would observe a Southeast motion as an East motion. This is also known as
the barber pole effect (You can see the rotation when zoomed out, but as it zooms in it looks more
like it’s going upwards and less like it’s going left).

Evaluation of various optical flow algorithms: http://vision.middlebury.edu/flow/

To compute distances using a single camera ‘orbiting’ around a scene, you need to have 5 point
correspondences. Slide states “possible to compute unknown R and T (up to a scale)”. (R,T) is the
annotation for the curved arrow indicating the ‘orbit’. Slide ‘section’ statement is “MR ego motion”,
‘subject’ is “Distance computation”.

Fun fact: Kinect has an RGB camera in the middle, and the two other apertures around it are an IR
projector and an IR camera; the projector emit the radiation and the camera picks up the reflected
light. I believe this is the first one, and after Apple bought the technology involved (or its provider?),
they had to change the design. Or this is the new one, I forget.