Chapter 13 Perspective Projection Geometry

13.1 Introduction

Computer vision problems often involve interpreting information on a
two-dimensional (2D) image of a three-dimensional (3D) world in order to
determine the placement of the 3D objects portrayed in the image.

To do this requires understanding the perspective transformation governing
the geometric way 3D information is projected onto the 2D image.

image formation on the retina, according to Descartes

scrape ox eye, observe from darkened room inverted image of scene

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 2.1=====

13.2 One-Dimensional Perspective Projection

: focal length of lens

: distance between object and lens center

: distance between image and lens center

thin-lens equation: lens law:

light passing lens center does not deflect

light parallel to optical axis will pass focal point

=====Nalwa,

perspective projection: object point projected along straight line --caption--

=====Nalwa,

image distortion: room ceiling appears bowed in the image

=====Nalwa,

pinhole camera: infinitesimally small aperture

pinhole camera: approximated by lens with aperture adjusted to the smallest

pinhole camera: simplest device to form image of 3D scene on 2D surface

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 2.2=====

aperture size decreased: image become sharper

diameter of aperture is 0.06 inch, 0.015 inch, 0.0025 inch

aperture below certain size: diffraction: bending of light rays around edge

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 2.10=====

lens oriented along the -axis and image plane parallel to -axis

perspective projection gives point coordinates on image

=====Fig. 13.1=====

: camera constant (different from above equation)

: homogeneous coordinate system for point

first linear transformation: translates by distance of

second linear transformation: takes perspective transformation to image line

1D image line coordinate:

lens: at origin and looks down -axis

image line: distance in front of lens and parallel to -axis

axes: the axes rotated anticlockwise by angle

=====Fig. 13.2=====

rewriting the relationship in terms of homogeneous coordinate system

13.3 The Perspective Projection in 3D

camera lens: along line parallel to -axis

position of lens: center of perspectivity:

: coordinates of perspective projection of on image plane

thus

13.3.1 Smaller Appearance of Farther Objects

without loss of generality: take center of perspectivity to be origin

perspective projection: objects appear smaller the farther they are

=====Fig. 13.3=====

foreshortening: line segments in plane parallel to image has maximum size

=====Fig. 13.4=====

13.3.2 Lines to Lines

lines in the 3D world transform to lines in the image plane

parallel lines in 3D with nonzero slope: meet in a vanishing point

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 2.4=====

13.3.3 Perspective Projection of Convex Polyhedra are Convex

proofs in textbook, simple but tedious, study as exercise by yourself

13.3.4 Vanishing Point

Perspective projections of parallel 3D lines having nonzero slope along the
optic -axis meet in a vanishing point on the image projection plane.

13.3.5 Vanishing Line

All lines lying in planes parallel to the slanted floor have vanishing points
that lie along a vanishing line.

=====Fig. 13.5=====

=====Tour Into Picture=====

=====Garfield 17:21=====

13.3.6 3D Lines-2D Perspective Projection Lines

There is a relationship between the parameters of a 3D line and the parameters
of the perspective projection of the line.

3D line L:

perspective projection of the line :

take camera lens as origin of coordinate system

for any given :

eliminate :

for any :

thus

13.4 2D to 3D Inference Using Perspective Projection

perspective projection on unknown 3D line: provides four of six constraints

additional constraints: 3D-world-model information about points, lines

13.4.1 Inverse Perspective Projection

: perspective projection of a point

: image plane distance from camera lens

thus : 3D coordinate of the point in image plane

camera lens: at the origin

line : inverse perspective projection of the point

13.4.2 Line Segment with Known Direction Cosines and Known Length

known:

- : line segment length
- : line segment direction cosine
- : perspective projections of endpoints

- : 3D coordinates of endpoints

13.4.3 Collinear Points with Known Interpoint Distances

known:

- : perspective projections of th collinear points,
- : distance between th point and first point

- : direction cosine of line
- : 3D coordinates of points

13.4.4 Parallel Lines

known:

- : perspective projections of th parallel line

- : direction cosine of line

13.4.5 Lines Intersecting at a Point with Known Angles

known:

- : perspective projection of intersecting point
- : perspective projections of th intersecting line
- : known angle between and

- : 3D th intersecting line

13.4.6 Intersecting Lines in a Known Plane

known:

- : perspective projection of intersecting point
- : perspective projections of th intersecting line
- : plane equation

- : 3D th intersecting line

normal vector: perpedicular to direction cosine

=====joke=====

13.4.7 Three Lines in a Plane with One Perpendicular to the Other Two

known:

- : perspective projection of line
- : perspective projection of line
- : perspective projection of line

- : 3D line
- : 3D line
- : 3D line

13.4.8 Point with Given Distance to a Known Point

known:

- : perspective projection of unknown point
- : known 3D point
- : distance between the two points

- : direction cosine between two points

13.4.9 Point in a Known Plane

known:

- : perspective projection of unknown point
- : known plane equation where point lies

- : 3D coordinate of the point

- : 3D coordinate of the point

13.4.10 Line in a Known Plane

known:

- : known plane equation where line lies
- : perspective projection of line

- : 3D line

: since line lies in plane

13.4.11 Angle

known:

- : perspective projection of the unknown line
- : direction cosine for the known line
- : angle between the 3D lines

- : direction cosine for the unknown line

13.4.12 Parallelogram

known:

- perspective projection of four corner points of a parallelogram

- : normal to the plane where the parallelogram lies

13.4.13 Triangle with One Vertex Known

known:

- : perspective projection of three vertices
- : one known 3D vertex of the three vertices
- : known length of the triangle in 3D

- : two unknown 3D vertices of the three vertices

13.4.14 Triangle with Orientation of One Leg Known

known:

- : perspective projection of three vertices
- : known length of the triangle in 3D
- : known direction cosines between the first two vertices

- : three unknown 3D vertices

13.4.15 Triangle: three-point spatial resection problem in photogrammetry

known:

- : perspective projection of three vertices
- : known length of the triangle in 3D

- : three unknown 3D vertices

13.4.16 Determining the Principal Point by Using Parallel Lines

principal point: point through which the optic axis passes

principal point: so far assumes origin of image reference frame

known:

- : perspective projection of th parallel line

- : coordinates of the principal point

13.5 Circles

known:

- perspective projection of a circle having known radius

- plane on which the circle lies
- the 3D center of the circle

13.6 Range from Structured Light

structured light: active visual sensing technique upon perspective geometry

structured light: controlled light source with regular pattern onto scene

regular pattern: stripes, grid, ...

light striping and a typical arrangement

=====Ballard and Brown, *Computer Vision*, Fig. 2.25=====

intensity and range images

=====Ballard and Brown, *Computer Vision*, Fig. 2.26=====

Two light sources with cylindrical lenses produce sheets of light that intersect
in a line lying on the surface of a conveyor belt.

A camera above the belt is aimed so that this line is imaged on a linear array
of photosensors.

When there is no object present, all the sensor cells are brightly
illuminated.

When part of an object interrupts the incident light, the corresponding
region on the linear array is darkened.

The motion of the belt scans the object past the sensor, generating the
second image dimension.

=====Horn, *Robot Vision*, Fig. 5.4=====

13.7 Cross-Ratio

cross-ratio: of perspective projection of 4 collinear points, takes same value

13.7.1 Cross-Ratio Definitions and Invariance

four collinear points:

: centers of perspectivity for two projection images

=====Fig. 13.7=====

Let
.
by perspective projection equations

cross-ratio:

cross-ratio: independent of reference frame, point , direction cosine

cross-ratio: depends only on directed distance of collinear points

13.7.2 Only One Cross-Ratio

each of 4! cross-ratios is a function of cross-ratio

13.7.3 Cross-Ratio in Three Dimensions

The cross-ratio derived from 1D perspective projections in
2D world can be generalized to 2D perspective
projections in 3D world.

five co-planar points

: cross-ratio for the line segment and

: cross-ratio for the line segment and

=====Fig. 13.8=====

13.7.4 Using Cross-Ratios

cross-ratios: to aid in establishing correspondences

=====joke=====

tentative term project problems, (30%) total grade:

submit one page in English explaining method, steps, expected results

submit report in a month; report progress every other week

all right to be the same problem with Master's thesis

objective: a working prototype with new, original, novel ideas

objective: not just literature survey

objective: not just straightforward implementation of existing algorithms

objective: all right to modify existing algorithms

1. Neural Network, Gonzalez, Sec. 9.3.3, p. 595

=====Gonzalez, *Digital Image Processing*, Fig. 9.14=====

2. Fuzzy Logic

3. Image Compression: JPEG, MPEG, Gonzalez, Sec. 6.6, p. 389

=====Gonzalez, *Digital Image Processing*, Plate X=====

4. wavelet transform

5. segmentation based on texture, Gonzalez, p. 594

=====Gonzalez, *Digital Image Processing*, Fig. 9.13=====

6. optical character reading: a, b, c, d, ... 0, 1, 2, .....

=====fonts=====

7. stereo vision, Nalwa, Chapter 7

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 7.9=====

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 7.10=====

8. handwriting recognition: zip code; on-line, off-line Chinese characters ...

=====zip=====

=====Chinese=====

9. histogram specification: Gonzalez, p. 182, Sec. 4.2.2

=====Gonzalez, *Digital Image Processing*, Fig. 4.14=====

10. Homomorphic filtering: Gonzalez, Sec. 4.4.3

=====Gonzalez, *Digital Image Processing*, Fig. 4.41=====

=====Gonzalez, *Digital Image Processing*, Fig. 4.42=====

11. real-time counting number of cars and their sizes.

12. calculating the sizes of stones, cells, cell nucleus.

=====t_pebbles.im=====

13. trademark resemblance, semi-automatic similarity classification

=====trademark.im=====

14. car plate recognition.

15. structured light 3-D reconstruction. Horn, p. 95

=====Horn, *Robot Vision*, Fig. 5.4=====

=====Ballard and Brown, *Computer Vision*, Fig. 2.25=====

=====Ballard and Brown, *Computer Vision*, Fig. 2.26=====

16. object classification with moments invariant to rotation,

scaling, translation, Gonzalez, p. 514, Sec. 8.3.4

=====Gonzalez, *Digital Image Processing*, Eq. 8.3-14=====

=====Gonzalez, *Digital Image Processing*, Fig. 8.24=====

17. photometric stereo, Sec. 12.3, p. 16, Eq. 12.14

18. shape from focus, defocus, Sec. 12.4.1, p. 21

19. shape from polarization, Sec. 12.5, p. 22

20. shape from shading, Horn, p. 226

=====Horn, *Robot Vision*, Fig. 10.18=====

=====Horn, *Robot Vision*, Fig. 10.19=====

21. shape from texture, Nalwa, p. 199

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 6.1=====

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 6.6=====

22. solving correspondence problem or optic flow field

=====truck.im=====

23. motion and shape parameter recovery

=====truck.im=====

24. segmentation of newspaper, documents into title, figure, caption,....

=====magazine.im=====

25. optical distortion correction, Nalwa, p. 46

=====Nalwa, *A Guided Tour of Computer Vision*, Fig. 2.13=====

26. line labeling of 2D line drawing of 3D objects, Sec. 17.5, p. 404

=====Fig. 17.13=====

27. Computer Tomography: 3D image reconstruction from 2D projections

=====Cho , *Foundations of Medical Imaging*, Fig. 6.6=====

=====Wicke, *Atlas of Radiologic Anatomy*, Fig. 159=====

=====Wicke, *Atlas of Radiologic Anatomy*, Fig. 160=====

28. road surface inspection: cracks, holes, ...

29. traffic counting: number of trucks, cars, motorcycles, ...

30. finger-print validation

31. face recognition

32. use robot to pick stones out of a bin

=====stone.im=====

33. digital morphing

=====morphing.im=====

34. model-based diagnosis

=====thorax.im=====

35. automatic identification and classification of video scenes for indexing

36. automatic break detection or video partitioning or scene change detection

37. automatic full-video search for objects of interest

e.g. find all frames which contain frogs

38. creating video or image database with functions

such as efficient querying, indexing, retrieval, browsing

39. recognition of video content invariant to viewing conditions

e.g. find all other shots of this scene

40. printed music sheet recognition and translation into MIDI format file

=====music.im=====

41. 360 degree image from several pictures (alignment, color interpolation)

42. wafer defect inspection

=====wafer.defect=====

43. wafer critical dimension measurement

=====Elliott, *Integrated Circuit Fabrication Technology*, Fig. 8.54=====

44. IC pin inspection: coplanarity of surface mount device

colinearity of dual-in-line package

=====IC.pin=====

45. IC mark printing inspection: smear, contract, scratch, ...

=====IC.mark=====

46. electrical contact point inspection

=====electrical.contact=====

47. digital watermarking: viewing but no printing due to copyright

48. auto focus in digital camera

49. auto exposure in digital camera

50. auto white balance in digital camera

51. color management: sRGB: standard Red, Green, Blue

52. 640X480 ==> 1280X960 from single image

53. super resolution: 640X480 ==> 1280X960 from multiple images

54. video stabilization for digital camcorder

Project due April 9:

camera calibration i.e compute #pixels/mm object displacement

calculate field of view in angles.

calculate and compare with theoretical values.

use lens of focal length: 16mm, 25mm, 50mm

object displacement of: 1mm, 5mm, 10mm, 20mm

object distance of: 0.5m, 1m, 2m

camera parameters: 8mm 6mm 512485?

Are pixels square or rectangular?

2002-02-26