Chapter 13 Perspective Projection Geometry

13.1 Introduction
Computer vision problems often involve interpreting information on a two-dimensional (2D) image of a three-dimensional (3D) world in order to determine the placement of the 3D objects portrayed in the image.
To do this requires understanding the perspective transformation governing the geometric way 3D information is projected onto the 2D image.
image formation on the retina, according to Descartes
scrape ox eye, observe from darkened room inverted image of scene
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.1=====

13.2 One-Dimensional Perspective Projection
: focal length of lens
: distance between object and lens center
: distance between image and lens center
thin-lens equation: lens law:

light passing lens center does not deflect
light parallel to optical axis will pass focal point
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.11=====
perspective projection: object point projected along straight line --caption--
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.3=====
image distortion: room ceiling appears bowed in the image
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.13=====

pinhole camera: infinitesimally small aperture
pinhole camera: approximated by lens with aperture adjusted to the smallest
pinhole camera: simplest device to form image of 3D scene on 2D surface
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.2=====
aperture size decreased: image become sharper
diameter of aperture is 0.06 inch, 0.015 inch, 0.0025 inch
aperture below certain size: diffraction: bending of light rays around edge
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.10=====

lens oriented along the -axis and image plane parallel to -axis
perspective projection gives point coordinates on image
=====Fig. 13.1=====
: camera constant (different from above equation)
: homogeneous coordinate system for point
first linear transformation: translates by distance of
second linear transformation: takes perspective transformation to image line

1D image line coordinate:

lens: at origin and looks down -axis
image line: distance in front of lens and parallel to -axis
axes: the axes rotated anticlockwise by angle
=====Fig. 13.2=====
rewriting the relationship in terms of homogeneous coordinate system

13.3 The Perspective Projection in 3D
camera lens: along line parallel to -axis
position of lens: center of perspectivity:
: coordinates of perspective projection of on image plane

thus

13.3.1 Smaller Appearance of Farther Objects
without loss of generality: take center of perspectivity to be origin
perspective projection: objects appear smaller the farther they are
=====Fig. 13.3=====
foreshortening: line segments in plane parallel to image has maximum size
=====Fig. 13.4=====

13.3.2 Lines to Lines
lines in the 3D world transform to lines in the image plane
parallel lines in 3D with nonzero slope: meet in a vanishing point
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.4=====

13.3.3 Perspective Projection of Convex Polyhedra are Convex
proofs in textbook, simple but tedious, study as exercise by yourself

13.3.4 Vanishing Point
Perspective projections of parallel 3D lines having nonzero slope along the optic -axis meet in a vanishing point on the image projection plane.

13.3.5 Vanishing Line
All lines lying in planes parallel to the slanted floor have vanishing points that lie along a vanishing line.
=====Fig. 13.5=====
=====Tour Into Picture=====
=====Garfield 17:21=====

13.3.6 3D Lines-2D Perspective Projection Lines
There is a relationship between the parameters of a 3D line and the parameters of the perspective projection of the line.
3D line L:

perspective projection of the line :

take camera lens as origin of coordinate system

for any given :

eliminate :

for any :

thus

13.4 2D to 3D Inference Using Perspective Projection
perspective projection on unknown 3D line: provides four of six constraints

13.4.1 Inverse Perspective Projection
: perspective projection of a point
: image plane distance from camera lens
thus : 3D coordinate of the point in image plane
camera lens: at the origin
line : inverse perspective projection of the point

13.4.2 Line Segment with Known Direction Cosines and Known Length
known:

• : line segment length
• : line segment direction cosine
• : perspective projections of endpoints
unknown:
• : 3D coordinates of endpoints

13.4.3 Collinear Points with Known Interpoint Distances
known:

• : perspective projections of th collinear points,
• : distance between th point and first point
unknown:
• : direction cosine of line
• : 3D coordinates of points

13.4.4 Parallel Lines
known:

• : perspective projections of th parallel line
unknown:
• : direction cosine of line

13.4.5 Lines Intersecting at a Point with Known Angles
known:

• : perspective projection of intersecting point
• : perspective projections of th intersecting line
• : known angle between and
unknown:
• : 3D th intersecting line
: known angle between line pair

13.4.6 Intersecting Lines in a Known Plane
known:

• : perspective projection of intersecting point
• : perspective projections of th intersecting line
• : plane equation
unknown:
• : 3D th intersecting line

normal vector: perpedicular to direction cosine
=====joke=====

13.4.7 Three Lines in a Plane with One Perpendicular to the Other Two
known:

• : perspective projection of line
• : perspective projection of line
• : perspective projection of line
unknown: three lines in same plane, one perpendicular to other two
• : 3D line
• : 3D line
• : 3D line
: since perpendicular to

13.4.8 Point with Given Distance to a Known Point
known:

• : perspective projection of unknown point
• : known 3D point
• : distance between the two points
unknown:
• : direction cosine between two points

13.4.9 Point in a Known Plane
known:

• : perspective projection of unknown point
• : known plane equation where point lies
unknown:
• : 3D coordinate of the point
solution:
• : 3D coordinate of the point

13.4.10 Line in a Known Plane
known:

• : known plane equation where line lies
• : perspective projection of line
unknown:
• : 3D line
: since line lies in plane
: since line lies in plane

13.4.11 Angle
known:

• : perspective projection of the unknown line
• : direction cosine for the known line
• : angle between the 3D lines
unknown:
• : direction cosine for the unknown line
: since is the angle between 3D lines

13.4.12 Parallelogram
known:

• perspective projection of four corner points of a parallelogram
unknown:
• : normal to the plane where the parallelogram lies
: direction cosines of two sides of parallelogram

13.4.13 Triangle with One Vertex Known
known:

• : perspective projection of three vertices
• : one known 3D vertex of the three vertices
• : known length of the triangle in 3D
unknown:
• : two unknown 3D vertices of the three vertices

13.4.14 Triangle with Orientation of One Leg Known
known:

• : perspective projection of three vertices
• : known length of the triangle in 3D
• : known direction cosines between the first two vertices
unknown:
• : three unknown 3D vertices

13.4.15 Triangle: three-point spatial resection problem in photogrammetry
known:

• : perspective projection of three vertices
• : known length of the triangle in 3D
unknown:
• : three unknown 3D vertices
four solutions

13.4.16 Determining the Principal Point by Using Parallel Lines
principal point: point through which the optic axis passes
principal point: so far assumes origin of image reference frame
known:

• : perspective projection of th parallel line
unknown:
• : coordinates of the principal point

13.5 Circles
known:

• perspective projection of a circle having known radius
unknown:
• plane on which the circle lies
• the 3D center of the circle
=====Oldie 33:70=====

13.6 Range from Structured Light
structured light: active visual sensing technique upon perspective geometry
structured light: controlled light source with regular pattern onto scene
regular pattern: stripes, grid, ...
light striping and a typical arrangement
=====Ballard and Brown, Computer Vision, Fig. 2.25=====
intensity and range images
=====Ballard and Brown, Computer Vision, Fig. 2.26=====
Two light sources with cylindrical lenses produce sheets of light that intersect in a line lying on the surface of a conveyor belt.
A camera above the belt is aimed so that this line is imaged on a linear array of photosensors.
When there is no object present, all the sensor cells are brightly illuminated.
When part of an object interrupts the incident light, the corresponding region on the linear array is darkened.
The motion of the belt scans the object past the sensor, generating the second image dimension.
=====Horn, Robot Vision, Fig. 5.4=====

13.7 Cross-Ratio
cross-ratio: of perspective projection of 4 collinear points, takes same value

13.7.1 Cross-Ratio Definitions and Invariance
four collinear points:
: centers of perspectivity for two projection images
=====Fig. 13.7=====
Let . by perspective projection equations

cross-ratio:

cross-ratio: independent of reference frame, point , direction cosine
cross-ratio: depends only on directed distance of collinear points

13.7.2 Only One Cross-Ratio
each of 4! cross-ratios is a function of cross-ratio

13.7.3 Cross-Ratio in Three Dimensions
The cross-ratio derived from 1D perspective projections in 2D world can be generalized to 2D perspective projections in 3D world.
five co-planar points
: cross-ratio for the line segment and

: cross-ratio for the line segment and

=====Fig. 13.8=====

13.7.4 Using Cross-Ratios
cross-ratios: to aid in establishing correspondences
=====joke=====

tentative term project problems, (30%) total grade:
submit one page in English explaining method, steps, expected results
submit report in a month; report progress every other week
all right to be the same problem with Master's thesis
objective: a working prototype with new, original, novel ideas
objective: not just literature survey
objective: not just straightforward implementation of existing algorithms
objective: all right to modify existing algorithms
1. Neural Network, Gonzalez, Sec. 9.3.3, p. 595
=====Gonzalez, Digital Image Processing, Fig. 9.14=====
2. Fuzzy Logic
3. Image Compression: JPEG, MPEG, Gonzalez, Sec. 6.6, p. 389
=====Gonzalez, Digital Image Processing, Plate X=====
4. wavelet transform
5. segmentation based on texture, Gonzalez, p. 594
=====Gonzalez, Digital Image Processing, Fig. 9.13=====
6. optical character reading: a, b, c, d, ... 0, 1, 2, .....
=====fonts=====
7. stereo vision, Nalwa, Chapter 7
=====Nalwa, A Guided Tour of Computer Vision, Fig. 7.9=====
=====Nalwa, A Guided Tour of Computer Vision, Fig. 7.10=====
8. handwriting recognition: zip code; on-line, off-line Chinese characters ...
=====zip=====
=====Chinese=====
9. histogram specification: Gonzalez, p. 182, Sec. 4.2.2
=====Gonzalez, Digital Image Processing, Fig. 4.14=====
10. Homomorphic filtering: Gonzalez, Sec. 4.4.3
=====Gonzalez, Digital Image Processing, Fig. 4.41=====
=====Gonzalez, Digital Image Processing, Fig. 4.42=====
11. real-time counting number of cars and their sizes.
12. calculating the sizes of stones, cells, cell nucleus.
=====t_pebbles.im=====
13. trademark resemblance, semi-automatic similarity classification
14. car plate recognition.
15. structured light 3-D reconstruction. Horn, p. 95
=====Horn, Robot Vision, Fig. 5.4=====
=====Ballard and Brown, Computer Vision, Fig. 2.25=====
=====Ballard and Brown, Computer Vision, Fig. 2.26=====
16. object classification with moments invariant to rotation,
scaling, translation, Gonzalez, p. 514, Sec. 8.3.4
=====Gonzalez, Digital Image Processing, Eq. 8.3-14=====
=====Gonzalez, Digital Image Processing, Fig. 8.24=====
17. photometric stereo, Sec. 12.3, p. 16, Eq. 12.14
18. shape from focus, defocus, Sec. 12.4.1, p. 21
19. shape from polarization, Sec. 12.5, p. 22
20. shape from shading, Horn, p. 226
=====Horn, Robot Vision, Fig. 10.18=====
=====Horn, Robot Vision, Fig. 10.19=====
21. shape from texture, Nalwa, p. 199
=====Nalwa, A Guided Tour of Computer Vision, Fig. 6.1=====
=====Nalwa, A Guided Tour of Computer Vision, Fig. 6.6=====
22. solving correspondence problem or optic flow field
=====truck.im=====
23. motion and shape parameter recovery
=====truck.im=====
24. segmentation of newspaper, documents into title, figure, caption,....
=====magazine.im=====
25. optical distortion correction, Nalwa, p. 46
=====Nalwa, A Guided Tour of Computer Vision, Fig. 2.13=====
26. line labeling of 2D line drawing of 3D objects, Sec. 17.5, p. 404
=====Fig. 17.13=====
27. Computer Tomography: 3D image reconstruction from 2D projections
=====Cho , Foundations of Medical Imaging, Fig. 6.6=====
=====Wicke, Atlas of Radiologic Anatomy, Fig. 159=====
=====Wicke, Atlas of Radiologic Anatomy, Fig. 160=====
28. road surface inspection: cracks, holes, ...
29. traffic counting: number of trucks, cars, motorcycles, ...
30. finger-print validation
31. face recognition
32. use robot to pick stones out of a bin
=====stone.im=====
33. digital morphing
=====morphing.im=====
34. model-based diagnosis
=====thorax.im=====
35. automatic identification and classification of video scenes for indexing
36. automatic break detection or video partitioning or scene change detection
37. automatic full-video search for objects of interest
e.g. find all frames which contain frogs
38. creating video or image database with functions
such as efficient querying, indexing, retrieval, browsing
39. recognition of video content invariant to viewing conditions
e.g. find all other shots of this scene
40. printed music sheet recognition and translation into MIDI format file
=====music.im=====
41. 360 degree image from several pictures (alignment, color interpolation)
42. wafer defect inspection
=====wafer.defect=====
43. wafer critical dimension measurement
=====Elliott, Integrated Circuit Fabrication Technology, Fig. 8.54=====
44. IC pin inspection: coplanarity of surface mount device
colinearity of dual-in-line package
=====IC.pin=====
45. IC mark printing inspection: smear, contract, scratch, ...
=====IC.mark=====
46. electrical contact point inspection
=====electrical.contact=====
47. digital watermarking: viewing but no printing due to copyright
48. auto focus in digital camera
49. auto exposure in digital camera
50. auto white balance in digital camera
51. color management: sRGB: standard Red, Green, Blue
52. 640X480 ==> 1280X960 from single image
53. super resolution: 640X480 ==> 1280X960 from multiple images
54. video stabilization for digital camcorder

2002-02-26