Chapter 14 Analytic Photogrammetry

14.1 Introduction

Analytic photogrammetry includes the body of techniques by which, from

measurements of one or more 2D-perspective projections of

a 3D object, one can make inferences about the 3D position,

orientation, and lengths of the observed 3D object parts

in a world reference frame.

inference problems: can be construed as nonlinear least-squares problems

Photogrammetry provides a collection of methods for determining the position

and orientation of cameras and range sensors in the scene and

relating camera positions and range measurements to
scene coordinates.

GIS: Geographic Information System

GPS: Global Positioning System

exterior orientation: determines position and orientation of camera in

absolute coordinate system from projections of calibration

points in scene

exterior orientation of camera: specified by all parameters of camera pose

pose parameters: perspectivity center position, optical axis direction...

exterior orientation specification: requires 3 rotation angles, 3 translations

interior orientation: determines internal geometry of camera

interior orientation of camera: parameters determining geometry of 3D rays

from measured image coordinates

The parameters of interior orientation relate the geometry of ideal

perspective projection to the physics of a camera.

parameters: camera constant, principal point, lens distortion ....

complete specification of camera orientation: interior and exterior orientation

relative orientation: determines relative position and orientation between

2 cameras from projections of calibration points in scene

relative orientation: to calibrate relation between two cameras for stereo

relative orientation: relates coordinate systems of two cameras to each other,

not knowing 3D points themselves, only their projections
in image

relative orientation of one camera to another constitutes stereo model:

specified by 5 parameters: 3 rotation angles, 2 translations

relative orientation: assumes interior orientation of each camera known

absolute orientation: determines transformation between 2 coordinate systems

or position and orientation of range sensor in absolute

coordinate system from coordinates of calibration points

absolute orientation: to convert depth measurements in viewer-centered

coordinates to absolute coordinate system for the scene

absolute orientation: orientation of stereo model in world reference frame

absolute orientation: determines scale, 3 translations, 3 rotations

absolute orientation: recovery of relation between two coordinate systems

:
a point in the world reference frame

:
camera lens position

left-hand coordinate system: clockwise angle of rotation

: angle of rotation around -axis of camera reference frame

: tilt angle

: angle of rotation around -axis

: pan angle

: angle of rotation around -axis

: swing angle

3 3 rotation matrix:

angles can be obtained directly from values :

point in world frame represented by in camera frame:

: denotes matrix or vector transpose

pinhole camera with image at distance from camera lens, projection:

: camera constant, related to focal length of lens

principal point: origin of measurement image plane coordinate

principal point : in image measurement plane coordinate system

general fundamental perspective projection equations: collinearity equation:

They show that the relationship between the measured 2D-perspective projection coordinates and the 3D coordinates is a nonlinear function of

and .

=====Garfield 17:27=====

14.2 Nonlinear Least-Squares Solutions

noise model:

: unknown parameters governing nonlinear transformations

: nonlinear transformations

: observed values of

: additive mean zero Gaussian random variables with covariance

maximum likelihood solution:
maximize

maximum likelihood solution minimizes least-squares criterion

where

assumption: overconstrained and unique solution,

first-order Taylor series expansion of taken around

: th iteration approximate solution

: adjustments added for a better approximate solution

: total derivative of : linear function of adjustment vector

: Jacobian given by

adjustments added to the current to form the next :

14.3 The Exterior Orientation Problem

one-camera exterior orientation problem:

known:

- : set of 3D points with known positions,
- : corresponding set of 2D-perspective projections,

- rotation and translation that put the camera reference frame in the world reference frame

object pose estimation problem: unknown object position in camera frame

spatial resection problem in photogrammetries: 3D positions from 2D orientation

: point 's coordinate in camera reference frame

observed perspective projection

in the notation of the previous section,

14.3.1 Standard Solution

by chain rule for partial differentiation

in matrix form

=====p. 134=====

..... (tedious mathematics, study by yourself)

14.3.2 Auxiliary Solution

Instead of iteratively adjusting the angles directly, we can reorganize the
calculation such that we iteratively adjust the three auxiliary parameters of
a skew symmetric matrix associated with the rotation matrix.

Then we determine the adjustments of the angles in terms of the adjustments of
the parameters of the skew symmetric matrix associated with the rotation
matrix.

14.3.3 Quaternion Representation

quaternion: one other representation for rotation matrices

from any skew symmetric matrix

a rotation matrix R can be constructed by choosing scalar

this definition guarantees that :

expanding the equation for , we have

parameters and can be constrained to satisfy

in terms of quaternion parameters

so that

calculating the matrices directly, we obtain

..... (tedious mathematics, study by yourself)

=====Oldie 33:76=====

14.4 Relative Orientation

The transformation from one camera station to another can be represented by
a rotation and a translation.

The relation between the coordinates, and of a point can be
given by means of a rotation matrix and an offset vector.

=====Horn, *Robot Vision*, Fig. 13.3=====

Relative orientation is typically concerned with the determination of the
position and orientation of one photograph with respect to another, given a set
of corresponding image points.

: 3D positions of left and right camera lenses

: rotation angles
specifying exterior orientation

: set of image points from left image

: corresponding set of image points from right image

separation of two camera lenses : as constant controlling scale

relative orientation: specified by five parameters

assumption:

- camera interior orientation known
- image positions expressed to identical scale and w.r.t. principal point

14.4.1 Standard Solution

rotation matrix associated with exterior orientation of left image

rotation matrix associated with exterior orientation of right image

: distance between right image plane and right lens

: distance between left image plane and left lens

from perspective collinearity equation

Hence

where and

..... (tedious mathematics, study by yourself)

14.4.2 Quaternion Solution

Hinsken sets up the problem in a slightly different way.

Instead of determining the relative orientation of the right image with respect
to the left image, he aligns a reference frame having its -axis along the
line from the left image lens to the right image lens.

The relative orientation is then determined by the angles
, which rotate the right image into
this reference
frame, and the angles
, which rotate the
left image into this reference frame.

14.5 Interior Orientation

interior orientation: inner orientation: of a camera is specified by

- camera constant : distance between image plane and camera lens
- principal point : intersection of optic axis with image plane in measurement reference frame located on image plane
- geometric distortion characteristics of the lens; assuming isotropic around the principal point

14.6 Stereo

optical axes parallel to one another and perpendicular to baseline

simple camera geometry for stereo photography

=====Horn, *Robot Vision*, Fig. 13.1=====

stereo pair taken by Viking Lander I on the surface of Mars

=====Horn, *Robot Vision*, Fig. 13.2=====

parallax: displacement in perspective projection by position translation

: 3D point position

: perspective projection on left image of stereo pair

: perspective projection on right image of stereo pair

: baseline length in -axis

position of the left camera:

position of the right camera:

, so that the -parallax is zero

-parallax: the difference

Hence

=====Example 14.3=====

relation
close to being useless in real-world, because

- observed perspective projections are subject to measurement errors

so that for corresponding points - left and right camera frames may have slightly different orientations
- when two cameras used, almost always

14.7 2D-2D Absolute Orientation

The relationship between two coordinate systems is easy to find if we can
measure the coordinates of a number of points in both systems.

It takes three measurements to tie two coordinate systems together uniquely.

(a) A single measurement leaves three degrees of freedom of motion.

(b) A second measurement removes all but one degree of freedom.

(c) third measurement rigidly attaches two coordinate systems to each other.

=====Horn, *Robot Vision*, Fig. 13.4=====

2D-2D absolute orientation problem: 2D-2D pose detection problem:

to determine from matched points more precise estimate of rotation matrix
and translation such that
:

determine and that minimize weighted sum of residual errors:

weights : satisfy and

if no prior knowledge: can be defined to be equal,

..... (tedious mathematics, study by yourself)

14.8 3D-3D Absolute Orientation

: 3D points relative to camera reference frame

: 3D points relative to object model reference frame

we must determine rotation matrix and translation vector satisfying

constrained least-squares problem to minimize

The least-squares problem can be modeled by a mechanical system in which corresponding points in the two coordinate systems are attached to each other by means of springs.

The solution to the least-squares problem corresponds to the equilibrium position of the system, which minimizes the energy stored in the springs.

=====Horn,

Lagrange multipliers can be introduced and a solution found

..... (tedious mathematics, study by yourself)

=====joke=====

14.9 Robust M-Estimation

least-squares techniques are ideal when random data perturbations

or measurement errors are Gaussian distributed

this section describes some robust techniques for nonlinear regression

14.9.1 Modified Residual Method

: residual of -th datum

: difference between -th observation and its fitted value

standard least-squares method: tries to minimize

standard least-squares method: unstable if outliears present in data

outlying data: strong effect to distort estmated parameters

M-estimators: try to reduce outlier effect by replacing squared residuals

M-estimators: min

: symmetric, positive-definite function with unique minimum at zero

: chosen to be less increasing than square

=====p. 168=====

=====p. 169=====

14.9.2 Modified Weights Method

14.9.3 Experimental Results

14.10 Error Propagation

: input parameters

: random errors

quantity depends on input parameters through known function :

quantities observed and quantity calculated:

error propagation analysis: determines expected value and variance of

known information about : mean, variance:

14.10.1 Implicit Form

error propagation: can also be done in implicit form

known function has the form

The quantities are observed, and the quantity is determined to satisfy .

14.10.2 Implicit Form: General Case

general case: is not a scalar but a vector

: are vectors representing true values

: are vectors representing
noisy observed values

: random perturbations

: is a vector representing unknown true parameters

noiseless model:

with noisy observations, the idealized model:

14.11 Summary

we have shown how to

- take a nonlinear least-squares problem
- linearize it
- solve by iteratively solving successive linearized least-squares problems

Project due April 19:

camera calibration: interior orientation:

to determine principal point

do camera motion in z-axis, find focus of expansion/contraction

use lens of focal length: 16mm, 25mm, 50mm

2002-02-26