The summer vision project is an attempt to use our summer workers effectively in the construction of a significant part of a visual system. The particular task was chosen partly because it can be segmented into sub-problems which will allow individuals to work independently and yet participate in the construction of a system complex enough to be a real landmark in the development of “pattern recognition”.
The basic structure is fixed for the first phase of work extending to some point in July. Everyone is invited to contribute to the discussion of the second phase. Sussman is coordinator of “Vision Project” meetings and should be consulted by anyone who wishes to participate.
Goals — General
The primary goal of the project is to construct a system of programs which will divide a vidisector picture into regions such as likely objects, likely background areas, and chaos. We shall call this part of its operation FIGURE-GROUND analysis. It will be impossible to do this without considerable analysis of shape and surface properties, so FIGURE-GROUND analysis is really inseparable in practice from the second goal which is REGION DESCRIPTION. The final goal is OBJECT IDENTIFICATION which will actually name objects by matching them with a vocabulary of known objects.
Goals — Specific
We plan to work by getting a simple form of the system going as soon as possible and then elaborating upon it. To keep the work reasonably coordinated there is a graduated scale of subgoals.
Subgoal for July
Analysis of scenes consisting of non-overlapping objects from the following set: balls; bricks with faces of the same or different colors or textures; cylinders. Each face will be of uniform and distinct color and/or texture. Background will be homogeneous.
Extensions for August
The first priority will be to handle objects of the same sort but with complex surfaces and backgrounds, e.g. cigarette pack with writing and bands of different color, or a cylindrical battery. Then extend class of objects to objects like tools, cups, etc.
Structure of System and Assignment of Responsibility
(a) Quality Functions — Speciner, in connection with Gosper
An original feature of this program is that it thinks in terms of surfaces rather than lines or contours. So the starting point is a set of functions for surface properties.
- Point properties: Three light intensities produced by the vidisector looking through color filters; color functions, e.g. ratios of the intensity functions.
- Local properties: Texture functions; integrals and derivatives of point properties.
(b) Quality Distribution Analysis — Simpson
Programs to select quality functions appropriate to the given scene.
(c) Preliminary Region Proposal — Sussman
Finds connected regions with “constant” surface properties.
(d) Symbolic Picture Model — Lamport
Keeps account of which parts of the picture have been investigated and what regions have been found. Calls (c) until satisfied.
(e) File of Property Lists of Regions — Sussman and Lamport
Parts (c) and (d) end up by proposing a set of regions and establishing a property list for each region. At this point the property list will contain at least the following information:
- Surface quality
- Size
- Size of boundary
- List of boundary points
- x-y rectangle
- Moments
- Contiguous regions
(f) Construction of Principal Boundary — Sussman, in consultation with Greenblatt and Gosper
The program now looks at a fine raster version of the x-y rectangle for each region and produces an ordered list of points on what it thinks is the outer boundary of the region. In later versions it will do the same for inner boundaries (holes).
(g) Curve Properties — White, Geffner, Simpson
Given a list of points, find properties (or measures) such as convex, linear, elliptical segment, smooth (no corners) etc.
(h) Curve Segmentation — White, Geffner, Simpson
The faces of a cylinder might be described as:
- Face 1: Convex, smooth.
- Face 2: 4 segments; line, convex smooth, line, concave smooth. 1 and 3 parallel, 2 and 4 parallel, etc.
This information will be added to the property list.
(i) Global Surface Properties — Benjamin, Speciner
- Elementary examples: shading, highlight detection, local texture gradients.
- Advanced examples: global texture, surface shape, orientation, etc.
(j) Object Identification and High Level Executive — Henneman, Guzman, Sussman
The general form of the system is dictated by a desire for maximal flexibility in the addition of “heuristic” rules. The current proposal is to have a list of heuristic rules of the form PATTERN-ACTION. The Pattern is matched against a property list of a region. The Action results in a (usually null) modification of the property list using a simple boolean rule or a complex CONVERT program calling any of the functions of the system. Particular objects or classes of objects are found by using models like the description of the cylinder in (h) as “patterns”.