XX. ARTIFICIAL INTELLIGENCE
Massachusetts Institute of Technology. Research Laboratory of Electronics. Quarterly Progress Report, no. 73
(1964)
Authors: (scroll to bottom for biographies)
| Prof. M. L. Minsky | Prof. H. Putnam | Prof. A. L. Samuel |
| Prof. C. E. Shannon | Dr. S. Papert | D. G. Bobrow |
| D. J. Edwards | T. P. Hart | M. I. Levin |
| D. C. Luckham | J. Moses | L. Norton |
A. STEREOSCOPIC SYNTHESIS AS A TECHNIQUE FOR LOCALIZING VISUAL MECHANISMS*
In the eye or in the brain? Localization questions of this sort can become dangerous verbal traps or the cornerstones of too simple models of neurological function. Nevertheless, they have a precise meaning if handled with sufficient care. For instance, the afterimages produced by bright lights can be attributed meaningfully to processes localized peripherally and can even be split into a component involving changes in the photoreceptors and a neural component. The research reported here contributes to the analogous question for two other visual “aftereffects,” namely the aftereffect of viewed motion known as the “waterfall illusion” and the aftereffects of overexposure to particular geometrical forms known as the figural aftereffects, which have been studied systematically by Gibson, Köhler and Wallach, and many others. We use a technique that, in 1961, enabled us to show that optical illusions (like the Mueller-Lyer arrows) cannot be primarily explained by peripheral mechanisms.1 Julia Hochberg at Cornell University has recently corroborated this conclusion by an essentially similar experiment.2
To explain the techniques, let us first recall the principle of stereoscopic vision. In each of two squares imagine a rectangular coordinate system placed in the usual way and with its origin at the center. Denote by P_L(x, y) the point in the left square with coordinates (x, y), and by P_R(x, y) the point in the right square with the same coordinates. If y₁ = y₂ and x₂ = x₁ + Δ we say that P_L(x₁, y₁) and P_R(x₂, y₂), i.e., P_L(x₁, y₁) and P_R(x₁ + Δ, y₁), are a stereoscopic pair with disparity Δ. The upper points of the figure form a stereoscopic pair with nonzero disparity, while the lower pair has zero disparity. The basic principle of stereoscopy is illustrated by the results of viewing these two squares in such a way that they are projected onto corresponding locations of the two retinas of a subject. The subject then sees two points, not four, and these two points will appear to be at different distances from him. Most readers should be able to see this by “squinting” sufficiently to produce double images of the squares and to cross these until the rightmost image of the left square coincides with the leftmost image of the right square (alternatively, use a stereoscope). A hidden figure can be brought out by stereoscopic fusion. The five points (0, 0), (±1, ±1) are placed in both squares, so that they form a set, A, of five stereopairs of zero disparity. The remaining points generate a set, B, of stereopairs with disparity 1/8. The separation into A and B is not clearly visible when the squares are inspected monocularly, though a “logical and” operation would reveal it immediately. If the two squares are fused binocularly, the cross formed by the five A-points stands out of the plane of the B-points and is thus directly visible. The “brain” sees a figure that is invisible to each eye.
For the present experiments we need two extensions of this phenomenon. The first was established by Julesz of Bell Telephone Laboratories, Inc.3 Stereoscopic vision remains possible even if the number of points in such figures is increased enormously. We established the second in October 1963 in collaboration with Marvin Minsky and a PDP-1 computer. Suppose that the two squares are displayed on a cathode-ray tube. Instead of having a fixed set of points as in our figure and those used by Julesz, we continuously generate points at random and display each one only once in the left square as a brief flash. In the right square we simultaneously display points obtained by transposing these points with a disparity that depends on the location of the original point (e.g., Δ if it was in a region A, and zero in the complementary region B). With a little tidying up to ensure uniform dot density we achieve a situation in which each eye sees continuously random changing noise, while stereoscopic fusion reveals the form of the hidden figure; the information is entirely contained in the correlation between two individually random signals. Although this finding has interesting implications for the theory of stereoscopic vision in general, our present interest is confined to using it as a tool for the investigation of other phenomena.
The first of these is the waterfall illusion. A subject views a textured surface (the inducing stimulus) which moves in a fixed direction as he looks at it during an interval of time T₀ (the induction time). The surface then stops moving or the subject looks away from it at a motionless surface (the test stimulus). If T₀ is large enough (of the order of seconds) the test stimulus appears to move in the opposite direction. The duration of this “aftereffect” (AE) is not precisely defined, since it fades out gradually, but it would be pedantic in the present context to quibble at the use of a measure T₁ (the AE time) of how long it takes for the AE to become imperceptible according to some criterion. The AE time is an increasing function of the induction time, and can easily be driven up to ten or fifteen seconds by induction times of a few minutes—the exact function obviously depends on the criterion used to define the AE time, but for our present purposes a qualitative statement is sufficient. What kind of mechanism produces this AE? The question does not have a simple unique answer. For example, eye movements induced by the induction stimulus play a role. But the role of this component can be minimized by displaying an induction stimulus with movement simultaneously in more than one direction. (With arrows showing direction of movement of vertical bars.) With these induction stimuli each “movement” induces its own AE, which cannot all be due to eye movements. So, we can split the AE into two components: a generalized one applying to the field as a whole, as would the effects of eye movements, and a localized one applying to that part of the visual field directly affected by the induction stimulus. We are interested only in the second here. Is it due to action on the retina, the brain, both or neither?
The PDP-1 computer has a large CRT on which points can be displayed in any position at a rate of 50 µsec per point. It is therefore easy to make random stereoscopic displays in real time so that the subject sees a pattern of continuously moving bars. Under these conditions we do see movement aftereffects. But further investigation shows how dangerous it is to give binary answers to this kind of question. This aftereffect of centrally produced movement differs from that induced by real movement on the retina in the following ways:
(i) The AE time with centrally produced inducing motion is very much shorter. Times of more than a second are rare with induction times (e.g., 30 seconds) that regularly give rise to long AE’s with real movement.
(ii) In particular the AE time is not a sensitive function of induction time; in fact, it looks more like a step function.
(iii) With real induction movement all subjects always report the AE. With centrally produced inducing motion the AE is sporadic; some subjects rarely see it, few always do.
(iv) The qualitative effect is very different. This is hard to measure or describe. Subjects often complain that the AE produced by real movement is unpleasant, produces a “sinking feeling,” and so forth. Not so in the other case.
We must therefore conclude that the AE has a dominant component induced only by stimuli involving a real movement on the retina. We note, though not without hesitation, that the characteristics of the real movement AE are what would be expected of a simple cellular adaptation, fatigue or exhaustion at a peripheral level, while the stereoscopically induced effect has the kind of property that would be associated with more complex brain processes.
The induction of the movement AE by “apparent” movements produced monocularly gives rise to a similar split of the nature of the AE, though intermediate cases are possible, and it is hard to draw a sharp line. A typical experiment continuously generates random points in a band, but displays them only if they fall in the AAA region. The boundary between AAA and BBB is then made to depend on time, so that the star-speckled A bands (of course, without the vertical boundary lines of the figure) are seen to move across the dark B background. With suitable speeds and dot density, subjects see a very beautiful and convincing apparent motion. The AE induced by it, however, is more like the stereo AE than the real motion AE. Thus the statement (which we have often heard from psychologists) that “phi movement produces a movement aftereffect” is true but misleading.
However debatable this discussion might be, the fact remains that the waterfall illusion is clearly distinguished by our experiments from geometrical illusions such as the Müller-Lyer illusion whose characteristics do not appear to change under stereo conditions. We turn therefore to a case that might appear to be intermediate: the figural aftereffects. The best known of these is to be read temporally as the frames of a comic strip: the subject sees successively the contents of frame 1 for t₁ seconds, frame 2 for t₂ seconds, … projected onto the same retinal area. (Note that these are not stereopairs but successive stimuli.) The illusion appears during the test time t₃: the vertical line appears tilted in the direction shown by the arrow, i.e., in the sense opposed to the tilt of the inducing stimulus. The extent of the effect can be measured by the usual psychophysical techniques. We systematically use a randomized constant method in these experiments: in each repetition of the cycle we present during t₃ a randomly chosen member of a set of lines with different slopes and force the subject to say whether he sees it as tilted left or right. After 20 repetitions the sense of rotation of the inducing line is reversed and so the experiment continues until a sufficient number of observations have been made. In almost every case the difference between a test after a left-sloping inducer is significantly different from both tests after a right-sloping inducer and without any inducer. We have, however, passed three subjects through 5 repetitions of this cycle for the experiment as described, and for the corresponding experiment with vertical comparison lines. The results are sufficiently clear qualitatively for any calculation of statistical significance to be only of anecdotal interest. The extent of the aftereffect varies from subject to subject: 1.5°–2° in the normal range for good subjects under our conditions.
In the stereoexperiments the entire sequence of events is programmed onto the computer which also records the subject’s responses. The sloping lines are pure edges without thickness which separate a region A of the field from a region B whose points have the correct disparity to cause them to appear in a separate plane. During the time t₂, no tilted line is visible, but the randomly starry background remains.
The experiments consist of varying the times t₁–t₄ and measuring the AE for each combination under the two conditions: real-line and stereo. We have found no systematic and significant difference. Thus the figural aftereffect, which is like the waterfall insofar as it is an aftereffect and like the Müller-Lyer insofar as it is geometrical, is more like the latter in its resistance to stereoscopic presentation. It is therefore certainly not primarily of retinal or near retinal origin.
In conclusion we might note that these two findings are highly relevant to the discussion of a family of explanations of AE’s and illusions by models based on postulating analogies with the findings of Lettvin, Maturana, McCulloch, Pitts, and others on frogs’ eyes, and of Hubel and Wiesel on cats. The movement AE might be simply explicable in some such fashion, but our findings exclude all of the simple and obvious models of this sort for figural aftereffects and illusions.
We have a routine research program in progress to check systematically all known illusions and aftereffects under conditions of stereoscopic synthesis of figures. Thus far, we can assert on the basis of qualitative findings that (as one might expect) spiral and expanding-circle AE’s behave as the waterfall illusion, concentric circle AE’s and the Delboeuf illusion behave as the sloping line AE and the Müller-Lyer illusion. The horizontal-vertical illusion is a particularly interesting case because it may contain a larger retinal component than the others; but our data are not yet sufficient to settle this problem.
S. Papert
References
1. S. Papert, Nature 191, 733 (1961).
2. Julia Hochberg, paper presented at the Psychonomics Meeting, Philadelphia, September 1963.
3. B. Julesz, Bell System Tech. J. 39, 1125-1162 (1960).
B. REGULARITIES IN THE TIME COURSES OF SOME VISUAL PROCESSES
We are interested in using quantitative features of interactive visual phenomena (this concept generalizes the illusions, the figural aftereffects, and so forth) to gain insights into the information-processing systems used in vision. To obtain a foothold, it is essential to have techniques to segregate the relatively peripheral and “simple” processes from nearly inextricable complex “higher order” events. We want measures relative to form perceptions that are as quantifiable and repeatable as the curves on which color theory was based. In the experiments reported here we draw attention to a neglected variable of these phenomena—their time constants—and try to score a methodological point: In addition to the essentially simple and essentially complex there is the quasi-complexity caused by the interaction of very few very simple processes. The time curves are a case in point; a proper dissection of the phenomena shows them to be simple and accessible.
By the time curve of an illusion we mean the curve obtained by measuring the size, I, of the deformation for different exposure times, t, of the illusion figure and plotting I against t. Workers in Geneva, Switzerland, (J. Piaget, Vinh Bang, B. Matalon) have published a number of such curves for different illusions and for different sizes of subjects. These curves seem to fall into two classes.
(i) For certain illusions, as t increases, I first increases to a maximum for some value t₀ and then decreases to an asymptotic level that may have the opposite sign. For adults t₀ has a value near 0.2 second; for children, t₀ is larger. We shall call these the regular illusions.
(ii) For other illusions the curve has a more complex form; we shall call these irregular.
Although highly suggestive, these results needed some verification on account of methodological considerations of which the chief is that they are based (as is usual in much contemporary psychological research) on averages over many subjects; in fact, in many cases, different subjects were used for different values of t. It is therefore impossible to say whether the form of the curve represents the characteristics of perceptual mechanisms or merely the distribution of types of reaction in the population. In all of our experiments we try to obtain enough observations from each subject to establish a significant curve for him; if necessary, we ask him to come back repeatedly over a period of weeks or months to avoid or to control interactions, massing effects, learning, etc. The results obtained by this method are the following. First, the regular form is found for many illusions on a subject-by-subject basis. Second, the published curves for the irregular illusions are not generally repeatable under these conditions. To illustrate the issues involved we shall single out an example of each type.
Our example of a regular illusion is one whose time curve has not, as far as we know, been previously studied but which we chose to examine in detail because we happen to know a great deal about it in other respects. The illusion figure consists of a subject who has to say whether the nearly vertical line slopes to the left or to the right. The line AB is at a fixed angle, 10°, from the vertical. The deviation of CD from the vertical is varied on different presentations between 3° clockwise and 3° anticlockwise. Out of 10 adult subjects, 6 gave curves with clear maxima, between 0.1 and 0.2 second, 2 gave curves with minor irregularities, and 2 curves that were completely aberrant with respect to the other 8 (e.g., one had a minimum!). We studied only one child, 7 years old, who gave a curve with a maximum at 0.5 second.
The curve that rises and falls suggests the additive behavior of two antagonistic processes, and some rather complicated hypotheses of this sort have been advanced. There is, however, a very simple hypothesis which should (following William of Occam) have prior claim. This is that the “contrast” or “repulsion” effect of AB on CD builds up, or spreads out, reaching nearly its asymptotic value in the course of at most a few tenths of a second, but since it is locally tied into the visual field, it is reduced by the first saccadic eye movement. The time position of the fall in the curve is consistent with this, as is the delayed maximum in children whose eye movements are less frequent. To check this hypothesis, we measured the illusion in a full-field grating format, for which the eye movement should be expected to have less or no effect. And indeed the maximum value of I for the single-line figure comes close to the value of I for free inspection of the full grating.
We turn next to one of the most difficult irregular illusions to see whether more order appears there when the role of eye movement is taken into account. In the Delboeuf illusion, the inner of the two concentrics appears larger than the free measuring circle when they are in fact objectively equal. The time curve published by Piaget, Bang, and Matalon for this illusion is rather complex; its form is irrelevant here because in fact we were unable to re-establish it for any individual subject, and even the average curve for our group resembles it only very vaguely. In fact, out of a dozen subjects we scarcely even found the same curve twice. Why? Again a simple hypothesis suggests itself: The form of the figure is a powerful incitement to shifts of attention and of the point of fixation from left to right. Either of these factors is sufficient to affect the relative apparent sizes of the circles. So, we studied a symmetric form of the illusion; the measuring circle was projected onto a screen for a fixed time, followed by the two concentric circles or, as a control against order effects, the inner circle alone. The experiment gave a simple curve only slightly variable from subject to subject: The illusion builds up to a plateau over the range t = 0 second to t = 1 second, most of the increase being (as before) in the first few tenths of a second.
There is, then, reason to believe that the time courses of certain elementary phenomena follow simple regular laws and this regularity is evidence in particular cases for their elementary nature. But there are interactions and we conclude by showing how confusing they can be unless properly recognized. We shall call the “repulsion” effect of a sloping line on another the K-effect (K for Köhler). A phenomenon that has often been confused with the K-effect we shall call the G-effect (G for Gibson). Make a subject fixate a single tilted line. After 10 seconds introduce a second line to obtain a parallel pair. If the new line is objectively parallel, the subject sees the pair as diverging upward; the fixated line drifts toward the vertical during the fixation. Moreover, it rotates neighboring lines with it, thus simulating the K-effect as measured on a vertical test line seen after the sloping induction line (see Sec. XX-A). But the phenomena are distinct and can be put into opposition by exploiting the fact that the K-effect has a time constant of the order of tenths of a second, while that of the G-effect is very much longer. Thus if an induction figure with an opposing tilt is used with a parallel test figure, the distortion on the test is clockwise for induction times of fractions of a second (because the K-effect dominates), and anticlockwise for induction times of tens of seconds (because the G-effect dominates). It is thus easy to generate endless research on whether contours “always repel” or “sometimes attract.” This particular case has not come up, to our knowledge, in psychological literature, but many discrepancies in reported results can easily be reconciled as soon as the time dimension (usually neglected in work on figural AE’s) is taken into account.
S. Papert
References
1. J. Piaget, Les Mécanismes Perceptifs (Presse Universitaire de France, Paris, 1962).
* This work is supported in part by the National Science Foundation (Grant G-16526); in part by the National Institutes of Health (Grant MH-04737-03); and in part by the National Aeronautics and Space Administration (Grant NsG-496).
† The computer facility for this work was provided by Project MAC, an M. I. T. research program sponsored by the Advanced Research Projects Agency, Department of Defense, under Office of Naval Research Contract Number Nonr-4102(01).
Biographical notes about the authors added in 2026
- Prof. Marvin Lee Minsky (Co-founder of the MIT AI Laboratory, pioneer in artificial intelligence and cognitive science)
- Prof. Hilary Whitehall Putnam (Prominent philosopher of mind, language, and mathematics; professor of philosophy at MIT in the early 1960s)
- Prof. Arthur Lee Samuel (Pioneer in machine learning and computer game-playing, creator of the Samuel Checkers-playing program)
- Prof. Claude Elwood Shannon (Father of information theory, MIT professor and pioneer in digital computing and chess automata)
- Dr. Seymour Aubrey Papert (Co-founder of the MIT AI Laboratory, co-author of Perceptrons, creator of the Logo programming language)
- Daniel Gurewich Bobrow (Creator of the STUDENT natural language algebra problem-solving system for his 1964 MIT Ph.D.)
- Daniel James Edwards (Co-creator of Spacewar!, contributor to early LISP, CTSS, and ARPANET cryptography/security)
- Timothy Paige Hart (Co-author of the LISP 1.5 Programmer’s Manual and early LISP compiler implementations)
- Michael Ira Levin (Lead author of the LISP 1.5 Programmer’s Manual and LISP 2 designer)
- David Christopher Luckham (Pioneer in automated theorem proving, program verification, and complex event processing)
- Joel Moses (Creator of the SIN and Macsyma symbolic mathematics systems, later Provost of MIT)
- Lewis Mark Norton (Author of ADEPT, an early heuristic automated theorem-proving system for group theory at MIT)