Surfaces & depth
How the brain turns edges into surfaces — figure–ground, border ownership, amodal and modal completion — and how it rebuilds a 3D world from flat retinal images using pictorial and non-pictorial depth cues.
1. Surface perception at Marr's three levels
V1 gives you edges. But the world is made of surfaces — the continuous areas of objects that define their shape — and objects overlap, hide each other, and sit under uneven light. The retinal image is a fragmented 2D mess.
| Level | Surface perception |
|---|---|
| Computational (why?) | Recover coherent surfaces and objects from a fragmented 2D image |
| Algorithmic (how?) | Assign borders to figure or ground; link separated regions by amodal and modal completion; interpret shading and lighting; form contextual predictions |
| Implementation (where?) | V1 encodes local edges. V2 integrates borders and represents real and illusory surfaces |
2. Figure–ground and border ownership
The lecture's other examples: Da Vinci's Last Supper (figure–ground separation in a painting) and a carved-wood puzzle where the borders can belong to the light wooden pieces (light wood in front) or to the dark deer shapes (deer in front, light wood becomes the table behind).
3. Amodal and modal completion
Objects get cut into pieces by whatever's in front of them. The brain has to glue the pieces back into one object — and it does it in two different ways.
When the predictions go wrong — and how knowledge steers them
Your brain is mostly making predictions, and sometimes gets them wrong: amodal completion gone wrong (a head "joined" to the wrong body behind an occluder) and modal completion gone wrong — the whole r/confusingperspective genre. Knowledge of objects and context refines the predictions: prime people with "young" or "old" and the same ambiguous drawing becomes a young woman or an old woman.
Where it happens: V1 edges → V2 surfaces
Neurophysiology: V1 cells respond to edges. V2 cells respond to real surfaces and illusory surfaces — a V2 cell fires for the edge of a Kanizsa-style illusory figure even though there is no luminance edge in its receptive field. That's why the implementation row reads "V1 edges, V2 surfaces".
4. Depth: rebuilding the third dimension
The retina is a 2D sheet. Depth is inferred. At Marr's levels:
| Level | Depth perception |
|---|---|
| Computational | Reach and grasp objects accurately, navigate safely, judge approaching objects |
| Algorithmic | Combine pictorial cues (occlusion, perspective, texture, shading…) with non-pictorial cues (binocular disparity, motion parallax, vergence, accommodation) |
| Implementation | V1–V5 are all involved in inferring depth |
5. The seven pictorial cues
| # | Cue | What it says |
|---|---|---|
| 1 | Occlusion | If A hides part of B, A is in front (Raphael's School of Athens) |
| 2 | Linear perspective | Parallel lines converge with distance to a vanishing point (trompe-l'œil murals — "deceive the eye") |
| 3 | Height in field | Assuming a flat ground, objects higher in the visual field are farther |
| 4 | Known size | If you know how big it is (a cat), its retinal size tells you its distance |
| 5 | Texture gradient | Assume texture elements are uniform; if they shrink, it's because they're farther |
| 6 | Atmospheric perspective | Distant surfaces are hazier and lower-contrast (the atmosphere imposes a haze) |
| 7 | Shadows | Ground objects to the surface; the gap between object and shadow gives the object's distance from the surface |
6. Size constancy and the Ames room
A man far away is small on your retina, but you don't see a small man. You compensate using distance:
$$\text{Perceived size} = \text{Size on retina} \times \text{Perceived distance}$$
The same equation runs in two directions — this is the part to get straight:
A 1.8 m person stands 10 m away, then walks back to 30 m. (a) How does their retinal size change? (b) With good distance cues, how big do they look? (c) In an Ames-style room that makes you think they're still at 10 m, how big do they look?
Step 1 — Retinal size
Retinal size ∝ size / distance: 1.8/10 = 0.18 → 1.8/30 = 0.06. It shrinks to one third.
Step 2 — With correct distance
Perceived size = 0.06 × 30 = 1.8 m. Size constancy: they look the same height.
Step 3 — With distance misjudged as 10 m
Perceived size = 0.06 × 10 = 0.6 m — a third of their real height.
The takeaway
Retinal size alone never determines perceived size. The equation's "perceived distance" term is the lever the Ames room pulls: the room's perspective and height-in-field cues are rigged to agree on one wrong distance, and they outvote your knowledge that people don't shrink.
7. Non-pictorial cues
| Cue | How it works |
|---|---|
| Motion parallax | As you move, near objects sweep past faster than far ones. A praying mantis sways its head side to side to judge distance to prey. (Ch. 12) |
| Accommodation | The lens changes shape to focus; the visual system senses how strained the lens is and translates that into distance |
| Vergence | Eyes rotate toward each other (convergence) for near things, apart (divergence) for far — the angle gives distance |
| Binocular disparity | The two eyes see slightly different images; the difference in separation → depth. Using it is stereopsis. Wheatstone's mirror stereoscope (1832) fakes 3D by showing each eye its own image |
Where: binocular disparity is processed across multiple visual areas, from V1 to V5/MT.