Colour, V4 & the two pathways
Why colour is an inference the brain has to un-multiply, how three cones and two opponent channels deliver it, what cortical damage takes away that retinal damage doesn't — and the double dissociation that splits vision into "what" and "where".
- Colour at Marr's three levels. Computational: finding food, finding mates, recognising objects. Algorithmic: perceived colour = illumination × surface reflectance, so the brain must factor a product. Implementation: cones → centre-surround retinal ganglion cells → V4.
- Illumination = the light source. Surface reflectance = which wavelengths a surface returns under white light — the property of the object itself. Perceived colour = the light that actually reaches the eye, which depends on both.
- Colour constancy: a surface looks roughly the same colour under changing light, because the brain estimates the illumination and divides it out. It must guess the illumination — one retinal signal, two unknowns. Adelson's checkerboard and the dress are the two canonical cases where the guess is exposed.
- Trichromacy: three cone types — S (short, ~420 nm, blue), M (medium, ~530 nm, green), L (long, ~560 nm, red). Perceived hue follows the ratio of their responses, so three numbers code the whole visible spectrum (400–700 nm).
- Colour blindness (colour-vision deficiency): loss or alteration of a cone type in the retina. Hues that differed only in the missing cone's contribution collapse together.
- Colour opponency: retinal ganglion cells are excited by one colour in the centre and inhibited by its opposite in the surround (red-vs-green, blue-vs-yellow channels). This codes colour contrast, not absolute wavelength. Colour adaptation — stare at purple, grey looks green — is the behavioural evidence.
- Cerebral achromatopsia: loss of colour vision from brain damage, commonly V4 (patient Mr. I, the colourblind painter). Cones are intact; the dissociation from retinal colour blindness is the exam point.
- Dorsal stream ("where", occipital → parietal): position of visual stimuli, visually guided action. Ventral stream ("what", occipital → temporal/IT): meaning and recognition. Full route: eye → LGN → V1 → V2 → V4 → IT.
- Double dissociation: patient DF (ventral/LOC lesion) → object agnosia, cannot recognise shape but posts a card through a slot accurately. Optic ataxia (dorsal lesion) → recognises objects but grasps inaccurately. Two patients with opposite profiles rule out "one task is just harder".
- Ventral specialisations: LOC = object shape (lesion → object agnosia), FFA = faces/fine discrimination, PPA = places (lesion → topographical agnosia), VWFA = written words. Dorsal lesions also give simultanagnosia — seeing only one object at a time.
1. Colour at Marr's three levels
Start with the thing that makes colour worth having. A ripe berry and an unripe one are the same shape and size; a peacock's tail carries no information at all in greyscale. Colour is how a visual system separates objects that geometry cannot separate — and the two uses the lecture names are the two that pay in evolutionary currency: finding food and finding mates.
Newton's notebook (1661) supplies the physics: white light decomposes into wavelengths, and the band we see runs from about 400 to 700 nm. But wavelength is not colour. Colour is what the brain builds, and the reason it's hard is the next line.
| Level | Colour perception |
|---|---|
| Computational (why?) | Finding food, finding mates, recognising objects — separating surfaces that differ in material but not in shape |
| Algorithmic (how?) | A light source illuminates a surface; the surface reflects some wavelengths. The light reaching the eye is the product of the two: perceived colour = illumination × reflectance. To recover the object's colour the brain must divide the illumination back out — colour constancy |
| Implementation (where?) | Cones (S, M, L) in the retina → centre-surround retinal ganglion cells with colour opponency → V4. The visual hierarchy now reads V1 edges, V2 surfaces, V1–V5 depth, V5/MT motion, V4 colour |
2. One equation, two unknowns
The retina measures only the product. Write it as a single number per channel and the problem becomes obvious: the eye receives \(r = i \times \rho\), where \(i\) is illumination and \(\rho\) is reflectance. The brain wants \(\rho\) — a berry's redness shouldn't change at dusk. It has one measurement and two unknowns. Every colour illusion on this page is that arithmetic refusing to have a unique answer.
--ink at varying opacity, so they track the theme.3. Colour constancy, and the two canonical illusions
Constancy is a feature, and the illusions are the feature being caught in the act. Both canonical examples work the same way: the retinal signal is fixed, the assumed illumination differs across two regions, so the inferred reflectance differs — and inferred reflectance is what you consciously see.
Adelson's checkerboard
A checkerboard sits under a cylinder that casts a shadow. Square A is on the lit part of the board; square B is inside the shadow. A and B send identical light to your eye — identical pixel values. They look nothing alike: B looks clearly lighter.
The brain's reasoning, written out: the light from B equals the light from A; but B is in shadow, so B is being illuminated less; to return the same amount of light under less illumination, B's surface must be a lighter colour. That inference is correct physics applied to a trick photograph. Divide a fixed numerator by a smaller denominator and the quotient goes up.
The dress
The 2015 photograph that split the internet. Some people see white and gold, others blue and black, from the same pixels. The photograph is badly exposed and gives almost no reliable cue to the light source — so the illumination term is genuinely ambiguous, and different viewers plug in different values.
- Assume the dress is in bluish shade / indoor light → the brain subtracts blue from the image → what's left reads white and gold.
- Assume it is brightly lit, lots of yellow light → the brain subtracts yellow → what's left reads blue and black.
Nobody is seeing it wrong. Two observers running the same correct algorithm on the same data with different priors about the lighting must reach different answers — that is what "illumination is ambiguous" means, and it's the sentence to write on the exam.
Two matte cards, photographed in one scene. Both send the same amount of light to the camera: a value of 0.27 on a scale where 1.0 is the brightest possible. Card A sits in a shadow where the illumination is 0.30. Card B sits in open light where the illumination is 0.90.
(a) What reflectance does each card have? (b) Which card is physically the lighter-coloured card? (c) An observer shown the photograph with the shadow cropped out reports that the two cards look the same. Are they wrong?
Step 1 — What are we actually being asked?
Not "which is brighter in the image" — that's given and identical. We're asked to invert perceived colour = illumination × reflectance for the surface property. Three quantities, and the question moves between them: the retinal signal (0.27, shared), the illumination (differs), the reflectance (to find).
Step 2 — Do the division
Reflectance = signal ÷ illumination. A: 0.27 ÷ 0.30 = 0.90. B: 0.27 ÷ 0.90 = 0.30. Multiply back: 0.90 × 0.30 = 0.27 and 0.30 × 0.90 = 0.27. Both reproduce the measurement, which is the whole difficulty.
Answer & the takeaway
(a) A is 0.90, B is 0.30. (b) A — reflectance 0.90 is a near-white card, B at 0.30 is dark grey. (c) The observer isn't wrong about the image; the patches really are identical there. Cropping removed the cue they needed to estimate illumination, so they can't run the division and fall back on reporting the raw signal. Constancy didn't fail — its input was removed.
The confusion this is built around: treating "the colour hitting your eye" and "the colour of the surface" as one quantity. They're a product and a factor, and the checkerboard, the dress and achromatopsia's spared cones all turn on keeping them apart.
4. Trichromacy: three numbers for a whole spectrum
In the 1850s Helmholtz asked people to match colours using adjustable lights. With two lights they could not match everything; with three they could match everything; four added nothing but difficulty. That is a behavioural measurement of a biological fact — the retina has three cone types — made decades before anyone could see them.
Colour addition follows directly. If hue is a ratio across three channels, then any ratio can be produced by mixing three suitably chosen lights — which is exactly what the screen you are reading this on does. A television has no yellow pixels; it drives your L and M cones in the ratio that 580 nm light would produce, and you see yellow. You are not seeing the wavelength. You are seeing the ratio.
Colour blindness
Colour blindness — more precisely a colour-vision deficiency — comes from losing or altering a cone type. Drop the L cones and every pair of wavelengths that differed only in how much more they excited L than M becomes indistinguishable: red and green collapse. The affected person is not seeing grey everywhere; they are seeing a world with one fewer dimension, and hues that were far apart for you land on the same point for them.
A participant is shown pairs of single-wavelength lights and asked whether they look the same or different. Their results:
- 530 nm vs 620 nm — "the same"
- 440 nm vs 530 nm — "different"
- 440 nm vs 620 nm — "different"
(a) Which cone class is most likely absent? (b) Is this deficit retinal or cortical, and what single extra test would tell you? (c) Why does the participant still see a difference in the other two pairs?
Step 1 — What are we actually being asked?
To reason from a pattern of confusions back to a missing channel. The useful move is to ask, for each pair, which cone comparison normally distinguishes those two wavelengths — then find the comparison that the "same" judgement has knocked out. Part (b) is a different question entirely: not which channel, but at which stage of the pathway.
Step 2 — Work out which comparison is dead
530 nm and 620 nm are both long enough that S cones barely respond to either (0.007 and essentially 0 in the figure above). What normally separates them is the L:M ratio — 0.88 versus 2.31. If those two look identical, the L-versus-M comparison has been lost, so either L or M is missing. The pairs involving 440 nm are still discriminable, which tells us S is intact and is doing that work.
Answer & the takeaway
(a) L or M — the red–green channel (losing L is protanopia, losing M deuteranopia); the data as given don't distinguish which. (b) Retinal. The signature is that the confusions are selective and probably lifelong: specific pairs collapse, everything else is normal. The one extra test: ask whether colour was lost suddenly and whether all hue is gone, including in memory and imagery — acquired, total loss with intact cones means cortex (next section). (c) S cones are intact, so any short-versus-long comparison still works and 440 nm still differs from both.
The transferable point: a missing cone removes a dimension of colour space, not colour itself, and which pairs collapse tells you which dimension went.
5. Colour opponency in retinal ganglion cells
Cones hand on to retinal ganglion cells, and those cells do not simply relay cone activity. They are centre-surround — the same architecture that gives luminance contrast in Chapter 10 — but wired across cone types. A cell might be excited by red in its centre and inhibited by green in its surround. Its output no longer says "there is red light here"; it says "there is more red here than around here".
The lecture's exit ticket, answered
The deck asks: why might a grey background appear green after you stare at a pink or purple image? Try it, then read the mechanism.
--bg-soft grey, the grey takes on a faint greenish cast for a few seconds — the
complement of what you adapted to. The bars on the right are the mechanism: two opponent populations whose
balance sets the percept. The adapting patch drives the violet side hard, that side fatigues, and when
neutral grey arrives — which should drive both sides equally — the unfatigued side wins. The effect is
deliberately subtle here because the page uses only its own palette (--accent-3 violet,
--accent-2 teal) rather than saturated primaries; the slides' purple-and-green demo is the stronger
version of the same thing.6. Cerebral achromatopsia
Set the two disorders side by side, because the contrast is the question:
| Retinal colour blindness | Cerebral achromatopsia | |
|---|---|---|
| Damage | Cone photoreceptors, in the retina | Cortex, commonly V4 |
| Onset | Usually congenital, lifelong | Acquired — a stroke or injury, with a before and after |
| What is lost | One dimension: specific hue pairs collapse (e.g. red–green) | Colour as a whole — the scene goes grey |
| Cones | Missing or altered | Intact and responding normally |
| Colour in memory, dreams, imagery | Was never there to lose | Can also be lost — Mr. I could not even imagine colour |
Why this matters beyond terminology: Mr. I's cones are fine, so trichromatic machinery alone is not sufficient for colour experience — something downstream must be intact too, and the lesion site says where. That is Chapter 5's logic, a selective loss localising a function, and it is the lecture's strongest evidence that V4 does the job.
7. Dorsal and ventral: "where" and "what"
Past V4 the hierarchy stops being a single ladder and splits. The same V1 output feeds two streams that ask different questions about the same image.
Occipital → parietal. Codes the position of visual stimuli and the spatial properties an action needs: where a thing is, how it is oriented, how to reach it. Vision for action.
Occipital → temporal (inferior temporal cortex, IT). Codes the meaning of visual stimuli: identity, category, recognition. Vision for perception.
The first evidence was monkey lesion studies: parietal lesions impaired tasks requiring the spatial location of a stimulus, while temporal lesions impaired tasks requiring which object it was. Human patient work made it sharper — next section.
What the ventral stream is made of
IT is a hierarchy of feature detectors with category-selective regions sitting at the top. These recur across the course, so learn them as a set with their lesion syndromes:
| Region | Prefers | Lesion gives |
|---|---|---|
| LOC — lateral occipital complex | Object shape; responds more to intact than scrambled objects, whereas early visual cortex responds about equally | Object agnosia |
| FFA — fusiform face area | Faces — or, on Tarr's account, fine-grained discrimination among similar-looking things, which is why it also responds in car, bird and greeble experts | Impaired face recognition |
| PPA — parahippocampal place area | Scenes and the geometry of surrounding space | Topographical agnosia — cannot recognise familiar places and landmarks |
| VWFA — visual word form area | Written words in the reader's own script | Impaired word reading |
VWFA carries an extra idea worth having ready. Writing is a few thousand years old — far too recent for a dedicated cortical area to have evolved for it. Neuronal recycling: circuits that evolved for one function get recruited for a newer cultural skill. The deck's other example is arithmetic, where ECoG electrodes over parietal cortex — the dorsal "where" stream — show strong activity during number processing. Spatial circuitry, repurposed for maths.
Dorsal lesions have a second syndrome besides optic ataxia: simultanagnosia, the inability to see more than one object at a time. Note how it differs from agnosia — each object is recognised fine; what fails is holding several in a spatial layout at once, which is a "where" problem, not a "what" problem.
8. The double dissociation, done properly
This is the reasoning pattern the course keeps returning to, and this is the cleanest instance of it. The general machinery is in the dissociations chapter; here is the vision version.
Patient DF — the ventral half
DF has a lesion in the ventral visual pathway (LOC). Her profile:
- Cannot identify line drawings, and cannot copy them.
- Can draw objects from memory — so the knowledge of what objects look like is intact.
- Cannot recognise the object she herself drew moments later.
That is object agnosia: an inability to recognise the shape of objects, with vision, memory and intelligence otherwise working. Drawing from memory but failing to recognise her own drawing rules out a loss of object knowledge and localises the failure to getting from image to identity.
Now the part that makes DF famous. Ask her to judge the orientation of a slot — a perceptual report — and she is at chance. Hand her a card and ask her to post it through the slot, and her hand rotates to the correct orientation on the way. Same slot, same visual information, two uses: visually guided action intact, perceptual judgement not. The lecture's clarinet patient shows the same split — he can't recognise it, but he picks it up and handles it correctly.
- Task difficulty. Maybe the perceptual judgement is simply harder than the posting action. Any brain injury hurts hard tasks first, so one patient failing the harder of two tasks tells you nothing about which systems exist.
- Ceiling and floor effects. If the posting task is easy enough that everyone succeeds, DF's "intact" performance may be the task's ceiling hiding a real impairment. The measure has to have room to show a deficit.
- General effects of brain injury. DF's lesion is not a scalpel. Fatigue, attention, slowed processing and the specific extent of her damage all offer alternative explanations for one failing task that have nothing to do with a "what" pathway.
Optic ataxia — the dorsal half
Put the two patients in a table and the argument completes itself:
| Patient | Lesion | Recognise the object ("what") | Grasp / post it ("where") |
|---|---|---|---|
| DF | Ventral (LOC) | Impaired | Intact |
| Optic ataxia | Dorsal (parietal) | Intact | Impaired |
You have two patient groups — Group V with ventral occipitotemporal lesions, Group D with dorsal parietal lesions — plus matched controls. You want to establish that visual recognition and visually guided action are separable.
(a) Name two tasks that use identical visual input but differ in which process they demand. (b) State the exact pattern of results that would support a double dissociation. (c) Both groups turn out to be impaired on both tasks. What explanations remain, and how would you change the experiment?
Step 1 — What are we actually being asked?
Not "what do the streams do" — that's the hypothesis. We're asked to design a test of it, which means the stimulus must be held constant while the demanded process varies. If the two tasks use different stimuli, any difference between them could be a stimulus effect and the design is dead before data collection.
Step 2 — Pick the two tasks, and the pattern you need
The slot-and-card pair is the model. Task 1 (recognition): report the slot's orientation, verbally or by rotating a handheld card to match without reaching. Task 2 (action): post the card through it, scored by hand orientation at insertion. Identical slot, lighting and viewing time; orientation varied trial to trial so neither task is solvable from memory.
The supporting pattern is crossed. Group V: impaired on the report, at control level on posting. Group D: at control level on the report, impaired on posting. Controls off both floor and ceiling on both tasks, so the measure has room to show impairment either way. A double dissociation is a crossover — not just a group difference, but each group failing the task the other handles.
Answer & the takeaway
(c) If both groups fail both tasks you have no dissociation, and three alternatives stay live: both tasks too hard (a floor effect), lesions large enough to hit both streams, or general injury effects swamping any selective deficit. Fixes: make both tasks easier until controls are comfortably off the floor; check lesion extent on imaging and exclude participants whose damage crosses both streams; add a task neither stream should affect as a general-impairment control; titrate both tasks to the same control accuracy before comparing.
The step students skip: a double dissociation is not "two patients" and not "two tasks" — it is the crossing, because no single difficulty dimension can produce a crossing. Carry that test to the aphasias, to rhythm versus melody, and to anything else in this course claiming two separable processes.