Imaging & perturbing the brain, told through faces
Every method in the course — behavior, fMRI, EEG/MEG, ECoG, sEEG, patient studies, electrical stimulation — introduced by the question about faces it can answer, with the benefits and limitations you need to know cold.
1. Five questions about faces
The lecture organizes all the methods around one topic, and each question sits at a Marr level. Which method you need depends on what you're asking:
| # | Question | Marr level | Method → answer |
|---|---|---|---|
| 1 | Why is face perception important? | Computational | Communication, social interaction, emotion perception |
| 2 | How do humans represent faces? | Algorithmic | Behavior → orientation-specific templates |
| 3 | Is face perception distinct from the rest of vision? | Algorithmic | Behavior → face inversion effect; fMRI → FFA is face-selective |
| 4 | How fast are faces detected? | Algorithmic | EEG/MEG/ECoG/single neuron → 150–170 ms after the face appears |
| 5 | Does a brain area play a causal role? | Implementation | Patients + electrical stimulation → strong evidence FFA is causal |
2. Behavior: how are faces represented?
Two hypotheses: (a) we hold one orientation-invariant representation of each face, or (b) we memorize many orientation-specific views (which needs a lot of experience).
A collage shows many photos of Dutch politicians from different angles and lighting. Two groups count how many distinct people are shown: people familiar with Dutch politicians and people who aren't. What does each hypothesis predict?
Step 1 — What would "orientation-invariant" predict?
If everyone extracts one view-independent representation, experience shouldn't matter much — both groups equally good.
Step 2 — And "orientation-specific"?
If recognition needs stored templates for each view, the familiar group — who have seen these faces from many angles — should be much better.
Answer & the takeaway
Correct answer: 2 people. Unfamiliar viewers typically say 7. Face recognition benefits hugely from experience → more exposure = more orientation-specific templates = better recognition. The method lesson: behavioral experiments characterize representations by making the hypotheses predict different performance.
The face inversion effect
Turning an image upside down disrupts face recognition much more than object (e.g. house) recognition. That disproportionate cost suggests faces are processed by specialized mechanisms. Try it — one version of this face has been tampered with:
- Characterizes representations (orientation-specific vs invariant)
- Dissociates representations (face vs object)
- Cheap and low-tech
- Sparse, limited data (one response per trial)
- Hard to tell the fine-grained stages of processing
- No relation to the brain
3. fMRI: is there a face region?
Hypothesis: some place in the brain responds more to faces than to objects. Method: fMRI. It measures the BOLD (Blood Oxygenation Level–Dependent) signal — changes in the ratio of oxygenated to deoxygenated hemoglobin. Active neurons pull in oxygen-rich blood; the scanner sees the blood, not the neurons.
Finding: the fusiform face area (FFA) on the underside of the temporal lobe.
- Responds to faces more than anything else tested
- Found in a similar region in every subject
- Shows a face-inversion effect for Mooney faces (two-tone faces) — same stimuli upright vs inverted, different response, so it isn't low-level image features
The FFA lights up more for faces than for objects. List alternative explanations, then say how the lecture ruled them out.
Alternatives
(1) It responds to any body part. (2) It responds to anything curvy. (3) Faces are more interesting, so it responds to anything attended.
How they were ruled out
Show other body parts (hands), curvy non-face objects, and make people attend to everything equally with a 1-back task: watch a stream of stimuli and press when the current one matches the one immediately before. Now every stimulus is attended. FFA still prefers faces.
Takeaway
A difference between two conditions has many possible causes; good design adds control conditions that each kill one alternative. Attention confounds are killed by giving a task that forces equal attention.
- How BOLD relates to neural activity is unclear and indirect
- Low temporal resolution: lags neural activity by 2–3 s
- ~1 mm spatial resolution — can't resolve cortical layers (~0.5 mm)
- Susceptibility artifact near ears and sinuses
- Expensive (about 600 dollars an hour)
- Can't tell if a region plays a causal role
4. EEG, MEG, ECoG, sEEG: how fast?
Why not just use reaction times? Because an RT bundles too much: processing might have started earlier, and the RT also contains motor planning and decision making (e.g. confidence). You need to watch the brain directly, with millisecond precision.
| Method | What it records | Face result | Benefits | Limitations |
|---|---|---|---|---|
| EEG (electroencephalography) | 12–100+ scalp electrodes, each summing over lots of neurons | Face-specific response at 170 ms; can't tell if it's from FFA | Excellent temporal resolution; non-invasive | Lousy spatial resolution; disrupted by small movements; can't show causality |
| MEG (magnetoencephalography) | Magnetic fields from neural currents | — | Excellent temporal; non-invasive; slightly better spatial than EEG | Still lousy spatial; disrupted by Earth's magnetic field and movement |
| ECoG (electrocorticography) | Electrodes on the brain surface in neurosurgery (seizure) patients | Small (2 mm) patches at FFA are face-selective from ~150 ms | High spatial and temporal resolution | Rare; invasive; hard to control in hospital; covers one patch, not whole brain; lots of neurons per electrode |
| sEEG (stereo-EEG) | Depth electrodes that can pick up individual neurons | Single neurons show face processing at ~150 ms | Very high spatial and temporal resolution | Same as ECoG |
5. The resolution map
6. Causality: patients and stimulation
Every recording method above shares one limitation: it can't tell if a region plays a causal role. To test whether X causes Y, you have to mess with X and see whether Y breaks.
Patient studies
- Impairs face recognition, not face detection
- Voice recognition intact (can identify people that way)
- Object recognition intact
- Not memory? Tactile object recognition is normal → it's a visual agnosia.
- Not lost knowledge? Asked to define a pipe: "a short, hollow object, larger on one end, 120° angle, for leisurely smoking using tobacco."
- Hard to find patients with the "right" lesion
- Lesions are messy — affect multiple functions
- Chronic lesions: the brain reorganizes during recovery, so effects may reflect recovery rather than the lesion
- Not a direct manipulation — you can't turn the area off then on
Direct electrical stimulation
Neurosurgeons treating seizures need to find the seizure's origin and check whether it's safe to operate (e.g. preserve language). They electrically stimulate the cortex. ECoG electrodes can both record and stimulate — turning an area off, then back on. Stimulating FFA electrodes distorts the faces patients see, establishing a causal role for FFA.
- Can't tell if disruption is due to FFA specifically or FFA's connections with other areas
- Special patient group (seizure patients) who may have a propensity for the function you're studying