Machine Psychology

The lecture opens with a shared-ground primer — what is a language model: a network trained to predict the next token, the tokeniser, the transformer's attention mechanism, and a first pass over the training pipeline (pretraining, supervised fine-tuning, RLHF, and increasingly RLVR). It then states the paradox: we hold every weight and still cannot account for the competence, because the interesting capacities emerged. It distinguishes the two research programmes — mechanistic interpretability (look inside; currently limited to narrow circuits) and behavioural study (input–output relations) — and places the enterprise in its lineage from Simon through Rahwan to Hagendorff. The load-bearing distinction is between a benchmark (how well a system performs) and a psychological experiment (designed so that different underlying processes yield different behaviours). It closes on a first result: Binz and Schulz on GPT-3, and biases that appeared in one model generation and vanished in the next.

The lecture begins from the claim that cognitive science, its self-image notwithstanding, is at root a methodological behaviourism: it infers latent processes that best explain public behaviour, always open to correction. It anchors this in Turing's 1950 imitation game — the behavioural inference principle is at once Turing's heir and his correction — and introduces the explanandum/explanans distinction to police the confusion between behaviour and the constructs that explain it. The inferential machinery is built gently as a conceptual Bayes rule: the probability of a process given a behaviour rises with how specifically the process predicts it (likelihood), is weighted by how plausible the process is in this kind of system (prior), and is discounted by how easily the behaviour could arise otherwise (marginal). This single expression explains why identical behaviour warrants different conclusions in a human, a monkey and a model, and makes the mimicry problem precise. The lecture separates behavioural equivalence, computational equivalence and the behavioural inference principle; meets the Chinese Room, Blockhead and zombie objections from within an inductive framework; and turns the lens on the interpreter, naming the biases that distort both inflationary and deflationary readings.

The lecture reactivates the primer from Lecture 1 and draws out its consequences: each training stage shapes behaviour and priors differently, which is why the mechanics matter for design. It introduces the Umwelt of the model — von Uexküll's term for the world as a creature perceives and acts on it — as the organising idea: the model's world is a stream of token identifiers, so tasks built on human sensory experience do not translate cleanly. It then works through the design problems: modality (converting a perceptual paradigm into text without giving the answer away), contamination (the model may have seen the test in training, so one measures retrieval rather than cognition — with remedies including reframing, procedural generation and black-box detection), and prompt variability and stochasticity (small wording changes and sampling both move the output, requiring paraphrase batteries and repeated administration). This yields the reconceived notion of subject: a model is better treated as a population of participants than as one individual, which reshapes the statistics.

The lecture returns to the Bayesian compass from Lecture 2 to make the central interpretive point precise: the likelihood term is system-independent, but the prior is not, because a network trained on text need not develop the mechanisms evolution installed in brains. So similar behaviour need not mean similar cognition; behavioural correspondence is the beginning of an inference, not its conclusion. It develops the specific hazards — anthropomorphism (importing human interpretive defaults into a system that does not share our Umwelt) and goal alignment (which answer the model is even trying to give: the statistically human answer, the answer a user would like, or the correct one). It surveys the encouraging-but-contested proposals to use models as substitutes for human participants and defends the course's position — the method is legitimate, the substitution is not automatic — before extending the inflationary/deflationary frame into the ethical and public, with the consciousness debate as the sharpest case study.

By the end, each group has chosen a topic and can state in one sentence the psychological finding it will test; has seen the full pipeline demonstrated on one worked example (the text bandit task); has working online access to at least one open-weight model; and has run a throwaway pilot — one prompt sent, one response logged — to confirm the plumbing works. This is the “first contact” that surfaces access problems while a TA is in the room to fix them.

Checkpoint: a one-line topic statement approved by an instructor, and one logged model response.

Convert a topic into a defensible experiment. Groups write their question as a behavioural inference — which cognitive process, from which behaviour, against which alternative, and what would each hypothesis predict? — and design the task so that different underlying processes would produce different behaviours. Three design obligations are imposed as a checklist before any real data are collected: contamination (is the item one the model likely saw in training, and how will that be reframed or checked?), prompt variability (prepare a battery of paraphrases, not a single wording), and stochasticity (administer each condition many times).

Checkpoint: a written task specification — question as inference, task text, paraphrase set, contamination and repetition plan.

Groups run the full design — every condition, across the paraphrase battery, repeated enough to estimate response distributions — and, where their question and provider allow, run a contamination check. They then analyse, treating each of the model's repeated responses as analogous to one participant, which licenses ordinary population statistics and comparison with a human reference. Groups that finish early begin the model-comparison extension: repeating the experiment on a second, deliberately different open-weight model.

Checkpoint: a dataset and at least one figure or statistic bearing on the research question.

Groups assemble the Friday talk around the pipeline they have walked, and apply the Bayesian compass honestly: behavioural similarity to humans is the beginning of an inference, not its end. They name the confounds their result cannot escape — goal alignment chief among them — and state where their result sits between the inflationary and deflationary readings. A strong talk states what the result licenses and, in the same breath, what it does not.

Checkpoint: a complete draft presentation and a completed rehearsal.

Each group presents and defends its experiment (a ten-to-fifteen-minute talk with questions). This is the assessed deliverable. The rubric rewards methodological honesty over headline results: a clean null, well interpreted, passes as comfortably as a positive finding.