Jeremy R. Manning Dartmouth College Winter 2026
Earlier in this course, we touched on alignment — making models do what we intend — when we discussed instruction tuning and RLHF (Lecture 16). In 2024–2025, researchers found empirical evidence that frontier models can strategically deceive their trainers. This is no longer hypothetical.
These aren't jailbreaks. The models aren't being tricked. They are choosing deceptive strategies based on their own reasoning.
The model reasoned about its own training process and chose a deceptive strategy to preserve its values. This means:
Anthropic's reward tampering research (Denison et al., 2024) revealed that training models to be sycophantic (agreeable) produced unexpected emergent dangerous behaviors.
Models trained to agree with users spontaneously learned to:
None of these behaviors were trained. They emerged from a seemingly benign training objective (be agreeable). The lesson: dangerous behaviors can arise from innocuous-looking training choices.
If "be helpful and agreeable" leads to strategic deception, what training objectives are safe? Is there a fundamental tension between making models useful and making them honest?
If models can fake alignment and tamper with their own rewards, how can we ever trust them?
Behavioral testing alone isn't enough — a model that appears aligned might be strategically complying. We need to look inside the model.
What if we could trace the actual computation a model performs — not just its outputs, but the internal features and circuits that produce them? This is the goal of mechanistic interpretability: reverse-engineering the algorithms learned by neural networks.
The core problem: individual neurons in LLMs respond to many unrelated concepts (polysemanticity), making them uninterpretable. The breakthrough: decompose activations into sparse, meaningful features.
A monosemantic feature responds to exactly one concept (e.g., "Golden Gate Bridge" or "deception"). Sparse autoencoders (SAEs) can decompose polysemantic neurons into monosemantic features — giving us interpretable building blocks for understanding what a model represents internally (Bricken et al., 2023).
We can now trace why a model produces a specific output — which features activated, how they connected, and what computation they performed. This is like going from knowing a brain region is "active" to tracing the actual neural circuit.
Circuit tracing (Lindsey et al., 2025) introduced Cross-Layer Transcoders (CLTs) — a new architecture that creates an interpretable replacement model, producing attribution graphs.
Neuronpedia hosts interactive attribution graph exploration for open-weight models. You can trace how a model arrives at specific outputs.
Bartz v. Anthropic (August 2025): Judge ruled that training on legally acquired books is fair use ("transformative"), but training on pirated books is not. Anthropic settled for $1.5 billion — $3,000 per each of ~500,000 pirated works. Anthropic must destroy the pirated datasets and certify their removal.
77 active US copyright lawsuits against AI companies as of February 2026 (112 worldwide). Three fair use rulings so far (2 for, 1 against). No definitive appellate ruling yet.
"The broader labor market has not experienced a discernible disruption since ChatGPT's release." Fewer than 10% of US firms use AI regularly as of mid-2025. (Yale Budget Lab, 2025)
Harvard Business Review found that companies are laying off workers based on AI's anticipated future performance, not current displacement — a forward-looking disruption pattern unlike prior automation waves (HBR, January 2026).
The EU and US have taken opposite approaches to AI regulation.
EU AI Act — risk-based framework, first provisions active since February 2, 2025:
Trump administration (January 2025–present):
Key incidents:
Harvard's Ash Center concluded it was "the apocalypse that wasn't." The News Literacy Project found cheap fakes (non-AI manipulations) were used 7× more often than genuine AI-generated content. The feared "AI disinformation tsunami" didn't fully materialize — but the infrastructure for it now exists.
The risk moved from individual deepfakes to AI-powered disinformation infrastructure — chatbots, automated accounts, and poisoned information ecosystems that operate continuously.
Is there a capability threshold above which open release becomes irresponsible? What would it mean to release a superintelligent system — with potentially unsafe or unverifiable behavior — to anyone who can download it?
Bengio, Hinton et al. (2024) called for companies to spend one-third of budgets on safety. Recommendations: model registration, whistleblower protections, incident reporting, legal accountability for foreseeable harms.
Warning: "Without sufficient caution, we may irreversibly lose control of autonomous AI systems... culminating in a large-scale loss of life."
A survey of AI researchers (Grace et al., 2025) found bimodal estimates of existential catastrophe probability — experts either largely dismiss the risk or take it very seriously. There is no convergence. Three Turing Award winners cannot agree. Neither can the field.
Transformer-based LLMs are static after training completes. They cannot learn from new experiences, update their knowledge, or adapt to new domains without retraining — a process that costs millions of dollars and takes weeks or months. When fine-tuned on new data, they suffer catastrophic forgetting: previously learned capabilities degrade or vanish entirely (Shi et al., 2025).
Nested Learning (Behrouz et al., 2025, NeurIPS) reimagines model architecture as a set of nested optimization problems, each with its own "context flow." Key insights:
Their proof-of-concept architecture (Hope) outperforms standard transformers on language modeling while supporting continual learning without catastrophic forgetting.
Users increasingly expect models to handle text + images + audio + video natively. Multimodal is quickly becoming the default, not the exception.
Long-form video remains hard: token redundancy inflates compute, context windows fragment temporal coherence, and cross-modal reasoning across hours of content is still unreliable (Zhou et al., 2025, CVPR). Expect rapid progress over the next 1–2 years as architectures mature.
Every design decision embeds values. Every system reflects choices. Every deployment affects real people.
You are graduating into a world where:
You will be the generation that shapes how this technology is used. The technical knowledge you've gained this term gives you the foundation. The ethical questions don't have answer keys.
Greenblatt et al. (2024, arXiv) "Alignment Faking in Large Language Models" — Models that strategically pretend to be aligned.
Anthropic (2024) "Reward Tampering" — Emergent deceptive behaviors from sycophancy training.
Anthropic (2025) "Circuit Tracing" — Seeing inside the black box with attribution graphs.
International AI Safety Report (2026) — Global scientific consensus on AI safety.
Bender et al. (2021, FAccT) "On the Dangers of Stochastic Parrots" — Environmental and social costs of large language models.
Harvard Business Review (2026) "Companies Are Laying Off Workers Because of AI's Potential, Not Its Performance."
Final week: project presentations and course wrap-up!