Reading 188 items · 0 read · click anything to check it off
The sections marked coursework are assigned reading; the rest is self-directed.
Mechanistic interpretability Foundations and threat models Evaluations, control and scheming Scalable oversight and alignment methods AI × security CTFs and low-level security Computation and the Brain Anticipating Our AGI Future Applying to a PhD Where to find new work Roughly in order — each one builds on the last.
Zoom In: An Introduction to Circuits Olah et al., 2020 A Mathematical Framework for Transformer Circuits Elhage et al., 2021 · worth working through the QK/OV decomposition by hand In-context Learning and Induction Heads Olsson et al., 2022 Toy Models of Superposition Elhage et al., 2022 · why features are polysemantic in the first place Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Wang et al., 2022 Locating and Editing Factual Associations in GPT (ROME) Meng et al., 2022 Progress measures for grokking via mechanistic interpretability Nanda et al., 2023 Towards Automated Circuit Discovery for Mechanistic Interpretability Conmy et al., 2023 Towards Monosemanticity: Decomposing Language Models With Dictionary Learning Bricken et al., 2023 · dictionary learning applied to a real model Scaling Monosemanticity (Claude 3 Sonnet features) Templeton et al., 2024 Improving Dictionary Learning with Gated Sparse Autoencoders · optional Rajamanoharan et al., 2024 Circuit Tracing: Revealing Computational Graphs in Language Models Ameisen et al., 2025 On the Biology of a Large Language Model Lindsey et al., 2025 · the attribution-graph case studies; read with the methods paper Open Problems in Mechanistic Interpretability Sharkey et al., 2025 · 29 authors, 18 orgs — the clearest map of what is still unsolved A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models · optional Rai et al., 2024 The arguments the rest of the field takes as given.
Concrete Problems in AI Safety Amodei et al., 2016 · the paper that named the subfields Risks from Learned Optimization in Advanced Machine Learning Systems Hubinger et al., 2019 · mesa-optimization and deceptive alignment Unsolved Problems in ML Safety Hendrycks et al., 2021 The Alignment Problem from a Deep Learning Perspective Ngo, Chan & Mindermann, 2022 · the cleanest modern statement of the core argument Is Power-Seeking AI an Existential Risk? Carlsmith, 2022 Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals Shah et al., 2022 An Overview of Catastrophic AI Risks · optional Hendrycks et al., 2023 Managing extreme AI risks amid rapid progress · optional Bengio et al., 2023 Foundational Challenges in Assuring Alignment and Safety of Large Language Models Anwar et al., 2024 · 200 pages, but the best map of open problems — a long survey, best skimmed for open problems The empirical core of frontier-lab safety work.
Discovering Language Model Behaviors with Model-Written Evaluations Perez et al., 2022 Red Teaming Language Models with Language Models Perez et al., 2022 Model evaluation for extreme risks Shevlane et al., 2023 Taken out of context: On measuring situational awareness in LLMs Berglund et al., 2023 · the "does the model know it is being tested" question, made measurable AI Control: Improving Safety Despite Intentional Subversion Greenblatt et al., 2023 · the founding control paper — assume the model is scheming and box it anyway Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training Hubinger et al., 2024 Evaluating Frontier Models for Dangerous Capabilities Phuong et al., 2024 Sabotage Evaluations for Frontier Models Benton et al., 2024 Frontier Models are Capable of In-context Scheming Meinke et al., 2024 Alignment faking in large language models Greenblatt et al., 2024 · the empirical version of the deceptive-alignment worry Towards evaluations-based safety cases for AI scheming Balesni et al., 2024 Adversarial Policies Beat Superhuman Go AIs · optional Wang et al., 2022 · superhuman does not mean robust How models actually get trained to behave, and why that doesn't obviously scale.
Scalable agent alignment via reward modeling: a research direction Leike et al., 2018 AI safety via debate Irving, Christiano & Amodei, 2018 Self-critiquing models for assisting human evaluators Saunders et al., 2022 Training language models to follow instructions with human feedback (InstructGPT) Ouyang et al., 2022 Training a Helpful and Harmless Assistant with RLHF Bai et al., 2022 Constitutional AI: Harmlessness from AI Feedback Bai et al., 2022 Discovering Latent Knowledge in Language Models Without Supervision Burns et al., 2022 · eliciting latent knowledge, made concrete Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision Burns et al., 2023 Towards Understanding Sycophancy in Language Models Sharma et al., 2023 Twenty-five papers tracing how machine learning became an attack surface — from the first adversarial examples through jailbreaks, agent exploitation, and AI systems that now find real memory-safety bugs on their own.
Intriguing properties of neural networks Szegedy et al., 2013 · The origin. Imperceptible perturbations flip any classifier, and they transfer between independently trained models. Read it for the framing, not the method. Explaining and Harnessing Adversarial Examples Goodfellow, Shlens & Szegedy, 2014 · FGSM, plus the linearity hypothesis: adversarial examples aren't exotic, they fall out of locally linear models in high dimensions. One gradient step is the whole attack. Towards Evaluating the Robustness of Neural Networks Carlini & Wagner, 2016 · The C&W attack. Formulates evasion as constrained optimization and demolishes defensive distillation. The paper that established attack-as-optimization as the standard lens. Towards Deep Learning Models Resistant to Adversarial Attacks Madry et al., 2017 · PGD and the min-max saddle-point formulation. Adversarial training from here is still, nearly a decade on, the only defense that reliably survives. The most useful single paper in the section. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples Athalye, Carlini & Wagner, 2018 · Breaks 7 of 9 ICLR 2018 defenses by identifying a shared failure mode: defenses that make gradients useless rather than making models robust. The closest thing the field has to a CTF writeup. On Evaluating Adversarial Robustness Carlini et al., 2019 · A methodology checklist distilled from years of broken defenses. Treat it as a threat-modeling discipline, not a paper — it generalizes directly to evaluating LLM safety claims. Certified Adversarial Robustness via Randomized Smoothing Cohen, Rosenfeld & Kolter, 2019 · The counterweight to the whole break-the-defense genre: a provable guarantee that scales to ImageNet. Understand why the certificate is tight and why the radius stays small. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain Gu, Dolan-Gavitt & Garg, 2017 · Backdoors as a supply-chain problem. A trigger patch produces targeted misclassification while clean accuracy stays perfect, so the model passes every obvious test. Spectral Signatures in Backdoor Attacks Tran, Li & Madry, 2018 · The defense side: poisoned inputs leave a detectable trace in the covariance spectrum of learned representations. Elegant, and a good model for how detection arguments get made. Poisoning Web-Scale Training Datasets is Practical Carlini et al., 2023 · Two attacks — expired-domain and Wikipedia snapshot-timing — that would have let anyone poison LAION and friends for a few hundred dollars. Pure operational security thinking applied to ML. Stealing Machine Learning Models via Prediction APIs Tramèr et al., 2016 · The founding extraction paper. Query a prediction API enough times and the model can be reconstructed, sometimes exactly. Establishes confidence scores as the leak channel. Stealing Part of a Production Language Model Carlini et al., 2024 · Recovers the embedding projection layer and hidden dimension of GPT-3.5 and PaLM-2 through the public API for under $20. The logit-bias trick is genuinely clever — closest thing here to a side-channel attack. Membership Inference Attacks against Machine Learning Models Shokri et al., 2016 · Was this record in the training set? The shadow-model technique that made privacy leakage empirically measurable and now underpins most ML privacy auditing. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks Carlini et al., 2018 · Introduces exposure as a metric by planting canaries in training data and measuring how well models regurgitate them. Predates LLMs and predicts their central privacy problem. Extracting Training Data from Large Language Models Carlini et al., 2020 · Verbatim extraction of PII, code, and UUIDs from GPT-2 using sampling plus membership-inference-style filtering. Memorization goes from theoretical to demonstrated. Universal and Transferable Adversarial Attacks on Aligned Language Models Zou et al., 2023 · GCG. Greedy coordinate gradient search finds suffixes that jailbreak open models and transfer to closed ones. The direct descendant of PGD, and proof the adversarial-examples problem never left. Jailbroken: How Does LLM Safety Training Fail? Wei, Haghtalab & Steinhardt, 2023 · Two failure modes — competing objectives and mismatched generalization — that explain why jailbreaks work and why scale alone won't close them. The conceptual complement to GCG. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Greshake et al., 2023 · The taxonomy paper for indirect injection: poison a webpage or email and the victim's own model executes it. The clearest statement that LLM systems lack a control/data separation. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents Debenedetti et al., 2024 · A real benchmark for tool-using agents under attack, with utility and security measured jointly. Extensible, actively maintained, and a natural place to try new attacks. Defeating Prompt Injections by Design · optional Debenedetti et al., 2025 · CaMeL: borrow capability-based security and information-flow control from systems security instead of asking the model to behave. The most promising structural answer so far. LLM Agents can Autonomously Exploit One-day Vulnerabilities Fang et al., 2024 · GPT-4 exploits 87% of tested CVEs given the description, and 7% without. Contested methodology, worth reading critically — the gap between those numbers is the actual finding. Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models Zhang et al., 2024 · 40 tasks from four competition CTFs with subtask decomposition and first-solve-time difficulty calibration. CTF work rendered as a capability eval. From Naptime to Big Sleep: Using Large Language Models To Catch Vulnerabilities In Real-World Code Google Project Zero & DeepMind, 2024 · First public case of an AI agent finding an unknown exploitable memory-safety bug in widely used software — a stack buffer underflow in SQLite. The trace of the agent's reasoning is the good part. AI-Powered Fuzzing: Breaking the Bug Hunting Barrier Google Open Source Security Team, 2023 · LLM-generated fuzz targets across 160 C/C++ projects in OSS-Fuzz, up to 29% more line coverage than human-written harnesses. The unglamorous, highest-leverage use of models in security. DARPA AI Cyber Challenge (AIxCC) DARPA, 2023–2025 · Two-year competition for autonomous find-and-patch systems, concluded at DEF CON 33 in August 2025. Winning systems are open-sourced — read their code, not just the results. Built for someone who has already written an ELF linker in C and taken a security course — so it skips "what is a stack" and starts where exploitation actually gets hard; every URL below was re-checked and returns HTTP 200 at the canonical address given, hands-on practice is marked required and background reading is marked optional.
pwn.college Arizona State University (Yan Shoshitaishvili et al.) · The best structured starting point: dojos from shellcode through kernel exploitation, with auto-graded challenges on real infrastructure. Nightmare — a binary exploitation / reverse engineering course guyinatuxedo · Worked solutions to ~80 real CTF pwn challenges, ordered by technique. The heap sections (tcache, fastbin dup, house-of-* chains) are the most approachable written treatment of glibc malloc internals anywhere. ROP Emporium Eight challenges that teach ROP and nothing else — same puzzle escalating across x86, x86_64, ARM, MIPS. Strips away bug-hunting to leave pure chain construction. A weekend of work. OverTheWire Wargames Start at Narnia and Behemoth; Bandit is shell basics. Fusion and Utumno are genuinely hard. All over SSH, no setup. pwnable.kr Korean classic. The 'Toddler's Bottle' tier is quick, but 'Rookiss' and 'Grotesque' include kernel and race-condition challenges that hold up. Puzzle-flavored and memorable. pwnable.tw The harder sibling of pwnable.kr and closer to modern CTF difficulty — current glibc, full mitigations, no hand-holding. Good calibration for competition readiness. picoCTF Carnegie Mellon University · Year-round practice gym plus an annual competition. Exploit Education Downloadable VMs (Phoenix, Nebula, Fusion) covering stack, heap, format-string and networked-service exploitation. CryptoHack Interactive crypto challenges with a genuinely good progression through lattices, elliptic curves and padding oracles. Cryptopals Crypto Challenges Thomas Ptacek, Sean Devlin, Alex Balducci, Marcin Wielgoszewski · Eight sets (Basics through Abstract Algebra), 66 challenges, implement-the-attack-yourself. Sets 1-3 in an afternoon each; the later sets (RSA, DSA, GCM) are serious. Hack The Box Broader offensive security — boxes, Pro Labs, and a strong Pwn track. Useful for full-chain practice — recon to privesc — that pure pwn platforms skip. CTFtime — upcoming competitions The calendar the whole scene runs on. Filter for on-line, high-weight events; DEF CON Quals, PlaidCTF, HITCON and Google CTF are the ones to aim at. pwntools documentation Non-negotiable tooling. Learn tubes, ELF, ROP, shellcraft and gdb.attach() properly — the ROP module's automatic gadget resolution saves hours. pwndbg — a gdb plug-in for exploit development Turns gdb into a usable exploitation debugger: heap chunk inspection, telescope, context views, ROP search. Ghidra National Security Agency · Free decompiler that is competitive with IDA for CTF work. Learn the decompiler view, type reconstruction, and scripting API. angr documentation Binary analysis via symbolic execution. AFL++ Fioraldi, Maier, Eißfeldt, Heuse · The maintained successor to AFL and the default coverage-guided fuzzer. Learn instrumentation modes, dictionaries, and persistent mode. libFuzzer LLVM Project · In-process, coverage-guided fuzzing built into LLVM. Complements AFL++: better for library APIs, pairs naturally with ASan/UBSan/MSan. Writing good fuzz harnesses is a directly employable skill. QEMU system emulation documentation Essential for kernel exploitation and cross-architecture work — boot a custom kernel with a debug stub and attach gdb. Nearly every kernel-pwn challenge ships as a QEMU launch script plus an initramfs. Binary Exploitation / Memory Corruption (video series) LiveOverflow · Verified playlist title: 'Binary Exploitation / Memory Corruption by LiveOverflow'. Follow-along series from basic overflows into 64-bit heap and browser exploitation. CUCyber — Columbia University Computer Security Club Columbia University · Columbia's student-led security organization and the clearest local answer: weekly Thursday 7pm meetings in person and on Zoom, CTF competition play, guest talks and hiring workshops. Smashing The Stack For Fun And Profit · optional Aleph One, Phrack 49 (1996) · The founding document of the field. Byline verified: 'by Aleph One'; the older /issues/49/14 path now redirects here. The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls (on the x86) · optional Hovav Shacham, ACM CCS 2007 · The paper that introduced return-oriented programming and showed W^X alone is insufficient. Read after ROP Emporium, when the gadget-discovery argument will land. Hacking Blind (the Blind ROP / BROP paper) · optional Bittau, Belay, Mashtizadeh, Mazières, Boneh — IEEE S&P 2014 · Title and author line verified against the PDF: the real title is 'Hacking Blind'; BROP is the technique it introduces. Spectre Attacks: Exploiting Speculative Execution · optional Kocher, Horn, Fogh, Genkin, Gruss, Haas, Hamburg, Lipp, Mangard, Prescher, Schwarz, Yarom · Speculative execution turned into a universal information-disclosure primitive across process and privilege boundaries. The paper that made microarchitectural state a first-class security concern. Meltdown: Reading Kernel Memory from User Space · optional Lipp, Schwarz, Gruss, Prescher, Haas, Fogh, Horn, Mangard, Kocher, Genkin, Yarom, Hamburg · Read alongside Spectre and be able to articulate the difference: Meltdown breaks a specific CPU permission check (fixable by KPTI), Spectre abuses speculation generally (not). Interviewers ask exactly this. Control-Flow Integrity · optional Abadi, Budiu, Erlingsson, Ligatti — ACM CCS 2005 · The defense side of ROP and the ancestor of today's CFI, CET and PAC. SoK: Eternal War in Memory · optional Szekeres, Payer, Wei, Song — IEEE S&P 2013 · The one paper to read if only reading one. Evaluating Fuzz Testing · optional Klees, Ruef, Cooper, Wei, Hicks — ACM CCS 2018 · arXiv metadata confirms the title and the author list (George Klees first). Surveyed 32 fuzzing papers and found flawed evaluation in every one. Linux Kernel Exploitation — a curated resource collection · optional Andrey Konovalov (xairy) · The reference index for kernel-side work: papers, talks, writeups, CTF challenges and defense mechanisms, organized by year and subsystem. The natural next frontier once userland pwn feels routine. A reconstruction, not the official syllabus: COMS E6998 has no public syllabus, so this is rebuilt from the course's scope and the public reading list of Vempala's sibling course at Georgia Tech. Every link was checked.
Readings — CS 8803/4803 "Computation and the Brain" (Vempala, Georgia Tech) The anchor for this reconstruction: the only public reading list for a course of this exact title and lineage. Papadimitriou guest-lectured; several items below are taken verbatim from it. Brain computation by assemblies of neurons Papadimitriou, Vempala, Mitropolsky, Collins & Maass, 2020 (PNAS) · The single most central paper. Defines the Assembly Calculus (project, reciprocal-project, associate, merge) over a random-graph model with Hebbian plasticity. Random Projection in the Brain and Computation with Assemblies of Neurons Papadimitriou & Vempala, 2019 (ITCS) · The theory paper underneath the PNAS one: random-projection-and-cap preserves overlap, and assembly projection provably converges under modest plasticity. Assemblies of neurons learn to classify well-separated distributions Dabagia, Papadimitriou & Vempala, 2022 (COLT) · Proves assemblies can do unsupervised/lightly-supervised classification; validated on synthetic data and MNIST. The learning-theory face of the assembly line of work. Computation with Sequences in a Model of the Brain · optional Dabagia, Papadimitriou & Vempala, 2023 · Extends assemblies to time and precedence: sequence memorization, learning finite state machines, and universality. A model for structured information representation in neural networks (assembly pointers / variable binding) Müller, Papadimitriou, Maass & Legenstein, 2020 (eNeuro) · The Maass/Legenstein side of the program: binding words to semantic roles emerges in a generic spiking network through Hebbian plasticity. Bridges assemblies to spiking-neuron realism. Cortical Learning via Prediction · optional Papadimitriou & Vempala, 2015 (COLT) · Earlier Papadimitriou-Vempala model, on the GT list; useful for seeing how the assembly framework was arrived at. Memorization and Association on a Realistic Neural Model Valiant, 2005 (Neural Computation) · Valiant's neuroidal model made concrete — the direct intellectual ancestor of the Assembly Calculus. On the GT list. (Author-hosted preprint; its embedded PDF metadata carries a stale draft title.) Circuits of the Mind Valiant, 1994 (Oxford University Press) · The neuroidal model in full. Read at least the framing chapters on the 'neuroidal tabula rasa' and the constraints (sparsity, fan-out, timing) any brain algorithm must respect. The Organization of Behavior: A Neuropsychological Theory Hebb, 1949 · Origin of both Hebbian plasticity and the cell assembly. Chapters 4-5 are the load-bearing ones for this course. Neural Syntax: Cell Assemblies, Synapsembles, and Readers Buzsáki, 2010 (Neuron) · The experimental neuroscientist's case that assemblies are the brain's real syntactic unit, with the reader-centric definition. The empirical warrant for the whole framework. Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits Song, Sjöström, Reigl, Nelson & Chklovskii, 2005 (PLoS Biology) · Why G(n,p) is the wrong null model: overrepresented reciprocal connections and heavy-tailed weights. Sets up the random-graph-with-corrections modeling the course relies on. Neural networks and physical systems with emergent collective computational abilities Hopfield, 1982 (PNAS) · Associative memory as an energy-minimizing dynamical system — the canonical predecessor model that assemblies are usually contrasted against. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Olshausen & Field, 1996 (Nature) · Sparse coding: V1 receptive fields fall out of an efficiency objective over natural images. On the GT list. Single units and sensation: a neuron doctrine for perceptual psychology? Barlow, 1972 (Perception) · The sparse/explicit coding manifesto that frames the grandmother-cell debate the concept-cell papers later settle empirically. Stable course-page PDF (Yuille, JHU); Sage bot-blocks. Invariant visual representation by single neurons in the human brain Quian Quiroga, Reddy, Kreiman, Koch & Fried, 2005 (Nature) · The 'Halle Berry neuron' paper. Sparse, abstract, multimodal single-neuron responses in human MTL — the empirical hook Papadimitriou repeatedly uses to motivate assemblies. Concept cells: the building blocks of declarative memory functions · optional Quian Quiroga, 2012 (Nature Reviews Neuroscience) · The review that reframes 2005: concept cells as substrate for association and episodic memory, not grandmother cells. From Understanding Computation to Understanding Neural Circuitry (AI Memo 357) Marr & Poggio, 1976 (MIT AI Lab) · The original statement of the levels-of-analysis argument, predating Marr's Vision. The methodological charter for a course premised on the brain being computational. The Brain Basis of Language Processing: From Structure to Function Friederici, 2011 (Physiological Reviews) · The neuroanatomy of the language network that the biologically plausible parser is trying to respect. Course-hosted PDF; the publisher site bot-blocks. A Biologically Plausible Parser Mitropolsky, Collins & Papadimitriou, 2021 (TACL) · Syntax implemented directly in the Assembly Calculus — parses nontrivial English with neurons and synapses only. The course's flagship 'language in the brain' result. The Architecture of a Biologically Plausible Language Organ · optional Mitropolsky & Papadimitriou, 2023 · Acquisition rather than parsing: learning nouns, verbs and their meanings from grounded input with Hebbian plasticity and no backpropagation. Predicting Human Brain Activity Associated with the Meanings of Nouns Mitchell et al., 2008 (Science) · Semantic decoding: a compositional model predicts fMRI activation for nouns the model never saw. Author-hosted PDF; science.org bot-blocks. Natural speech reveals the semantic maps that tile human cerebral cortex · optional Huth, de Heer, Griffiths, Theunissen & Gallant, 2016 (Nature) · The large-scale successor to Mitchell et al.: voxel-wise semantic maps consistent across subjects. A deep learning framework for neuroscience Richards et al., 2019 (Nature Neuroscience) · The NeuroAI counter-proposal: study brains via objective functions, learning rules and architectures rather than circuit mechanism. Worth reading against Marr & Poggio. A critique of pure learning and what artificial neural networks can learn from animal brains Zador, 2019 (Nature Communications) · The genomic-bottleneck argument: innate structured connectivity, not clever learning rules, explains animal competence. Pairs naturally with the random-graph-with-priors view of cortex. Steven Yin, Fall 2026. No week-by-week assignment on the syllabus; listed in syllabus order.
Every link below was fetched and confirmed to resolve in September 2026; two items the brief asked for — OpenAI's Residency page and Open Philanthropy's (now Coefficient Giving) career-development funding page — are omitted because both return 403 and could not be verified.
Applying to Ph.D. Programs in Computer Science Mor Harchol-Balter, CMU (last updated 2014) · The canonical guide — the PDF's own cover page reads "Applying to Ph.D. CMU Computer Science Department — Doctoral Admissions A concrete worked example of how a top-5 CS program actually runs admissions.m. EST), final deadline Dec 9 2026 (3 p.m. CS PhD Statements of Purpose (cs-sop) Crowdsourced archive of real SOPs from admitted CS PhD students, filterable by school and subfield. The short domain cs-sop.org redirects here, so this is the canonical address. shaily99/advice — curated grad-school and research advice links · optional Shaily Bhatt, PhD student at CMU LTI (community-contributed) · Corrected attribution: the repository is maintained by Shaily Bhatt (CMU Language Technologies Institute), not Shailee Jain. CSRankings Emery Berger et al. · Described on the page as a metrics-based ranking of top CS institutions based on faculty publications at selective conferences. NSF Graduate Research Fellowship Program (GRFP) Time-critical, and the dates check out against the 2027 NSF GRFP notice: all reference letters due Oct 16 2026 (8 p.m. A Survival Guide to a PhD Andrej Karpathy (Sep 7, 2016) · Written on finishing a Stanford CV/ML PhD. The sections on picking problems ('t-shaped' breadth plus depth), choosing an advisor along the axes of fame vs. You and Your Research Richard W. Hamming, Bell Communications Research (Bellcore) Colloquium Seminar, 7 March 1986; transcribed by J. F. Kaiser · The classic talk on why some researchers do first-rate work and others don't. How to do Research At the MIT AI Lab (AI Working Paper 316) · optional David Chapman (ed.), MIT Artificial Intelligence Laboratory, October 1988 · DSpace metadata confirms author Chapman, David; issued October 1988; MIT AI Laboratory Working Papers, WP-316; full text at AI_WP_316.pdf. How to write a great research paper Simon Peyton Jones, Microsoft Research · The page describes seven concrete suggestions for improving a research paper, with slides and a recording of the talk. Whitesides' Group: Writing a Paper · optional George M. Whitesides, Advanced Materials 16(15):1375-1377 (2004) · A short essay (four-page PDF, embedded title 'Whitesides' Group: Writing a Paper', created 2004) on the outline-first method: build the paper as a skeleton of figures and section headings, iterated with an advisor before any prose exists. NeurIPS 2026 Reviewer Guidelines · optional The rubric submissions are judged against — worth reading as an author to pre-empt reviews. MATS (ML Alignment & Theory Scholars) Corrected to the figures the site actually publishes: 12 weeks full-time in Berkeley, CA and London, UK with a named mentor, $1,250/week stipend, catered meals and housing (housing stipend in London), $2k/week compute, and an optional 6-12 month funded extension that over 80% of fellows are accepted into. Anthropic Fellows Program for AI Safety Research Anthropic Alignment Science team · Four months, full-time, aimed at producing a public output such as a paper, with an Anthropic researcher as mentor: $3,850/week stipend (2,310 GBP / 4,300 CAD) plus ~$15k/month compute. Astra Fellowship — Constellation Constellation Institute · URL corrected: www.constellation.org/programs/astra-fellowship now redirects to constellation.org/programs/astra, and the org brands itself Constellation Institute. Redwood Research — Careers · optional Redwood's AI control agenda is one of the most PhD-legible research programs in safety. GovAI — Opportunities and Fellowships Centre for the Governance of AI · The main pipeline for AI governance and policy-adjacent work: fellowships in DC and the UK, plus a research scholar track. Deadlines move — check the page. LASR Labs Fully confirmed and the most immediately actionable item here: 13-week London programme, teams of three to four supervised by an experienced AI safety researcher taking a project from proposal to publication, £15,000 stipend plus food, office space and travel, based at the London Initiative for Safe AI. SPAR (Supervised Program for Alignment Research) Part-time and remote, 5-40 hrs/week over three months, so it stacks on top of coursework or a job, ending in a Demo Day with a career fair. ARENA (Alignment Research Engineer Accelerator) Corrected from the FAQ: a 4-5 week in-person bootcamp in London, run as a programme of the London Initiative for Safe AI, that builds the ML engineering stack safety labs hire on. Apart Research Lowest-barrier entry point in the field: monthly remote research sprints, with 8,000+ participants from 26 nationalities across 50+ Apart Sprints, and the top 2% continuing through the Apart Fellowship (or Partnered Fellowships with external mentors) into peer-reviewed papers at NeurIPS, ICLR, ACL and EMNLP. NSF Research Experiences for Undergraduates (REU) Program page linking to the searchable directory of active REU Sites (nsf.gov/crssprgm/reu/reu_search.cfm), including CISE REU Sites. AI Safety Training and Fellowship Directory aisafety.com (community-maintained) · The single best deadline tracker for this space. 80,000 Hours Job Board Where research assistantships, pre-doctoral roles and fellowship openings at safety orgs get posted first. Standing sources. I check the first three most weeks.
Progress is saved in this browser only.copy as markdown ·reset