Workshop

The winter school is organized and supported by the Mathmatical AI Safety Institute (MAISI) and the Alignment Research Center (ARC), and will take place at the Simons Laufer Mathematical Sciences Institute (SLMath) in Berkeley, California. Abstract: Neural networks are becoming increasingly capable, but are still treated primarily as black boxes, making it hard to predict how they will behave in rare or unusual circumstances. Mechanistic estimation aims to make such predictions much more reliably, not by running samples through the network, but instead by exploiting structural properties of the network's weights, and performing a deductive, heuristic analysis. In this winter school, researchers from the Alignment Research Center will lead a series of lectures and discussions covering the idea of mechanistic estimation, existing mechanistic estimators for random structures, conjectures about the behavior of mechanistic estimators in general, and how mechanistic estimation fits into an overall strategy for AI alignment. Description: Each day will include a mixture of lectures, Q&A and group discussion sessions led by researchers from the Alignment Research Center. A preliminary high-level schedule is as follows (this is subject to change, and a finalized schedule will be released closer to the time): Day 1: Introduction. Philosophical definitions of mechanistic estimation; simple examples of mechanistic estimators; and basic motivation from the perspective of AI alignment. Day 2: Random structures. Mechanistic estimators for wide, randomly-initialized multilayer perceptrons, and for expectations of random products; and the matching sampling principle for random structures. Day 3: Frontiers and applications. The matching sampling principle for trained and worst-case structures; handling empirical input data; and applications to AI alignment such as mechanistic training and mechanistic anomaly detection. Day 4: Open problems. Tour of open problems and opportunity for small-group discussion.