These papers, books, and essays serve as a shared basis of knowledge for the MIT Probabilistic Computing Project.

### Non-Technical Background

- The Structure of Scientific Revolutions by Thomas Kuhn [Amazon]
- The Innovator’s Dilemma: The Revolutionary Book That Will Change the Way You Do Business by Clayton Christensen [Amazon]
- Epigrams on Programming by Alan Perlis [html] [Turing award page][Foreword to SICP]
- The Effective Executive: The Definitive Guide to Getting the Right Things Done by Peter F. Drucker [Amazon]
- Technology and Courage by Ivan Sutherland [pdf]
- Mathematical Writing by Don Knuth [pdf]
- You and Your Research by Richard Hamming [html]
- Whirlwind I Computer [Wikipedia]
- Worse Is Better by Richard Gabriel [html]
- Who Inspired the Leamer-Rosenthal Prizes? Part II by Ed Leamer [html]
- The Laws of Medicine: Field Notes From an Uncertain Science by Siddhartha Mukherjee [Interview][Amazon]
- The War of Art: Winning the Inner Creative Battle by Steven Pressfield [Amazon]
- Study of Telemedicine Finds Misdiagnoses of Skin Problems by Melinda Beck [html]
- Many Analysts, One Dataset: Making Transparent How Variations in Analytical Choices Affect Results by Silberzahn et al. [pdf]
- What Is It Like To Be A Bat? by Thomas Nagel [pdf]
- When U.S. air force discovered the flaw of averages [html]
- The origins of SageMath – creating a viable open source alternative to Magma, Maple, Mathematica, and Matlab [pdf]
- Politics and the English Language by George Orwell [html]
- The Sciences of the Artificial by Herbert Simon [Amazon]
- Building Robust Systems an essay by Gerald Jay Sussman [pdf]

### Technical Background

- Mathematics: Its Content, Methods and Meaning: Chapter 11. The Theory of Probability by A. N. Kolmogorov [Amazon]
- Structure and Interpretation of Computer Programs by Harold Abelson, Gerald Jay Sussman, and Julie Sussman [Amazon] [pdf]
- Classifier Technology and the Illusion of Progress by David J. Hand [pdf]
- Machine Learning: The High-Interest Credit Card of Technical Debt by Sculley et al. [pdf]
- (Skim) On Computable Numbers and the Entscheidungsproblem by Alan Turing [pdf]
- (Skim) Noncomputable Conditional Distributions by Ackerman, Freer, and Roy [pdf]
- Natively Probabilistic Computation; PhD Dissertation by Vikash Mansinghka [pdf]
- Design of LISP-Based Processors by Steele and Sussman [pdf]
- Lisp: A Language for Stratified Design by Abelson and Sussman [pdf]
- Steps Toward Artificial Intelligence by Minsky [pdf]
- On Chomsky and the Two Cultures of Statistical Learning by Peter Norvig [html]
- Statistical Modeling: The Two Cultures by Leo Breiman [pdf]
- Critique of Paper by “Deep Learning Conspiracy” (Nature 521 p 436) by Jürgen Schmidhuber [html]
- Artificial Intelligence A Modern Approach (Introduction p. 3-31) by Russell and Norvig [pdf]
- Confessions of a Used Programming Language Salesman by Erik Meijer [pdf]

### Probabilistic Programming

- VentureScript Tutorial [html]
- BayesDB Documentation [html]
- Picture: an imperative probabilistic programming language for scene perception by Kulkarni, Kohli, Tenenbaum, and Mansinghka, [webpage]43
- Bayesian Statistics Without Tears: A Sampling-Resampling Perspective by Smith and Gelfand. [pdf]
- When are probabilistic programs probably computationally tractable? Freer, Mansinghka, and Roy [pdf]
- Bayesian Optimization as a Probabilistic Meta-Program; MEng thesis by Ben Zinberg [pdf]

### Probabilistic Programming (Optional, Outdated)

- Venture: a general purpose probabilistic programming platform with efficient stochastic inference. Mansinghka, Selsam, and Perov [pdf]
- A stochastic programming perspective on nonparametric Bayes by Roy, Mansinghka, Goodman and Tenenbaum [pdf]

### Foundations of Probabilistic Modeling (Required)

- Bayesianism and Causality, or, Why I am Only a Half-Bayesian by Judea Pearl [pdf]
- Why I Am Not a Bayesian by Clark Glymour [pdf]
- Probability Theory: The Logic of Science (Chapters 1-3) by E. T. Jaynes [pdf] [Amazon]
- Information Theory, Inference, and Learning Algorithms (Chapter 28, Model Comparison and Occam’s Razor) by David MacKay [webpage]

### Standard Library of Modeling Techniques

- Information Theory, Inference, and Learning Algorithms (Chapter 45, Gaussian Processes) by David MacKay [webpage]
- Unifying Rational Models of Categorization via the Hierarchical Dirichlet Process by Griffiths, Canini, Sanborn, and Navarro [pdf]
- Discovering latent classes in relational data by Kemp, Griffiths, and Tenenbaum [pdf]
- Topics in Semantic Representation by Griffiths and Steyvers [pdf]
- Finding Scientific Topics by Griffiths and Steyvers [pdf]
- Continuous Time Dynamic Topic Models by Wang, Blei, and Heckerman [pdf]
- Practical Bayesian Optimization of Machine Learning Algorithms by Snoek, Larochelle, and Adams [webpage]
- A Unifying Review of Linear Gaussian Models by Roweis and Ghahramani [pdf]

### Standard Library of Inference Techniques

- Composable Probabilistic Inference with Blaise by Keith A. Bonawitz. [pdf]
- An Introduction to MCMC for Machine Learning by Andrieu, De Freitas, Doucet, and Jordan [pdf]
- The Markov Chain Monte Carlo Revolution by Persi Diaconis [pdf]
- Convergence of Sequential Monte Carlo-based Sampling Methods by Huggins and Roy [pdf]

### Useful Examples (Optional)

- MrBayes: Bayesian Inference of Phylogenetic Trees by Huelsenbeck and Ronquist [pdf] [webpage]
- FastSLAM: FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem [pdf]
- First few chapters of Bayesian Spectrum Analysis and Parameter Estimation by Bretthorst [pdf]
- Recovering Intrinsic Images from a Single Image by Tappen, Freeman, and Adelson [pdf]
- Bayesian Inference for PCFGs via Markov chain Monte Carlo by Johnson, Griffiths and Goldwater [pdf]
- Global Seismic Monitoring as Probabilistic Inference by Arora, Russell, Kidwell, and Sudderth [pdf]

### Automatic Model Discovery (Optional)

- CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data by Mansinghka et. al.
- Structure Discovery in Nonparametric Regression through Compositional Kernel Search by Duvenaud, Lloyd, Grosse, Tenenbaum, Ghahramani [pdf]
- Exploiting compositionality to explore a large space of model structures by Grosse et al [pdf]
- Model Selection in Compositional Spaces; PhD Thesis by Roger Grosse [pdf]
- The discovery of structural form by Kemp and Tenenbaum [pdf]
- Structured priors for structure learning by Mansinghka, Kemp, Tenenbaum, and Griffiths [pdf]
- Information Theory, Inference, and Learning Algorithms (Chapter 28, Model Comparison and Occam’s Razor) by David MacKay [webpage]

### Reference Material on Approximate Inference (Optional)

- Probabilistic Inference using Markov chain Monte Carlo Methods by Neal [pdf]
- Hamiltonian Monte Carlo tutorial by Neal [pdf]
- Exact and approximate inference via systematic stochastic search by Mansinghka et al. [pdf]
- Factor graphs and the sum-product algorithm by Kschischang, Frey, and Loeliger [pdf]
- An Architecture for Parallel Topic Models by Smola and Narayanamurthy [pdf]
- LeCun, Bottou, Orr and Muller: Efficient BackProp, in Orr, and Muller (Eds), Neural Networks: Tricks of the trade [pdf]
- Black Box Variational Inference by Ranganath, Gerrish and Blei [pdf]