9.S915: Introduction to Probabilistic Programming (Fall 2016)

Introduces probabilistic programming, a computational formulation of probability theory. Covers how to formalize key ideas from probabilistic modeling and inference as probabilistic meta-programs, and provides hands-on probabilistic programming experience with Venture, an open-source research platform. Emphasizes practical AI-based techniques for probabilistic data analysis while also surveying applications to computer vision, robotics, and the exploration and modeling of complex databases in domains such as public health and neuroscience. Illustrates connections with other approaches to engineering and reverse-engineering intelligence.

Course Information

Grading: 20% participation, 40% problem sets, 40% final project

  1. Participation (lecture & recitation): 20%

    Students are expected to attend all lectures; the problem sets do not cover all the critical material, and there is no written reference for much of the course. Students will also be expected to scribe 3 lectures, and to revise their scribe notes by merging them with others scribing the same lectures; the final quality of the scribe notes will be considered as part of the participation grade.

    Recitation is an optional clinic-style meeting where students get together in a cluster and work on problem sets and questions about course material. Vikash and/or his research assistants will be available for questions and support. Recitation participation is strongly encouraged, but you will not be penalized for not attending.

  2. Problem Sets : 40%

    There are three problem sets, due 10/13, 11/3, and 11/18. Each problem set is worth 13.3% of the total course grade. These problem sets emphasize (i) probabilistic programming skills, (ii) technical communication skills, and (iii) reading comprehension for both technical and non-technical background material.

  3. Final Project : 40%

    You will be responsible for producing a research workshop quality poster in four stages:

    (i) Proposal for your final project topic and scope. You will discuss these in small groups with members of the MIT Probabilistic Computing Project during the week of 10/10. You will have the option of selecting a poster topic from a list or developing your own.

    (ii) Stub poster outlining what you will demonstrate using synthetic results and figures, to enable in-depth feedback on the story and ensure project viability before the research itself is complete. You will hand this in by 11/22.

    (iii) Poster presentation where you show a revised draft poster in class on 12/6.

    (iv) Final poster must be turned in by 12/13.

This structure is designed to help you learn how to organize work on projects whose goals and formulation are evolving as the work unfolds and communicate preliminary results. Exemplary projects may lead to additional mentorship from MIT ProbComp members towards external presentation and/or publication, but this is not required.

Learning Outcomes

After taking this course, students will be able to:

  1. Explore sparse databases and solve predictive modeling problems using BayesDB BQL and MML.

  2. Diagnose model and inference quality in functional terms — e.g. predictive accuracy — and normative terms — e.g. how close to ``fully Bayesian'' was the algorithmic process that produced the model.

  3. Use key invariants of Bayesian probability — chiefly, the Bayesian Occam's Razor and asymptotic concentration — to guide the testing and debugging of probabilistic programs.

  4. Formulate simple textbook models from Bayesian statistics as generative programs in VentureScript.

  5. Use inference programming to apply them to draw approximately Bayesian inferences from real data.

  6. Formulate core conceptual problems in computer vision and robotics in terms of probabilistic programming, and solve simple instances of them.

Students will also (i) gain experience using the probabilistic programming framework to critically assess technical proposals for engineering and reverse engineering intelligence; (ii) get exposure to several core theoretical and conceptual issues in probabilistic programming and probabilistic computing; and (iii) learn about the research frontier in these fields.

Topics and Weekly Schedule

Fundamentals of probabilistic programming

Probabilistic data analysis

Probabilistic meta-programming

Applications to computer vision and robotics

Advanced topics

Relationships to Other Courses