Harvard Intelligent Probabilistic Systems

Building Intelligent Probabilistic Systems

In the HIPS group, we are interested in building intelligent algorithms. What makes a system intelligent? Our philosophy is that "intelligence" means making decisions under uncertainty, adapting to experience, and discovering structure in high-dimensional noisy data. The unifying theme for research in these areas is developing new approaches to statistical inference: uncovering the coherent structure that we cannot directly observe and using it for exploration and to make decisions or predictions. We develop new models for data, new tools for performing inference, and new computational structures for representing knowledge and uncertainty.

Discovering Structure with Bayesian Nonparametrics

A perpetual challenge in statistical modelling is trying to find the parsimonious complexity in the data. That is, balancing simplicity in our explanations of the world with the flexibility that is required to capture the rich variation that occurs in real data. One remarkable class of mathematical tools for balancing these extremes are Bayesian nonparametric model, which enable one to specify an infinite-dimensional model, while still manipulating it tractably on a finite computer. Such models mean that our explanations for the world can grow in complexity precisely to the extent that the data allow it.

Point Processes as Primitives of Computation

Modern machine learning methods have proved remarkably successful at inferring statistical structure from data, something that any intelligent system must be able to perform. However, there is a disconnect between how our algorithms are represented in computer hardware and what we understand about the hardware of natural neural systems. In particular, we are still trying to understand how action potentials (neural spikes) can be used to implement adaptive computation. An ongoing project in the HIPS group is to try to formalize such computation in terms of powerful statistical objects called point processes.

Learning Latent Feature Representations

Many of the empirical successes of machine learning can be characterized as "simple discrimination functions applied to complex representations". The question is: how do we automatically find these representations? In probabilistic modelling, we view this as a problem of finding latent variables, which provide a simpler and often lower-dimensional representation of our high-dimensional data. In the HIPS group, we are constantly developing new ways to construct these kinds of models and apply them in different domains.

New Computational Tools for Statistical Inference

Powerful mathematical models and representations are only useful if we can perform the computation necessary to manipulate them. In the context of intelligent probabilistic systems, this can often be viewed as the problem of performing statistical inference. We are interested in building new computational tools that enable this inference, most often by developing new Monte Carlo methods, with potential impact both within computer science and statistics, but also across the broader sciences, such as biology and physics.

Check out Linderman and Ryan Adams' abstract "Inferring functional connectivity with priors on network topology" at Cosyne '13!

Scott Linderman and Ryan Adams' poster (III-9) and abstract, "Inferring functional connectivity with priors on network topology" will be presented at Cosyne '13 in Salt Lake City, February 28-March 3rd. They construct a Bayesian approach to modeling the latent functional network underlying observed spike trains, and represent top-down hypotheses via prior distributions over random graphs.

Announcing a Group Blog

We're excited to announce a new collaborative blog, written by members of the HIPS group. The idea is to use this as a venue to discuss interesting ideas and results — new and old — about probabilistic modeling, inference, artificial intelligence, theoretical neuroscience, or anything else research-related that strikes our fancy. There will be posts from folks at both Harvard and MIT, in computer science, mathematics, biophysics, and BCS departments, so expect a wide variety of interests.

Embryo development research published in National Academy of Sciences

Michael Gelbart, a PhD student in the Harvard Intelligent Probabilistic Systems group, publishes research on fruit fly embryo development in the Proceedings of the National Academy of Sciences.

Call for Applications: 2013-14 Harvard CRCS Postdoctoral Fellows and Visiting Scholars

The Harvard Center for Research on Computation and Society (CRCS) solicits applications for its Postdoctoral Fellows and Visiting Scholars Programs for the 2013-2014 academic year. Postdoctoral Fellows are given an annual salary of approximately $60,000 for one year (with the possibility of renewal) to engage in a program of original research, and are provided with additional funds for travel and research support. Visiting Scholars often come with their own support, but CRCS can occasionally offer supplemental funding.

Recent Publications

Gaussian Process Kernels for Pattern Discovery and Extrapolation. Wilson, AG, Adams RP. Proceedings of the 30th International Conference on Machine Learning. 2013. { arXiv:1302.4245 [stat.ML] | PDF }

wilson-extrapolation-icml-2013.pdf

Learning Outcome-Discriminative Dynamics in Multivariate Physiological Cohort Time Series. Nemati, S, Lehman L-wei, Adams RP. Proceedings of the 35th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) . 2013. { PDF }

nemati-discriminative-embc-2013.pdf

Tracking Progression of Patient State of Health in Critical Care Using Inferred Shared Dynamics in Physiological Time Series. Lehman, L-wei, Nemati S, Adams RP, Moody G, Malhotra A, Mark R. Proceedings of the 35th International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2013. { PDF }

lehman-progression-embc-2013.pdf

Bootstrap Learning Via Modular Concept Discovery. Dechter, E, Malmaud J, Adams RP, Tenenbaum JB. 23rd International Joint Conference on Artificial Intelligence. 2013. { PDF }

dechter-bootstrap-ijcai-2013.pdf

Cardinality Restricted Boltzmann Machines. Swersky, K, Tarlow D, Sutskever I, Salakhutdinov R, Zemel RS, Adams RP. Advances in Neural Information Processing Systems 25. 2012. { PDF }

swersky-crbm-nips-2012.pdf