Harvard Intelligent Probabilistic Systems

Building Intelligent Probabilistic Systems

In the HIPS group, we are interested in building intelligent algorithms. What makes a system intelligent? Our philosophy is that "intelligence" means making decisions under uncertainty, adapting to experience, and discovering structure in high-dimensional noisy data. The unifying theme for research in these areas is developing new approaches to statistical inference: uncovering the coherent structure that we cannot directly observe and using it for exploration and to make decisions or predictions. We develop new models for data, new tools for performing inference, and new computational structures for representing knowledge and uncertainty.

Discovering Structure with Bayesian Nonparametrics

A perpetual challenge in statistical modelling is trying to find the parsimonious complexity in the data. That is, balancing simplicity in our explanations of the world with the flexibility that is required to capture the rich variation that occurs in real data. One remarkable class of mathematical tools for balancing these extremes are Bayesian nonparametric model, which enable one to specify an infinite-dimensional model, while still manipulating it tractably on a finite computer. Such models mean that our explanations for the world can grow in complexity precisely to the extent that the data allow it.

Point Processes as Primitives of Computation

Modern machine learning methods have proved remarkably successful at inferring statistical structure from data, something that any intelligent system must be able to perform. However, there is a disconnect between how our algorithms are represented in computer hardware and what we understand about the hardware of natural neural systems. In particular, we are still trying to understand how action potentials (neural spikes) can be used to implement adaptive computation. An ongoing project in the HIPS group is to try to formalize such computation in terms of powerful statistical objects called point processes.

Learning Latent Feature Representations

Many of the empirical successes of machine learning can be characterized as "simple discrimination functions applied to complex representations". The question is: how do we automatically find these representations? In probabilistic modelling, we view this as a problem of finding latent variables, which provide a simpler and often lower-dimensional representation of our high-dimensional data. In the HIPS group, we are constantly developing new ways to construct these kinds of models and apply them in different domains.

New Computational Tools for Statistical Inference

Powerful mathematical models and representations are only useful if we can perform the computation necessary to manipulate them. In the context of intelligent probabilistic systems, this can often be viewed as the problem of performing statistical inference. We are interested in building new computational tools that enable this inference, most often by developing new Monte Carlo methods, with potential impact both within computer science and statistics, but also across the broader sciences, such as biology and physics.

Ryan Adams Wins DARPA Young Faculty Award

Ryan Adams, Assistant Professor of Computer Science at the Harvard School of Engineering and Applied Sciences (SEAS), has won a Defense Advanced Research Project Agency (DARPA) Young Faculty Award.

New England Machine Learning Day

We are thrilled to announce the first New England Machine Learning Day (NEML), which will be held May 16th, 2012 at Microsoft Research New England. The event will bring together local researchers in machine learning and those who use machine learning in applications. There will be a lively poster session during lunch.

Invited speakers:

* Edo Airoldi (Harvard)
* Regina Barzilay (MIT)
* Pedro Felzenszwalb (Brown)
* Tommi Jaakkola (MIT)
* Ce Liu (MSR)
* Andrew McCallum (UMass Amherst)
* Ohad Shamir (MSR)
* Leslie Valiant (Harvard)

Call for Papers: IEEE PAMI Special Issue

Call for Papers
IEEE Transaction on Pattern Analysis and Machine Intelligence Special Issue on Bayesian Nonparametrics

IMS/ASA Spring Research Conference

This year's IMS/ASA Spring Research Conference ("Enabling the Interface Between Statistics and Engineering") will be occurring here at Harvard, hosted jointly by SEAS and the Department of Statistics.

NIPS Workshop on Bayesian Nonparametrics

Ryan Adams and Emily B. Fox (University of Pennsylvania) will be organizing a workshop entitled "Bayesian Nonparametric Methods: Hope or Hype?" with speakers such as Zoubin Ghahramani, Alex Smola, and Chris Holmes. The workshop will be associated with the Neural Information Processing Systems conference in Granada, Spain. The workshop is currently accepting contributions.

New Course: CS281 Advanced Machine Learning

Prof. Ryan Adams will be offering a new course this fall in SEAS: CS281 - Advanced Machine Learning. This course is targeted at graduate students and advanced undergraduates. It will focus on probabilistic approaches to machine learning, with particular attention to Bayesian methods. Topics covered will include Markov chain Monte Carlo, variational inference, Bayesian nonparametrics, matrix factorization models, and more. Students taking the class should feel comfortable with basic linear algebra and probability.

Recent Publications

Revisiting Uncertainty in Graph Cut Solutions. Tarlow, D, Adams RP. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2012. { PDF }

tarlow-revisiting-cvpr-2012.pdf

tarlow-revisiting-cvpr-2012-supp.pdf

Learning Structural Element Patch Models with Hierarchical Palettes. Chua, J, Givoni I, Adams RP, Frey B. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2012. { PDF }

chua-stel-cvpr-2012.pdf

chua-stel-cvpr-2012-supp.pdf

Randomized Optimum Models for Structured Prediction. Tarlow, D, Adams RP, Zemel RS. Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS). 2012. { PDF }

tarlow-randoms-aistats-2012.pdf

On Nonparametric Guidance for Learning Autoencoder Representations. Snoek, J, Adams RP, Larochelle H. Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS). 2012. { arXiv:1102.1492v4 [stat.ML] | PDF }

snoek-nglar-aistats-2012.pdf

Discussion of the Paper: Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods by Girolami and Calderhead. Murray, I, Adams RP. Journal of the Royal Statistical Society, Series B. 73:1-37. 2011. { PDF }

murray-riemann-jrssb-2011.pdf