INB-Lunch-Seminar

The Bayesian inference (or Free Energy Minimization) view on decision
making and goal-directed behavior

Marc Toussaint

Classical Computer Science approaches to behavioral problems -- such
as Reinforcement Learning, control and behavior planning -- are
dominated by Bellman's principle and dynamic programming. However,
some fundamental questions are hard to address in this framework: How
can planning and decision making be realized on distributed
representations? How on hierarchies and mixed (discrete, continuous)
representations? How can appropriate representations be learnt? And
what is a coherent computational paradigm that solves behavioral
problems equally to state estimation and sensor processing problems?
Recently there have been a series of papers showing that behavioral
problems can be reformulated as a problem of Bayesian inference or
Free Energy Minimization in graphical models where actions, states,
observations and rewards are equally represented as coupled random
variables.  The most important implication of this view is that
existing Machine Learning methods such as inference on factored and
hierarchical representations, likelihood maximization, and
unsupervised learning (which were classically associated to sensor
processing and learning sensor representations) can now be transferred
to the realm of behavior organization. In this talk I will introduce
to the general approach and give examples from our recent work,
focussing on two application domains: first, using graphical model
techniques to find hierarchies in Partially Observable Markov Decision
Processes; and second, learning and planning in stochastic relational
worlds, which builds on recent advances in the field of statistical
relational learning. In both cases the graphical model view opens the
door to fundamentally novel algorithmic solutions that go beyond the
state-of-the-art by exploiting the structure of the problems.