# A Bayesian view on motor control and planning

Paper

Marc Toussaint, Christian Goerick (2010): A Bayesian view on motor control and planning. In Olivier Sigaud, Jan Peters (Eds.): From motor to interaction learning in robots, Springer, print expected in 2010.

Summary

Summary here.

Comments on the relationship between linearized belief propagation and covariant gradient descent

Earlier we derived an algorithm that used belief propagation to efficiently compute the covariant gradient for trajectory optimization under a kinetic energy metric.  A special case of the kinetic energy term, in which we assume all of the robot’s mass is concentrated at the end-effector, reduces to

$\phi(\xi) = \frac{\lambda}{2}\sum_{i=1}^{n-1} \|\phi(q_{i+1}) - \phi(q_i)\|^2$,

where $\xi = (q_1,\ldots,q_n)$ and $\phi(q)$ is the forward kinematics function.  Covariant gradient descent under the kinetic energy metric (for this special case) is extremely closely related to a the algorithm proposed in this paper, particularly the variant discussed in Section 4.4 which additionally incorporates collision constraints.  There are four stages to the algorithm (I’ve grouped them slightly differently from the paper):

1. Forward-backward in workspace with cost integration.
2. Propagate down to the configuration space.
3. Forward-backwith in configuration space.
4. Propagate up to workspace.

The analogy under covariant gradient descent is

1. Compute the workspace cost gradient.  The arc-length parameterization CHOMP uses for the cost functional produces a workspace gradient that is in a sense qualitatively related to smoothing the gradient across the workspace trajectory (although the two operations are quantitatively different).  This point is the biggest difference between the two algorithms and I have yet to completely reconcile it.
2. Backpropagate the gradient down to the configuration space using the chain rule.
3. Compute the covariant gradient by solving the linear system.
4. Push the new configurations up to the workspace using the forward kinematics.

Verifying the details of these points mathematically by showing that the updates are equivalent is tedious and dependent on the correct choice of cost function and metric (e.g. Toussaint’s cost function cares only about the end-effector currently, although the extension is straightforward), so my question is whether there is a higher level way to relate the algorithms to one another.

Writing out the equivalent objective is straightforward, but behavior of belief propagation is not well understood.  Optimization, on the other hand, has a strong history and recent results on optimization over Riemannian manifolds (see the literature listed in the reading group post) have made strides toward a formal understanding within this generalized context.  My belief is that linearized belief propagation, such as the algorithm proposed in this paper, has general connections to covariant gradient descent on a Riemannian manifold.  (Additionally, it’s well known that even for Gaussian graphical models with loops, the estimates of covariance matrices are not accurate under loopy belief propagation, so this algorithm may be more accurately viewed as primarily an optimization procedure.)  Formalizing this connection may provide tools for analyzing the convergence and convergence rate of continuous belief propagation algorithms, and on the flip side, may give us more computationally efficient ways of exploiting structure in covariant gradient descent.

## Author: nratliff

Research scientist at Intel Labs, Pittsburgh.

## 3 thoughts on “A Bayesian view on motor control and planning”

1. I found the notational difference between the moment parametrization and the natural parametrization hard to follow. Perhaps a better notation might be
$\mathcal{N} (m, \Sigma)$ and $\bar{\mathcal{N}} (j, P)$,
with the bar denoting the difference.

2. It is strange that the covariance of the forward kinematics is used to signify “how tight we want to follow a given task constraint” (page 11), rather than what the math says it should signify: the uncertainty in the forward kinematics, perhaps due to joint flexion or backlash. I feel like the prior on x should be used to signify “I don’t care about where the intermediate x’s end up. I only care about the final end-effector position”

1. Covariance on the forward kinematics make sense to me only in special circumstances. If you think about it in terms of the map estimate (in particular, the corresponding non-probabilistic objective function suggested by the model), we can view the forward kinematics covariance as signifying the strength of a spring force pulling the random variable $x$ toward the actual end-effector location $\phi(q)$. The bigger the covariance, the smaller that spring force. The constraints on the problem are only that the variables $x$ are collision free, so it seems clear that large forward kinematics covariances can easily lead to collisions (which are essentially violated task constraints). One interpretation of the forward kinematics covariance might be that it models (or at least is correlated with) a notion of compliance in the arm. Compliant arms may successfully violate collision constraints by brushing up against a wall without compromising success. This scenario is the only one that makes sense to me, though, in the presence of obstacles.