Week 3 Project Update

Stuff I did this week:

• I've implemented the E2C model and the (synthetic) dataset for the 2D plane navigation task. There are some bugs in the code that I need to work out, but I have a pretty good grasp on the paper now.
• I've started implementing a couple stochastic optimal control algorithms (iLQG, AICO) from this paper. I've written them in MATLAB but will translate to Python later for actual simulation.
• I have a better formulation for the project I am trying to tackle, which I describe below:

In the Muscle-based Locomotion Paper, the robot is controlled by determining robot joint angles (corresponding to locomotive pose), then using a modified Jacobian-transpose controller to determine muscle activations that approximately effect the corresponding joint torques. This might be suitable for generating walking animations, but is not applicable to real robotics for the following reasons:

• The controller is not model-free: i.e. we need explicit to explicitly model rigid-body dynamics before we can solve for muscle activation. If our robot has deforming parts (like an octopus or even a squishy backpack), this gets difficult.
• Robots are not able to accurately measure their localization, pose, and other state variables precisely, which makes planning difficult.

For my project, I propose Model-Free Stochastic Optimal Control for Muscle-Based Robots.

The construction is as follows:

1. Given a fixed robot body plan (muscle routing) and environment (same as task environment), generate a dataset $D$ of initial state observations (images), random action by the robot, and resulting state transition (another image).
2. Train E2C on $D$ to learn dynamics of how the muscle system interacts with the surrounding environment.
3. Use SOC algorithm to perform optimal control (moving from one side of the floor to the other). The robot does not have proprioceptive access to it's "true" state; planning only utilizes an inferred state from it's observations of the environment.

This is useful because we can use this framework to control soft robots in a model-free manner, letting the robot learn it's own body dynamics implicitly.

I hope to finish this in the next two weeks (with the majority of implementation taking place over Spring Break). The remainder of the semester will be examining how to scale up the model to a more complex robot (bipedal walker, 3D, end-to-end refinement), as well as examining the topological latent space formed by E2C.