E2C is working... sort of

Finally got E2C implementation working.

Here's the visualization of the latent space of the plane task (moving randomly around in a simple 2D maze) unfolding over time for the first few iterations.

For the first ~5000 iterations, the latent space is completely mixed, but the colors separate somewhat during training.

Unfortunately, the results do not look as pretty as those of the paper (latent space becomes nearly identical to true space), but perhaps it is because my data points are less densely populated at the bottom of the image? I.e. the state space is not uniformly sampled (the trajectories sample more from the top region than the bottom one.

The next steps:

• Get iLQG control working with this agent. The task space hasn't properly unfolded, which is kind of concerning. 
• Resume work on Box2D + LiquidFun implementation of muscle robot.

A rather daunting difficulty right now is the static nature of the observations. The environment isn't moving, and I'm uncertain how "accurate" the latent manifold will be in the presence of visual noise.

For example, can I get the worm to cross a moving platform across a chasm?