Here is a still image:
Intuitively, the correct topology is formed by the "stitching" of continuous topological patches in latent space. Because we are uniformly sampling trajectories and not possible positions, the reconstruction fidelity is biased towards places where the agent is seen more often. The topology gets messed up if two adjacent points are difficult to transition between (such as a narrow passageway).
Consequently, increasing the step size "forces" the reconstructions to be meaningful not just between adjacent points, but also points up to some maximum distance away. In other words, instead of just stitching together adjacent points (corresponding to 1 unit away), points are also implicitly stitched to other points within some radius from the dataset samples.
Given this result, I hypothesize that a multi-step reconstruction loss (i.e. minimizing the reconstruction loss over trajectory forecasts, rather than single steps) will be better. I also think that this will improve performance during planning.
If the agent has $T=10$ horizon for use in iLQR, then it seems like a good idea for it's forecasting abilities to be accurate up to around $T=10$ rather than $T=1$. Otherwise, the costs computed far away may correspond to really blurry images.
But... actual reconstructions are substantially worse than the max_dist=1 step size:
There are number of parallel tasks for me at this point:
- Multi-step (recurrent) E2C
- Actual SOC algorithm (this can proceed now that I am reasonably confident about the fidelity of the latent space reconstruction)
- Implement simulation framework for the worm
- This simulation code will generate images that are stored in a database and used to train E2C offline.
- Online training / adaptive exploration would be awesome, but I haven't thought too much about it yet.
No comments:
Post a Comment