Sim2Real Transfer in Robotics: Thoughts on Model Pruning and Robust Visual Transfer

We consider the problem of transferring robotic control from simulation to the real world. In particular, we consider two important sub-problems that are often faced: model size and visual robustness.

To decrease the size of our trained models, we develop a one shot pruning technique for recurrent and time series models that significantly reduces our model footprint while maintaining accuracy. This smaller model size is crucial for overcoming hardware limitations in robotics. For our vision system, we develop a new a statistical process, Invariance Through Inference, for adapting visual systems from simulation into the real world. This process shows how we can use statistical inference at test time to extract robust visual features that are constant across simulated and real world models.

Please report video problems to itsupport@slmath.org.

See more of our Streaming videos on our main VMath Videos page.