Snip20190225_3.png

code: https://bitbucket.csiro.au/scm/˜col549/quantifyingthe-reality-gap-in-robotic-manipulation-tasks.git

Abstract

Goal--quantify the accuracy of various simulators compared to a real world robotic reaching and interaction task.

Problem--The ‘reality gap’ prevents solutions developed or learnt in simulation from performing well, or at at all, when transferred to real-world hardware.

Method--Making use of a Kinova robotic manipulator and a motion capture system, we record a ground truth enabling comparisons with various simulators, and present quantitative data for various manipulation-oriented robotic tasks.

Introduction

simulation的好处：

No wear or damage to real-world hardware
Many instantiations of a simulation can run in parallel
(Often) Faster than real-time operation
Instant access to robots without having to purchase
Human intervention is not required

但是simulation也有坏处：

Learning-based approaches难以tranfer到real-world中。这个毛病在Evolutionary Robotics中被称为‘Reality Gap’[4]。

Gap的产生主要是由于这几点不确定的影响：

actuators (i.e. torque characteristics, gear backlash, ...),
sensors (i.e. sensor noise, latencies, and faults)
temporal dynamics
the physics that govern interactions between robots and objects in their environment (i.e. deformable objects, fluid dynamics, ...)

本paper中的方法：使用6DOF Kinova Mico2 arm，在多个模拟引擎中设置相同的环境进行模拟，将获得的数据和真实平台上采集到的数据（groud truth）进行比较。如Fig. 1

Snip20190225_4.png

Answering:

What are the differences between the chosen physics engines when simulating the same scenario?
Are there specific types of interactions that some simulators can accurately model, compared to other simulators we test?

Related Work

A. Bridging the Reality Gap

1. Domain randomisation 域随机化

在机器视觉中很常用，给一个trained model输入随机的input（包括colour, shading, rendering, camera position, etc.）

[12]S. James, A. J. Davison, and E. Johns, “Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task,” 7 2017\. [Online]. Available: http://arxiv.org/abs/1707.02267 [13]J. Borrego, R. Figueiredo, A. Dehban, P. Moreno, A. Bernardino, and J. Santos-Victor, “A generic visual perception domain randomisation framework for Gazebo,” in 18th IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2018\. IEEE, 4 2018, pp. 237–242\. [Online]. Available: https://ieeexplore.ieee.org/document/8374189/

它的sim-to-real随着domain adaptation和 Generative Adversarial Networks (GAN)等方法的出现而被解决。[3][15]

[16]使用domain adaptation，在simulation中训练，面对 a dataset of unseen objects，在真实抓取中能达到76.7%的成功率。

[16]K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping,” 2017. [Online]. Available: http://arxiv.org/abs/1709.07857

2. Optimisation of the simulated environments

收集真实环境的数据，是的simulator和real world的observation更加接近

[17]J. C. Zagal, J. Ruiz-del Solar, and P. Vallejos, “Back to reality: Crossing the reality gap in evolutionary robotics,” IFAC Proceedings Volumes, vol. 37, no. 8, pp. 834–839, 7 2004\. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1474667017320840 [18]T. Laue and M. Hebbel, “Automatic Parameter Optimization for a Dynamic Robot Simulation,” in RoboCup 2008: Robot Soccer World Cup XII, L. Iocchi, H. Matsubara, A. Weitzenfeld, and C. Zhou, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 121–132.

3. 使用多个simulators来克服单个simulator的不足

[21]created the Physics Abstraction Layer (PAL), a unified interface between multiple physics engines and successfully evolved a PID controller for an Autonomous Underwater Vehicle. [22]evolved behaviour for a Nao robot using first the V-Rep simulator and then for successful controllers the Webots simulator to remove controllers that were exploiting unrealistic scenarios.

B. Physics Engines

Boeing et al. [23] compared PhysX, Bullet, JigLib, Newton, Open Dynamics Engine (ODE), Tokamak and True Axis;

他们发现Bullet表现最好，但是没有一个能够胜任所有的tasks

Chung et al. [24] likewise found when testing Bullet, Dynamic Animation and Robotics Toolkit (DART), MuJoCo, and ODE

他们发现在不同的任务和不同的环境中，有各自不同的物理引擎更加适合

Gonzalez-Badillo et al. [25] showed that PhysX performs better than Bullet for non-complex geometries but is unable to simulate more complex geometries to the same degree as Bullet.

他们发现对于几何简单的，PhysiX比bullet表现更好，但是几何复杂的，还是bullet表现更优

Aim：选出最适合learning to grasp任务的物理引擎

C. Simulation Selection

选择simulator的条件：

mature, well maintained simulators with active communities and good documentation practices to facilitate the development of robotics research.
simulators that provided a common programming language interface whilst also providing access to the Robot Operating System (ROS).

选取出来的simulator：V-Rep, MuJoCo and PyBullet

相关的Physics Engine：Bullet, ODE, Vortex, Newton and Mujoco.

Method

实验设备：Kinova Mico2 6DOF arm with an attached KG-3 gripper固定在0.75 * 1.8 米的桌子边缘，有一个可抓取的ABS 3D打印的cube（边长7.5cm，88.4g，上有4 tracking markers）放于旁边，Kinova爪子上装有4个tracking markers，底盘上装有4 markers，世界坐标系在桌子的另一侧。

Snip20190225_5.png

simulator中使用的是Kinova官方的URDF文件。

Kinova使用joint velocities进行控制——因为比position controller的保真度高，防止position controller和sent motion commands的interfering。WHY？

controller的更新速率是5Hz。

在不同的simulator和真实环境下执行相同的movements。

A. Real World Ground Truth

使用有24个相机的motion capture system，装在8 * 8 * 4 metre gantry上，用于记录real-world data。校准后的精度为1mm

Snip20190225_6.png

记录6DOF的poses，包括x, y and z positions and orientation as Euler angles相对于rogid bases。

使用Kinova的ROS官方包，支持joint rotations in degrees 和100Hz的joint velocity commands。

B. Simulation

实验中比较的三种simulators：

V-Rep [28] 场景有V-Rep的插件导入，生成.tt的二进制文件。Joint settings were changed to “Lock motor when target velocity is zero”.
PyBullet [30] PyBullet’s time step was explicitly fixed to the value of 0:01 seconds.
Mujoco [29] The mujoco-py Python wrapper maintained by OpenAI was used as the interface for MuJoCo. URDF文件需转换成XML文件（用MuJoCo compile script 完成）。手动添加驱动器和传感器with the only altered parameter in the XML file being the kv velocity feedback gain. The simulator time step of the simulation was set at 0:0001 seconds as this provided a stable simulation.

Experiments and Results

所有的实验都重复了20遍。

3个场景：

(a) beginning with a very basic robotic movement of one joint单个关节动作

(b) moving onto more complex multijoint movement tasks多个关节动作

A. Scene 1: Single Joint Movement

Joint #2 在6秒内，从初始位置旋转100度到终止位置(i.e. 120 control cycles at 5Hz)。所有其他joints都设置为0度。

Fig.4中纵坐标是euclidean distance error，可以发现Vortex, Newton and PyBullet最后误差最小，在Table1中有数值比较。

Bullet283 and Bullet278到达峰值的时间最短，但是相对于其他引擎振荡较大，调节时间较长。

Snip20190225_7.png

Snip20190225_8.png

B. Scene 2: Multi Joint Movement

20秒内移动

Joint #2 between 0 | 90 | 0 | 90 | 0 degrees

Joint #5 between 0 | 90 | 0 | -90 | 0 degrees

Newton and Vortex紧随motion capture path，累计误差正负5.5 - 6 metres, PyBullet也表现不错，累计误差在 7 metres。

MuJoCo, Bullet283 and Bullet278 model the motion capture closer between 0-5 seconds and 10-15seconds, 因为顺着重力运动。当逆着重力运动时，只有MoJoCo可以达到终点。

Snip20190225_9.png

C. Scene 3: Interaction with the cube

Joint #2 #3 #4按照一定的顺序移动来沿着桌面推动cube

Snip20190225_10.png

在Table 1 的Total列中显示，Pybullet在chosen scene中累积误差最小，只有正负45.5 metres，而V-Rep的累积误差达到了正负131 metres。可能的原因有两点：

the default values being set differently (i.e. we set the timestep of PyBullet to be 0:01 seconds while V-Rep generically uses 0:05 seconds),
the underlying implementation between the V-Rep simulator and the physics engine at a scale inaccessible to the user.

在不同的阶段使用不同的物理引擎，可以这样选择：

使用Newton, PyBullet and Vortex来控制机械臂，to model the kinematics,但是不使用其结果。
结合使用MuJoCo, Bullet283 and Bullet278进行cube的interaction计算

Conclusion

差不多总结了一下，这里不赘述了～

Quantifying the Reality Gap in Robotic Manipulation Tasks