SenseAct: 现实世界机器人学习任务计算框架

SenseAct: 现实世界机器人学习任务计算框架

SenseAct: A computational framework for real-world robot learning tasks

This repository provides the implementation of several reinforcement learning tasks with multiple real-world robots. These tasks come with an interface similar to OpenAI-Gym so that learning algorithms can be plugged in easily and in a uniform manner across tasks. All the tasks here are implemented based on a computational framework of robot-agent communication proposed by Mahmood et al. (2018a), which we call SenseAct. In this computational framework, agent and environment-related computations are ordered and distributed among multiple concurrent processes in a specific way. By doing so, SenseAct enables the following:

Timely communication between the learning agent and multiple robotic devices with reduced latency,Easy and systematic design of robotic tasks for reinforcement learning agents,Facilitate reproducible real-world reinforcement learning.

This repository provides the following real-world robotic tasks, which are proposed by Mahmood et al. (2018b) as benchmark tasks for reinforcement learning algorithms:

Universal-Robots (UR) robotic arms:

Tested on UR Software v. 3.3.4.310

UR-Reacher (both 2 joint and 6 joint control)

Dynamixel (DXL) actuators:

Currently we only support MX-64AT.

DXL-ReacherDXL-Tracker

iRobot Create 2 robots:

Create-MoverCreate-Docker

Mahmood et al. (2018b) provide extensive results comparing multiple reinforcement learning algorithms on the above tasks, and Mahmood et al. (2018a) show the effect of different task-setup elements in learning. Their results can be reproduced by using this repository (see documentation for more information).

Versions

The branch master is the latest official release and dev is current development branch.

Installation

SenseAct uses Python3 (>=3.5), and all other requirements are automatically installed via pip.

On Linux and Mac OS X, run the following:

git clone https://github.com/kindredresearch/SenseAct.gitcd SenseActpip install -e . or pip3 install -e . depends on your setup

To replicate experimental results from the paper please install the tag v0.1.1 from the git repo.

git fetch --all --tagsgit checkout tags/v0.1.1

Additional instruction for installing OpenAI Baselines needed for running the advanced examples is given in the corresponding readme.

Additional installation steps for Dynamixel-based tasks (Linux only)

Dynamixels can be controlled by drivers written using either ctypes by Robotis or pyserial, which can be chosen by passing either True (ctypes) or False (pyserial) as an argument to the use_ctypes_driver parameter of a Dynamixel-based task (e.g., see examples/advanced/dxl_reacher.py). We found the ctypes-based driver to provide substantially more timely and precise communication compared to the pyserial-based one.

In order to use the CType-based driver, we need to install gcc and relevant packages for compiling the C libraries:

sudo apt-get install gcc-5 build-essential gcc-multilib g++-multilib

Then run the following script to download and compile the Dynamixel driver C libraries:

sudo bash setup_dxl.sh

For additional setup and troubleshooting information regarding Dynamixels, please see DXL Docs.

Tests

You can check whether SenseAct is installed correctly by running the included unit tests.

cd SenseActpython -m unittest discover -b

Support

Installation problems? Feature requests? General questions?

read through examples and API documentationcreate github issues on the SenseAct projectjoin the mailing list https://groups.google.com/forum/#!forum/senseact

Acknowledgments

This project is developed by the Kindred AI Research team. Rupam Mahmood, Dmytro Korenkevych, and Brent Komer originally developed the computational framework and the UR tasks. William Ma developed the Create 2 tasks and contributed substantially by adding new features to SenseAct. Gautham Vasan developed the DXL tasks. Francois Hogan developed the simulated task.

James Bergstra provided support and guidance throughout the development. Adrian Martin, Scott Rostrup, and Jonathan Yep developed the pyserial DXL driver for a Kindred project, which was used for the SenseAct DXL Communicator. Daniel Snider, Oliver Limoyo, Dylan Ashley, and Craig Sherstan tested the framework, provided thoughtful suggestions, and confirmed the reproducibility of learning by running experiments on real robots.

Citing SenseAct

For the SenseAct computational framework and the UR-Reacher tasks, please cite Mahmood et al. (2018a). For the DXL and the Create 2 tasks, please cite Mahmood et al. (2018b).

Mahmood, A. R., Korenkevych, D., Komer,B. J., Bergstra, J. (2018a). Setting up a reinforcement learning task with a real-world robot. In IEEE/RSJ International Conference on Intelligent Robots and Systems. Mahmood, A. R., Korenkevych, D., Vasan, G., Ma, W., Bergstra, J. (2018b). Benchmarking reinforcement learning algorithms on real-world robots. In Proceedings of the 2nd Annual Conference on Robot Learning.

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。