dm_control

Google DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo.

3,713

660

3,713

View on GitHub

Top Related Projects

mujoco

7,802

Multi-Joint dynamics with Contact. A general purpose physics simulator.

gym

34,461

A toolkit for developing and comparing reinforcement learning algorithms.

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

bullet3

12,417

Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.

roboschool

2,115

DEPRECATED: Open-source software for robot simulation, integrated with OpenAI Gym.

Quick Overview

dm_control is a Python library developed by DeepMind for physics-based control tasks and reinforcement learning environments. It provides a suite of challenging control tasks and a flexible framework for creating custom environments, all built on top of the MuJoCo physics engine.

Pros

Rich set of pre-built control tasks and environments
Highly customizable and extensible framework
Efficient and accurate physics simulation using MuJoCo
Seamless integration with machine learning libraries like TensorFlow and JAX

Cons

Requires MuJoCo, which may have licensing restrictions
Steeper learning curve compared to simpler RL libraries
Limited documentation for advanced customization
May be overkill for simple control tasks

Code Examples

Loading and running a pre-built environment:

from dm_control import suite

env = suite.load(domain_name="cartpole", task_name="swingup")
time_step = env.reset()
while not time_step.last():
    action = env.action_spec().generate_value()
    time_step = env.step(action)

Creating a custom environment:

from dm_control import mujoco
from dm_control.rl import control

class MyEnvironment(control.Environment):
    def __init__(self):
        physics = mujoco.Physics.from_xml_string("<mujoco>...</mujoco>")
        super().__init__(physics, control_timestep=0.01)

    def step(self, action):
        # Implement step logic
        pass

env = MyEnvironment()

Rendering an environment:

from dm_control import viewer

env = suite.load("humanoid", "stand")
viewer.launch(env)

Getting Started

To get started with dm_control:

Install MuJoCo (follow instructions on the MuJoCo website)
Install dm_control:
```
pip install dm_control
```

Run a simple example:

from dm_control import suite
import numpy as np

env = suite.load(domain_name="cartpole", task_name="swingup")
time_step = env.reset()
for _ in range(1000):
    action = np.random.uniform(-1, 1, size=env.action_spec().shape)
    time_step = env.step(action)
    if time_step.last():
        break

This example loads the cartpole swingup task, runs it for 1000 steps or until termination, taking random actions.

Competitor Comparisons

mujoco

7,802

Multi-Joint dynamics with Contact. A general purpose physics simulator.

Pros of mujoco

Direct access to MuJoCo physics engine, allowing for more low-level control and customization
Potentially faster simulation speed due to closer integration with the physics engine
More comprehensive documentation and examples for advanced usage

Cons of mujoco

Steeper learning curve for beginners due to lower-level API
Less focus on reinforcement learning tasks out-of-the-box
Requires more setup and configuration for complex environments

Code Comparison

dm_control example:

from dm_control import suite
env = suite.load(domain_name="cartpole", task_name="swingup")
timestep = env.reset()
action = env.action_spec().generate_value()
next_timestep = env.step(action)

mujoco example:

import mujoco
model = mujoco.load_model_from_path("model.xml")
data = mujoco.MjData(model)
mujoco.mj_step(model, data)
qpos = data.qpos
qvel = data.qvel

The dm_control repository provides a higher-level interface for reinforcement learning tasks, while mujoco offers more direct access to the physics engine. dm_control is generally easier for beginners and provides pre-built environments, whereas mujoco allows for more customization and potentially faster simulations at the cost of increased complexity.

gym

34,461

A toolkit for developing and comparing reinforcement learning algorithms.

Pros of Gym

Wider variety of environments, including classic control, robotics, and Atari games
Larger community and more third-party extensions
Simpler API, making it easier for beginners to get started

Cons of Gym

Less focus on physics-based simulations compared to dm_control
Fewer built-in visualization tools
Some environments may be less realistic or detailed than those in dm_control

Code Comparison

Gym:

import gym
env = gym.make('CartPole-v1')
observation = env.reset()
for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, done, info = env.step(action)

dm_control:

from dm_control import suite
env = suite.load(domain_name='cartpole', task_name='swingup')
time_step = env.reset()
for _ in range(1000):
    action = env.action_spec().generate_value()
    time_step = env.step(action)

Both libraries provide similar functionality for reinforcement learning environments, but with different focuses and strengths. Gym offers a broader range of environments and a simpler API, while dm_control excels in physics-based simulations and provides more detailed control over the environment.

ml-agents

16,887

Pros of ml-agents

Integrated with Unity game engine, allowing for easy creation of complex 3D environments
Supports a wide range of learning algorithms, including PPO, SAC, and imitation learning
Provides a user-friendly interface for non-experts to train and deploy AI agents

Cons of ml-agents

Limited to Unity environment, which may not be suitable for all research scenarios
Potentially slower performance compared to specialized frameworks like dm_control
Steeper learning curve for those unfamiliar with Unity development

Code Comparison

ml-agents:

from mlagents_envs.environment import UnityEnvironment
env = UnityEnvironment(file_name="MyUnityEnvironment")
behavior_name = list(env.behavior_specs)[0]
decision_steps, terminal_steps = env.get_steps(behavior_name)

dm_control:

from dm_control import suite
env = suite.load(domain_name="cartpole", task_name="swingup")
time_step = env.reset()
action = env.action_spec().generate_value()
next_time_step = env.step(action)

Both repositories offer powerful tools for reinforcement learning research, with ml-agents focusing on game-like environments and dm_control specializing in physics-based control tasks. The choice between them depends on the specific research requirements and the user's familiarity with Unity or Python-based frameworks.

bullet3

12,417

Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.

Pros of bullet3

More comprehensive physics simulation, including soft body dynamics and fluid simulation
Wider range of supported platforms, including mobile devices and game consoles
Larger and more active community, with frequent updates and contributions

Cons of bullet3

Steeper learning curve due to its broader scope and more complex API
Less focus on reinforcement learning tasks compared to dm_control
May require more computational resources for advanced simulations

Code Comparison

dm_control example:

from dm_control import suite
env = suite.load(domain_name="cartpole", task_name="swingup")
timestep = env.reset()
action = env.action_space.sample()
next_timestep = env.step(action)

bullet3 example:

import pybullet as p
p.connect(p.DIRECT)
p.loadURDF("r2d2.urdf")
for i in range(1000):
    p.stepSimulation()

Both libraries offer Python bindings for ease of use, but dm_control provides a more streamlined interface for reinforcement learning tasks, while bullet3 offers more flexibility for general physics simulations.

roboschool

2,115

DEPRECATED: Open-source software for robot simulation, integrated with OpenAI Gym.

Pros of Roboschool

Simpler installation process, fewer dependencies
Supports a wider range of operating systems
Includes more diverse environments and tasks

Cons of Roboschool

Less actively maintained, with fewer recent updates
Lower-quality physics simulation compared to dm_control
Limited documentation and community support

Code Comparison

dm_control example:

from dm_control import suite
env = suite.load(domain_name="cartpole", task_name="swingup")
timestep = env.reset()
action = env.action_space.sample()
next_timestep = env.step(action)

Roboschool example:

import gym
import roboschool
env = gym.make('RoboschoolCartPole-v1')
observation = env.reset()
action = env.action_space.sample()
observation, reward, done, info = env.step(action)

Both libraries provide similar functionality for creating and interacting with reinforcement learning environments. dm_control offers more precise physics simulation and better documentation, while Roboschool provides a wider range of environments and broader system compatibility. The choice between them depends on specific project requirements and the desired level of simulation fidelity.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

`dm_control`: Google DeepMind Infrastructure for Physics-Based Simulation.

Google DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.

An introductory tutorial for this package is available as a Colaboratory notebook:

Overview

This package consists of the following "core" components:

dm_control.mujoco: Libraries that provide Python bindings to the MuJoCo physics engine.
dm_control.suite: A set of Python Reinforcement Learning environments powered by the MuJoCo physics engine.
dm_control.viewer: An interactive environment viewer.

Additionally, the following components are available for the creation of more complex control tasks:

dm_control.mjcf: A library for composing and modifying MuJoCo MJCF models in Python.
dm_control.composer: A library for defining rich RL environments from reusable, self-contained components.
dm_control.locomotion: Additional libraries for custom tasks.
dm_control.locomotion.soccer: Multi-agent soccer tasks.

If you use this package, please cite our accompanying publication:

@article{tunyasuvunakool2020,
         title = {dm_control: Software and tasks for continuous control},
         journal = {Software Impacts},
         volume = {6},
         pages = {100022},
         year = {2020},
         issn = {2665-9638},
         doi = {https://doi.org/10.1016/j.simpa.2020.100022},
         url = {https://www.sciencedirect.com/science/article/pii/S2665963820300099},
         author = {Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and
                   Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and
                   Timothy Lillicrap and Nicolas Heess and Yuval Tassa},
}

Installation

Install dm_control from PyPI by running

pip install dm_control

Note: dm_control cannot be installed in "editable" mode (i.e. pip install -e).

While dm_control has been largely updated to use the pybind11-based bindings provided via the mujoco package, at this time it still relies on some legacy components that are automatically generated from MuJoCo header files in a way that is incompatible with editable mode. Attempting to install dm_control in editable mode will result in import errors like:
ImportError: cannot import name 'constants' from partially initialized module 'dm_control.mujoco.wrapper.mjbindings' ...
The solution is to pip uninstall dm_control and then reinstall it without the -e flag.

Versioning

Starting from version 1.0.0, we adopt semantic versioning.

Prior to version 1.0.0, the dm_control Python package was versioned 0.0.N, where N was an internal revision number that increased by an arbitrary amount at every single Git commit.

If you want to install an unreleased version of dm_control directly from our repository, you can do so by running pip install git+https://github.com/google-deepmind/dm_control.git.

Rendering

The MuJoCo Python bindings support three different OpenGL rendering backends: EGL (headless, hardware-accelerated), GLFW (windowed, hardware-accelerated), and OSMesa (purely software-based). At least one of these three backends must be available in order render through dm_control.

Hardware rendering with a windowing system is supported via GLFW and GLEW. On Linux these can be installed using your distribution's package manager. For example, on Debian and Ubuntu, this can be done by running sudo apt-get install libglfw3 libglew2.0. Please note that:
- dm_control.viewer can only be used with GLFW.
- GLFW will not work on headless machines.
"Headless" hardware rendering (i.e. without a windowing system such as X11) requires EXT_platform_device support in the EGL driver. Recent Nvidia drivers support this. You will also need GLEW. On Debian and Ubuntu, this can be installed via sudo apt-get install libglew2.0.
Software rendering requires GLX and OSMesa. On Debian and Ubuntu these can be installed using sudo apt-get install libgl1-mesa-glx libosmesa6.

By default, dm_control will attempt to use GLFW first, then EGL, then OSMesa. You can also specify a particular backend to use by setting the MUJOCO_GL= environment variable to "glfw", "egl", or "osmesa", respectively. When rendering with EGL, you can also specify which GPU to use for rendering by setting the environment variable MUJOCO_EGL_DEVICE_ID= to the target GPU ID.

Additional instructions for Homebrew users on macOS

The above instructions using pip should work, provided that you use a Python interpreter that is installed by Homebrew (rather than the system-default one).
Before running, the DYLD_LIBRARY_PATH environment variable needs to be updated with the path to the GLFW library. This can be done by running export DYLD_LIBRARY_PATH=$(brew --prefix)/lib:$DYLD_LIBRARY_PATH.

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

Top Related Projects

Quick Overview

Pros

Cons

Code Examples

Getting Started

Competitor Comparisons

Pros of mujoco

Cons of mujoco

Code Comparison

Pros of Gym

Cons of Gym

Code Comparison

Pros of ml-agents

Cons of ml-agents

Code Comparison

Pros of bullet3

Cons of bullet3

Code Comparison

Pros of Roboschool

Cons of Roboschool

Code Comparison

Convert designs to code with AI

README

dm_control: Google DeepMind Infrastructure for Physics-Based Simulation.

Overview

Installation

Versioning

Rendering

Additional instructions for Homebrew users on macOS

Top Related Projects

Convert designs to code with AI

`dm_control`: Google DeepMind Infrastructure for Physics-Based Simulation.