Genesis: A Generative and Universal Physics Engine for Robotics and Beyond

Zhou Xian*,1 , Yiling Qiao*,2, Zhenjia Xu*,3,4,6, Tsun-Hsuan Wang*,5, Zhehuan Chen*,7,8, Juntian Zheng*,1,9, Ziyan Xiong*,9, Yian Wang*,7, Mingrui Zhang*,10,11, Pingchuan Ma*,5, Yufei Wang*,1, Zhiyang Dou*,13,14, Byungchul Kim5, Yunsheng Tian5, Yipu Chen12, Xiaowen Qiu7, Chunru Lin7, Tairan He1, Zilin Si1, Yunchu Zhang16, Zhanlue Yang11, Tiantian liu11, Tianyu Li12, Kashu Yamazaki1, Hongxin Zhang7, Huy Ha3, 4, Yu Zhang15, Michael Liu1,17, Shaokun Zheng9, Zipeng Fu4, Qi Wu4, Yiran Geng8, Feng Chen9,13, Milky, Yuanming Hu11, Chelsea Finn4, Guanya Shi1, Lingjie Liu14, Taku Komura13, Zackory Erickson1, David Held1, Minchen Li1, Linxi "Jim" Fan6, Yuke Zhu6,18, Wojciech Matusik5, Dan Gutfreund19, Shuran Song3,4, Daniela Rus5, Ming Lin2, Bo Zhu12, Katerina Fragkiadaki1, Chuang Gan7,19
* Core Contributors
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Code Documentation Paper (Coming Soon)

Genesis is a comprehensive physics simulation platform designed for general purpose Robotics, Embodied AI, & Physical AI applications. It is simultaneously multiple things:

  • A universal physics engine re-built from the ground up, capable of simulating a wide range of materials and physical phenomena.
  • A lightweight, ultra-fast, pythonic, and user-friendly robotics simulation platform.
  • A powerful and fast photo-realistic rendering system.
  • A generative data engine that transforms user-prompted natural language description into various modalities of data.

Powered by a universal physics engine re-designed and re-built from the ground up, Genesis integrates various physics solvers and their coupling into a unified framework. This core physics engine is further enhanced by a generative agent framework that operates at an upper level, aiming towards fully automated data generation for robotics and beyond. Our generative framework aims to automate generating data including the following modalities:

  • Physically-accurate & spatially consistent videos
  • Camera motion & parameters
  • Human and animal character motion
  • Robotic manipulation & locomotion policy, deployable to real-world
  • Fully interactive 3D scene
  • Open-world articulated object generation
  • Speech audio, facial animation & emotion

Currently, we are open-sourcing the underlying physics engine and the simulation platform. Access to the generative framework will be rolled out gradually in the near future.
This webpage serves as a gallery showcasing different capabilities of the Genesis platform. Please refer to our documentation site to for detailed installation steps, tutorials and API references.


Optimized Performance

Genesis is a highly-optimized physics engine that leverages GPU-accelerated parallel computation, with features like optimized collision checking, auto-hibernation, contact island, etc. Genesis delivers an unprecedented simulation speed in various scenes.

When simulating a manipulation scene (with a single plane and a Franka arm), Genesis runs at 43 millision FPS, which is 430,000 times faster than in real time.

In large-scale simulation, Genesis utilizied auto-hibernation to speed up simulation of entities that are in converged and static states. (This feature is under testing and will be release in version 0.1.1).



Speed comparison with popular CPU and GPU-based robotics simulator.



Generating 4D dynamical & physical world

Genesis's physics engine is empowered by a VLM-based generated agent that uses the APIs provided by the simulation infrastructure as tools to create 4D dynamic worlds, which can then be used as a foundational data source for extracting various modalities of data. Together with modules for generating camera and object motion, we are able to generate physically-accurate and view-consistent videos and other modalities of data.




Character Motion Generation


"A miniature Wukong holding a stick in his hand sprints across a table surface for 3 seconds, then jumps into the air, and swings his right arm downward during landing. The camera begins with a close-up of his face, then steadily follows the character while gradually zooming out. When the monkey leaps into the air, at the highest point of the jump, the motion pauses for a few seconds. The camera circles around the character for 360 degrees, and slowly ascends, before the action resumes."



"A Japanese samurai performs boxing."

"A Chinese soldier performs the Gangnam Style dance."

"A Roman soldier walks forward like a zombie."

Robotic Policy Generation

Genesis aims to use generative robotic agent and physics engine to automatically generate robotic policies and demonstration data for various skills under different scenarios. For the high-level motivation and more details behind the module, see RoboGen and our upcoming paper.

"A mobile franka arm heats the corn with the bowl and the microwave."

"A mobile franka arm throws all the objects on the floor into the basket."

"A mobile franka arm re-organizes the books on the table by pushing the brown and the white books to align with the red one."

"A fleet of 24 drones (arranged in 4x6) take off together from the ground and perform a flip together."

"A Unitree Go2 Quadraped runs under the rain." (Sim)

"A Unitree H1 humanoid performs single-foot hopping." (Sim, because our robot is broken :( )

"A Unitree H1-2 humanoid walks forward." (Sim2Real)

"A robotic leap hand walks forward like a quadrapad using its all four fingers." (Sim)

"A robotic shadow hand walks forward like a human using only its index and middle fingers." (Sim)

Quadraped locomotion with various gaits (Sim2Real)

Quadraped 2-feet hopping (Sim2Real)

"Quadraped performs handstand with its 2 front legs." (Sim2Real)

Quadraped locomotion with various base heights (Sim2Real)

More Sim2Real Policy Transfer

Loco-manipulation of large & underactuated objects

Quadraped single-backflip


Quadraped double-backflip

3D & Fully Interactive Scene Generation

"A home interior scene with a living room (including a dinning space), a restroom, a study and a bedroom."

"A restaurant interior."

Open-world Articulated Object Generation

We generate articulated of open-set mesh assets, extending beyond categories contained in human-annotated articulated object asset. For more details, see our upcoming paper (under submission now).

Soft Robot

Genesis can simulate generated soft or hybrid (soft skin and rigid skeleton) robot. To get started, check tutorial.


Worm (MPM muscle).

"A walking teapot." (hybrid))

"A gripper that can rotate a box.", attached to a UR5. (MPM + rigid)

A soft gripper navigates a box and picks up a pen cap. (MPM + rigid)

Speech Audio, Facial Animation & Emotion Generation

"Facial animation and speech of the text [Genesis is a physical platform designed for general purpose Robotics/Embodied AI/Physical AI applications]. Emotion transitions from neutral to angry, then to happy".

Generalize across different facial characters and emotions

BibTeX

@software{Genesis,
          author = {Genesis Authors},
          title = {Genesis: A Universal and Generative Physics Engine for Robotics and Beyond},
          month = {December},
          year = {2024},
          url = {https://github.com/Genesis-Embodied-AI/Genesis}
        }