Abstract
Generating realistic and robust motion for virtual characters under complex physical conditions, such as irregular terrain, real-time control scenarios, and external disturbances, remains a key challenge in computer graphics. While deep reinforcement learning has enabled high-fidelity physics-based character animation, such methods often suffer from limited generalizability, as learned controllers tend to overfit to the environments they were trained in. In contrast, simplified models, such as single rigid bodies, offer better adaptability, but traditionally require hand-crafted heuristics and can only handle short motion segments. In this paper, we present a general learning framework that trains a single-rigid-body (SRB) character controller from long and unstructured datasets, without the reliance on human-crafted rules. Our method enables zero-shot adaptation to diverse environments and unseen motion styles. The resulting controller generates expressive and physically plausible motions in real time and seamlessly integrates with high-level kinematic motion planners without retraining, enabling a wide range of downstream tasks.
Overview
The SRB tracking policy, trained using a combination of reinforcement and supervised learning on flat terrain, generalizes to uneven terrain at inference. A QP solver computes contact forces from predicted actions, while a full-body motion predictor outputs future states. The states are refined via momentum-mapped space–time optimization for rendering.
Terrain Adaptation
All blinded! All executed by a single policy! All trained on flat terrain only!
Hopping on hill.
Walking on pyramid with ball interaction.
Walking on stepping stones.
Unseen Motion Tracking
Trained only on basic locomotion (walking, running, and jumping) over flat terrain, our method generalizes to a wide range of unseen motions beyond the training distribution, as well as to unseen terrains.
Kicking on flat terrain.
Playing volleyball on flat terrain.
Playing basketball on rough terrain.
Fighting on hill terrain.