Paper

{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2603.03279v1</id>\n    <title>ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation</title>\n    <updated>2026-03-03T18:59:29Z</updated>\n    <link href='https://arxiv.org/abs/2603.03279v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2603.03279v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>Achieving autonomous and versatile whole-body loco-manipulation remains a central barrier to making humanoids practically useful. Yet existing approaches are fundamentally constrained: retargeted data are often scarce or low-quality; methods struggle to scale to large skill repertoires; and, most importantly, they rely on tracking predefined motion references rather than generating behavior from perception and high-level task specifications. To address these limitations, we propose ULTRA, a unified framework with two key components. First, we introduce a physics-driven neural retargeting algorithm that translates large-scale motion capture to humanoid embodiments while preserving physical plausibility for contact-rich interactions. Second, we learn a unified multimodal controller that supports both dense references and sparse task specifications, under sensing ranging from accurate motion-capture state to noisy egocentric visual inputs. We distill a universal tracking policy into this controller, compress motor skills into a compact latent space, and apply reinforcement learning finetuning to expand coverage and improve robustness under out-of-distribution scenarios. This enables coordinated whole-body behavior from sparse intent without test-time reference motions. We evaluate ULTRA in simulation and on a real Unitree G1 humanoid. Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception, consistently outperforming tracking-only baselines with limited skills.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.RO'/>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.CV'/>\n    <published>2026-03-03T18:59:29Z</published>\n    <arxiv:comment>Project Page: https://ultra-humanoid.github.io/</arxiv:comment>\n    <arxiv:primary_category term='cs.RO'/>\n    <author>\n      <name>Xialin He</name>\n    </author>\n    <author>\n      <name>Sirui Xu</name>\n    </author>\n    <author>\n      <name>Xinyao Li</name>\n    </author>\n    <author>\n      <name>Runpei Dong</name>\n    </author>\n    <author>\n      <name>Liuyu Bian</name>\n    </author>\n    <author>\n      <name>Yu-Xiong Wang</name>\n    </author>\n    <author>\n      <name>Liang-Yan Gui</name>\n    </author>\n  </entry>"
}