Paper

{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2603.17947v1</id>\n    <title>Unified Policy Value Decomposition for Rapid Adaptation</title>\n    <updated>2026-03-18T17:19:56Z</updated>\n    <link href='https://arxiv.org/abs/2603.17947v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2603.17947v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>Rapid adaptation in complex control systems remains a central challenge in reinforcement learning. We introduce a framework in which policy and value functions share a low-dimensional coefficient vector - a goal embedding - that captures task identity and enables immediate adaptation to novel tasks without retraining representations. During pretraining, we jointly learn structured value bases and compatible policy bases through a bilinear actor-critic decomposition. The critic factorizes as Q = sum_k G_k(g) y_k(s,a), where G_k(g) is a goal-conditioned coefficient vector and y_k(s,a) are learned value basis functions. This multiplicative gating - where a context signal scales a set of state-dependent bases - is reminiscent of gain modulation observed in Layer 5 pyramidal neurons, where top-down inputs modulate the gain of sensory-driven responses without altering their tuning. Building on Successor Features, we extend the decomposition to the actor, which composes a set of primitive policies weighted by the same coefficients G_k(g). At test time the bases are frozen and G_k(g) is estimated zero-shot via a single forward pass, enabling immediate adaptation to novel tasks without any gradient update. We train a Soft Actor-Critic agent on the MuJoCo Ant environment under a multi-directional locomotion objective, requiring the agent to walk in eight directions specified as continuous goal vectors. The bilinear structure allows each policy head to specialize to a subset of directions, while the shared coefficient layer generalizes across them, accommodating novel directions by interpolating in goal embedding space. Our results suggest that shared low-dimensional goal embeddings offer a general mechanism for rapid, structured adaptation in high-dimensional control, and highlight a potentially biologically plausible principle for efficient transfer in complex reinforcement learning systems.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.LG'/>\n    <category scheme='http://arxiv.org/schemas/atom' term='q-bio.NC'/>\n    <published>2026-03-18T17:19:56Z</published>\n    <arxiv:primary_category term='cs.LG'/>\n    <author>\n      <name>Cristiano Capone</name>\n    </author>\n    <author>\n      <name>Luca Falorsi</name>\n    </author>\n    <author>\n      <name>Andrea Ciardiello</name>\n    </author>\n    <author>\n      <name>Luca Manneschi</name>\n    </author>\n  </entry>"
}