Paper
TIMID: Time-Dependent Mistake Detection in Videos of Robot Executions
Authors
Nerea Gallego, Fernando Salanova, Claudio Mannarano, Cristian Mahulea, Eduardo Montijano
Abstract
As robotic systems execute increasingly difficult task sequences, so does the number of ways in which they can fail. Video Anomaly Detection (VAD) frameworks typically focus on singular, low-level kinematic or action failures, struggling to identify more complex temporal or spatial task violations, because they do not necessarily manifest as low-level execution errors. To address this problem, the main contribution of this paper is a new VAD-inspired architecture, TIMID, which is able to detect robot time-dependent mistakes when executing high-level tasks. Our architecture receives as inputs a video and prompts of the task and the potential mistake, and returns a frame-level prediction in the video of whether the mistake is present or not. By adopting a VAD formulation, the model can be trained with weak supervision, requiring only a single label per video. Additionally, to alleviate the problem of data scarcity of incorrect executions, we introduce a multi-robot simulation dataset with controlled temporal errors and real executions for zero-shot sim-to-real evaluation. Our experiments demonstrate that out-of-the-box VLMs lack the explicit temporal reasoning required for this task, whereas our framework successfully detects different types of temporal errors. Project: https://ropertunizar.github.io/TIMID/
Metadata
Related papers
Gen-Searcher: Reinforcing Agentic Search for Image Generation
Kaituo Feng, Manyuan Zhang, Shuang Chen, Yunlong Lin, Kaixuan Fan, Yilei Jian... • 2026-03-30
On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers
Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or • 2026-03-30
Graphilosophy: Graph-Based Digital Humanities Computing with The Four Books
Minh-Thu Do, Quynh-Chau Le-Tran, Duc-Duy Nguyen-Mai, Thien-Trang Nguyen, Khan... • 2026-03-30
ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining
Anuj Diwan, Eunsol Choi, David Harwath • 2026-03-30
RAD-AI: Rethinking Architecture Documentation for AI-Augmented Ecosystems
Oliver Aleksander Larsen, Mahyar T. Moghaddam • 2026-03-30
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2603.09782v1</id>\n <title>TIMID: Time-Dependent Mistake Detection in Videos of Robot Executions</title>\n <updated>2026-03-10T15:17:34Z</updated>\n <link href='https://arxiv.org/abs/2603.09782v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2603.09782v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>As robotic systems execute increasingly difficult task sequences, so does the number of ways in which they can fail. Video Anomaly Detection (VAD) frameworks typically focus on singular, low-level kinematic or action failures, struggling to identify more complex temporal or spatial task violations, because they do not necessarily manifest as low-level execution errors. To address this problem, the main contribution of this paper is a new VAD-inspired architecture, TIMID, which is able to detect robot time-dependent mistakes when executing high-level tasks. Our architecture receives as inputs a video and prompts of the task and the potential mistake, and returns a frame-level prediction in the video of whether the mistake is present or not. By adopting a VAD formulation, the model can be trained with weak supervision, requiring only a single label per video. Additionally, to alleviate the problem of data scarcity of incorrect executions, we introduce a multi-robot simulation dataset with controlled temporal errors and real executions for zero-shot sim-to-real evaluation. Our experiments demonstrate that out-of-the-box VLMs lack the explicit temporal reasoning required for this task, whereas our framework successfully detects different types of temporal errors. Project: https://ropertunizar.github.io/TIMID/</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.RO'/>\n <published>2026-03-10T15:17:34Z</published>\n <arxiv:comment>8 pages, 5 figures , IROS submission</arxiv:comment>\n <arxiv:primary_category term='cs.RO'/>\n <author>\n <name>Nerea Gallego</name>\n <arxiv:affiliation>University of Zaragoza</arxiv:affiliation>\n </author>\n <author>\n <name>Fernando Salanova</name>\n <arxiv:affiliation>University of Zaragoza</arxiv:affiliation>\n </author>\n <author>\n <name>Claudio Mannarano</name>\n <arxiv:affiliation>University of Zaragoza</arxiv:affiliation>\n <arxiv:affiliation>University of Torino</arxiv:affiliation>\n </author>\n <author>\n <name>Cristian Mahulea</name>\n <arxiv:affiliation>University of Zaragoza</arxiv:affiliation>\n </author>\n <author>\n <name>Eduardo Montijano</name>\n <arxiv:affiliation>University of Zaragoza</arxiv:affiliation>\n </author>\n </entry>"
}