Paper
Vibe Coding XR: Accelerating AI + XR Prototyping with XR Blocks and Gemini
Authors
Ruofei Du, Benjamin Hersh, David Li, Nels Numan, Xun Qian, Yanhe Chen, Zhongyi Zhou, Xingyue Chen, Jiahao Ren, Robert Timothy Bettridge, Steve Toh, David Kim
Abstract
While large language models have accelerated software development through "vibe coding", prototyping intelligent Extended Reality (XR) experiences remains inaccessible due to the friction of complex game engines and low-level sensor integration. To bridge this gap, we contribute XR Blocks, an open-source, modular WebXR framework that abstracts spatial computing complexities into high-level, human-centered primitives. Building upon this foundation, we present Vibe Coding XR, an end-to-end rapid prototyping workflow that leverages LLMs to translate natural language intent directly into functional XR software. Using a web-based interface, creators can transform high-level prompts (e.g., "create a dandelion that reacts to hand") into interactive WebXR applications in under a minute. We provide a preliminary technical evaluation on a pilot dataset (VCXR60) alongside diverse application scenarios highlighting mixed-reality realism, multi-modal interaction, and generative AI integrations. By democratizing spatial software creation, this work empowers practitioners to bypass low-level hurdles and rapidly move from "idea to reality." Code and live demos are available at https://xrblocks.github.io/gem and https://github.com/google/xrblocks.
Metadata
Related papers
Comparing Developer and LLM Biases in Code Evaluation
Aditya Mittal, Ryan Shar, Zichu Wu, Shyam Agarwal, Tongshuang Wu, Chris Donah... • 2026-03-25
The Stochastic Gap: A Markovian Framework for Pre-Deployment Reliability and Oversight-Cost Auditing in Agentic Artificial Intelligence
Biplab Pal, Santanu Bhattacharya • 2026-03-25
Retrieval Improvements Do Not Guarantee Better Answers: A Study of RAG for AI Policy QA
Saahil Mathur, Ryan David Rittner, Vedant Ajit Thakur, Daniel Stuart Schiff, ... • 2026-03-25
MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
Zhuo Li, Yupeng Zhang, Pengyu Cheng, Jiajun Song, Mengyu Zhou, Hao Li, Shujie... • 2026-03-25
Anti-I2V: Safeguarding your photos from malicious image-to-video generation
Duc Vu, Anh Nguyen, Chi Tran, Anh Tran • 2026-03-25
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2603.24591v1</id>\n <title>Vibe Coding XR: Accelerating AI + XR Prototyping with XR Blocks and Gemini</title>\n <updated>2026-03-25T17:58:56Z</updated>\n <link href='https://arxiv.org/abs/2603.24591v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2603.24591v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>While large language models have accelerated software development through \"vibe coding\", prototyping intelligent Extended Reality (XR) experiences remains inaccessible due to the friction of complex game engines and low-level sensor integration. To bridge this gap, we contribute XR Blocks, an open-source, modular WebXR framework that abstracts spatial computing complexities into high-level, human-centered primitives. Building upon this foundation, we present Vibe Coding XR, an end-to-end rapid prototyping workflow that leverages LLMs to translate natural language intent directly into functional XR software. Using a web-based interface, creators can transform high-level prompts (e.g., \"create a dandelion that reacts to hand\") into interactive WebXR applications in under a minute. We provide a preliminary technical evaluation on a pilot dataset (VCXR60) alongside diverse application scenarios highlighting mixed-reality realism, multi-modal interaction, and generative AI integrations. By democratizing spatial software creation, this work empowers practitioners to bypass low-level hurdles and rapidly move from \"idea to reality.\" Code and live demos are available at https://xrblocks.github.io/gem and https://github.com/google/xrblocks.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.HC'/>\n <published>2026-03-25T17:58:56Z</published>\n <arxiv:primary_category term='cs.HC'/>\n <author>\n <name>Ruofei Du</name>\n </author>\n <author>\n <name>Benjamin Hersh</name>\n </author>\n <author>\n <name>David Li</name>\n </author>\n <author>\n <name>Nels Numan</name>\n </author>\n <author>\n <name>Xun Qian</name>\n </author>\n <author>\n <name>Yanhe Chen</name>\n </author>\n <author>\n <name>Zhongyi Zhou</name>\n </author>\n <author>\n <name>Xingyue Chen</name>\n </author>\n <author>\n <name>Jiahao Ren</name>\n </author>\n <author>\n <name>Robert Timothy Bettridge</name>\n </author>\n <author>\n <name>Steve Toh</name>\n </author>\n <author>\n <name>David Kim</name>\n </author>\n </entry>"
}