Paper
Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models
Authors
Liangwei Yang, Shiyu Wang, Haolin Chen, Rithesh Murthy, Ming Zhu, Jielin Qiu, Zixiang Chen, Juntao Tan, Jianguo Zhang, Zhiwei Liu, Wenting Zhao, Silvio Savarese, Caiming Xiong, Huan Wang, Shelby Heinecke
Abstract
As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customize LLM behavior, we argue that text-only prompting does not constitute a suitable control interface for scalable, stable, and inference-only customization. This position paper argues that model providers should expose \emph{vector prompt inputs} as part of the public interface for customizing LLMs. We support this position with diagnostic evidence showing that vector prompt tuning continues to improve with increasing supervision whereas text-based prompt optimization saturates early, and that vector prompts exhibit dense, global attention patterns indicative of a distinct control mechanism. We further discuss why inference-only customization is increasingly important under realistic deployment constraints, and why exposing vector prompts need not fundamentally increase model leakage risk under a standard black-box threat model. We conclude with a call to action for the community to rethink prompt interfaces as a core component of LLM customization.
Metadata
Related papers
Gen-Searcher: Reinforcing Agentic Search for Image Generation
Kaituo Feng, Manyuan Zhang, Shuang Chen, Yunlong Lin, Kaixuan Fan, Yilei Jian... • 2026-03-30
On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers
Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or • 2026-03-30
Graphilosophy: Graph-Based Digital Humanities Computing with The Four Books
Minh-Thu Do, Quynh-Chau Le-Tran, Duc-Duy Nguyen-Mai, Thien-Trang Nguyen, Khan... • 2026-03-30
ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining
Anuj Diwan, Eunsol Choi, David Harwath • 2026-03-30
RAD-AI: Rethinking Architecture Documentation for AI-Augmented Ecosystems
Oliver Aleksander Larsen, Mahyar T. Moghaddam • 2026-03-30
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2603.04292v1</id>\n <title>Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models</title>\n <updated>2026-03-04T17:08:47Z</updated>\n <link href='https://arxiv.org/abs/2603.04292v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2603.04292v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customize LLM behavior, we argue that text-only prompting does not constitute a suitable control interface for scalable, stable, and inference-only customization. This position paper argues that model providers should expose \\emph{vector prompt inputs} as part of the public interface for customizing LLMs. We support this position with diagnostic evidence showing that vector prompt tuning continues to improve with increasing supervision whereas text-based prompt optimization saturates early, and that vector prompts exhibit dense, global attention patterns indicative of a distinct control mechanism. We further discuss why inference-only customization is increasingly important under realistic deployment constraints, and why exposing vector prompts need not fundamentally increase model leakage risk under a standard black-box threat model. We conclude with a call to action for the community to rethink prompt interfaces as a core component of LLM customization.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.CL'/>\n <published>2026-03-04T17:08:47Z</published>\n <arxiv:primary_category term='cs.CL'/>\n <author>\n <name>Liangwei Yang</name>\n </author>\n <author>\n <name>Shiyu Wang</name>\n </author>\n <author>\n <name>Haolin Chen</name>\n </author>\n <author>\n <name>Rithesh Murthy</name>\n </author>\n <author>\n <name>Ming Zhu</name>\n </author>\n <author>\n <name>Jielin Qiu</name>\n </author>\n <author>\n <name>Zixiang Chen</name>\n </author>\n <author>\n <name>Juntao Tan</name>\n </author>\n <author>\n <name>Jianguo Zhang</name>\n </author>\n <author>\n <name>Zhiwei Liu</name>\n </author>\n <author>\n <name>Wenting Zhao</name>\n </author>\n <author>\n <name>Silvio Savarese</name>\n </author>\n <author>\n <name>Caiming Xiong</name>\n </author>\n <author>\n <name>Huan Wang</name>\n </author>\n <author>\n <name>Shelby Heinecke</name>\n </author>\n </entry>"
}