Research

Paper

TESTING March 10, 2026

Distribution-free screening of spatially variable genes in spatial transcriptomics

Authors

Changhu Wang, Qiyun Huang, Zihao Chen, Jin Liu, Ruibin Xi

Abstract

Spatial transcriptomics (ST) technologies enable transcriptome-wide gene expression profiling while preserving spatial resolution, offering unprecedented opportunities to uncover complex spatial structures. Due to the ultra-high dimensionality of ST data, identifying spatially variable genes (SVGs) associated with unknown spatial clusters has become a central task in ST data analysis. Here, we develop a distribution-free SVG screening method based on a novel quasi-likelihood ratio statistic, the MM-test, combined with a knockoff procedure to control the false discovery rate (FDR). MM-test leverages auxiliary information, such as spatial distances, about the unknown spatial domains for SVG screening. Notably, in addition to two-dimensional ST datasets, MM-test is well-suited for increasingly common three-dimensional (3D), multi-slice ST datasets. Extensive benchmarking using simulations and 34 real ST datasets demonstrates that MM-test consistently outperforms existing SVG detection methods. In a 3D mouse brain dataset, MM-test accurately delineates fine-scale structures that are challenging for other methods, such as the 3D architecture of the pyramidal layer of the hippocampal cornu ammonis and the dentate gyrus. Theoretical guarantees-including selection consistency, FDR control, and an error bound for post-selection clustering-are also established.

Metadata

arXiv ID: 2603.09061
Provider: ARXIV
Primary Category: stat.AP
Published: 2026-03-10
Fetched: 2026-03-11 06:02

Related papers

Raw Data (Debug)
{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2603.09061v1</id>\n    <title>Distribution-free screening of spatially variable genes in spatial transcriptomics</title>\n    <updated>2026-03-10T01:07:36Z</updated>\n    <link href='https://arxiv.org/abs/2603.09061v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2603.09061v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>Spatial transcriptomics (ST) technologies enable transcriptome-wide gene expression profiling while preserving spatial resolution, offering unprecedented opportunities to uncover complex spatial structures. Due to the ultra-high dimensionality of ST data, identifying spatially variable genes (SVGs) associated with unknown spatial clusters has become a central task in ST data analysis. Here, we develop a distribution-free SVG screening method based on a novel quasi-likelihood ratio statistic, the MM-test, combined with a knockoff procedure to control the false discovery rate (FDR). MM-test leverages auxiliary information, such as spatial distances, about the unknown spatial domains for SVG screening. Notably, in addition to two-dimensional ST datasets, MM-test is well-suited for increasingly common three-dimensional (3D), multi-slice ST datasets. Extensive benchmarking using simulations and 34 real ST datasets demonstrates that MM-test consistently outperforms existing SVG detection methods. In a 3D mouse brain dataset, MM-test accurately delineates fine-scale structures that are challenging for other methods, such as the 3D architecture of the pyramidal layer of the hippocampal cornu ammonis and the dentate gyrus. Theoretical guarantees-including selection consistency, FDR control, and an error bound for post-selection clustering-are also established.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='stat.AP'/>\n    <category scheme='http://arxiv.org/schemas/atom' term='stat.ME'/>\n    <published>2026-03-10T01:07:36Z</published>\n    <arxiv:primary_category term='stat.AP'/>\n    <author>\n      <name>Changhu Wang</name>\n    </author>\n    <author>\n      <name>Qiyun Huang</name>\n    </author>\n    <author>\n      <name>Zihao Chen</name>\n    </author>\n    <author>\n      <name>Jin Liu</name>\n    </author>\n    <author>\n      <name>Ruibin Xi</name>\n    </author>\n  </entry>"
}