Paper
How Fair is Software Fairness Testing?
Authors
Ann Barcomb, Mariana Pinheiro Bento, Giuseppe Destefanis, Sherlock Licorish, Cleyton Magalhães, Ronnie de Souza Santos, Mairieli Wessel
Abstract
Software fairness testing is a central method for evaluating AI systems, yet the meaning of fairness is often treated as fixed and universally applicable. This vision paper positions fairness testing as culturally situated and examines the problem across three dimensions. First, fairness metrics encode particular cultural values while marginalizing others. Second, test datasets are predominantly designed from Western contexts, excluding knowledge systems grounded in oral traditions, Indigenous languages, and non-digital communities. Third, fairness testing raises ethical concerns, including the reliance on low-paid data labeling in the Global South, and associated with this, the environmental costs of training and deploying large-scale models, which disproportionately affect climate-vulnerable populations. Addressing these issues requires rethinking fairness testing beyond universal metrics and moving toward evaluation frameworks that respect cultural plurality and acknowledge the right to refuse algorithmic mediation.
Metadata
Related papers
Gen-Searcher: Reinforcing Agentic Search for Image Generation
Kaituo Feng, Manyuan Zhang, Shuang Chen, Yunlong Lin, Kaixuan Fan, Yilei Jian... • 2026-03-30
On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers
Omer Dahary, Benaya Koren, Daniel Garibi, Daniel Cohen-Or • 2026-03-30
Graphilosophy: Graph-Based Digital Humanities Computing with The Four Books
Minh-Thu Do, Quynh-Chau Le-Tran, Duc-Duy Nguyen-Mai, Thien-Trang Nguyen, Khan... • 2026-03-30
ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining
Anuj Diwan, Eunsol Choi, David Harwath • 2026-03-30
RAD-AI: Rethinking Architecture Documentation for AI-Augmented Ecosystems
Oliver Aleksander Larsen, Mahyar T. Moghaddam • 2026-03-30
Raw Data (Debug)
{
"raw_xml": "<entry>\n <id>http://arxiv.org/abs/2603.12511v1</id>\n <title>How Fair is Software Fairness Testing?</title>\n <updated>2026-03-12T23:02:49Z</updated>\n <link href='https://arxiv.org/abs/2603.12511v1' rel='alternate' type='text/html'/>\n <link href='https://arxiv.org/pdf/2603.12511v1' rel='related' title='pdf' type='application/pdf'/>\n <summary>Software fairness testing is a central method for evaluating AI systems, yet the meaning of fairness is often treated as fixed and universally applicable. This vision paper positions fairness testing as culturally situated and examines the problem across three dimensions. First, fairness metrics encode particular cultural values while marginalizing others. Second, test datasets are predominantly designed from Western contexts, excluding knowledge systems grounded in oral traditions, Indigenous languages, and non-digital communities. Third, fairness testing raises ethical concerns, including the reliance on low-paid data labeling in the Global South, and associated with this, the environmental costs of training and deploying large-scale models, which disproportionately affect climate-vulnerable populations. Addressing these issues requires rethinking fairness testing beyond universal metrics and moving toward evaluation frameworks that respect cultural plurality and acknowledge the right to refuse algorithmic mediation.</summary>\n <category scheme='http://arxiv.org/schemas/atom' term='cs.SE'/>\n <published>2026-03-12T23:02:49Z</published>\n <arxiv:primary_category term='cs.SE'/>\n <author>\n <name>Ann Barcomb</name>\n </author>\n <author>\n <name>Mariana Pinheiro Bento</name>\n </author>\n <author>\n <name>Giuseppe Destefanis</name>\n </author>\n <author>\n <name>Sherlock Licorish</name>\n </author>\n <author>\n <name>Cleyton Magalhães</name>\n </author>\n <author>\n <name>Ronnie de Souza Santos</name>\n </author>\n <author>\n <name>Mairieli Wessel</name>\n </author>\n </entry>"
}