Paper

{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2603.12044v1</id>\n    <title>HPC Containers for EBRAINS: Towards Portable Cross-Domain Software Environment</title>\n    <updated>2026-03-12T15:19:45Z</updated>\n    <link href='https://arxiv.org/abs/2603.12044v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2603.12044v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>Deploying complex, distributed scientific workflows across diverse HPC sites is often hindered by site-specific dependencies and complex build environments. This paper investigates the design and performance of portable HPC container images capable of encapsulating MPI- and CUDA-enabled software stacks without sacrificing bare-metal performance. This work is part of recent work performed within the EBRAINS Research Infrastructure, to evaluate the implementation of portable HPC (Apptainer-based) container images targeting the EBRAINS Software Distribution (ESD) -- a Spack-based software ecosystem comprising approximately 80 top-level packages (and 800 dependencies). We evaluate a hybrid, PMIx-based containerization strategy using Apptainer that seamlessly bypasses the need for site-specific builds by dynamically leveraging host-level specialized hardware, such as network interfaces and GPUs, on two production HPC clusters: Karolina and Jureca-DC. We demonstrate the feasibility of building portable, MPI- and CUDA-enabled scientific software into container images that correctly leverage site-installed drivers and hardware to reproduce bare-metal communication behavior. Using communication microbenchmarks (e.g., OSU and NCCL) alongside performance metrics of applications from neuroscience, we measure and verify their performance against bare-metal deployments. Crucially, our verification approach extends beyond top-level runtime measurements; we highlight the analysis of underlying debug logs to actively detect misbehavior and misconfigurations, such as suboptimal transport pathways. Ultimately, this investigation demonstrates the feasibility of a simple and reproducible methodology for decoupling software environments from underlying infrastructures, paving the way for automated pipelines that ensure optimized, performance-verified execution across varied HPC architectures.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.DC'/>\n    <published>2026-03-12T15:19:45Z</published>\n    <arxiv:primary_category term='cs.DC'/>\n    <author>\n      <name>Krishna Kant Singh</name>\n    </author>\n    <author>\n      <name>Eric Müller</name>\n    </author>\n    <author>\n      <name>Eleni Mathioulaki</name>\n    </author>\n    <author>\n      <name>Wouter Klijn</name>\n    </author>\n    <author>\n      <name>Lena Oden</name>\n    </author>\n  </entry>"
}