Buildtest (https://buildtest.readthedocs.io/) is an HPC testing framework to aid HPC facilities to perform acceptance testing for their system. HPC systems are growing in complexity, with a tightly coupled software and system stack that requires a degree of automation and continuous testing. In the past decade, two build frameworks (Spack, EasyBuild) have emerged and widely used in HPC community for automating build & installation process for scientific software. On the contrary, testing frameworks for HPC systems are limited to a few handful (ReFrame, Pavilion2, buildtest) that are in active development.

In buildtest, users will write test recipes in YAML called buildspecs that buildtest process to generate a shell script. buildtest utilizes versioned-based JSON Schema for validating buildspecs and currently, we support two main schemas (compiler, script). The script schema and compiler schema are used for writing traditional shell-scripts (bash, sh, csh), python-scripts and single source compilation test. In this talk we will present an overview of buildtest and how one can write buildspecs. Furthermore, we will discuss Cori Testsuite (https://github.com/buildtesters/buildtest-cori) in buildtest with several real examples on testing various components for Cori system at NERSC.