Skip to content
Docs

Apptainer

For users that would like to run a containerized environment. You can utilize Apptainer on the CS cluster computers. Apptainer is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Apptainer on your laptop, and then run it on many of the largest HPC clusters in the world, local university or company clusters, a single server, in the cloud, or on a workstation down the hall. Your container is a single file, and you don’t have to worry about how to install all the software you need on each different operating system.

Why use containers?

Containers are used to get software to run reliably across multiple computing platforms. They can also be used to simplify software installations across different groups or even members of the same group by packing all dependencies of an application within a single image. Since the entire user space portion of the Linux environment, including programs, custom configurations, and environment, are bundled into a single file, providing the following benefits:

  • portability - the container file can be moved to different systems and run without modification.
  • reproducibility - the container file contains all the software and dependencies needed to run the application.
  • shareable - containers can be shared with collaborators, or published in repositories.
  • usability - you can run software that is not installed on the system, or that is difficult to install.
  • legacy software - you can run older software that is no longer supported on newer operating systems.

Getting and building Apptainer containers

Pre-built Docker and Apptainer containers are often available within scientific repositories and on GitHub.

Docker containers

Many docker containers can be run using Apptainer. You can pull a Docker container from Docker Hub and convert it to a Apptainer container using the apptainer pull or apptainer build commands.

Useful Apptainer Container repositories

Pre-built containers are also available in external repositories and can be loaded to the CSC using apptainer pull or apptainer build commands.

Creating your own Apptainer containers

Apptainer containers can be created and uploaded to the CS cluster. To do this you will need root access to a Linux system (free through Duke VCM, or use the OIT GitLab CI process.

Once your container is built you can move it to the cluster using Globus, scp, or sftp.

Using OIT GitLab CI

Users may also create their own Apptainer containers by using the OIT Gitlab CI process. Using GitLab CI, your build process for the Linux environment is automated through GitLab by providing a apptainer definition file and a .gitlab-ci.yml file. Using this process, the basic steps are:

  1. Create a project in GitLab. Each apptainer container should have its own project and your Container image will be named after the project.
  2. Create a apptainer.def) file in your project. You can create from scratch or start from a copy from a repository.
  3. Copy the .gitlab-ci.yml file from the sample project into your project. Once this file is added, every time you commit to your project, the GitLab Continuous Integration pipeline will be initiated and a apptainer image is created for your project.
  4. Pull your image down to the CS cluster from GitLab.

For more details see this: Sample Project

Running Apptainer on the CSC

Apptainer is available on the cluster and can be used as part of Slurm interactive or batch jobs.

Sample interactive session

In this session, user321 requests an interactive session through Slurm and executes a shell using the apptainer image for Tensorflow. Using apptainer in this way lets you interact with the environment like it is a virtual machine.

❯ srun -p compsci-gpu --gres=gpu:1 --mem=70g -w linux53 --pty zsh -l
❯ w
 04:56:38 up 19:02,  1 user,  load average: 0.15, 0.04, 0.01
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.138.72.30     Wed09   21.00s  0.12s  0.12s -bash
❯ apptainer run --nvccli --nv docker://tensorflow/tensorflow:latest-gpu
INFO:    Using cached SIF image
INFO:    Setting 'NVIDIA_VISIBLE_DEVICES=all' to emulate legacy GPU binding.
INFO:    Setting --writable-tmpfs (required by nvidia-container-cli)


________                               _______________
___  __/__________________________________  ____/__  /________      __
__  /  _  _ \_  __ \_  ___/  __ \_  ___/_  /_   __  /_  __ \_ | /| / /
_  /   /  __/  / / /(__  )/ /_/ /  /   _  __/   _  / / /_/ /_ |/ |/ /
/_/    \___//_/ /_//____/ \____//_/    /_/      /_/  \____/____/|__/




You are running this container as user with ID 28892 and group 50,
which should map to the ID and group for your user on the Docker host. Great!


Nothing
Apptainer> python3 -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"
2024-04-04 04:56:57.948470: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-04 04:56:59.068150: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
Apptainer> exit

Sample batch session

The sample below illustrates how to use apptainer as part of a batch session.

Sample batch script(slurm_apptainer.sh):

# Submit to a random node with a command e.g.
#   sbatch slurm_apptainer.sh
#SBATCH --job-name=slurm_apptainer
#SBATCH --partition=common
apptainer exec /usr/xtmp/user321/detectron2.sif cat /etc/os-release

Submitting the batch script:

user321@login  /usr/xtmp/user321 $ sbatch slurm_apptainer.sh
Submitted batch job 1518992
user321@login  /usr/xtmp/user321 $ cat slurm-1518992.out
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
user321@login  /usr/xtmp/user321 $