Images Supported by Singularity
Singularity supports many types of containers, which are also known as images. These include:
| Image Type | Description |
|---|---|
| simg | standard image |
| directory | Unix Directory containing root container image |
| tar.gz | Gzip compressed tar archive |
| tar.bx2 | Bzip2 compressed tar archive |
| tar | uncompressed tar archive |
| cpio.gz | Gzip compressed CPIO archive |
| cpio | Uncompressed CPIO archive |
Standard image and directory are the most commonly used formats. The others tend to be archive of the directory format that are decompressed on the fly.
Docker Integration
Singularity supports Docker natively and directly. This means that nearly all Docker images can be used directly as long as the required functionality does not require root access; however, most applications do not require this. We provide an example in the next section, Obtaining pre-built images.
Obtaining Pre-built Images
Images built by others users, sometimes the developers of the desired software itself, are among the best choices for a base image and, in many cases, may be all a researcher needs to do to have an image that gives the desired outcome.
Here is a common-example, the Docker image, lolcow.
[cjb47@comp-sc-0161 ~]$ singularity run docker://godlovedc/lolcow
** NOTE: Singularity is in a testing phase on ACI-b and is currently unsupported.
Docker image path: index.docker.io/godlovedc/lolcow:latest
Cache folder set to /storage/home/c/cjb47/.singularity/docker
[1/1] |===================================| 100.0%
Creating container runtime…
Exploding layer:
˓→sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118.tar.gz
Exploding layer:
˓→sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a.tar.gz
Exploding layer:
˓→sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2.tar.gz
Exploding layer:
˓→sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e.tar.gz
Exploding layer:
˓→sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9.tar.gz
Exploding layer:
˓→sha256:8e860504ff1ee5dc7953672d128ce1e4aa4d8e3716eb39fe710b849c64b20945.tar.gz
Exploding layer:
˓→sha256:736a219344fbca3099ce5bd1d2dbfea74b22b830bac0e85ecca812c2983390cd.tar.gz
WARNING: Non existent bind point (directory) in container: ‘/storage/home’
WARNING: Non existent bind point (directory) in container: ‘/storage/work’
WARNING: Non existent bind point (directory) in container: ‘/gpfs/scratch’
WARNING: Non existent bind point (directory) in container: ‘/gpfs/group’
WARNING: Non existent bind point (directory) in container: ‘/var/spool/torque’
WARNING: Could not chdir to home: /storage/home/cjb47
________________________________________
/ Tonight you will pay the wages of sin; \
\ Don’t forget to leave a tip. /
—————————————-
\ ^__^
\ (oo)\_______
(__)\ )\/\
||—-w |
|| ||
This example presents problems in certain cases, because the image does not have access to the data on the Roar system:
[cjb47@comp-sc-0185 singularity]$ singularity shell docker://godlovedc/lolcow
** NOTE: Singularity is in a testing phase on ACI-b and is currently unsupported.
Docker image path: index.docker.io/godlovedc/lolcow:latest
Cache folder set to /storage/home/c/cjb47/.singularity/docker
Creating container runtime…
Exploding layer:
˓→sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118.tar.gz
Exploding layer:
˓→sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a.tar.gz
Exploding layer:
˓→sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2.tar.gz
Exploding layer:
˓→sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e.tar.gz
Exploding layer:
˓→sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9.tar.gz
Exploding layer:
˓→sha256:8e860504ff1ee5dc7953672d128ce1e4aa4d8e3716eb39fe710b849c64b20945.tar.gz
Exploding layer:
˓→sha256:736a219344fbca3099ce5bd1d2dbfea74b22b830bac0e85ecca812c2983390cd.tar.gz
WARNING: Non existent bind point (directory) in container: ‘/storage/home’
WARNING: Non existent bind point (directory) in container: ‘/storage/work’
WARNING: Non existent bind point (directory) in container: ‘/gpfs/scratch’
WARNING: Non existent bind point (directory) in container: ‘/gpfs/group’
WARNING: Non existent bind point (directory) in container: ‘/var/spool/torque’
WARNING: Could not chdir to home: /storage/home/cjb47
Singularity: Invoking an interactive shell within container…
Singularity lolcow:/> cd ~
bash: cd: /storage/home/cjb47: No such file or directory
We can note the first issue, the non-existent bind paths. For this type of image, add the bind paths by slightly modifying the images. This can be done by writing a simple set of instructions, which are used to create an image. This set of instructions is called a recipe, in Singularity terminology.
Handling the Bind Paths on Roar
In order to handle the bind paths, it is very helpful to start with a recipe to set the data paths, even if the image is mostly a prepared image from another source. We will take a look at how to do this, and then discuss the building process in more detail.
We can also start with a recipe and add the correct binding paths for Roar. Here is a simple example to handle the bind points using the lolcow image.
BOOTSTRAP: docker
FROM: godlovedc/lolcow
%post
#ACI mappings so you can access your files.
mkdir -p /storage/home
mkdir -p /storage/work
mkdir -p /gpfs/group
mkdir -p /gpfs/scratch
mkdir -p /var/spool/torque
We build the image in an environment where we have sudo access (not Roar):
[cjb47@localhost simple_bind]$ sudo singularity build ./lolcow.simg ./lolcow.recipe
Building into existing container: ./lolcow.simg
Using container recipe deffile: ./lolcow.recipe
Sanitizing environment
Adding base Singularity environment to container
Running post scriptlet
+ mkdir -p /storage/home
+ mkdir -p /storage/work
+ mkdir -p /gpfs/group
+ mkdir -p /gpfs/scratch
+ mkdir -p /var/spool/torque
Found an existing definition file
Adding a bootstrap_history directory
Finalizing Singularity container
Calculating final size for metadata…
Skipping checks
Building Singularity image…
Singularity container built: ./lolcow.simg
Cleaning up…
Next, transfer the image to Roar using scp, and run it:
[cjb47@comp-sc-0185 images]$ singularity run ./lolcow.simg
** NOTE: Singularity is in a testing phase on ACI-b and is currently unsupported.
_________________________________________
/ The Priest’s grey nimbus in a niche \
| where he dressed discreetly. I will not |
| sleep here tonight. Home also I cannot |
| go. |
| |
| A voice, sweetened and sustained, |
| called to him from the sea. Turning the |
| curve he waved his hand. A sleek brown |
| head, a seal’s, far out on the water, |
| round. Usurper. |
| |
\ — James Joyce, “Ulysses” /
—————————————–
\ ^__^
\ (oo)\_______
(__)\ )\/\
||—-w |
|| ||
Note that we are now able to access our data on the local Roar locations.
[cjb47@comp-sc-0185 images]$ singularity shell ./lolcow.simg
** NOTE: Singularity is in a testing phase on ACI-b and is currently unsupported.
Singularity: Invoking an interactive shell within container…
Singularity lolcow.simg:/gpfs/group/dml129/default/cjb47/singularity/images> cd ~
Singularity lolcow.simg:~> pwd
/storage/home/cjb47