Skip to content

Commit 977adb8

Browse files
author
preminger
authored
document OCI-mode SCIF support (#222)
1 parent 9e8c3e1 commit 977adb8

File tree

3 files changed

+166
-2
lines changed

3 files changed

+166
-2
lines changed

‎build_a_container.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,8 @@ containers. The containers are decrypted at runtime entirely in kernel space,
270270
meaning that no intermediate decrypted data is ever written to disk. See
271271
:ref:`encrypted containers <encryption>` for more details.
272272

273+
.. _dockerfile:
274+
273275
*************************
274276
Building from Dockerfiles
275277
*************************
@@ -339,11 +341,11 @@ in the expected way.
339341
.. code:: console
340342
341343
$ singularity exec --oci ./debian.oci.sif uname -a
342-
Linux nueve 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 GNU/Linux
344+
Linux myhost 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 GNU/Linux
343345
344346
$ singularity shell --oci ./debian.oci.sif uname
345347
Singularity> uname -a
346-
Linux nueve 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 GNU/Linux
348+
Linux myhost 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023 x86_64 GNU/Linux
347349
Singularity>
348350
349351
.. note::

‎definition_files.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1191,6 +1191,13 @@ containers.
11911191
{Singularity} implements SCIF, and you can read more about how to use it
11921192
below.
11931193

1194+
.. note::
1195+
1196+
The OCI mode introduced in version 4 of {Singularity} also includes support
1197+
for SCIF, but in a way that is in line with the behavior of SCIF in other OCI
1198+
container runtimes (e.g. Docker). See the :ref:`documentation on SCIF in
1199+
OCI-mode <oci_scif>` for more information.
1200+
11941201
SCIF is not specific to {Singularity}. To learn more, take a look at the
11951202
project's site at https://sci-f.github.io/, which includes extended
11961203
tutorials, a detailed specification of the SCIF standard, and other

‎oci_runtime.rst‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -472,6 +472,161 @@ files:
472472
definition JSON files. If omitted,
473473
default will be: /etc/cdi,/var/run/cdi
474474
475+
.. _oci_scif:
476+
477+
****************
478+
SCIF in OCI mode
479+
****************
480+
481+
`SCIF <https://sci-f.github.io/>`__ is a standard for encapsulating multiple
482+
apps into a container. Support for SCIF in the native runtime is discussed
483+
:ref:`here <apps>`; but the behavior of SCIF in OCI-mode is different, and is in
484+
line with how SCIF is used in other OCI container runtimes, such as Docker, as
485+
discussed & demonstrated in `this SCIF tutorial
486+
<https://sci-f.github.io/tutorial-preview-install>`__.
487+
488+
In brief, SCIF in OCI containers relies on the container having the `scif
489+
executable <https://pypi.org/project/scif/>`__ as its CMD / ENTRYPOINT, as shown
490+
for example in this Dockerfile:
491+
492+
.. code:: console
493+
494+
$ cat Dockerfile.scif
495+
FROM continuumio/miniconda3
496+
RUN pip install scif
497+
ADD my_recipe /
498+
RUN scif install /my_recipe
499+
CMD ["scif"]
500+
501+
.. note::
502+
503+
Starting with version 4.1, {Singularity} includes support for building OCI-SIF
504+
images directly from Dockerfiles, and so a Dockerfile like the one above can
505+
be compiled directly into an OCI-SIF image. (In this particular case, the
506+
``my_recipe`` file would have to be present in the current directory and be a
507+
well-formed SCIF recipe.) See :ref:`here <dockerfile>` for details on building
508+
OCI-SIF images from Dockerfiles, and see the `SCIF documentation
509+
<https://sci-f.github.io/tutorial-preview-install>`__ for more information on
510+
SCIF recipes.
511+
512+
The main difference between SCIF support in native- and OCI-modes is the
513+
location of the SCIF "recipe" (``%appinstall``, ``%appenv``, ``%apprun``,
514+
``%apphelp`` and ``%applabels`` sections). In native mode, the SCIF recipe is
515+
:ref:`part of the {Singularity} definition file <apps>`. In OCI mode, on the
516+
other hand, the SCIF recipe is typically included in a separate file, and
517+
processed using the ``scif install <recipefile>`` command inside the container,
518+
to be executed *after* the ``scif`` executable has been installed (in this case,
519+
using ``pip``).
520+
521+
Including the SCIF recipe as a separate file is not the only option, however.
522+
The SCIF recipe file can be constructed on-the-fly as part of the OCI container
523+
build, as well, as in the following example:
524+
525+
.. code::
526+
527+
$ cat Dockerfile.scif2
528+
FROM continuumio/miniconda3
529+
530+
RUN pip install scif
531+
532+
RUN echo $'\n\
533+
%apprun hello-world-one\n\
534+
echo "'Hello world!'"\n\
535+
\n\
536+
%apprun hello-world-two\n\
537+
echo "'Hello, again!'"\n\
538+
' > /my_recipe
539+
540+
RUN scif install /my_recipe
541+
542+
CMD ["scif"]
543+
544+
Once you have built a SCIF-compliant OCI-SIF image, you can use {Singularity}'s
545+
``--app`` option to interact with individual SCIF apps in the container using
546+
the ``run / shell / exec`` commands:
547+
548+
.. code::
549+
550+
$ cat Dockerfile.scif
551+
FROM continuumio/miniconda3
552+
RUN pip install scif
553+
ADD my_recipe /
554+
RUN scif install /my_recipe
555+
CMD ["scif"]
556+
557+
$ cat my_recipe
558+
%appenv hello-world-echo
559+
THEBESTAPP=$SCIF_APPNAME
560+
export THEBESTAPP
561+
%apprun hello-world-echo
562+
echo "The best app is $THEBESTAPP"
563+
564+
%appinstall hello-world-script
565+
echo "echo 'Hello World!'" >> bin/hello-world.sh
566+
chmod u+x bin/hello-world.sh
567+
%appenv hello-world-script
568+
THEBESTAPP=$SCIF_APPNAME
569+
export THEBESTAPP
570+
%apprun hello-world-script
571+
/bin/bash hello-world.sh
572+
573+
$ singularity build --oci scif.oci.sif Dockerfile.scif
574+
INFO: Did not find usable running buildkitd daemon; spawning our own.
575+
INFO: cfg.Root for buildkitd: /home/myuser/.local/share/buildkit
576+
INFO: Using "crun" runtime for buildkitd daemon.
577+
INFO: running buildkitd server on /run/user/1000/buildkit/buildkitd-8508905943414043.sock
578+
[+] Building 1.8s (8/9)
579+
[+] Building 1.9s (9/9) FINISHED
580+
=> [internal] load build definition from Dockerfile.scif 0.0s
581+
=> => transferring dockerfile: 206B 0.0s
582+
=> [internal] load metadata for docker.io/continuumio/miniconda3 0.5s
583+
=> [internal] load .dockerignore 0.0s
584+
=> => transferring context: 2B 0.0s
585+
=> [1/4] FROM docker.io/continuumio/miniconda3:latest@sha256:db9 0.0s
586+
=> => resolve docker.io/continuumio/miniconda3:latest@sha256:db9 0.0s
587+
=> [internal] load build context 0.0s
588+
=> => transferring context: 89B 0.0s
589+
=> CACHED [2/4] RUN pip install scif 0.0s
590+
=> CACHED [3/4] ADD my_recipe / 0.0s
591+
=> CACHED [4/4] RUN scif install /my_recipe 0.0s
592+
=> exporting to docker image format 1.2s
593+
=> => exporting layers 0.0s
594+
=> => exporting manifest sha256:5fa6d77d3e0f9190088d57782bbe52dc 0.0s
595+
=> => exporting config sha256:a9ffa234dd97432b0bc74fa3ee7fa46bfd 0.0s
596+
=> => sending tarball 1.2s
597+
Getting image source signatures
598+
Copying blob e67fdae35593 done |
599+
Copying blob 62aa66a9c405 done |
600+
Copying blob 129bc9a4304f done |
601+
Copying blob 9eeb7d589f05 done |
602+
Copying blob d4ef55d3a44b done |
603+
Copying blob 81edcff80a6f done |
604+
Copying config 2f162fba3f done |
605+
Writing manifest to image destination
606+
INFO: Converting OCI image to OCI-SIF format
607+
INFO: Squashing image to single layer
608+
INFO: Writing OCI-SIF image
609+
INFO: Cleaning up.
610+
INFO: Build complete: scif.oci.sif
611+
612+
$ singularity run --oci --app hello-world-script scif.oci.sif
613+
[hello-world-script] executing /bin/bash /scif/apps/hello-world-script/scif/runscript
614+
Hello World!
615+
616+
$ singularity exec --oci --app hello-world-script scif.oci.sif env | grep APPDATA
617+
SCIF_APPDATA=/scif/data/hello-world-script
618+
SCIF_APPDATA_hello_world_script=/scif/data/hello-world-script
619+
SCIF_APPDATA_hello_world_echo=/scif/data/hello-world-echo
620+
621+
$ singularity shell --oci --app hello-world-script scif.oci.sif
622+
[hello-world-script] executing /bin/bash
623+
myuser@myhost:/scif/apps/hello-world-script$ echo $SCIF_APPNAME
624+
hello-world-script
625+
myuser@myhost:/scif/apps/hello-world-script$
626+
627+
See the `SCIF homepage <https://sci-f.github.io/>`__ for more information and
628+
links to further documentation on SCIF itself.
629+
475630
.. _oci_command:
476631

477632
*****************

0 commit comments

Comments
 (0)