4 |Gromacs| is transitioning to GitLab for source code management, issue tracking,
5 and integrated automation for testing and documentation.
7 The repository contains DockerFiles and GitLab Runner configuration
8 files to support automated testing and documentation builds.
9 General information on configuring GitLab CI pipelines can be found
10 in the official `Gitlab documentation <https://docs.gitlab.com/ee/ci/yaml/>`_.
12 The GitLab CI configuration entry point is the :file:`.gitlab-ci.yml` file
13 at the root of the source tree.
14 Configuration templates are found in the files in the
15 :file:`admin/ci-templates/` directory.
17 Docker images used by GitLab Runner are available on `Docker Hub <https://hub.docker.com/u/gromacs>`__.
18 Images are (re)built manually from DockerFiles in :file:`admin/dockerfiles`.
20 This documentation is incomplete, pending resolution of :issue:`3275`.
22 .. todo:: Expand this documentation to resolve :issue:`3275`
27 .. todo:: Discuss the distinct characteristics of |Gromacs| CI pipelines to relevant to job configuration.
29 .. todo:: Comment on the number of pipelines that can be or which are likely to be running at the same time.
34 At the root of the repository, :file:`.gitlab-ci.yml` defines the stages and
35 some default parameters, then includes files from :file:`admin/gitlab-ci/` to
36 define jobs to be executed in the pipelines.
38 Note that job names beginning with a period (``.``) are
39 `"hidden" <https://docs.gitlab.com/ee/ci/yaml/#hidden-keys-jobs>`_.
40 Such jobs are not directly eligible to run, but may be used as templates
41 via the `*extends* job property <https://docs.gitlab.com/ee/ci/yaml/#extends>`_.
46 Refer to https://docs.gitlab.com/ee/ci/yaml for complete documentation on
47 GitLab CI job parameters, but note the following GROMACS-specific conventions.
52 Used by several of our templates to prepend shell commands to
53 a job *script* parameter.
54 Avoid using *before-script* directly, and be cautious
55 about nested *extends* overriding multiple *before_script* definitions.
58 There is no global default, but jobs that build software will likely
59 set *cache*. To explicitly unset *cache* directives, specify a job
60 parameter of ``cache: {}``.
61 Refer to `GitLab docs <https://docs.gitlab.com/ee/ci/yaml/#cache>`__
62 for details. In particular, note the details of cache identity according
63 to `cache:key <https://docs.gitlab.com/ee/ci/yaml/#cachekey>`__
66 Part of the tool chain configuration. Instead of setting *image*
67 directly, *extend* a *.use_<toolchain>* template from
68 :file:`admin/gitlab-ci/global.gitlab-ci.yml`
74 *Job* parameters for controlling the circumstances under which jobs run.
75 (Some key words may have different meanings when occurring as elements
76 of other parameters, such as *archive:when*, to which this note is not
78 Instead of setting any of these directly in a job definition, try to use
79 one of the pre-defined behaviors (defined as ``.rules:<something>`` in
80 :file:`admin/gitlab-ci/global.gitlab-ci.yml`).
81 Errors or unexpected behavior will occur if you specify more than one
82 *.rules:...* template, or if you use these parameters in combination
83 with a *.rules...* template.
84 To reduce errors and unexpected behavior, restrict usage of these controls
85 to regular job definitions (don't use in "hidden" or parent jobs).
88 Many job definitions will add or override keys in *variables*.
89 Refer to `GitLab <https://docs.gitlab.com/ee/ci/yaml/#variables>`__
90 for details of the merging behavior. Refer to :ref:`variables` for local usage.
92 Schedules and triggers
93 ~~~~~~~~~~~~~~~~~~~~~~
95 Pipeline `schedules <https://gitlab.com/help/ci/pipelines/schedules>`__ are
96 configured through the GitLab web interface.
97 Scheduled pipelines may provide different variable definitions through the
98 environment to jobs that run under the ``schedules``
99 `condition <https://gitlab.com/help/ci/pipelines/schedules#using-only-and-except>`__.
101 Nightly scheduled pipelines run against ``master`` and *release* branches in
102 the GROMACS repository.
104 Pending resolution of :issue:`3272`,
105 the repository hosted on Gerrit is mirrored to GitLab.
106 Mirror updates trigger pipeline builds.
111 In addition to the templates in the main job definition files,
112 common "mix-in" functionality and behavioral templates are defined in
113 :file:`admin/gitlab-ci/global.gitlab-ci.yml`.
115 Jobs beginning with ``.use-`` provide mix-in behavior, such as boilerplate for
116 jobs using a particular tool chain.
118 Jobs beginning with a `parameter <https://docs.gitlab.com/ee/ci/yaml>`__
119 name allow parameters to be set in a single place for common job characteristics.
120 If providing more than a default parameter value, the job name should be suffixed
121 by a meaningful descriptor and documented within
122 :file:`admin/gitlab-ci/global.gitlab-ci.yml`
129 1. Indicate the purpose of the job.
130 2. Indicate relationships between multi-stage tasks.
131 3. Distinguish jobs in the same stage.
132 4. Distinguish job definitions throughout the configuration.
134 Jobs may be reassigned to different stages over time, so including the stage
135 name in the job name is not helpful, generally. If tags like "pre" and "post,"
136 or "build" and "test" are necessary to distinguish phases of, say, "webpage,"
137 then such tags can be buried at the end of the job name.
139 Stylistically, it is helpful to use delimiters like ``:`` to distinguish the
140 basic job name from qualifiers or details. Also consider
141 `grouping jobs <https://docs.gitlab.com/ee/ci/pipelines/index.html#grouping-jobs>`__
148 The GitLab CI framework, GitLab Runner, plugins, and our own scripts set and
149 use several `variables <https://docs.gitlab.com/ee/ci/variables/README.html>`__.
151 Default values are available from the ``.variables:default`` definition in
152 :file:`admin/gitlab-ci/global.gitlab-ci.yml`.
153 Many of the mix-in / template jobs provide additional or overriding definitions.
154 Other variables may be set when making final job definitions.
156 Variables may control the behvior of GitLab-CI (those beginning with ``CI_``),
157 GitLab Runner and supporting infrastructure, or may be used by job definitions,
158 or passed along to the environment of executed commands.
160 *variables* keys beginning with ``KUBERNETES_`` relate to the GitLab Runner
161 `Kubernets executor <https://docs.gitlab.com/runner/executors/kubernetes.html#the-kubernetes-executor>`__
163 Other important variable keys are as follows.
166 COMPILER_MAJOR_VERSION
167 Integer version number provided by toolchain mix-in for convenience and
170 CMAKE_COMPILER_SCRIPT
171 CMake command line options for a tool chain. A definition is provided by
172 the mix-in toolchain definitions (e.g. ``.use-gcc8``) to be appended to
173 :command:`cmake` calls in a job's *script*.
176 Provide CMake command line arguments to define GROMACS MPI build options.
179 Read-only environment variable that can be checked to see if a job is
180 executing in a pipeline for preparing a tagged release.
181 Can be set when launching pipelines via the GitLab web interface.
182 For example, see *rules* mix-ins in :file:`admin/gitlab-ci/global.gitlab-ci.yml`.
185 List additional OS package requirements. Used in *before_script* for some
186 mix-in job definitions to install additional software dependencies. If
187 using such a job with *extends*, override this variable key with a
188 space-delimited list of packages (default: ``""``). Consider proposing a
189 patch to the base Docker images to include the dependency to reduce
190 pipeline execution time.
192 .. todo:: Define common variables.
193 ``BUILD_DIR``, ``INSTALL_DIR``, ``CACHE_FALLBACK_KEY``, ...