Paul Bauer [Wed, 29 Apr 2020 09:23:54 +0000 (09:23 +0000)]
Use test conversion script from source
No longer install the script to convert CTest results to JUnit and
instead use the files from the cloned git source for it.
Fixes #3514
Change-Id: Ida4f2ab563601153b0b218c9bc6b7cfdb4969f01
Paul Bauer [Thu, 30 Apr 2020 12:03:15 +0000 (14:03 +0200)]
Fix incorrect header and linter scripts
Wrong grep flag meant that messages have not been printed.
Change-Id: I74b2088a12753d584929f9aa75ed915d400d8091
Pascal Merz [Thu, 30 Apr 2020 19:49:14 +0000 (13:49 -0600)]
Catch errors in v-rescale and Parrinello-Rahman checkpointing
This adds additional tests to catch checkpoints missing the v-rescale
thermostat integral or the Parrinello-Rahman box velocities. This
also make the number of steps explicit (useful in further investigations
of #3518).
In partial fulfillment of #3518
Gilles Gouaillardet [Thu, 30 Apr 2020 13:56:22 +0000 (22:56 +0900)]
cmake: declare (and hence initialize) GMX_USE_SIMD_KERNELS before it is used
Fixes gromacs/gromacs#3516
Mark Abraham [Thu, 30 Apr 2020 11:28:39 +0000 (11:28 +0000)]
Remove bundled FindCUDA.cmake
We bundle FindCUDA so we can suppress warnings in CI testing. In user
space, we always use the FindCUDA from the cmake they used, because
the tarballs never had these files.
M. Eric Irrgang [Fri, 24 Apr 2020 13:15:42 +0000 (16:15 +0300)]
Update header sorting.
* Add missing C++ standard library headers to includesorter.py
* Work around some logic errors in the way includesorter.py enforces
the stated scoping of sorted include blocks. Document.
* Apply updated includesorter.py
Refs #3297
M. Eric Irrgang [Wed, 29 Apr 2020 08:21:14 +0000 (11:21 +0300)]
Require C++17 standard and STL features.
Closes #3297
* Update project CMAKE_CXX_STANDARD.
* Migrate to std::optional and std::string_view.
* Update requirements in client packages.
* Provide some tool chain hints.
* Replace deprecated use of std::uncaught_exception() with
std::uncaught_exceptions().
* Update install-guide.
ejjordan [Wed, 29 Apr 2020 08:38:16 +0000 (10:38 +0200)]
Add EOF newlines to eliminate compiler warnings
ejjordan [Wed, 29 Apr 2020 08:54:53 +0000 (10:54 +0200)]
Remove cmake logic that is outdated by requiring cmake 3.13
Mark Abraham [Wed, 29 Apr 2020 12:07:35 +0000 (14:07 +0200)]
Update cmake version for docs container
Currently the nightly docs build fails because of
the bumped cmake requirement.
Paul Bauer [Wed, 29 Apr 2020 12:47:58 +0000 (12:47 +0000)]
Print base commit for source linters
Added a printout for the branch used for comparing.
Will use to see if branch point selection is working properly and to see
if the clang-tidy script picks up the warnings it should.
Change-Id: I791d2a11f98cd8e0cf0d06e92a9980cd9d038874
M. Eric Irrgang [Tue, 28 Apr 2020 16:47:36 +0000 (19:47 +0300)]
Apply more granular CTest time-out policy.
Separate some test macro logic:
* Decouple IGNORE_LEAKS from INTEGRATION_TEST and SLOW_TEST.
* Use SLOW_TEST without IGNORE_LEAKS for some newly relaxed time-outs.
Give longer time out to some tests.
Resolves #3513
Mark Abraham [Tue, 28 Apr 2020 18:08:18 +0000 (20:08 +0200)]
Fixed several clang-tidy issues
A static variable was vulnerable to static-init-order-fiasco. Tests
will still perform OK even with constructing the default value
multiple times, so converted to a getter.
Closes #3498
ejjordan [Tue, 28 Apr 2020 13:55:28 +0000 (15:55 +0200)]
Address review comments
ejjordan [Mon, 27 Apr 2020 18:17:00 +0000 (20:17 +0200)]
Simplify calc_listed signature
In order to remove a number of parameters from the signature of
calc_listed, function calls for position restraints, dispersion
correction, and orientation restraints were moved out of the
calc_listed function.
ejjordan [Tue, 28 Apr 2020 11:54:03 +0000 (13:54 +0200)]
Fix issues with check-source
M. Eric Irrgang [Mon, 27 Apr 2020 17:57:27 +0000 (17:57 +0000)]
Resolve static analyzer warnings from google tests.
Exclude a couple of problematic lines from consideration by the clang
static analyzer by surrounding with precompiler conditionals
(`#ifndef __clang_analyzer__`).
Fixes #3503
M. Eric Irrgang [Mon, 27 Apr 2020 17:19:35 +0000 (17:19 +0000)]
Avoid temporary strings.
Resolve a compiler warning about temporary strings created through a
string concatenation written as a chain of addition operators.
Instead, use `string.append()`.
Artem Zhmurov [Mon, 27 Apr 2020 14:24:10 +0000 (14:24 +0000)]
Move PBC setup out of the launchKernel(...) in GPU bondeds
The PBC setup should not be performed on every step, hence it
should be separated from the force computation launch. This
is a preparation step, that does not change the logic, only the
separates the PBC setup and changes kernel launch signature.
Mark Abraham [Mon, 27 Apr 2020 12:08:49 +0000 (12:08 +0000)]
Improved some logic in do_force
This helps the analyzer to understand that it should not just assume
that stateGpu can be a nullptr. The new logic is equivalent to
that for constructing stateGpu.
Removed the GMX_MPI preprocessing, which is less robust than plain
logic and probably contributed to using the wrong logic to construct
localXReadyOnDevice
M. Eric Irrgang [Fri, 24 Apr 2020 13:48:32 +0000 (16:48 +0300)]
Note CUDA builds in job names.
Identify CI jobs that involve CUDA builds. Include `cuda-10.1` in the
tool chain slug for the job name for easier navigation through the
pipeline web interface.
M. Eric Irrgang [Fri, 24 Apr 2020 16:16:08 +0000 (16:16 +0000)]
Update bundled pybind.
The pybind 2.5 release has better build system support for C++ 2017.
Otherwise there is no direct impact to our existing usage.
M. Eric Irrgang [Fri, 24 Apr 2020 11:19:52 +0000 (14:19 +0300)]
Only set GMX_PYTHON_PACKAGE=ON where needed.
Remove GMX_PYTHON_PACKAGE=ON from default CMAKE_GMXAPI_OPTIONS. Only
provide it in the jobs supporting documentation builds (which use a
gmxapi Python package staged in the build tree).
Remove a "TODO" that is confusing and presumably outdated.
Paul Bauer [Thu, 23 Apr 2020 20:24:50 +0000 (20:24 +0000)]
Use branch point also for format and copyright check
Change-Id: I16a9df11223426a40e6e138ee9f3793e700ae06e
Mark Abraham [Tue, 21 Apr 2020 07:31:24 +0000 (09:31 +0200)]
Mark HWLOC cmake cache variable as advanced
There's no need to put this internal-use-only cache variable
in the non-advanced list, which we forgot to do when we
introduced it.
Pascal Merz [Thu, 23 Apr 2020 17:10:46 +0000 (17:10 +0000)]
Make boxdeformation independent of t_commrec
boxdeformation requested a pointer to the full t_commrec,
but only used on communicator and information on whether
the current rank is master and whether the simulation
is run in parallel. This has been made explicit, simplifying
subsequent changes splitting up t_commrec.
Refs #2395
M. Eric Irrgang [Thu, 23 Apr 2020 14:43:55 +0000 (14:43 +0000)]
Use CMake 3.12+ facilities for detecting Python.
Consolidate Python detection in the main CMakeLists.txt before the
first usage in a subdirectory. Use FindPython3.cmake instead of
FindPythonInterp.cmake. Populate the legacy PYTHON_EXECUTABLE variable
for compatibility.
Fixes #2998
Paul Bauer [Thu, 23 Apr 2020 13:14:50 +0000 (13:14 +0000)]
Make clang-tidy script more robust
Changed script to be more robust and in line with other source
checking scripts.
Modified clang-tidy:test job to check against the merge fork point for a
commit instead of the previous git magic, as this did not work.
Change-Id: Ic806439727c5317db0c18964afd6138eeeea0427
Paul Bauer [Thu, 23 Apr 2020 12:07:38 +0000 (12:07 +0000)]
Fix parallel testing
Change testing to be more robust in threaded setups.
Also change on merge test to use real MPI to make sure this works
correctly.
Change-Id: Id78b213291bf7cc225fc2564a23f0d85362467fd
Pascal Merz [Wed, 22 Apr 2020 17:21:14 +0000 (17:21 +0000)]
Make init_dires independent of t_commrec
init_disres was requesting a full pointer to the commrec,
but only uses a single communicator and checks for master rank
and whether the run is parallel. This information is now passed
in explicitly, simplifying the planned splitting of t_commrec.
Note that passing a nullptr for commrec was (mis)used by
gmx_disre only - effectively signalling that init_disres was
called from an analysis tool and not from mdrun. This has
been made explicit.
Refs #2395
Mark Abraham [Tue, 21 Apr 2020 07:28:16 +0000 (09:28 +0200)]
Fix warnings with clang 9
Use of GMX_USE_SIMD_KERNELS left some code declared and unused,
which is now fixed.
Printf formats for int64_t are not very portable, so used the
traditional string work around again.
Artem Zhmurov [Mon, 20 Apr 2020 19:00:28 +0000 (19:00 +0000)]
Fix the number of steps check when checkpoint is loaded
The check ignores the initial step in the config file, thust only
works when loading for the trajectory that was originally started
from step zero.
Paul Bauer [Mon, 20 Apr 2020 17:51:33 +0000 (17:51 +0000)]
Make tests check code for warnings same as the main code
This has been inconsistent since a while.
Change-Id: I5cbe0d768620f82bd19ff8741df6b75b517200ed
Pascal Merz [Fri, 3 Apr 2020 22:28:40 +0000 (16:28 -0600)]
Make global topology const in ISimulator
The global topology in constant in all simulator level functions
with the exception of the setup of MiMiC runs and reruns.
Follow-up work will move the topology-altering MiMiC setup out of
the simulator level functions. It is, however, worth to already
enforce the current state, avoiding subsequent changes to undermine
these efforts. This commit changes the pointer to the global
topology in ISimulator to be const, and casts the const-ness away
for the two known cases it needs to be non-const.
Refs #3467
Christian Blau [Fri, 17 Apr 2020 13:57:50 +0000 (13:57 +0000)]
proposed fix for redmine bug #3403
Fixes #3403 by preventing the free energy kernel from skipping pairs
that are beyond the cutoff but also in the exclusion list generated by
couple-intramol=no. Forces calculation of the reciprocal-space Ewald
component for those pairs in the free energy kernel. Also extends the
size of the relevant tables to rcoulomb+tabext to avoid getting nonsense
energies/forces beyond the cutoff. Molecules being decoupled in this way
should be smaller than rcoulomb+tabext.
Paul Bauer [Fri, 17 Apr 2020 10:05:10 +0000 (10:05 +0000)]
Fix check source not parsing errors and warnings correctly
Fixes #3487
Change-Id: I6161e02f66a9c34a56a7475f986ab94a5cc83d91
Artem Zhmurov [Thu, 16 Apr 2020 16:30:05 +0000 (18:30 +0200)]
New cool quote
From Petter PhD defence. Congratulations!
Mark Abraham [Thu, 16 Apr 2020 15:36:33 +0000 (15:36 +0000)]
Introduce GMX_USE_SIMD_KERNELS cmake option
Most GROMACS development does not need to recompile the SIMD nbnxm
(and fep) kernels whenever their dependencies change. These
dependencies are large in number, and include frequently changed
files, including config.h and various utility and nbnxm module
headers. This flag permits people to efficiently recompile while
working on code that doesn't directly target changes to the SIMD
kernels.
It also means that CI builds not aimed at efficient mdrun execution
times can instead minimize compilation times and ccache db sizes. This
will also have the side effect of testing more of the reference NBNXM
kernels.
There's other SIMD code (particularly PME, bonded, LINCS, update)
which still compiles and runs in the usual way. Currently these are
less costly to compile and harder to disable. That could change in
future.
Artem Zhmurov [Thu, 16 Apr 2020 11:38:32 +0000 (11:38 +0000)]
Unify CUDA and OpenCL lookup-table creation
In CUDA code, textures are used for the lookup-tables,
whereas in OpenCL they are created as a read-only
buffers. This commit hides these differences behind a
unified wrapper.
Refs #3318
Refs #3311
Change-Id: I003e0c982c2452a2753e331b46fc59f0b7e1b711
Paul Bauer [Thu, 16 Apr 2020 07:31:21 +0000 (07:31 +0000)]
Use GPU resources for build types again
Change-Id: Ibc2629b2ae29ba5099c4565f94adeccc6cb5c270
M. Eric Irrgang [Tue, 14 Apr 2020 16:17:06 +0000 (19:17 +0300)]
Change default CMake version to 3.13.0 for CI images.
Refs #3484
Pascal Merz [Tue, 14 Apr 2020 18:31:08 +0000 (18:31 +0000)]
Make bonded interaction checking more robust (modular simulator)
Choice on whether the number of bonded interactions should be checked
during calls to the ComputeGlobalsElement used to be done at scheduling
time. This works in the current setup, as domain decomposition (which
might require such a check) is always done right before scheduling.
As this check is requested through callbacks, it is however more robust
to decide whether to perform this test at run time, not at scheduling
time. This commit switches to this more robust behavior.
Change-Id: I2c54bf0152e185c10d582cf0f06ecfc911c6c68b
M. Eric Irrgang [Fri, 6 Mar 2020 11:28:15 +0000 (14:28 +0300)]
Require CMake >=3.13.
Fixes #3290
Mark Abraham [Tue, 14 Apr 2020 07:16:12 +0000 (07:16 +0000)]
Fix more clang warnings
M. Eric Irrgang [Tue, 14 Apr 2020 06:09:54 +0000 (06:09 +0000)]
Fix invalid gitlab-ci.yml
Use consistent rules for dependent jobs.
gmxapi related jobs were eligible to be added to a pipeline when job
dependencies were not.
Fixes #3483
Paul Bauer [Mon, 13 Apr 2020 14:51:18 +0000 (14:51 +0000)]
The ugliest hack to fix rebase issues ever
Change-Id: I1b8202c4f9f36d03d29679850fd2e0672c823968
Paul Bauer [Mon, 13 Apr 2020 06:59:11 +0000 (08:59 +0200)]
Fix clang-tidy script and code issue
The script would not pick up issues correctly.
Also fixed code issue found by nightly build.
Change-Id: Ia51d7b92f9f024ace254f5b98cc44e7f00702521
Mark Abraham [Mon, 13 Apr 2020 05:23:00 +0000 (07:23 +0200)]
Fix warnings for no-mpi build
This build configuration gave warnings when combined with
clang-7 in the docs build.
Szilárd Páll [Sun, 12 Apr 2020 16:16:43 +0000 (16:16 +0000)]
Fix clang 10 -Wimplicit-int-float-conversion warnings
Mark Abraham [Sun, 12 Apr 2020 12:58:59 +0000 (14:58 +0200)]
Removed unused variable in force table generation
Also reduced the scope of a related variable.
Mark Abraham [Sun, 12 Apr 2020 10:08:16 +0000 (10:08 +0000)]
Finish removing ENCAD shifts and table generation
The force field was removed in 2014, and the functional forms can't be
used any more since the group scheme is removed. The table-generation
functions were also broken in several ways. Anybody wanting to use
this force field in the recent past (or perhaps future) needs to use
user tables.
The tpr-facing enums have their numerical values preserved, along with
changes to indicate that the symbol values are unused. This means that
old tpr files can still be read correctly, while maintainers don't
have extra work to understand what is going on. The internal code can
be fully removed.
Fixes #3477
Mark Abraham [Sun, 12 Apr 2020 08:55:43 +0000 (08:55 +0000)]
Update to clang-tidy-9
Code fixes have already taken place, it remains to update the
CI configuration to use the existing llvm-9 Docker image and
adjust the documentation to match.
Paul Bauer [Sat, 11 Apr 2020 08:27:44 +0000 (08:27 +0000)]
Add Image for GCC 7 and AMD OpenCL
Change-Id: Ic884c1db337576c6bcabd1bec48545f26527de9f
Christoph Junghans [Sat, 4 Apr 2020 01:27:14 +0000 (19:27 -0600)]
cmake: fix exported target
Paul Bauer [Thu, 9 Apr 2020 17:21:24 +0000 (17:21 +0000)]
Add clang-tidy script
Change-Id: I4e3ff318681b562c0941c88360d2381747348517
M. Eric Irrgang [Thu, 9 Apr 2020 15:57:28 +0000 (15:57 +0000)]
Allow useful CI to run in forks
* Move the fast jobs with no dependencies to the first stage.
* Remove the global KTH-specific job runner tag from jobs in the pre-build stage.
* Use the `pre-build` stage as the dependency for all later stages, rather than the `simple-build` job, specifically.
* Convert rule sets to new *rules* syntax.
* Use '$CI_PROJECT_NAMESPACE == "gromacs"' to distinguish jobs created with access to GROMACS GitLab infrastructure.
Fixes #3458
Mark Abraham [Thu, 19 Mar 2020 08:42:47 +0000 (09:42 +0100)]
Fixes for clang-tidy-9
Mostly suggestions for risky repeat branches and more readable use of
methods that use no member variables and could be static. This means
that some bad, incompletely implemented, or excessively implemented
code is now gone or uglier than it was before.
The CPU-build classes for GPU management objects didn't use the only
member variable impl_, so they were also recommended to become
static. That doesn't work for our use case, but swapping the assertion
to be !impl_ does all jobs well enough.
eg2cHartree_e probably has had the wrong behavior ever since it
was introduced, but I don't know and it's not a big problem.
LJ12 Encad table is probably fixed, but it's unused, so who cares.
Christoph Junghans [Sat, 4 Apr 2020 15:16:48 +0000 (09:16 -0600)]
cmake: add support of OSX in GMXRC.*
M. Eric Irrgang [Wed, 8 Apr 2020 11:34:35 +0000 (14:34 +0300)]
Explicitly set UTF-8 encoding for gmxapi CLI interaction.
The detected locale may not imply UTF-8 character encoding, though the
gmx CLI tool may still issue non-ASCII characters. This change forces
assumption of UTF-8 character encoding for wrapped commmand line tools.
Refs #3474
Pascal Merz [Mon, 10 Feb 2020 23:08:46 +0000 (16:08 -0700)]
Merge force and shellFC elements
In the previous implementation, ForceElement was handling the call to
do_force, while ShellFCElement was handling the call to relax_shell_flexcon.
This lead to two nearly identical elements. The current change merges
the two elements minimizing code duplication.
Change-Id: Ief6e962e3780153d43ba4ea07eea40ed5b8d4ef8
Mark Abraham [Tue, 7 Apr 2020 21:02:10 +0000 (21:02 +0000)]
Replace EnumOption with EnumerationArrayOption
Avoids code needing to mis-use the m_elements member of the
EnumerationArray. Encourages using class enums for command-line
options. Removes a bunch of clang-tidy suppressions and will be needed
for dealing properly with even more warnings coming with clang-tidy 9.
Removes a bunch of comments about how this list has to match that enum
somewhere else, because the compiler will tell us that.
Fixed time-unit conversion bug introduced a decade ago in
bc23d95b8c54b383cc731666f692f03237f8e949. That's exactly the kind
of bug that now gets caught at compile time.
Removed strange first field of legacy enums time_unit_t and
xvg_format_t that forced complicated casting and searching. It didn't
seem to do anything useful.
Removed unnecessary EnumIntOption.
Some test code is now in gmx::test namespace for brevity (consistent
with other test code), now that the enum values are much longer.
Change-Id: I9fd398d8981115d2317614912e64a9129ff1253a
Pascal Merz [Tue, 7 Apr 2020 19:10:44 +0000 (19:10 +0000)]
Require explicit MPI_COMM for gmx_bcast and gmx_barrier
This changes gmx_bcast and gmx_barrier to take the MPI communicator
explicitly instead of taking a pointer to t_commrec and using
mpi_comm_mygroup. This also allows to remove gmx_bcast_sim and leave
the responsibility of passing the right communicator to the caller.
This is a first step in breaking up t_commrec. These functions
are the subset of low-level networking functions which are used
before domain decomposition (and hence PP/PME ranks) is set up.
Refs #2395
M. Eric Irrgang [Tue, 7 Apr 2020 11:17:58 +0000 (11:17 +0000)]
Prepare Python environments for GitLab CI pipeline.
* Install Python interpreters.
* Prepare and stash Python virtual environments.
Refs #2756
Change-Id: I1b68e039018a72865d1cc6f60557104026328285
Pascal Merz [Tue, 24 Mar 2020 02:50:56 +0000 (20:50 -0600)]
Make global topology constant in energyOutput.printAnnealingTemperatures()
energyOutput.printAnnealingTemperatures() takes a non-const pointer
to the SimulationGroups object stored in the global topology, but
only ever reads its information. This makes the pointer const.
Refs #3467
Pascal Merz [Tue, 24 Mar 2020 03:14:45 +0000 (21:14 -0600)]
Make global topology const in minimization
Added some trivial const qualifiers to the minimization code to
make global topology const throughout the minimization code.
Refs #3467
Pascal Merz [Tue, 24 Mar 2020 02:40:54 +0000 (20:40 -0600)]
Make global topology constant in mdoutf
init_mdoutf and gmx_mdoutf required non-const pointers to the
global topology. This is not needed, so this is changed to a const
pointer.
Refs #3467
Paul Bauer [Mon, 6 Apr 2020 15:24:29 +0000 (15:24 +0000)]
Add AMD image build, v2
Change-Id: I93c8df07c0bd1c9c03b04a06a2aaba24423278a0
Berk Hess [Mon, 6 Apr 2020 11:27:31 +0000 (11:27 +0000)]
Move force buffer resizing to mdsetup
Consolidate resizing of the force buffer passed to do_force() from
many places in the code to mdAlgorithmsSetupAtomData().
Now only the modular simulation does it's own resizing when not using
domain decomposition. Some refactoring is needed to avoid this.
Change-Id: I1320dffb7b42150f65c5c6a260a7e94e2f6806b5
Pascal Merz [Mon, 2 Mar 2020 23:24:54 +0000 (16:24 -0700)]
Make bonded interaction checking more robust (modular simulator)
Choice on whether the number of bonded interactions should be checked
during calls to the ComputeGlobalsElement used to be done at scheduling
time. This works in the current setup, as domain decomposition (which
might require such a check) is always done right before scheduling.
As this check is requested through callbacks, it is however more robust
to decide whether to perform this test at run time, not at scheduling
time. This commit switches to this more robust behavior.
Change-Id: I2c54bf0152e185c10d582cf0f06ecfc911c6c68b
M. Eric Irrgang [Fri, 3 Apr 2020 18:51:09 +0000 (18:51 +0000)]
Improve gmxapi Python package dependency resolution.
Adopt PEP-517 and PEP-518. Replace setup_requires with pyproject.toml
specification. Add install_requires for run-time requirements. Sort
and improve annotations in requirements*.txt files.
Fixes #3271
M. Eric Irrgang [Fri, 3 Apr 2020 10:07:34 +0000 (10:07 +0000)]
Prune change-management.rst and update links.
Replaced more URLS and updated or removed most references to
Redmine and Gerrit.
References to Gerrit that occur in conjunction with references to
Jenkins are left untouched, since there are other works in progress
related to updating the releng documentation that presumably will
update the Jenkins-related documentation.
M. Eric Irrgang [Thu, 2 Apr 2020 13:46:44 +0000 (13:46 +0000)]
Modernize some Python subprocess calls.
subprocess.run() is now the recommended high-level interface for the
subprocess module. It trivially replaces subprocess.check_output and
usually simplifies subprocess.Popen use cases. It has less error prone
semantics for str vs. bytes I/O. subprocess.run() is less prone to
PIPE buffer deadlocks and we should use the modern interface in order to
benefit from future improvements, as well.
A handful of more intricate subprocess.Popen() usages are left, but I
think we can say that, along with
d80618cb, this change
Fixes #3154
Change-Id: I88f4eaabd8504d3800692c9617571103b2528cf9
M. Eric Irrgang [Wed, 1 Apr 2020 12:45:36 +0000 (15:45 +0300)]
Avoid accidentally initializing pyenv in CI images.
Don't initialize pyenv automatically in root login shells: don't copy
the updated `.bashrc` from the intermediate build images to the final
image. This can cause confusion about which Python installation can or
will be detected by CMake in a CI job.
The primary use case for `pyenv` is to prepare Python venvs, which are
activated explicitly when needed, and which (once created) do not
require pyenv to be initialized in order to be activated.
M. Eric Irrgang [Tue, 31 Mar 2020 17:50:16 +0000 (17:50 +0000)]
Prepare Python environments for GitLab CI pipeline.
* Install Python interpreters.
* Prepare and stash Python virtual environments.
Refs #2756
Change-Id: I1b68e039018a72865d1cc6f60557104026328285
M. Eric Irrgang [Tue, 31 Mar 2020 15:05:05 +0000 (15:05 +0000)]
Update scripts for all of the requested pre-commit test images.
Remove old Dockerfiles (no longer used).
M. Eric Irrgang [Mon, 30 Mar 2020 12:46:58 +0000 (12:46 +0000)]
Require Python 3.6.
Python 3.5 will reach end-of-life 2020-09-13.
Ref: https://devguide.python.org/#status-of-python-branches
Refs #3047
M. Eric Irrgang [Fri, 27 Mar 2020 10:46:17 +0000 (10:46 +0000)]
Compartmentalize hpccm build stages.
Move logic for each building block to a separate function to improve
readability and reusability. Rearrange package installations to improve
build and rebuild time.
Change-Id: I2d33f69af6e57a4b80141ab35ccfb424ed86403b
M. Eric Irrgang [Fri, 27 Mar 2020 08:48:37 +0000 (08:48 +0000)]
Be friendlier w.r.t. linkchecker.
* Add external link to tool web site.
* Make command-line copy-paste friendly in CI job definition.
* Use explicit absolute path for config file in CI job CLI invocation.
M. Eric Irrgang [Mon, 16 Mar 2020 18:13:56 +0000 (21:13 +0300)]
Mock hpccm import so infrastructure docs build easily.
Change-Id: Ib5744ea762b68de0d039cdcf8fd46e73f72210e1
Artem Zhmurov [Mon, 24 Feb 2020 09:22:40 +0000 (10:22 +0100)]
Make use of the DeviceStreamManager
Use the DeviceStreamManager throughout the code. The manager is
owned by the runner and created when GPU is active. The consumers
get the context and streams if needed.
TODOs:
1. Make builders and move the selection on whether the stream should
be created there. The builders should take the manager and pass
the context and the stream to the consumer. Builders should have
the option to create a stream.
2. Makefile in ewald tests uses old infrastructure. Also, the device
context management should be lifted from there and utilized in
all the tests that can run on GPU hardware.
Refs #3316
Refs #3311
Change-Id: I0d08adbe1dee19c1890e55f0e0cf79cea97d39bd
Paul Bauer [Fri, 20 Mar 2020 07:03:11 +0000 (08:03 +0100)]
Fix release builds on Gitlab with regressiontests
Change-Id: Id6465d545d350b8c0d89aecd37d3568dca510d8b
M. Eric Irrgang [Sat, 21 Mar 2020 18:29:53 +0000 (21:29 +0300)]
Extract logic for when CI jobs execute.
Introduce `.rules:<condition>` mix-ins to assert when jobs should run.
Change-Id: I1bd74f347fb8e19ddc49dee2d173520dcb436820
M. Eric Irrgang [Sat, 21 Mar 2020 15:32:56 +0000 (18:32 +0300)]
Normalize some more job names.
Also moves a few jobs between files as seems appropriate.
Eliminates some trivially inherited jobs.
Change-Id: Ib8e989ef560cc14387d14eb0083b5d678ee28651
M. Eric Irrgang [Sat, 21 Mar 2020 14:01:03 +0000 (17:01 +0300)]
Simplify archive.gitlab-ci.yml
Merge some trivially inherited jobs.
Change-Id: Id0a3a998bf5fa3f08db0dccb7da1aadb4f2902a0
M. Eric Irrgang [Sat, 21 Mar 2020 12:58:56 +0000 (15:58 +0300)]
Remove redundant jobs.
Remove distinction between jobs that do and do not stage a build of the
gmxapi Python package in the build tree for API breakage testing and
full documentation builds.
Set a default CMake flag of "-DGMX_PYTHON_PACKAGE=ON" and install some
Python package dependencies with the default EXTRA_INSTALLS. The extra
installs will go away in the near future with updated Docker images.
Removes (up to) 12 jobs from the pipeline.
Change-Id: I13a60b7323ac0bf240f00b3b1360bedd2886e902
M. Eric Irrgang [Sat, 21 Mar 2020 12:25:01 +0000 (15:25 +0300)]
Fix duplicate variables parameter.
Defining the variables parameter twice hurts readability and may have
undefined behavior.
Change-Id: I2b22f89628237277b9596ad6a1cec84295849ce1
M. Eric Irrgang [Sat, 21 Mar 2020 11:27:50 +0000 (14:27 +0300)]
Add required Infrastructure for Gitlab merges
Also adds runs for regressiontests to normal CI.
Change-Id: I7bcb62cd3f2120f1d2e80b2f8a2ec1e328626902
(cherry picked from commit
f518d72b2bea9a0dba02695dc93ca55ef1b09139)
M. Eric Irrgang [Fri, 20 Mar 2020 18:40:25 +0000 (21:40 +0300)]
Normalize naming for gromacs build/testing jobs.
* Rename testing-matrix.gitlab-ci.yml to gromacs.gitlab-ci.yml
* Use a prefix of "gromacs" for jobs that manage the CMake-driven build
and testing procedure to improve readability and make it easier to
locate the sources of jobs mentioned in other files.
* Merge a few more trivial templates.
Change-Id: I6899dcf5a389190f6e973105c47d7743bf0ec802
M. Eric Irrgang [Fri, 20 Mar 2020 17:34:58 +0000 (20:34 +0300)]
Remove some unnecessary indirection in job definitions.
Some job templates were only used once or only existed to extend a
single parent. Other templates became trivial when accounting for
job parameter merging behavior. Additionally, removed some ineffective
or unnecessary parameter or variable specifications.
Note: in the case of the 'configure' jobs, KUBERNETES variables that
were previously masked by job inheritance are now exposed, in accordance
with the apparent original intent.
Change-Id: Idd4100d729a08c47b83c42ca9d7e0257a4029c6f
M. Eric Irrgang [Fri, 20 Mar 2020 13:40:16 +0000 (16:40 +0300)]
Consolidate .use-clangX CI job mix-in.
* Merge .build-clang-template and .clang-before-script-template into
.use-clang:base
* Replace .clangX-template with .use-clangX, intended to provide
everything needed for a gccX build environment.
Change-Id: I5b78cddb47befd2dceb68c144cf427906aeeac36
M. Eric Irrgang [Wed, 18 Mar 2020 18:26:57 +0000 (21:26 +0300)]
Consolidate .use-gccX CI job mix-in.
* Remove some unused templates.
* Merge .build-gcc-template and .gcc-before-script-template into
.use-gcc:base
* Replace .gccX-template with .use-gccX, intended to provide everything
needed for a gccX build environment.
* Document EXTRA_INSTALLS variable and default value.
* Document COMPILER_MAJOR_VERSION variable.
* Document CMAKE_COMPILER_SCRIPT variable.
Change-Id: I07fa648200095ed5e97dea6638031cbb412a615c
Artem Zhmurov [Thu, 20 Feb 2020 15:50:29 +0000 (16:50 +0100)]
Introduce DeviceStreamManager
Make a separate object that will be handling the creation,
management and destruction of the GPU context and streams.
It is detached from the rest of the code in this patch,
but will be attached in the follow-up.
Refs #3316
Refs #3311
Change-Id: I2c59b930ac266d89fafe9e0172b83f07e9858f0b
M. Eric Irrgang [Thu, 19 Mar 2020 16:47:26 +0000 (19:47 +0300)]
Adopt CI job naming guidelines.
Refs #3275
Change-Id: Ib1fdf4eb468fe2a4620e523f265226c09051bba2
M. Eric Irrgang [Fri, 20 Mar 2020 12:35:52 +0000 (15:35 +0300)]
Normalize GitLab-CI *cache* parameter.
* Document in gitlab.rst
* Remove unnecessary indirection for null *cache* parameter.
* Remove *.pull-cache-template* no-op.
* Rename *.build-cache-template* to *.use-ccache*
Change-Id: I3623d6834360498c69bf968b926139c884cab867
M. Eric Irrgang [Mon, 16 Mar 2020 16:02:38 +0000 (19:02 +0300)]
Merge environment support into global.gitlab-ci.yml
Change-Id: I03fdf0b43088db04e2beb32d0a2b9df3c6a25c54
M. Eric Irrgang [Mon, 16 Mar 2020 16:01:37 +0000 (19:01 +0300)]
Move .build-template.yml definitions closer to point of use.
Change-Id: I2d4d5e6063495b658550640c7cdf6655e0f03f6e
M. Eric Irrgang [Mon, 16 Mar 2020 15:17:59 +0000 (18:17 +0300)]
Merge *-test-template.yml into testing-matrix.gitlab-ci.yml
Change-Id: I339dd467b082000aaeffc0d65fdc5cbdde39c5b8
M. Eric Irrgang [Fri, 20 Mar 2020 11:19:54 +0000 (14:19 +0300)]
Merge .test-runner-template.yml into testing-matrix.gitlab-ci.yml
Change-Id: Id12c8b3922f737552b08887a0533db22b1cebe06
M. Eric Irrgang [Fri, 20 Mar 2020 11:14:13 +0000 (14:14 +0300)]
Merge .test-script-template.yml into testing-matrix.gitlab-ci.yml
Also remove unused template.
Change-Id: Ibac0b13f8a62c6ed3d9992247d85ff34a5b45dcf
M. Eric Irrgang [Fri, 20 Mar 2020 11:06:05 +0000 (14:06 +0300)]
Merge .regressiontest-template.yml into archive config.
Change-Id: I0659399f4ec9e4e3a651e5f6818e56ed1820ee62