Mark Abraham [Fri, 20 Sep 2019 15:46:06 +0000 (17:46 +0200)]
Improve PME testing
The change to PME coordinate management uglified these tests even
further. Creation semantics are now more clear.
Change-Id: I309a487cdd218d85dbe4620e1d94e2f04f809a8d
Berk Hess [Wed, 25 Sep 2019 09:42:29 +0000 (11:42 +0200)]
Use gmx::Range in domdec
Renamed gmx_domdec_ns_ranges_t to DDPairInteractionRanges
and made it use gmx::Range. This is now used in an std::vector
in gmx_domdec_zones_t.
Note: more refactoring should be done in gmx_domdec_zones_t.
Change-Id: Iff61096ee8ce1e45a998a905fd56e0c2f43fa319
Artem Zhmurov [Tue, 3 Sep 2019 12:23:40 +0000 (14:23 +0200)]
StatePropagatorDataGpu object to manage GPU forces, positions and velocities buffers
In current version the positions and forces on the GPU are managed by different
modules, depending of the offload scenario for a particular run. This makes
management of the buffers complicated and fragile. This commit adds the object
responsible for management of the GPU buffers of coordinates, forces and
velocities. The object is connected to all clients that use coordinates, forces
and velocities buffers, while keeping the existing logic intact where its
possible.
Since the H2D and D2H copies are now done in nullptr stream, some of implicit
synchronization is lost. Consequently this commit does not always work
properly with newly introduced buffer ops / halo exchange features. To avoid
the confusion, GPU buffer ops are disabled by the assertion. There will be
a separate commit with all copies done synchronously, which will work
with the buffer ops. The stream- and event-based synchronization will be
introduced in the follow-up commits.
Refs. #2816.
Change-Id: I2e2ba1b6436f087d1f2fef4ff876445814a724e7
Mark Abraham [Mon, 30 Sep 2019 18:06:20 +0000 (20:06 +0200)]
Move GPU task assignment
This commit changes no functionality except that it does re-order some
log-file reporting, with GPU task assignment reporting now preceding
OpenMP and thread affinity reporting.
Now that there is multi-stage DD construction, the GPU task assignment
can be done between the stages. This will permit the construction of
GPU resources in time to build domain objects with GPU stream(s),
where appropriate.
Change-Id: I1734afa65ee5b54995ccf23a69ff25ec6587a019
Mark Abraham [Tue, 1 Oct 2019 13:54:06 +0000 (15:54 +0200)]
Fix icc warnings
This makes the intent clearer to see
Change-Id: I633d3921d125edf4cb529554b04fd8667a76843a
Berk Hess [Thu, 19 Sep 2019 14:59:26 +0000 (16:59 +0200)]
Completely remove charge groups
Charge groups remain in the .top and .tpr formats for backward
compatibility.
Change-Id: I9410d464dae07e0dac5bddfcc0e551c821963547
Berk Hess [Mon, 30 Sep 2019 20:13:27 +0000 (22:13 +0200)]
Simplify DD exclusion counting
Change-Id: I7004b6860fc0c21fc4d82378b4a86d29a94d9391
Christoph Junghans [Mon, 30 Sep 2019 23:46:46 +0000 (17:46 -0600)]
cmake: add missing header to legacy api
range.h is now needed by block.h
Change-Id: I372ddbe6475511d3e6e711e2f677792d568ac853
Pascal Merz [Tue, 1 Oct 2019 04:28:08 +0000 (22:28 -0600)]
Output correct kinetic at last step (modular simulator)
This replicates the fix in I9a0bc228 for the modular simulator.
Fixes a bug which would write an incorrect kinetic energy and
temperature to the energy and log files when using the leap-frog
integrator and having a last step not coinciding with an energy
calculation step.
Refs #2950
Change-Id: I9969fa8114240145d9aa84f381a898bbb7570b75
Mark Abraham [Tue, 21 May 2019 14:45:17 +0000 (16:45 +0200)]
Removed dependency on commrec of mdrun setup
Changes no functionality.
Setup is now parameterized directly on MPI_COMM_WORLD, which we will
want later for letting library-based callers pass in an
MPI_Communicator. This permits commrec to be initialized later, once
the threads have been spawned for the thread-MPI ranks.
The initialization of multi-simulations moves from LegacyMdrunOptions
to SimulationContext, which is more appropriate for
ensemble-parallelism established directly by the user.
Before the decision about the duty of a rank, there is no difference
between MASTER(cr) and SIMMASTER(cr), so several calls to macros
taking a t_commrec pointer are replaced by booleans. Introduced
findIsSimulationMasterRank to compute that value. This eliminates
early use of t_commrec that has necessitated other hacks and
workarounds.
Removed redundant check for replica exchange when the number of multi
simulations is less than two, because gmx_multisim_t constructor
already prohibits that.
Resolves several TODO items and improves modularity, too.
Refs #2587, #2605, #3081
Change-Id: I48bd3b713bc181b5c1e4cbcd648706a9f00eab96
Carsten Kutzner [Mon, 30 Sep 2019 12:28:29 +0000 (14:28 +0200)]
Move ForceProviders class into the gmx namespace
Since the C group kernels have been removed there is no need any more for
having ForceProviders as a struct outside the gmx namespace.
Change-Id: I2c91fbd06be289e3eb8acd74436d931cf3e994de
Mark Abraham [Mon, 30 Sep 2019 08:36:35 +0000 (10:36 +0200)]
Make a builder for DD
This creates the seam for task assignment to use the rank duty
assignment to choose GPU devices and construct resources like streams
before completing building the DD object with such resouces in place.
Used a pimpl class to keep the DD setup implementation details
private.
Change-Id: I6dbfe0122777bb529f8007386ff06b227e355263
Szilárd Páll [Mon, 30 Sep 2019 10:54:52 +0000 (12:54 +0200)]
Move PME GPU program building later
Moves the code closer to where it is needed and later during
initialization so even without a high-level manager, context between
nonbonded and PME could be be shared.
Change-Id: I47d9645dc0ee5574960964dfd3f79479ff21bcea
Berk Hess [Thu, 19 Sep 2019 14:42:07 +0000 (16:42 +0200)]
Remove charge group code from forcerec
Removed sevearal charge group members from t_forcerec.
Removed the cginfo intra-cg exclusion flags, as it is always true
for single atoms.
Removed loops over charge groups from init_cginfo_mb().
Change-Id: Ic91ea78dc341dc9972567a488294cb3d262ef9f6
Carsten Kutzner [Mon, 30 Sep 2019 12:56:18 +0000 (14:56 +0200)]
Corrected a few typos in documentation and comments
Change-Id: Ic697080a7c1f1a01bb8370563c1256990935d05e
Szilárd Páll [Tue, 10 Sep 2019 15:40:01 +0000 (17:40 +0200)]
Make the wait on PME GPU results conditional
When the PME forces are reduced on-GPU and no energy/virial output is
produced, we can avoid blocking waiting on the CPU for the PME GPU
taks to complete.
This however would break the timing accounting which needs to happen
after PME tasks completed. Hence the accounting is moved to the PME
output clearing.
Refs #3029, #2817
Change-Id: I4e7f3aa43754a187fe5d6b584803444967516958
Berk Hess [Sun, 29 Sep 2019 20:21:15 +0000 (22:21 +0200)]
Increase ti3p5 minimization test tolerance
This is needed because of larger differences when running
this test on multiple ranks.
Change-Id: I3056ae5933800e30e274791e8d27e2b8e79f6e04
Berk Hess [Sat, 28 Sep 2019 21:22:26 +0000 (23:22 +0200)]
Fix CG minimization with DD
A force for determining the CG direction was not communicated
correctly with domain decomposition to recent refactoring change
5ed943bc.
Fixes #3112
Change-Id: Ic33062f4aad0ccd7c1296c7351bc71d735ebba24
Artem Zhmurov [Fri, 27 Sep 2019 11:58:40 +0000 (13:58 +0200)]
Slight improvements to GPU update/constraints initialization
1. Assertions are updated to better correspond to the supported
conditions.
2. The message in the log now depends on whether or not there are
constraints in the system.
3. Redundant set(...)/setPbc(...) before the MD loop removed.
The getter for the total number of constraints is also added to
the Constraints class.
TODO: The assertions can be removed once the supported
functionality is expanded.
Change-Id: I96e2b993f79cf721f7aa48b9af0eab6500c593ba
Szilárd Páll [Fri, 27 Sep 2019 13:11:32 +0000 (15:11 +0200)]
Switch GPU update off by default
The code-path is untested and not fit for being on by default, so better
kept off until it becomes stable and better tested.
Change-Id: Ia651b0ae2f0f6751c13ea6857cdf6d176b83e34e
Artem Zhmurov [Thu, 26 Sep 2019 00:01:31 +0000 (02:01 +0200)]
Introduce stream management to GPU Update-Constraints
The GPU stream in the GPU versions of Leap Frog, LINCS and SETTLE
is now managed by the UpdateConstraints module and passed to the
members. This prepares Update-Constraints to be switched to
non-nullptr stream.
Change-Id: I95b9e4112874eb02ea646975760922ab31ce6b27
Berk Hess [Wed, 25 Sep 2019 09:19:31 +0000 (11:19 +0200)]
Extract gmx::Range
Added a range class for general use by extracting
gmx::RangePartinioning::Block to gmx::Range in range.h and renaming
and adding some methods.
Change-Id: Id9384c139d4daf1a31240e28333c9e9801c2dffc
Berk Hess [Fri, 27 Sep 2019 13:51:09 +0000 (15:51 +0200)]
Fix location of angle in vsite figure
The virtual site figure in the manual had an angle circle
in the wrong location (after shifting part of figure).
Fixes #3109
Change-Id: Ieb19485f231cff3e15bfd4158d64b82f7b69fac5
Mark Abraham [Fri, 27 Sep 2019 07:38:38 +0000 (09:38 +0200)]
Improve MiMiC test implementation
These tests were not using callGrompp in the intended way, and
inadvertently relying on behavior that might change when follow-up
work on #3081 changes mdrun thread-MPI initialization behavior
Change-Id: Icdbb6049a7b609ddaabc8e073f87e8bbb00da4f2
Mark Abraham [Wed, 25 Sep 2019 17:30:36 +0000 (19:30 +0200)]
Stop checking CUDA version reported by nvcc
We should trust FindCUDA.cmake to do its job (and not find headers
that don't match nvcc), and also prepare to use the native CUDA
support in CMake (where nvcc isn't used).
Removed unused GMX_CUDA_VERSION config variable
Change-Id: I8959c16c7e1247dc114bae818e0b867cf6b8ae71
Mark Abraham [Thu, 26 Sep 2019 17:54:20 +0000 (19:54 +0200)]
Fix gro file output with index groups
Fixes #3107
Change-Id: I3fc586c69066a354b3210d9616125ef666f1ce26
Mark Abraham [Thu, 26 Sep 2019 17:50:00 +0000 (19:50 +0200)]
Add tests for editconf file conversion with indexing
The indexed output to gro files illustrates bug #3107
Fixed two memory leaks necessary to let the tests pass
Refs #3107
Change-Id: Ie61430210ac8a804fb86783c8b76c41333b8d3cf
Mark Abraham [Tue, 13 Aug 2019 12:31:57 +0000 (14:31 +0200)]
Remove leftover support for pre-9.0 CUDA
Refs #2831
Change-Id: I7ec33bb3582006123e745d06da27c9eed12fbfc2
Berk Hess [Thu, 26 Sep 2019 21:26:33 +0000 (23:26 +0200)]
Made mdrun appending work again
Fix that mdrun would only append the log file and create new files
for all other file types.
Fixes #3108
Change-Id: I0fe517f4ce3f6630e589a65d106ca67bcb14765b
Paul Bauer [Wed, 25 Sep 2019 15:42:38 +0000 (17:42 +0200)]
Revert "Remove update flag again"
This reverts commit
85ef063c53f77a3423b2943d16d1d2f3759aa1bf.
Reason for revert: We want to have the user interface available again and tested.
Change-Id: I976531f37a9aaf2782b22f22653804ad0f303b95
Szilárd Páll [Wed, 25 Sep 2019 14:05:49 +0000 (16:05 +0200)]
Merge remote-tracking branch 'origin/release-2019'
Change-Id: Ia822fecb18b63d8a4e4408e056850a42875d57e8
Paul Bauer [Mon, 9 Sep 2019 09:04:37 +0000 (11:04 +0200)]
GROMACS 2020 first beta release
Updated regressiontest hash.
TODO branch 2020 from master and set correct regressiontest branch.
Change-Id: Ibe58dfbec933aa6166a7573d2d371614dbacb52f
Mark Abraham [Fri, 20 Sep 2019 16:55:32 +0000 (18:55 +0200)]
Fix log file flag output
Fixes #3099
Change-Id: I5049e8807b87c83959a0a2a5b3439b904d5edc8b
Paul Bauer [Tue, 24 Sep 2019 10:25:36 +0000 (12:25 +0200)]
Add test for noappend
New test covers behaviour where users always set the -noappend flag.
Also adds documentation for checkpoint header.
Change-Id: I012bb6d20eb297bb654cbddfe0c2530161fc902b
Mark Abraham [Tue, 24 Sep 2019 14:09:27 +0000 (16:09 +0200)]
Fix no-MPI CUDA build
Change-Id: I32407bf373acca24dd7250503ab78387fa6b8404
Mark Abraham [Tue, 24 Sep 2019 16:03:08 +0000 (18:03 +0200)]
Merge "Merge branch release-2019 into master"
Magnus Lundborg [Tue, 24 Sep 2019 09:43:44 +0000 (11:43 +0200)]
Add .part0001. to files from the first run with -noappend.
There was a bug where only the runs after the first got
.partxxxx. added when running with -noappend.
Fixes #3103
Change-Id: I36fddfca058216a50f9910c548b8a106bf460f7b
Paul Bauer [Mon, 23 Sep 2019 07:44:00 +0000 (09:44 +0200)]
Merge branch release-2019 into master
Resolved Conflicts:
cmake/gmxDetectCpu.cmake
src/gromacs/gmxpreprocess/topshake.cpp
src/gromacs/listed-forces/bonded.cpp
src/gromacs/mdlib/nbnxn_cuda/nbnxn_cuda_data_mgmt.cu
src/gromacs/mdlib/nbnxn_gpu.h
src/gromacs/mdlib/nbnxn_ocl/nbnxn_ocl.cpp
src/gromacs/mdlib/nbnxn_ocl/nbnxn_ocl_data_mgmt.cpp
This does not include the change to the MPI_COMPILE flags handling,
as the code in master has been reorganized in a way that is incompatible
to that.
Change-Id: Idb1da69ce68f3121442859e8cbe24495c7894726
Paul Bauer [Fri, 14 Jun 2019 14:50:22 +0000 (16:50 +0200)]
Read in TPR char buffer as vector
Perform the I/O of the TPR char buffer as xdr_vector operation instead
of using single bytes.
Also use the xdr vector specialization for unsigned char and rvecs.
Refs #2971
Change-Id: I20534985fbdee8108792f676b3cb4264ab74c456
Mark Abraham [Tue, 24 Sep 2019 10:01:29 +0000 (12:01 +0200)]
Fix clang unused-function warnings
There are many places inline functions in headers are declared that
won't be used so we should suppress the clang warning about that.
Change-Id: I01c26f68473543007427e5baafc74eee966c0e68
Berk Hess [Thu, 19 Sep 2019 20:02:43 +0000 (22:02 +0200)]
Remove obsolete dd_get_ns_ranges()
Change-Id: Id64f2d51d6684cf4c22ebd770963cc69f7af087b
Paul Bauer [Tue, 17 Sep 2019 09:26:13 +0000 (11:26 +0200)]
Add release notes for modular simulator
Change-Id: I0f9081df842ce952ed40bf3844f7f2c10a59761c
Paul Bauer [Mon, 23 Sep 2019 12:09:17 +0000 (14:09 +0200)]
Fix issues when PME rank request becomes negative
Change-Id: I5f9888693983ed732c59a8fe62d596453b6a80dd
Berk Hess [Mon, 23 Sep 2019 11:33:21 +0000 (13:33 +0200)]
Limit Zen nbnxm kernel choice to Zen 1
Changed the Zen flag in gmx_hw_info_t to a Zen1 flag, so Zen 2,
and newer, is treated as a generic x86 AVX2 architecture.
Change-Id: I5a0fd73973416bbd8cd46422a6f22f1ad95d9924
Pascal Merz [Mon, 23 Sep 2019 00:16:44 +0000 (18:16 -0600)]
Parrinello-Rahman for md-vv
Change-Id: I3e49ad4d54fa9533e8a576661b0da3d6239bfd28
Pascal Merz [Sun, 22 Sep 2019 07:37:29 +0000 (01:37 -0600)]
Parrinello-Rahman barostat for modular simulator (leap-frog)
This introduces the Parrinello-Rahman barostat for the leap-frog
version of the modular simulator. Mirroring the implementation in
the legacy do_md, this does not support md-vv. Implementing the
pressure coupling requires the virial calculation to be synchronized
with the pressure coupling, so this is included in the energy
signaller.
As the new integrator framework is not used by default, this
patch adds equivalence tests for md Parrinello-Rahman pressure
coupling.
Change-Id: Iaadd78a3da1ee2b04978ab7c7a7caecbb03181e2
Paul Bauer [Fri, 14 Jun 2019 09:42:28 +0000 (11:42 +0200)]
Change MPI setup to communicate TPR as buffer
Changed the initial setup of nodes to communicate the full tpr file
buffer instead of using the individual calls for the fields.
Now non-master nodes receive the inputrec and mtop
and populate them themselves.
Refs #2971
Change-Id: Id4f3739a978ca507dacc45c78a8a75368cfe86fd
Pascal Merz [Sun, 22 Sep 2019 07:29:24 +0000 (01:29 -0600)]
Zero pressure and virial tensors at beginning of step
This sets pressure and virial tensors to zero when they are
accessed for the first time in a step. Not doing this does not
yield wrong behavior currently, but carries the risk of
hard-to-spot bugs moving forward.
Change-Id: I2553bd52fa56a497a61c31738ec4436f7f587ece
Pascal Merz [Mon, 23 Sep 2019 00:07:37 +0000 (18:07 -0600)]
Introduce FEP into modular simulator
This introduces free energy calculations in the modular simulator.
The lambda vector and the current FEP state are held by the
FreeEnergyPerturbationElement, offering access to its values via getter
functions. The FreeEnergyPerturbationElement does update the lambda
values during the simulation run if lambda is non-static. It does
implement the checkpointing client interface to save its current
state for restart.
Elements using the FEP getters include
* force
* shell/fc
* compute globals
* constraint element (only for initial constraining)
* state propagator data (only for trajectory writing)
The free energy itself is calculated by the energy element.
Change-Id: Iee12697a7ef54a490208fb1457528dc4368c5d5b
Christian Blau [Fri, 20 Sep 2019 09:50:33 +0000 (11:50 +0200)]
Grompp error for mismatching nst for energy calulation and densityfitting
When using multiple-time-stepping with density-guided simulations, the
energy evaluation frequency of the density-guided simulations must
match the "nstcalcenergy" option.
Change-Id: Ifcde8cada8a91540d69c9f91079bfc91c4415ba5
Christian Blau [Mon, 16 Sep 2019 08:49:11 +0000 (10:49 +0200)]
Amendments to density fitting manual section
- Fixed a sign error in the energy and force definition
- Added performance considerations
- Fixed whitespace
- Changed vector notation to mathbf as in the other parts of the manual
- Added pressure-coupling considerations
- Added considerations when using multiple-time-stepping
refs #2282
Change-Id: I8421ccf09ac960fa04508234e738967f51a27fab
Mark Abraham [Wed, 7 Aug 2019 13:03:51 +0000 (15:03 +0200)]
Fixes to broken CUDA compiler flags
Because FindCUDA.cmake is non-idiomatic CMake, recent changes to CUDA
compilation lost most of our custom compilation options for host code.
These are now restored.
Compilation options for build configurations deriving from the Release
build configuration are also restored.
Refs #3099
Change-Id: If030a256481faed425379dab00ea658e3c215867
Pascal Merz [Mon, 23 Sep 2019 00:04:57 +0000 (18:04 -0600)]
Make CheckpointHelper safer and easier to use
This commit improves the CheckpointHelper in two ways:
* Allow the vector of checkpoint helpers to contain nullptr. This
makes the building process easier. Moving forward, this could
be replaced by a builder.
* Reset the legacy t_state flags at every checkpointing step. This
makes sure that only relevant data is checkpointed, and avoids
hiding potential bugs.
Change-Id: I4f2a58f281d0f3d8b3819825df3c52850090d42b
Mark Abraham [Fri, 20 Sep 2019 15:50:08 +0000 (17:50 +0200)]
Fix 1D-1pulse support
Passed the correct rank count to the new checking function
Change-Id: Ib27ba5112a50894dbb0f1a2eb68bba53204270e3
Pascal Merz [Sun, 22 Sep 2019 22:51:46 +0000 (16:51 -0600)]
Fix vrescale thermostat integral (modular simulator)
The v-rescale thermostat was calculating the thermostat integral,
but this was neither written out nor saved into checkpoints. The
current commit fixes this.
Note that this is a good example of why the energy element needs
to become more modular - it should not need to know about the
details of other elements. This is a todo moving forward though,
not something we can solve immediately now.
Change-Id: I69541cb8ffa3c07e294e583f2c20423952e3ec01
Christoph Junghans [Thu, 19 Sep 2019 22:54:41 +0000 (16:54 -0600)]
workaround for rhel bug #
1749463, cmake bug #18349
MPI_C_COMPILE_FLAGS is a list instead of a string
see: https://gitlab.kitware.com/cmake/cmake/issues/18349
We mainly want to workaround this issue as RHEL 8 triggers it:
https://bugzilla.redhat.com/show_bug.cgi?id=
1749463
This issue is fixed in cmake-3.12.3
Change-Id: I14c1c0a4f33f26b128d55baa1b0455f8a463a999
Christian Blau [Fri, 13 Sep 2019 08:32:50 +0000 (10:32 +0200)]
Add cross-correlation as density simlarity measure
Add the canonical cross-correlation as similarity measure for densities
refs #2282
Change-Id: Idec820f38dab07c5ee8281944855a532a587a2ff
Paul Bauer [Fri, 20 Sep 2019 12:03:08 +0000 (14:03 +0200)]
Remove update flag again
Disables assignment of update task to different hardware and removes
user option.
Change-Id: I7780ac54f65809cbd91386a0d6a92b7bee35ab26
Berk Hess [Thu, 19 Sep 2019 15:50:54 +0000 (17:50 +0200)]
Remove cgs_gl from gmx_domdec_comm_t
This is no longer needed now we no longer support charge groups.
This reduced cache pressure, especially with energy minimization.
Change-Id: Ibf51fc4d70b929463f8a77a82cbe0316bdedeab2
Artem Zhmurov [Mon, 16 Sep 2019 14:23:48 +0000 (16:23 +0200)]
Disable GPU update/constraints when neither PME nor buffer ops are offloaded.
Using the GPU-version of update makes sense if forces are already on the GPU,
i.e. if at least:
1. PME is on the GPU (there should be a copy of coordinates on a GPU in rvec
format for PME spread).
2. Non-bonded interactions and buffer ops are on the GPU.
This is temporary solution, needed because the buffer ops offload switch is
operated by the environment variable. More favorable behavior would be to
switch on the GPU buffer ops in the second case rather then disabling the
GPU update.
Refs. #2816.
Change-Id: I37a9969dd6c74dcfa41a95da13ae54d014c9ea60
Berk Hess [Sat, 21 Sep 2019 09:36:32 +0000 (11:36 +0200)]
Fix incorrect shift forces for CMAP
The shift force indices were inverted for the second and third
atom in the CMAP term, leading to incorrect virial and pressure
contributions when these atoms resided in different periodic images.
Fixes #2845 and #2867
Change-Id: I1946a1d375d6c62e4e6d23ee25b92b42a3d4a6f7
Pascal Merz [Fri, 20 Sep 2019 21:01:04 +0000 (15:01 -0600)]
Fix restarts in modular simulator
The modular simulator constraint element was mistakenly performing
an initial constraiinng when restarting from checkpoints.
Change-Id: Icf310ff6d67fcd7f4b8aeb4f9da2749ad098a513
Pascal Merz [Fri, 20 Sep 2019 04:26:19 +0000 (22:26 -0600)]
Fix domdec for md-vv modular simulator
Domdec might have changed the local velocity vector after the state
object made a backup to restore after the first md-vv half step.
This change makes sure that the backup is updated if this happens.
Note that the half-step resetting will be eliminated in the course
of the transition to the modular simulator, but is currently kept
for backwards compatibility.
Change-Id: Ifb6346055a14d34a4efcae43b7b41399b1444bca
Berk Hess [Fri, 20 Sep 2019 14:22:02 +0000 (16:22 +0200)]
Fix DD 1 pulse checks without DLB
The check for requesting single pulse DD communication only
worked correctly without DLB.
Also added a check for 1 pulse with PME tuning.
Todo: PME tuned runs might still end up with multiple pulses
due to pressure scaling. This needs to be checked.
Fixes #3100
Change-Id: Icac5e37ef79385fd7c7cf5c19c7b40b2d685b95b
Mark Abraham [Thu, 19 Sep 2019 14:51:21 +0000 (16:51 +0200)]
Refactor to create a GpuTaskAssignments object
This change is purely reorganization, nothing is reordered or
functionality changed.
This expands on the previous vector of vectors of task assignments
into an object with the invariant of a completed assignment. It has
very complex construction logic, so it has a separate builder class
for the data that describes the decision about which GPUs are assigned
to tasks running on ranks on each physical node.
This refactoring prepares for re-ordering mdrun setup
so that the decision of rank duty by DD can lead to
task assignment, device initialization and the creation
of GPU contexts and streams with which to initialize
the various modules that use them to do their work.
Minor improvements to documentation and naming in a few places.
Change-Id: Idec71a3016212a4125b23b780e1059dbb134d7fb
Pascal Merz [Fri, 20 Sep 2019 04:59:59 +0000 (22:59 -0600)]
Run simulator equivalence tests in double precision only
The simulator equivalence tests are a great help in the effort of
rewriting the simulator framework, but might be too sensitive for
general use. They have been shown to be extremely sensitive to
order of operations which inevitably changes in day-by-day
development. Most recent test failures could be traced back to
the absence of a parenthesis in otherwise identical code (for
md-vv), and to the differences between the SIMD and non-SIMD
code paths (for md), respectively. Double precision builds seem to
be less affected by such re-ordering, so this commit changes the
simulator equivalence tests to be only enabled for double precision
builds. As the simulator rewriting progresses, and the new code paths
get covered by the existing tests, we might want to move away from
these type of equivalence tests altogether.
Change-Id: Ia4ad269959efd207242795ecedb88eaf7a870589
Berk Hess [Thu, 19 Sep 2019 13:00:13 +0000 (15:00 +0200)]
Remove bLocalCG from DD code
Replaced the use of bLocalCG in gmx_domdec_comm_t by ga2la,
which can be done since we no longer support charge groups.
Change-Id: I601e4a9c63e685b21f764fc99dc71c0080e84a43
Berk Hess [Thu, 19 Sep 2019 10:26:42 +0000 (12:26 +0200)]
Remove obsolete make_exclusions_zone_cg()
Change-Id: I1d18a4228f2a9a1f41266587f6b4c66f8d200f81
Christian Blau [Thu, 19 Sep 2019 13:37:50 +0000 (15:37 +0200)]
Check input record also in tests
When checking .mdp option reading and writing, the check_ir routine was
omitted that would potentially alter the result of the .mdp file
reading.
This patch fixes this behaviour, so that also checks and rectifications
of the input record are performed.
refs #3098
Change-Id: I450188442579bc11d71eda99ba1f64f33730345b
Mark Abraham [Mon, 16 Sep 2019 06:38:06 +0000 (08:38 +0200)]
Use GPU halo exchange only when compatible DD is available
Change-Id: I624861e61e6f1b6854172102b12a4f9b51b94101
Berk Hess [Thu, 19 Sep 2019 12:09:38 +0000 (14:09 +0200)]
Remove charge group data from t_forcerec
Removed the solvent optimization and natoms flags from cginfo.
Removed obsolete environment variables from the code and manual.
Change-Id: Ib2c92a8297837c444a6a1169fefdcae80579fca1
Mark Abraham [Tue, 17 Sep 2019 10:38:05 +0000 (12:38 +0200)]
Further improve getDDGridSetup
Merged and simplified two similar setup functions that were sharing
the setting up of the number of PME-only ranks and the DD grid used on
the PP ranks. No functionality changes here, although some user
messages are slightly improved.
This helps create the seam that GPU halo exchange setup will need in
order to do a good job of triggering that communication path, namely
that there is a clear point when the number of PP ranks is known, so
that the search for a DD cell decomposition can be made in a way that
might suit GPU halo exchange.
It also permits more of the DD setup routines to be called without
leading to a fatal error or output confusing to a user, which will also
be needed when introducing support for GPU halo exchange.
Functions called in init_domain_decomposition no longer look up the
total number of ranks from cr, but instead accept an integer
parameter. This will enable thread-MPI checking for various
possibilities to work smoothly if we ever do that.
Used better variable naming, IVec and more const
correctness to improve readability, and improved comments.
Noted TODOs for further work at some future time.
Improved havePPDomainDecomposition (and some of its users) so
that we are less likely to erroneously use it before we have
initialized DD.
Change-Id: I65e285cc10d946c7c5bf298d7ff08c81ba05f3d8
Paul Bauer [Thu, 5 Sep 2019 14:01:47 +0000 (16:01 +0200)]
Add basic interface to run update on GPU
Added basic command line option to direct update on a GPU,
as well as adding the necessary checks.
Since now the default code path is moved to the GPU, overly
tight tolerances in simulator tests lead to the failure of
one of the tests. This was a result of a slightly low
absolute value of the kinetic energy specified in the
definition of tolerances, which is also corrected by this
patch.
Change-Id: Ieca885d935aa1e0d592735bc7027f8261288b635
ejjordan [Thu, 19 Sep 2019 10:01:00 +0000 (12:01 +0200)]
Remove or update comments containing 'group scheme'
Change-Id: Ic2dea073b7cb2a9e3ed6dbb4fbeb14dfedd5cac3
Berk Hess [Thu, 19 Sep 2019 08:11:58 +0000 (10:11 +0200)]
Add haveFreeEnergyWork to DomainLifetimeWorkload flags
Change-Id: Ic0a09e2acafedc8c2caefb6935da88177d4c87ee
Paul Bauer [Wed, 18 Sep 2019 13:34:34 +0000 (15:34 +0200)]
Split up do_tpx_body functions
Will facilitate only communicating the parts of the TPR file needed on
ranks other than master.
Refs #2971
Change-Id: Ia5a5fe4f1c9bda1340e0776a0a2d9e96a90d4d07
Kevin Boyd [Thu, 19 Sep 2019 02:52:41 +0000 (19:52 -0700)]
Remove note about cutoff-scheme from mdrun docs
Change-Id: Ie7c6cd8fb5855cf7af1dd3fdd99586e133625cd2
Christian Blau [Wed, 18 Sep 2019 10:11:43 +0000 (12:11 +0200)]
Consistently change densityfitting spreading weight to charge
In the mdp options, the density-guided-simulation-atom-spreading weight
has been changed from charges to "charge" but within the code and in an
.mdp file comment the plural charges was still used. Changed now all
over to make this consistent.
refs #2282
Change-Id: I055743c8301e4ad18ceed5670c62be0a93c2d5f1
Berk Hess [Wed, 18 Sep 2019 12:52:49 +0000 (14:52 +0200)]
Added free-energy kernel performance note
Change-Id: Iea5d2b124633c4188753c7b1ebb6f964edb2f644
Berk Hess [Wed, 18 Sep 2019 12:35:17 +0000 (14:35 +0200)]
Fix incorrect DD ndim assertion
An incorrect assertion was introduced in recent commit
ed91e159.
Merged that assertion with a similar one nearby
Change-Id: Idff2a0d8a1985a19019233d29c4804294824297c
Pascal Merz [Fri, 16 Aug 2019 20:39:09 +0000 (14:39 -0600)]
v-rescale thermostat for modular simulator
This commit introduces the v-rescale thermostat for the modular
simulator. It also implements general velocity scaling in the
propagators.
Note that since the modular simulator is the default code path for
all supported input using md-vv as an integrator, this automatically
moves over some of the test coverage from the legacy simulator to
the modular simulator. This does, however, also extend equivalence
testing between the modular and the legacy code paths to v-rescale
thermostatted simulations.
Change-Id: Ia22931ccc3f9340f7f6272e395df149bd73d9b21
Paul Bauer [Thu, 13 Jun 2019 09:52:36 +0000 (11:52 +0200)]
Read TPR file body in buffer
For now only reads the buffer and uses it to write new files.
Refs #2971
Change-Id: I77a18ca50e96486d688db8b0d7acdbedf29d613d
Szilárd Páll [Fri, 13 Sep 2019 00:02:45 +0000 (02:02 +0200)]
Set up workload data structures
This change estabilishes three data structures for describing the
computational workload within a run. Flags related to the workload are
categorized in three groups based on their lifetime:
- per-step flags in the StepWorkload class
- domain liftime / nslist steps in DomainLifetimeWorkload
- constant over the entire simulation in SimulationWorkload
The present change only introduces naming by: renaming of the
former ForceFlags to StepWorkload and PpForceWorkload to
DomainLifetimeWorkload.
Moving flags from e.g. the force schedule into these will follow.
Change-Id: I2bcc911091e2dd8ca0bcbf53a40dfcda09ba368b
Alan Gray [Tue, 3 Sep 2019 08:00:13 +0000 (01:00 -0700)]
GPU Force Halo Exchange
Activate with GMX_GPU_DD_COMMS environment variable.
Extends GPU Halo exchange feature to provide GPU Force halo exchange
functionality. Does not yet support virial steps, which require an
extra shift force reduction - these are currently performed on the
non-buffer ops / non direct-comm path. Also has same limitations as
coordinate halo exchange.
Performs part of #2890. Future work to improve synchronization towards
a more one-sided scheme (#3092) and to make depenencies more
explicit (#3093)
Change-Id: Ifc23cc8db2655f7258e68b34e7cdc7b71994e1e8
Paul Bauer [Tue, 17 Sep 2019 12:21:56 +0000 (14:21 +0200)]
Fix MSVC warning
Should fix:
cl : Command line warning D9014
Change-Id: I24501ee27308df8abdbe3526d4035e74eb248286
Christian Blau [Fri, 13 Sep 2019 11:29:00 +0000 (13:29 +0200)]
Add mdp option documentation for density fitting.
Documents the .mdp options for the density fitting module.
refs #2282
Change-Id: Ic2cb616ca13627553a36d739d86192b0fcb0e239
Pascal Merz [Tue, 17 Sep 2019 02:32:57 +0000 (20:32 -0600)]
Minor tweaks to sum_dhdl and setCurrentLambdasLocal
setCurrentLambdasLocal() is taking the entire t_state as input,
but only uses the lambda vector and the free energy state. The
function arguments were changed to reflect that. This facilitates
rewriting the state data structure.
This change also adds a trivial const qualifier to sum_dhdl.
Change-Id: I50cb2b77711c13c1d77f6d55cfd32a324f94f76e
Pascal Merz [Tue, 17 Sep 2019 15:53:53 +0000 (09:53 -0600)]
Fix position pinning for modular simulator
This replicates Ia2b97d22 for the modular simulator, pinning the
position also in the non-DD case.
Fixes #3091
Change-Id: Ifb4b5a8897099152b7086fe700faa9a048c110a3
Berk Hess [Tue, 17 Sep 2019 14:14:35 +0000 (16:14 +0200)]
Remove spurious newline from log file
The annealing print function always added a newline.
Made calls to annealing and energy/log print conditional,
as they create needless overhead.
Change-Id: I63c0c6c9f1803aa8d42f59b40d7e2bff6fb6ac10
Paul Bauer [Tue, 17 Sep 2019 09:52:14 +0000 (11:52 +0200)]
Add deprecation note for 32 bit
Refs #3080
Change-Id: I033350ce0de76f30d11c44b5a73bf9b41ae91e97
Berk Hess [Tue, 17 Sep 2019 12:57:40 +0000 (14:57 +0200)]
Fix todos for update groups
Update groups allow for less PBC operations with domain decomposition
and allow Ewald dipole correction to work propely for small molecules.
Added checks and an assertion for Ewald dipole correction in grompp
and mdrun.
Change-Id: If072c6b0fe5afd591c3e01b1238d28820128c7af
Pascal Merz [Tue, 17 Sep 2019 10:00:53 +0000 (04:00 -0600)]
Fix modular simulator MPI bugs
This fixes two small errors which are not directly related to MPI, but
only show up in MPI runs:
* The checkpoint helper now receives the global number of atoms from the
global topology rather than the global state. This is identical to the
legacy implementation, and has the advantage that the global topology
is defined on every rank
* The state flags of the legacy state were not written correctly, or,
in some cases, not at all. This lead to errors using DD where the local
state was not properly allocated.
Change-Id: I6b6a48550065f16f415f46f20f16ee9c49da3dbe
Pascal Merz [Wed, 11 Sep 2019 18:44:14 +0000 (12:44 -0600)]
Make modular simulator the default code path for md-vv NVE
This change makes the modular simulator the default code path for
md-vv, if the options chosen are implemented. Effectively, this
means that currently, modular simulator will be chosen for md-vv
with NVE. The testing suite will hence use the new code path by
default for any applicable system.
Change-Id: I2635d58f8fc777a8efa04e216a5c612b7ef3a96b
Pascal Merz [Tue, 17 Sep 2019 03:05:30 +0000 (21:05 -0600)]
Fix force element
As FEP is currently turned off for the modular simulator, the force
element was using an empty ArrayRef as an argument for the lambda
vector in the call to the legacy do_force(). Further tests revealed,
however, that some force calculations use the lambda vector without
checking if FEP is needed. To avoid problems related to this, an
allocated array is used. Note that this change gets superseeded
by Iee12697a, so this is just a temporary fix.
Change-Id: Ibd09e83a8ee61f6da8c7f6ab8989777218407d94
Pascal Merz [Thu, 12 Sep 2019 09:57:12 +0000 (03:57 -0600)]
Enable checkpointing for modular simulator
This introduces the checkpoint helper and checkpoint helper clients.
The `CheckpointHelper` is responsible to write checkpoints. In the
longer term, it will also be responsible to read checkpoints, but this
is not yet implemented.
Writing checkpoints is done just before neighbor-searching (NS) steps,
or before the last step. Checkpointing occurs periodically (by default,
every 15 minutes), and needs two NS steps to take effect - on the first
NS step, the checkpoint helper on master rank signals to all other ranks
that checkpointing is about to occur. At the next NS step, the checkpoint
is written. On the last step, checkpointing happens immediately before the
step (no signalling). To be able to react to last step being signalled,
the CheckpointHelper does also implement the `ISimulatorElement` interface,
but does only register a function if the last step has been called.
Checkpointing happens at the top of a simulation step, which gives a
straightforward re-entry point at the top of the simulator loop. Moving
the checkpointing position requires to sum the kinetic energy (in
compute globals) a step earlier for md-vv.
In the current implementation, the clients of CheckpointHelper fill a
legacy t_state object (passed via pointer) with whatever data they need
to store. The CheckpointHelper then writes the t_state object to file.
This is an intermediate state of the code, as the long-term plan is for
modules to read and write from a checkpoint file directly, without the
need for a central object. The current implementation allows, however,
to define clearly which modules take part in checkpointing, while using
the current infrastructure for reading and writing to checkpoint.
This change also adds a lambda vector to the force element, since in
some cases, the force routine access the lambda vector even if FEP is
off, which leads to segmentation faults. It also moves the ownership
of needToSumEkinhOld_ from the compute globals element to the energy
element. It reintroduces the observablesHistory, which is owned by
the energy element. It also allows the CheckpointHelper to access
the outf private variable of TrajectoryElement.
Finally, this change also fixes a bug in the NeighborSearchSignaller
which would fail to signal the first step of the simulation run as a
NS step if it didn't fulfill do_per_step(step, nstlist). This was
not a problem as long as we only allowed new simulations but may
cause errors on restarts from checkpoints.
Change-Id: I15066fa66d653567f680a1c616a13ccfb7e3e955
Berk Hess [Thu, 12 Sep 2019 07:56:27 +0000 (09:56 +0200)]
Remove obsolete mdp option ns-type
Also removed unused member ndelta from t_inputrec.
Change-Id: Idde132176582001bdffafa6fa64af2eed9d0c242
ejjordan [Fri, 13 Sep 2019 08:55:11 +0000 (10:55 +0200)]
Add some cool quotes
Change-Id: Ib6a1565fb8cacf65523ec05e9fda52645d85a3de
Paul Bauer [Mon, 16 Sep 2019 11:14:45 +0000 (13:14 +0200)]
Formatting fixes for reference manual
Fixes #3090
Change-Id: I477e7422e29bc3dbccc91ba5a02f5bc90354d9fa
M. Eric Irrgang [Wed, 11 Sep 2019 08:50:06 +0000 (11:50 +0300)]
Add gmxapi release notes.
Change-Id: I12ee7c5707dcb17282aa29c30ec42271242b2a37
Szilárd Páll [Wed, 11 Sep 2019 17:07:00 +0000 (19:07 +0200)]
Refactor GPU buf ops-related launches in do_force()
- Consolidate branchy-ness into better organized code;
- Rename and clarify conditionals related to CPU force contributions:
for non-local forces only the presence of bonded interactions should
be a dependency, while for locals, the presence of any local force
contribtion from the CPU.
Change-Id: I91921d5d7425206b5980ec2b96032790a5d54408