Divide default communicator from DD communicators
authorMark Abraham <mark.j.abraham@gmail.com>
Wed, 15 Jul 2020 17:09:31 +0000 (17:09 +0000)
committerPascal Merz <pascal.merz@me.com>
Wed, 15 Jul 2020 17:09:31 +0000 (17:09 +0000)
commite0f481ae943b4fac5ddd3c6ed7f1af981f1d7de6
tree8c501bcd2ef9a44ac24b8ec159bf4d03a3bde796
parentbbe309502c792978b82fc1d2385e31235c365768
Divide default communicator from DD communicators

The communicators mpi_comm_mysim and mpi_comm_mygroup inside
t_commrec got initialized in init_commrec (to MPI_COMM_WORLD
if no multisim, to a subset otherwise). These communicators
were then used in subsequent setup work, before they got
reassigned during the construction of the DDBuilder object
and the construction of the actual domain decomposition object.
Effectively, this means that the same communicators (and, hence,
identical function calls) do very different things depending on
whether they get used before or after the setup of domain
decomposition. It also means that before DD set up, mpi_comm_mysim
and mpi_comm_mygroup are *identical*.

This change introduces an additional communicator within
t_commrec, mpiDefaulCommunicator, which helps to make these
implicit assumptions explicit. Consequently, this also redefines
PAR(cr), MASTER(cr), and SIMMASTER(cr).

This change will allow to move the sim and group communicators,
which are now only created at DD time, into the DD object,
logically separating the DD object from t_commrec.

Refs #2395
23 files changed:
src/gromacs/applied_forces/tests/electricfield.cpp
src/gromacs/awh/awh.cpp
src/gromacs/awh/biassharing.cpp
src/gromacs/awh/biasstate.cpp
src/gromacs/gmxlib/network.cpp
src/gromacs/gmxlib/network.h
src/gromacs/listed_forces/disre.cpp
src/gromacs/listed_forces/orires.cpp
src/gromacs/mdlib/lincs.cpp
src/gromacs/mdlib/md_support.cpp
src/gromacs/mdlib/mdoutf.cpp
src/gromacs/mdlib/tests/constrtestrunners.cpp
src/gromacs/mdrun/replicaexchange.cpp
src/gromacs/mdrun/runner.cpp
src/gromacs/mdrun/simulationcontext.cpp
src/gromacs/mdrunutility/handlerestart.cpp
src/gromacs/mdrunutility/multisim.cpp
src/gromacs/mdrunutility/multisim.h
src/gromacs/taskassignment/resourcedivision.cpp
src/gromacs/tools/pme_error.cpp
src/gromacs/utility/futil.cpp
src/gromacs/utility/mpiinplacebuffers.cpp [deleted file]
src/gromacs/utility/mpiinplacebuffers.h [deleted file]