2 * This file is part of the GROMACS molecular simulation package.
4 * Copyright (c) 2017, by the GROMACS development team, led by
5 * Mark Abraham, David van der Spoel, Berk Hess, and Erik Lindahl,
6 * and including many others, as listed in the AUTHORS file in the
7 * top-level source directory and at http://www.gromacs.org.
9 * GROMACS is free software; you can redistribute it and/or
10 * modify it under the terms of the GNU Lesser General Public License
11 * as published by the Free Software Foundation; either version 2.1
12 * of the License, or (at your option) any later version.
14 * GROMACS is distributed in the hope that it will be useful,
15 * but WITHOUT ANY WARRANTY; without even the implied warranty of
16 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
17 * Lesser General Public License for more details.
19 * You should have received a copy of the GNU Lesser General Public
20 * License along with GROMACS; if not, see
21 * http://www.gnu.org/licenses, or write to the Free Software Foundation,
22 * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
24 * If you want to redistribute modifications to GROMACS, please
25 * consider that scientific software is very special. Version
26 * control is crucial - bugs must be traceable. We will be happy to
27 * consider code for inclusion in the official distribution, but
28 * derived work must not be called official GROMACS. Details are found
29 * in the README & COPYING files - if they are missing, get the
30 * official version at http://www.gromacs.org.
32 * To help us fund GROMACS development, we humbly ask that you cite
33 * the research papers on the package. Check out http://www.gromacs.org.
35 /*! \libinternal \file
36 * \brief Declares functionality for deciding whether tasks will run on GPUs.
38 * \author Mark Abraham <mark.j.abraham@gmail.com>
39 * \ingroup module_taskassignment
43 #ifndef GMX_TASKASSIGNMENT_DECIDEGPUUSAGE_H
44 #define GMX_TASKASSIGNMENT_DECIDEGPUUSAGE_H
50 enum class EmulateGpuNonbonded : bool;
55 //! Record where a compute task is targetted.
56 enum class TaskTarget : int
63 /*! \brief Decide whether this thread-MPI simulation will run
64 * nonbonded tasks on GPUs.
66 * The number of GPU tasks and devices influences both the choice of
67 * the number of ranks, and checks upon any such choice made by the
68 * user. So we need to consider this before any automated choice of
69 * the number of thread-MPI ranks.
71 * \param[in] nonbondedTarget The user's choice for mdrun -nb for where to assign short-ranged nonbonded interaction tasks.
72 * \param[in] gpuIdsToUse The compatible GPUs that the user permitted us to use.
73 * \param[in] userGpuTaskAssignment The user-specified assignment of GPU tasks to device IDs.
74 * \param[in] emulateGpuNonbonded Whether we will emulate GPU calculation of nonbonded interactions.
75 * \param[in] usingVerletScheme Whether the nonbondeds are using the Verlet scheme.
76 * \param[in] nonbondedOnGpuIsUseful Whether computing nonbonded interactions on a GPU is useful for this calculation.
77 * \param[in] numRanksPerSimulation The number of ranks in each simulation.
79 * \returns Whether the simulation will run nonbonded tasks on GPUs.
81 * \throws std::bad_alloc If out of memory
82 * InconsistentInputError If the user requirements are inconsistent. */
83 bool decideWhetherToUseGpusForNonbondedWithThreadMpi(const TaskTarget nonbondedTarget,
84 const std::vector<int> &gpuIdsToUse,
85 const std::vector<int> &userGpuTaskAssignment,
86 const EmulateGpuNonbonded emulateGpuNonbonded,
87 const bool usingVerletScheme,
88 const bool nonbondedOnGpuIsUseful,
89 const int numRanksPerSimulation);
91 /*! \brief Decide whether this thread-MPI simulation will run
94 * The number of GPU tasks and devices influences both the choice of
95 * the number of ranks, and checks upon any such choice made by the
96 * user. So we need to consider this before any automated choice of
97 * the number of thread-MPI ranks.
99 * \param[in] useGpuForNonbonded Whether GPUs will be used for nonbonded interactions.
100 * \param[in] pmeTarget The user's choice for mdrun -pme for where to assign long-ranged PME nonbonded interaction tasks.
101 * \param[in] gpuIdsToUse The compatible GPUs that the user permitted us to use.
102 * \param[in] userGpuTaskAssignment The user-specified assignment of GPU tasks to device IDs.
103 * \param[in] canUseGpuForPme Whether the form of PME chosen can run on a GPU
104 * \param[in] numRanksPerSimulation The number of ranks in each simulation.
105 * \param[in] numPmeRanksPerSimulation The number of PME ranks in each simulation.
107 * \returns Whether the simulation will run PME tasks on GPUs.
109 * \throws std::bad_alloc If out of memory
110 * InconsistentInputError If the user requirements are inconsistent. */
111 bool decideWhetherToUseGpusForPmeWithThreadMpi(const bool useGpuForNonbonded,
112 const TaskTarget pmeTarget,
113 const std::vector<int> &gpuIdsToUse,
114 const std::vector<int> &userGpuTaskAssignment,
115 const bool canUseGpuForPme,
116 const int numRanksPerSimulation,
117 const int numPmeRanksPerSimulation);
119 /*! \brief Decide whether the simulation will try to run nonbonded
122 * The final decision cannot be made until after the duty of the rank
123 * is known. But we need to know if nonbonded will run on GPUs for
124 * setting up DD (particularly rlist) and determining duty. If the
125 * user requires GPUs for the tasks of that duty, then it will be an
126 * error when none are found.
128 * With thread-MPI, calls have been made to
129 * decideWhetherToUseGpusForNonbondedWithThreadMpi() and
130 * decideWhetherToUseGpusForPmeWithThreadMpi() to help determine
131 * the number of ranks and run some checks, but the final
132 * decision is made in this routine, along with many more
133 * consistency checks.
135 * \param[in] nonbondedTarget The user's choice for mdrun -nb for where to assign short-ranged nonbonded interaction tasks.
136 * \param[in] userGpuTaskAssignment The user-specified assignment of GPU tasks to device IDs.
137 * \param[in] emulateGpuNonbonded Whether we will emulate GPU calculation of nonbonded interactions.
138 * \param[in] usingVerletScheme Whether the nonbondeds are using the Verlet scheme.
139 * \param[in] nonbondedOnGpuIsUseful Whether computing nonbonded interactions on a GPU is useful for this calculation.
140 * \param[in] gpusWereDetected Whether compatible GPUs were detected on any node.
142 * \returns Whether the simulation will run nonbonded and PME tasks, respectively, on GPUs.
144 * \throws std::bad_alloc If out of memory
145 * InconsistentInputError If the user requirements are inconsistent. */
146 bool decideWhetherToUseGpusForNonbonded(const TaskTarget nonbondedTarget,
147 const std::vector<int> &userGpuTaskAssignment,
148 const EmulateGpuNonbonded emulateGpuNonbonded,
149 const bool usingVerletScheme,
150 const bool nonbondedOnGpuIsUseful,
151 const bool gpusWereDetected);
153 /*! \brief Decide whether the simulation will try to run tasks of
154 * different types on GPUs.
156 * The final decision cannot be made until after the duty of the rank
157 * is known. But we need to know if nonbonded will run on GPUs for
158 * setting up DD (particularly rlist) and determining duty. If the
159 * user requires GPUs for the tasks of that duty, then it will be an
160 * error when none are found.
162 * With thread-MPI, calls have been made to
163 * decideWhetherToUseGpusForNonbondedWithThreadMpi() and
164 * decideWhetherToUseGpusForPmeWithThreadMpi() to help determine
165 * the number of ranks and run some checks, but the final
166 * decision is made in this routine, along with many more
167 * consistency checks.
169 * \param[in] useGpuForNonbonded Whether GPUs will be used for nonbonded interactions.
170 * \param[in] pmeTarget The user's choice for mdrun -pme for where to assign long-ranged PME nonbonded interaction tasks.
171 * \param[in] userGpuTaskAssignment The user-specified assignment of GPU tasks to device IDs.
172 * \param[in] canUseGpuForPme Whether the form of PME chosen can run on a GPU
173 * \param[in] numRanksPerSimulation The number of ranks in each simulation.
174 * \param[in] numPmeRanksPerSimulation The number of PME ranks in each simulation.
175 * \param[in] gpusWereDetected Whether compatible GPUs were detected on any node.
177 * \returns Whether the simulation will run nonbonded and PME tasks, respectively, on GPUs.
179 * \throws std::bad_alloc If out of memory
180 * InconsistentInputError If the user requirements are inconsistent. */
181 bool decideWhetherToUseGpusForPme(const bool useGpuForNonbonded,
182 const TaskTarget pmeTarget,
183 const std::vector<int> &userGpuTaskAssignment,
184 const bool canUseGpuForPme,
185 const int numRanksPerSimulation,
186 const int numPmeRanksPerSimulation,
187 const bool gpusWereDetected);