2 % This file is part of the GROMACS molecular simulation package.
4 % Copyright (c) 2013,2014, by the GROMACS development team, led by
5 % Mark Abraham, David van der Spoel, Berk Hess, and Erik Lindahl,
6 % and including many others, as listed in the AUTHORS file in the
7 % top-level source directory and at http://www.gromacs.org.
9 % GROMACS is free software; you can redistribute it and/or
10 % modify it under the terms of the GNU Lesser General Public License
11 % as published by the Free Software Foundation; either version 2.1
12 % of the License, or (at your option) any later version.
14 % GROMACS is distributed in the hope that it will be useful,
15 % but WITHOUT ANY WARRANTY; without even the implied warranty of
16 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
17 % Lesser General Public License for more details.
19 % You should have received a copy of the GNU Lesser General Public
20 % License along with GROMACS; if not, see
21 % http://www.gnu.org/licenses, or write to the Free Software Foundation,
22 % Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
24 % If you want to redistribute modifications to GROMACS, please
25 % consider that scientific software is very special. Version
26 % control is crucial - bugs must be traceable. We will be happy to
27 % consider code for inclusion in the official distribution, but
28 % derived work must not be called official GROMACS. Details are found
29 % in the README & COPYING files - if they are missing, get the
30 % official version at http://www.gromacs.org.
32 % To help us fund GROMACS development, we humbly ask that you cite
33 % the research papers on the package. Check out http://www.gromacs.org.
35 \chapter{Technical Details}
37 \section{Installation}
38 The entire {\gromacs} package is Free Software, licensed under the GNU
39 Lesser General Public License; either version 2.1 of the License, or
40 (at your option) any later version.
41 The main distribution site is our WWW server at {\wwwpage}.
43 The package is mainly distributed as source code, but others provide
44 packages for Linux and Mac. Check your Linux distribution tools
45 (search for gromacs). On Mac OS X the {\bf port} tool will allow you
46 to install a recent version.
47 On the home page you will find all the information you need to
48 \normindex{install} the package, mailing lists with archives,
49 and several additional on-line resources like contributed topologies, etc.
50 %% The default installation action is simply to unpack the source code and
57 %% The configuration script should automatically determine the best options
58 %% for your platform, and it will tell you if anything is missing on
59 %% your system. You will also find detailed step-by-step installation
60 %% instructions on the website. There is a \normindex{cmake} based
61 %% installation route as well:
67 %% which is being tested in the wild since {\gromacs} version 4.5.
69 \section{Single or Double precision}
70 {\gromacs} can be compiled in either single\index{single
71 precision|see{precision, single}}\index{precision, single} or
72 \pawsindex{double}{precision}. It is very important to note here that
73 single precision is actually mixed precision. Using single precision
74 for all variables would lead to a significant reduction in accuracy.
75 Although in single precision all state vectors, i.e. particle coordinates,
76 velocities and forces, are stored in single precision, critical variables
77 are double precision. A typical example of the latter is the virial,
78 which is a sum over all forces in the system, which have varying signs.
79 In addition, in many parts of the code we managed to avoid double precision
80 for arithmetic, by paying attention to summation order or reorganization
81 of mathematical expressions. The default choice is single precision,
82 but it is easy to turn on double precision by adding the option
83 {\tt -DGMX_DOUBLE=on} to {\tt cmake}. Double precision
84 will be 20 to 100\% slower than single precision depending on the
85 architecture you are running on. Double precision will use somewhat
86 more memory and run input, energy and full-precision trajectory files
87 will be almost twice as large. SIMD (single-instruction multiple-data)
88 intrinsics non-bonded force and/or energy kernels are available for x86
89 hardware in single and double precision in different SSE and AVX flavors;
90 the minimum requirement is SSE2.
91 IBM Blue Gene Q intrinsics will be available soon. Some other parts of
92 the code, especially PME, also employ x86 SIMD intrinsics. All other
93 hardware will use optimized C kernels. The Verlet non-bonded scheme
94 uses SIMD non-bonded kernels that are C pre-processor macro driven,
95 therefore it is straightforward to implement SIMD acceleration
96 for new architectures; a guide is provided on {\wwwpage}.
98 The energies in single precision are accurate up to the last decimal,
99 the last one or two decimals of the forces are non-significant.
100 The virial is less accurate than the forces, since the virial is only one
101 order of magnitude larger than the size of each element in the sum over
102 all atoms (\secref{virial}).
103 In most cases this is not really a problem, since the fluctuations in the
104 virial can be two orders of magnitude larger than the average.
105 Using cut-offs for the Coulomb interactions cause large errors
106 in the energies, forces, and virial.
107 Even when using a reaction-field or lattice sum method, the errors
108 are larger than, or comparable to, the errors due to the single precision.
109 Since MD is chaotic, trajectories with very similar starting conditions will
110 diverge rapidly, the divergence is faster in single precision than in double
113 For most simulations single precision is accurate enough.
114 In some cases double precision is required to get reasonable results:
116 \item normal mode analysis,
117 for the conjugate gradient or l-bfgs minimization and the calculation and
118 diagonalization of the Hessian
119 \item long-term energy conservation, especially for large systems
122 \section{Porting {\gromacs}}
123 The {\gromacs} system is designed with portability as a major design
124 goal. However there are a number of things we assume to be present on
125 the system {\gromacs} is being ported on. We assume the following
129 \item A UNIX-like operating system (BSD 4.x or SYSTEM V rev.3 or higher)
130 or UNIX-like libraries running under {\eg} Cygwin
131 \item an ANSI C compiler
134 There are some additional features in the package that require extra
135 stuff to be present, but it is checked for in the configuration script
136 and you will be warned if anything important is missing.
138 That's the requirements for a single node system. If you want
139 to compile {\gromacs} for running a single simulation across multiple nodes,
140 you also need an MPI library (Message-Passing Interface) to perform the
141 parallel communication. This is always shipped with supercomputers, and
142 for workstations you can find links to free MPI implementations through
143 the {\gromacs} homepage at {\wwwpage}.
146 \section{Environment Variables}
147 {\gromacs} programs may be influenced by the use of
148 \normindex{environment variables}. First of all, the variables set in
149 the {\tt \normindex{GMXRC}} file are essential for running and
150 compiling {\gromacs}. Some other useful environment variables are
151 listed in the following sections. Most environment variables function
152 by being set in your shell to any non-NULL value. Specific
153 requirements are described below if other values need to be set. You
154 should consult the documentation for your shell for instructions on
155 how to set environment variables in the current shell, or in config
156 files for future shells. Note that requirements for exporting
157 environment variables to jobs run under batch control systems vary and
158 you should consult your local documentation for details.
164 \item {\tt GMX_CONSTRAINTVIR}: print constraint virial and force virial energy terms.
165 \item {\tt GMX_MAXBACKUP}: {\gromacs} automatically backs up old
166 copies of files when trying to write a new file of the same
167 name, and this variable controls the maximum number of
168 backups that will be made, default 99.
169 \item {\tt GMX_NO_QUOTES}: if this is explicitly set, no cool quotes
170 will be printed at the end of a program.
171 \item {\tt GMX_SUPPRESS_DUMP}: prevent dumping of step files during
172 (for example) blowing up during failure of constraint
174 \item {\tt GMX_TPI_DUMP}: dump all configurations to a {\tt .pdb}
175 file that have an interaction energy less than the value set
176 in this environment variable.
177 \item {\tt GMX_VIEW_XPM}: {\tt GMX_VIEW_XVG}, {\tt
178 GMX_VIEW_EPS} and {\tt GMX_VIEW_PDB}, commands used to
179 automatically view \@ {\tt .xvg}, {\tt .xpm}, {\tt .eps}
180 and {\tt .pdb} file types, respectively; they default to {\tt xv}, {\tt xmgrace},
181 {\tt ghostview} and {\tt rasmol}. Set to empty to disable
182 automatic viewing of a particular file type. The command will
183 be forked off and run in the background at the same priority
184 as the {\gromacs} tool (which might not be what you want).
185 Be careful not to use a command which blocks the terminal
186 ({\eg} {\tt vi}), since multiple instances might be run.
187 \item {\tt GMX_VIRIAL_TEMPERATURE}: print virial temperature energy term
188 \item {\tt LOG_BUFS}: the size of the buffer for file I/O. When set
189 to 0, all file I/O will be unbuffered and therefore very slow.
190 This can be handy for debugging purposes, because it ensures
191 that all files are always totally up-to-date.
192 \item {\tt LOGO}: set display color for logo in {\tt \normindex{ngmx}}.
193 \item {\tt LONGFORMAT}: use long float format when printing
195 \item {\tt GMX_COMPELDUMP}: Applies for computational electrophysiology setups
196 only (see section \ref{sec:compel}). The initial structure gets dumped to
197 {\tt .pdb} file, which allows to check whether multimeric channels have
198 the correct PBC representation.
206 \item {\tt DUMPNL}: dump neighbor list.
207 If set to a positive number the {\em entire}
208 neighbor list is printed in the log file (may be many megabytes).
209 Mainly for debugging purposes, but may also be handy for
210 porting to other platforms.
211 \item {\tt WHERE}: when set, print debugging info on line numbers.
213 % At this point, all environment variables should be described
214 %\item There are a number of extra environment variables like these
215 % that are used in debugging - check the code!
219 {\bf Performance and Run Control}
223 \item {\tt DISTGCT}: couple distances between two atoms when doing general coupling
224 theory processes. The format is a string containing two integers, separated by a space.
225 \item {\tt GALACTIC_DYNAMICS}: planetary simulations are made possible (just for fun) by setting
226 this environment variable, which allows setting {\tt epsilon_r = -1} in the {\tt .mdp}
227 file. Normally, {\tt epsilon_r} must be greater than zero to prevent a fatal error.
228 See {\wwwpage} for example input files for a planetary simulation.
229 \item {\tt GMX_ALLOW_CPT_MISMATCH}: when set, runs will not exit if the
230 ensemble set in the {\tt .tpr} file does not match that of the
232 \item {\tt GMX_CAPACITY}: the maximum capacity of charge groups per
233 processor when using particle decomposition.
234 \item {\tt GMX_CUDA_NB_EWALD_TWINCUT}: force the use of twin-range cutoff kernel even if {\tt rvdw} =
235 {\tt rcoulomb} after PP-PME load balancing. The switch to twin-range kernels is automated,
236 so this variable should be used only for benchmarking.
237 \item {\tt GMX_CUDA_NB_ANA_EWALD}: force the use of analytical Ewald kernels. Should be used only for benchmarking.
238 \item {\tt GMX_CUDA_NB_TAB_EWALD}: force the use of tabulated Ewald kernels. Should be used only for benchmarking.
239 \item {\tt GMX_CUDA_STREAMSYNC}: force the use of cudaStreamSynchronize on ECC-enabled GPUs, which leads
240 to performance loss due to a known CUDA driver bug present in API v5.0 NVIDIA drivers (pre-30x.xx).
241 Cannot be set simultaneously with {\tt GMX_NO_CUDA_STREAMSYNC}.
242 \item {\tt GMX_CYCLE_ALL}: times all code during runs. Incompatible with threads.
243 \item {\tt GMX_CYCLE_BARRIER}: calls MPI_Barrier before each cycle start/stop call.
244 \item {\tt GMX_DD_ORDER_ZYX}: build domain decomposition cells in the order
245 (z, y, x) rather than the default (x, y, z).
246 \item {\tt GMX_DETAILED_PERF_STATS}: when set, print slightly more detailed performance information
247 to the {\tt .log} file. The resulting output is the way performance summary is reported in versions
248 4.5.x and thus may be useful for anyone using scripts to parse {\tt .log} files or standard output.
249 \item {\tt GMX_DISABLE_CPU_ACCELERATION}: disables CPU architecture-specific SIMD-optimized (SSE2, SSE4, AVX, etc.)
250 non-bonded kernels thus forcing the use of plain C kernels.
251 \item {\tt GMX_DISABLE_CUDA_TIMING}: timing of asynchronously executed GPU operations can have a
252 non-negligible overhead with short step times. Disabling timing can improve performance in these cases.
253 \item {\tt GMX_DISABLE_GPU_DETECTION}: when set, disables GPU detection even if {\tt \normindex{mdrun}} was compiled
255 \item {\tt GMX_DISABLE_PINHT}: disable pinning of consecutive threads to physical cores when using
256 Intel hyperthreading. Controlled with {\tt \normindex{mdrun} -nopinht} and thus this environment
257 variable will likely be removed.
258 \item {\tt GMX_DISRE_ENSEMBLE_SIZE}: the number of systems for distance restraint ensemble
259 averaging. Takes an integer value.
260 \item {\tt GMX_EMULATE_GPU}: emulate GPU runs by using algorithmically equivalent CPU reference code instead of
261 GPU-accelerated functions. As the CPU code is slow, it is intended to be used only for debugging purposes.
262 The behavior is automatically triggered if non-bonded calculations are turned off using {\tt GMX_NO_NONBONDED}
263 case in which the non-bonded calculations will not be called, but the CPU-GPU transfer will also be skipped.
264 \item {\tt GMX_ENX_NO_FATAL}: disable exiting upon encountering a corrupted frame in an {\tt .edr}
265 file, allowing the use of all frames up until the corruption.
266 \item {\tt GMX_FORCE_UPDATE}: update forces when invoking {\tt \normindex{mdrun} -rerun}.
267 \item {\tt GMX_GPU_ID}: set in the same way as the {\tt \normindex{mdrun}} option {\tt -gpu_id}, {\tt GMX_GPU_ID}
268 allows the user to specify different GPU id-s, which can be useful for selecting different
269 devices on different compute nodes in a cluster. Cannot be used in conjunction with {\tt -gpu_id}.
270 \item {\tt GMX_IGNORE_FSYNC_FAILURE_ENV}: allow {\tt \normindex{mdrun}} to continue even if
272 \item {\tt GMX_LJCOMB_TOL}: when set to a floating-point value, overrides the default tolerance of
273 1e-5 for force-field floating-point parameters.
274 \item {\tt GMX_MAX_MPI_THREADS}: sets the maximum number of MPI-threads that {\tt \normindex{mdrun}}
276 \item {\tt GMX_MAXCONSTRWARN}: if set to -1, {\tt \normindex{mdrun}} will
277 not exit if it produces too many LINCS warnings.
278 \item {\tt GMX_NB_GENERIC}: use the generic C kernel. Should be set if using
279 the group-based cutoff scheme and also sets {\tt GMX_NO_SOLV_OPT} to be true,
280 thus disabling solvent optimizations as well.
281 \item {\tt GMX_NB_MIN_CI}: neighbor list balancing parameter used when running on GPU. Sets the
282 target minimum number pair-lists in order to improve multi-processor load-balance for better
283 performance with small simulation systems. Must be set to a positive integer, the default value
284 is optimized for NVIDIA Fermi and Kepler GPUs, therefore changing it is not necessary for
285 normal usage, but it can be useful on future architectures.
286 \item {\tt GMX_NBLISTCG}: use neighbor list and kernels based on charge groups.
287 \item {\tt GMX_NBNXN_CYCLE}: when set, print detailed neighbor search cycle counting.
288 \item {\tt GMX_NBNXN_EWALD_ANALYTICAL}: force the use of analytical Ewald non-bonded kernels,
289 mutually exclusive of {\tt GMX_NBNXN_EWALD_TABLE}.
290 \item {\tt GMX_NBNXN_EWALD_TABLE}: force the use of tabulated Ewald non-bonded kernels,
291 mutually exclusive of {\tt GMX_NBNXN_EWALD_ANALYTICAL}.
292 \item {\tt GMX_NBNXN_SIMD_2XNN}: force the use of 2x(N+N) SIMD CPU non-bonded kernels,
293 mutually exclusive of {\tt GMX_NBNXN_SIMD_4XN}.
294 \item {\tt GMX_NBNXN_SIMD_4XN}: force the use of 4xN SIMD CPU non-bonded kernels,
295 mutually exclusive of {\tt GMX_NBNXN_SIMD_2XNN}.
296 \item {\tt GMX_NO_ALLVSALL}: disables optimized all-vs-all kernels.
297 \item {\tt GMX_NO_CART_REORDER}: used in initializing domain decomposition communicators. Node reordering
298 is default, but can be switched off with this environment variable.
299 \item {\tt GMX_NO_CUDA_STREAMSYNC}: the opposite of {\tt GMX_CUDA_STREAMSYNC}. Disables the use of the
300 standard cudaStreamSynchronize-based GPU waiting to improve performance when using CUDA driver API
301 ealier than v5.0 with ECC-enabled GPUs.
302 \item {\tt GMX_NO_INT}, {\tt GMX_NO_TERM}, {\tt GMX_NO_USR1}: disable signal handlers for SIGINT,
303 SIGTERM, and SIGUSR1, respectively.
304 \item {\tt GMX_NO_NODECOMM}: do not use separate inter- and intra-node communicators.
305 \item {\tt GMX_NO_NONBONDED}: skip non-bonded calculations; can be used to estimate the possible
306 performance gain from adding a GPU accelerator to the current hardware setup -- assuming that this is
307 fast enough to complete the non-bonded calculations while the CPU does bonded force and PME computation.
308 \item {\tt GMX_NO_PULLVIR}: when set, do not add virial contribution to COM pull forces.
309 \item {\tt GMX_NOCHARGEGROUPS}: disables multi-atom charge groups, {\ie} each atom
310 in all non-solvent molecules is assigned its own charge group.
311 \item {\tt GMX_NOPREDICT}: shell positions are not predicted.
312 \item {\tt GMX_NO_SOLV_OPT}: turns off solvent optimizations; automatic if {\tt GMX_NB_GENERIC}
314 \item {\tt GMX_NSCELL_NCG}: the ideal number of charge groups per neighbor searching grid cell is hard-coded
315 to a value of 10. Setting this environment variable to any other integer value overrides this hard-coded
317 \item {\tt GMX_PME_NTHREADS}: set the number of OpenMP or PME threads (overrides the number guessed by
318 {\tt \normindex{mdrun}}.
319 \item {\tt GMX_PME_P3M}: use P3M-optimized influence function instead of smooth PME B-spline interpolation.
320 \item {\tt GMX_PME_THREAD_DIVISION}: PME thread division in the format ``x y z'' for all three dimensions. The
321 sum of the threads in each dimension must equal the total number of PME threads (set in
322 {\tt GMX_PME_NTHREADS}).
323 \item {\tt GMX_PMEONEDD}: if the number of domain decomposition cells is set to 1 for both x and y,
324 decompose PME in one dimension.
325 \item {\tt GMX_REQUIRE_SHELL_INIT}: require that shell positions are initiated.
326 \item {\tt GMX_REQUIRE_TABLES}: require the use of tabulated Coulombic
327 and van der Waals interactions.
328 \item {\tt GMX_SCSIGMA_MIN}: the minimum value for soft-core $\sigma$. {\bf Note} that this value is set
329 using the {\tt sc-sigma} keyword in the {\tt .mdp} file, but this environment variable can be used
330 to reproduce pre-4.5 behavior with respect to this parameter.
331 \item {\tt GMX_TPIC_MASSES}: should contain multiple masses used for test particle insertion into a cavity.
332 The center of mass of the last atoms is used for insertion into the cavity.
333 \item {\tt GMX_USE_GRAPH}: use graph for bonded interactions.
334 \item {\tt GMX_VERLET_BUFFER_RES}: resolution of buffer size in Verlet cutoff scheme. The default value is
335 0.001, but can be overridden with this environment variable.
336 \item {\tt GMX_VERLET_SCHEME}: convert from group-based to Verlet cutoff scheme, even if the {\tt cutoff_scheme} is
337 not set to use Verlet in the {\tt .mdp} file. It is unnecessary since the {\tt -testverlet} option of
338 {\tt \normindex{mdrun}} has the same functionality, but it is maintained for backwards compatibility.
339 \item {\tt GMXNPRI}: for SGI systems only. When set, gives the default non-degrading priority (npri)
340 for {\tt \normindex{mdrun}}, {\tt \normindex{g_covar}} and {\tt \normindex{g_nmeig}},
341 {\eg} setting {\tt setenv GMXNPRI 250} causes all runs to be performed at near-lowest priority by default.
342 \item {\tt GMXNPRIALL}: same as {\tt GMXNPRI}, but for all processes.
343 \item {\tt MPIRUN}: the {\tt mpirun} command used by {\tt \normindex{g_tune_pme}}.
344 \item {\tt MDRUN}: the {\tt \normindex{mdrun}} command used by {\tt \normindex{g_tune_pme}}.
345 \item {\tt GMX_NSTLIST}: sets the default value for {\tt nstlist}, preventing it from being tuned during
346 {\tt \normindex{mdrun}} startup when using the Verlet cutoff scheme.
347 \item {\tt GMX_USE_TREEREDUCE}: use tree reduction for nbnxn force reduction. Potentially faster for large number of
348 OpenMP threads (if memory locality is important).
352 {\bf Analysis and Core Functions}
356 \item {\tt ACC}: accuracy in Gaussian L510 (MC-SCF) component program.
357 \item {\tt BASENAME}: prefix of {\tt .tpr} files, used in Orca calculations
358 for input and output file names.
359 \item {\tt CPMCSCF}: when set to a nonzero value, Gaussian QM calculations will
360 iteratively solve the CP-MCSCF equations.
361 \item {\tt DEVEL_DIR}: location of modified links in Gaussian.
362 \item {\tt DSSP}: used by {\tt \normindex{do_dssp}} to point to the {\tt dssp}
363 executable (not just its path).
364 \item {\tt GAUSS_DIR}: directory where Gaussian is installed.
365 \item {\tt GAUSS_EXE}: name of the Gaussian executable.
366 \item {\tt GKRWIDTH}: spacing used by {\tt \normindex{g_dipoles}}.
367 \item {\tt GMX_MAXRESRENUM}: sets the maximum number of residues to be renumbered by
368 {\tt \normindex{grompp}}. A value of -1 indicates all residues should be renumbered.
369 \item {\tt GMX_FFRTP_TER_RENAME}: Some force fields (like AMBER) use specific names for N- and C-
370 terminal residues (NXXX and CXXX) as {\tt .rtp} entries that are normally renamed. Setting
371 this environment variable disables this renaming.
372 \item {\tt GMX_PATH_GZIP}: {\tt gunzip} executable, used by {\tt \normindex{g_wham}}.
373 \item {\tt GMXFONT}: name of X11 font used by {\tt \normindex{ngmx}}.
374 \item {\tt GMXTIMEUNIT}: the time unit used in output files, can be
375 anything in fs, ps, ns, us, ms, s, m or h.
376 \item {\tt MEM}: memory used for Gaussian QM calculation.
377 \item {\tt MULTIPROT}: name of the {\tt multiprot} executable, used by the
378 contributed program {\tt \normindex{do_multiprot}}.
379 \item {\tt NCPUS}: number of CPUs to be used for Gaussian QM calculation
380 \item {\tt OPENMM_PLUGIN_DIR}: the location of OpenMM plugins, needed for
381 {\tt \normindex{mdrun-gpu}}.
382 \item {\tt ORCA_PATH}: directory where Orca is installed.
383 \item {\tt SASTEP}: simulated annealing step size for Gaussian QM calculation.
384 \item {\tt STATE}: defines state for Gaussian surface hopping calculation.
385 \item {\tt TESTMC}: perform 1000 random swaps in Monte Carlo clustering method
386 within {\tt \normindex{g_cluster}}.
387 \item {\tt TOTAL}: name of the {\tt total} executable used by the contributed
388 {\tt \normindex{do_shift}} program.
389 \item {\tt VERBOSE}: make {\tt \normindex{g_energy}} and {\tt \normindex{eneconv}}
391 \item {\tt VMD_PLUGIN_PATH}: where to find VMD plug-ins. Needed to be
392 able to read file formats recognized only by a VMD plug-in.
393 \item {\tt VMDDIR}: base path of VMD installation.
394 \item {\tt XMGR}: sets viewer to {\tt xmgr} (deprecated) instead of {\tt xmgrace}.
398 \section{Running {\gromacs} in parallel}
399 By default {\gromacs} will be compiled with the built-in threaded MPI library.
400 This library supports MPI communication between threads instead of between
401 processes. To run {\gromacs} in parallel over multiple nodes in a cluster
402 of a supercomputer, you need to configure and compile {\gromacs} with an external
403 MPI library. All supercomputers are shipped with MPI libraries optimized for
404 that particular platform, and if you are using a cluster of workstations
405 there are several good free MPI implementations;
406 Open MPI is usually a good choice. Once you have an MPI library
407 installed it's trivial to compile {\gromacs} with MPI support: Just pass
408 the option {\tt -DGMX_MPI=on} to {\tt cmake} and (re-)compile. Please see
409 {\wwwpage} for more detailed instructions.
410 Note that in addition to MPI parallelization, {\gromacs} supports
411 thread-parallelization through \normindex{OpenMP}. MPI and OpenMP parallelization
412 can be combined, which results in, so called, hybrid parallelization.
413 See {\wwwpage} for details on use and performance of the parallelization
416 For communications over multiple nodes connected by a network,
417 there is a program usually called {\tt mpirun} with which you can start
418 the parallel processes. A typical command line could look like:
419 {\tt mpirun -np 10 mdrun_mpi -s topol -v}
421 With the implementation of threading available by default in {\gromacs} version 4.5,
422 if you have a single machine with multiple processors you don't have to
423 use the {\tt mpirun} command, or compile with MPI. Instead, you can
424 allow {\gromacs} to determine the number of threads automatically, or use the {\tt mdrun} option {\tt -nt}:
425 {\tt mdrun -nt 8 -s topol.tpr}
427 Check your local manuals (or online manual) for exact details
428 of your MPI implementation.
430 If you are interested in programming MPI yourself, you can find
431 manuals and reference literature on the internet.
434 \section{Running {\gromacs} on \normindex{GPUs}}
435 As of version 4.6, {\gromacs} has native GPU support through CUDA.
436 Note that {\gromacs} only off-loads the most compute intensive parts
437 to the GPU, currently the non-bonded interactions, and does all other
438 parts of the MD calculation on the CPU. The requirements for the CUDA code
439 are an Nvidia GPU with compute capability $\geq 2.0$, i.e. at
441 In many cases {\tt cmake} can auto-detect GPUs and the support will be
442 configured automatically. To be sure GPU support is configured, pass
443 the {\tt -DGMX_GPU=on} option to {\tt cmake}. The actual use of GPUs
444 is decided at run time by {\tt mdrun}, depending on the availability
445 of (suitable) GPUs and on the run input settings. A binary compiled
446 with GPU support can also run CPU only simulations. Use {\tt mdrun -nb cpu}
447 to force a simulation to run on CPUs only. Only simulations with the Verlet
448 cut-off scheme will run on a GPU. To test performance of old tpr files
449 with GPUs, you can use the {\tt -testverlet} option of {\tt mdrun},
450 but as this doesn't do the full parameter consistency check of {\tt grommp},
451 you should not use this option for production simulations.
452 Getting good performance with {\gromacs} on GPUs is easy,
453 but getting best performance can be difficult.
454 Please check {\wwwpage} for up to date information on GPU usage.
456 % LocalWords: Opteron Itanium PowerPC Altivec Athlon Fortran virial bfgs Nasm
457 % LocalWords: diagonalization Cygwin MPI Multi GMXHOME extern gmx tx pid buf
458 % LocalWords: bufsize txs rx rxs init nprocs fp msg GMXRC DUMPNL BUFS GMXNPRI
459 % LocalWords: unbuffered SGI npri mdrun covar nmeig setenv XPM XVG EPS
460 % LocalWords: PDB xvg xpm eps pdb xmgrace ghostview rasmol GMXTIMEUNIT fs dssp
461 % LocalWords: mpi distclean ing mpirun goofus doofus fred topol np
462 % LocalWords: internet gromacs DGMX cmake SIMD intrinsics AVX PME XN
463 % LocalWords: Verlet pre config CONSTRAINTVIR MAXBACKUP TPI ngmx mdp
464 % LocalWords: LONGFORMAT DISTGCT CPT tpr cpt CUDA EWALD TWINCUT rvdw
465 % LocalWords: rcoulomb STREAMSYNC cudaStreamSynchronized ECC GPUs sc
466 % LocalWords: ZYX PERF GPU PINHT hyperthreading DISRE NONBONDED ENX
467 % LocalWords: edr ENER gpu FSYNC ENV LJCOMB TOL MAXCONSTRWARN LINCS
468 % LocalWords: SOLV NBLISTCG NBNXN XNN ALLVSALL cudaStreamSynchronize
469 % LocalWords: USR SIGINT SIGTERM SIGUSR NODECOMM intra PULLVIR multi
470 % LocalWords: NOCHARGEGROUPS NOPREDICT NSCELL NCG NTHREADS OpenMP CP
471 % LocalWords: PMEONEDD Coulombic der Waals SCSIGMA TPIC GMXNPRIALL
472 % LocalWords: GOMP KMP pme NSTLIST ENVVAR nstlist startup OMP NUM ps
473 % LocalWords: ACC SCF BASENAME Orca CPMCSCF MCSCF DEVEL EXE GKRWIDTH
474 % LocalWords: MAXRESRENUM grompp FFRTP TER NXXX CXXX rtp GZIP gunzip
475 % LocalWords: GMXFONT ns MEM MULTIPROT multiprot NCPUS CPUs OPENMM
476 % LocalWords: PLUGIN OpenMM plugins SASTEP TESTMC eneconv VMD VMDDIR
477 % LocalWords: XMGR xmgr parallelization nt online Nvidia nb cpu
478 % LocalWords: testverlet grommp