1 ==============================
2 Full installation instructions
3 ==============================
7 This document provides detailed documentation about building and installing
8 the gmxapi Python package.
10 GROMACS is a high performance computational science tool that is optimized for
11 a variety of specialized hardware and parallel computing environments.
12 To make the best use of a computing environment, GROMACS is usually built from
15 Users of Python based molecular science tools may have various requirements and
16 use a variety of Python distributions,
17 so gmxapi extension code is most useful when built from source code for a specific
18 GROMACS installation and Python environment.
20 This document has a lot of detail.
21 If you don't need a lot of reference material, you may just jump ahead to the :ref:`installation`.
23 Command line examples assume the `bash <https://www.gnu.org/software/bash/>`_ shell.
25 .. note:: Regarding multiple GROMACS installations:
26 Many GROMACS users switch between multiple GROMACS installations on the same
27 computer using an HPC module system and/or a GMXRC configuration script.
28 For the equivalent sort of environment switching with the ``gmxapi`` Python package,
29 we recommend installing it in a different
30 `Python virtual environment <https://www.google.com/search?q=python+virtual+environment>`_
31 for each GROMACS installation.
32 Once built, a particular copy of the ``gmxapi`` Python package always refers to the
33 same GROMACS installation.
35 .. contents:: Contents
41 The following documentation contains frequent references to the ``pip`` tool
42 for installing Python packages. In some cases, an unprivileged user should
43 use the ``--user`` command line flag to tell ``pip`` to install packages
44 into the user site-packages directory rather than the default site-packages
45 directory for the Python installation. This flag is not appropriate when
46 running ``pip`` in a virtual environment (as recommended) and is omitted in
47 this documentation. If you need the ``--user`` flag, you should modify the
48 example commands to look something like ``pip install --upgrade somepackage --user``
52 These instructions use the executable names ``python`` and ``pip``
53 instead of ``python3`` or ``pip3``. Some Python installations require the ``3``
54 suffix, but it is usually not necessary if you have already activated a Python
55 virtual environment (recommended).
61 ``gmxapi`` comes in three parts:
63 * GROMACS gmxapi library for C++.
64 * This Python package, supporting Python 3.5 and higher
65 * MD restraint plugins and sample gmxapi client code
67 Build system requirements
68 -------------------------
70 gmxapi can be built for Python 3.5 and higher.
72 You will need a C++ 14 compatible compiler and a reasonably up-to-date version
74 Full gmxapi functionality may also require an MPI compiler (e.g. ``mpicc``).
76 The Python package requires a GROMACS installation.
78 `GROMACS <http://manual.gromacs.org/documentation/current/install-guide/index.html>`_
79 before proceeding. Be sure to configure CMake with the ``GMXAPI=ON`` option.
81 Then, "source" the GMXRC file from the GROMACS installation as you normally would
82 before using GROMACS, or note its installation location so that you can pass it
83 to the build configuration.
87 If you are using a managed computing resource, such as a research HPC cluster,
88 GROMACS may already be installed, but you will need GROMACS 2020 or later, and
89 it must be configured with ``GMXAPI=ON``.
91 Important: To build a module that can be imported by Python, you need a Python
92 installation that includes the Python headers. Unfortunately, it is not always
93 obvious whether these headers are present or where to find them. The simplest
94 answer is to just try to build the Python package using these instructions, and
95 if gmxapi is unable to find the Python tools it needs, try a different Python
96 installation or install the additional development packages.
98 On a Linux system, this may require installing packages such as ``python-dev``
99 and/or ``python3-dev``.
100 If you are building Python, either from scratch or with a tool like
101 ``pyenv install`` (see
102 `wiki entry <https://github.com/pyenv/pyenv/wiki#how-to-build-cpython-with---enable-shared>`_
104 be sure to enable installation of the Python C library with the
105 ``--enable-shared`` flag.
106 Alternatively, various Python distributions provide a
107 sufficient build environment while only requiring installation into a user
108 home directory. (Some examples below.)
110 If you are using an HPC system with software available through modules you may
111 be able to just ``module load`` a different Python installation and find one
114 Python environment requirements
115 -------------------------------
117 gmxapi requires Python 3.5 or higher. Check your version with
118 ``python3 --version`` or ``python --version``.
122 The following documentation assumes you do not need to use a trailing '3' to
123 access a Python 3 interpreter on your system.
124 The default Python interpreter on your system may use ``python3`` and ``pip3``
125 instead of ``python`` and ``pip``. You can check the version with
126 ``python3 --version`` or ``python --version`` and ``pip --version``.
128 To build and install, you also need the packages ``cmake``,
129 ``setuptools``, ``networkx``, and ``scikit-build``.
131 For full functionality, you should also have ``mpi4py`` and ``numpy``.
132 These requirements and version numbers are listed in ``requirements.txt``.
134 The easiest way to make sure you have the requirements installed, first update
135 ``pip``, then use the ``requirements.txt`` file provided with the repository.
136 File paths in this section are relative to the root directory of your local copy
137 of the GROMACS source.
139 Confirm that ``pip`` is available, install ``pip`` if it is missing, or get
140 instructions on how to install ``pip``::
142 python -m ensurepip --default-pip
144 Install or upgrade required components::
146 python -m pip install --upgrade pip
147 pip install --upgrade setuptools
149 "requirements" files in GROMACS source tree
150 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
152 If you are building from source code in a local copy of the GROMACS source
153 repository, some helpful files allow you to preinstall the Python requirements
154 before installing the ``gmxapi`` package.
156 pip install -r python_packaging/src/requirements.txt
158 If building documentation or running tests,
159 ``pip install -r python_packaging/requirements-docs.txt`` or
160 ``pip install -r python_packaging/requirements-test.txt``,
161 respectively, or see below.
163 Documentation build requirements
164 --------------------------------
166 See :ref:`gmxapi_package_documentation`
168 .. _testing_requirements:
173 Testing is performed with `pytest <https://docs.pytest.org/en/latest/>`_.
174 Tests also require ``numpy``.
175 You can probably install both with ``pip``::
177 pip install pytest numpy
179 To test the full functionality also requires an MPI parallel environment.
180 You will need the ``mpi4py`` Python package and an MPI launcher
181 (such as ``mpiexec``, ``mpirun``, a launcher provided by your HPC queuing system,
182 or whatever is provided by your favorite MPI package for your operating system).
184 .. _mpi_requirements:
189 For the ensemble simulations features, you will need an MPI installation. On an HPC system, this means you will
190 probably have to use ``module load`` to load a compatible set of MPI tools and compilers. Check your HPC
191 documentation or try ``module avail`` to look for an ``openmpi``, ``mpich``, or ``mvapich`` module and matching compiler
192 module. This may be as simple as
198 Note that the compilers loaded might not be the first compilers discovered automatically by the build tools we will use
199 below, so you may have to specify compilers on the command line for consistency. It may be necessary to require that
200 GROMACS, gmxapi, and the sample code are built with the same compiler(s).
202 Note that strange errors have been known to occur when ``mpi4py`` is built with
203 different a different tool set than has been used to build Python and gmxapi.
204 If the default compilers on your system are not sufficient for GROMACS or gmxapi,
205 you may need to build, e.g., OpenMPI or MPICH, and/or build ``mpi4py`` with a
206 specific MPI compiler wrapper. This can complicate building in environments such
209 Set the MPICC environment variable to the MPI compiler wrapper and forcibly
210 reinstall ``mpi4py``.
213 export MPICC=`which mpicc`
214 pip install --no-cache-dir --upgrade --no-binary \":all:\" --force-reinstall mpi4py
216 Installing the Python package
217 =============================
219 We recommend you install the gmxapi package in a Python virtual environment
220 (``virtualenv`` or ``venv``). There are several ways to do this, and it is also
221 possible to install without a virtual environment. If installing without a
222 virtual environment as an un-privileged user, you may need to set the CMake
223 variable ``GMXAPI_USER_INSTALL`` (``-DGMXAPI_USER_INSTALL=ON`` on the ``cmake``
224 command line) and / or use the ``--user`` option with ``pip install``.
226 Sometimes the build environment can choose a different Python interpreter than
227 the one you intended.
228 You can set the ``PYTHON_EXECUTABLE`` CMake variable to explicitly choose the
229 Python interpreter for your chosen installation.
230 For example: ``-DPYTHON_EXECUTABLE=\`which python\```
234 Recommended installation
235 ------------------------
237 We recommend using Python's `pip <https://pip.pypa.io/en/stable/>`_
238 package installer to automatically download, build, and install the latest
239 version of the gmxapi package into a Python
240 `virtual environment <https://docs.python.org/3/tutorial/venv.html>`_.
241 gmxapi requires an existing GROMACS installation.
243 The instructions in this section assume that *pip* is able to download files
246 Locate or install GROMACS
247 ^^^^^^^^^^^^^^^^^^^^^^^^^
249 You need a GROMACS installation that includes the gmxapi headers and library.
250 If GROMACS 2020 or higher is already installed,
251 *and* was configured with ``GMXAPI=ON`` at build time,
252 you can just source the GMXRC
253 (so that the Python package knows where to find GROMACS)
254 and skip to the next section.
256 Otherwise, install a supported version of GROMACS.
257 When building GROMACS from source, be sure to configure cmake with the flag
258 ``-DGMXAPI=ON`` (default).
260 Set the environment variables for the GROMACS installation so that the gmxapi
261 headers and library can be found when building the Python package.
262 If you installed to a ``gromacs-gmxapi`` directory in your home directory as
263 above and you use the ``bash`` shell, do::
265 source $HOME/gromacs-gmxapi/bin/GMXRC
267 Set up a Python virtual environment
268 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
270 We recommend installing the Python package in a virtual environment.
271 If not installing in a virtual environment, you may not be able to install
272 necessary prerequisites (e.g. if you are not an administrator of the system you are on).
274 Create a Python 3 virtual environment.
276 For Python 3, use the ``venv`` module.
277 Depending on your computing environment, the Python 3 interpreter may be accessed
278 with the command ``python`` or ``python3``. Use ``python --version`` and
279 ``python3 --version`` to figure out which you need to use. The following assumes
280 the Python 3 interpreter is accessed with ``python3``.
284 After activating the venv, ``python`` and ``pip`` are sufficient. (The '3'
285 suffix will no longer be necessary and will be omitted in the rest of this
290 python3 -m venv $HOME/myvenv
292 Activate the virtual environment. Your shell prompt will probably be updated with the name of the environment you
293 created to make it more obvious.
297 $ source $HOME/myvenv/bin/activate
300 Activating the virtual environment changes your shell prompt to indicate the
301 environment is active. The prompt is omitted from the remainging examples, but
302 the remaining examples assume the virtualenv is still active.
303 (Don't do it now, but you can deactivate the environment by running ``deactivate``.)
308 It is always a good idea to update ``pip`` and ``setuptools`` before installing
309 new Python packages::
311 pip install --upgrade pip setuptools
313 The gmxapi installer requires a few additional packages. It is best to make sure
314 they are installed and up to date before proceeding.
318 pip install --upgrade cmake scikit-build
320 For MPI, we use mpi4py.
321 Make sure it is using the same MPI installation that we are building
322 GROMACS against and building with compatible compilers.
326 python -m pip install --upgrade pip setuptools
327 MPICC=`which mpicc` pip install --upgrade mpi4py
329 Install the latest version of gmxapi
330 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
332 Fetch and install the latest version of gmxapi from the Python Packaging Index::
336 If ``pip`` does not find your GROMACS installation, use one of the following
337 environment variables to provide a hint.
342 If you have a single GROMACS installation at ``/path/to/gromacs``, it is usually
343 sufficient to provide this location to ``pip`` through the ``gmxapi_DIR``
344 environment variable.
348 gmxapi_DIR=/path/to/gromacs pip install gmxapi
353 If you have multiple builds of GROMACS distinguished by suffixes
354 (e.g. *_d*, *_mpi*, etcetera), or if you need to provide extra hints to ``pip``
355 about the software tools that were used to build GROMACS, you can specify a
356 directory in which the installer can find a CMake "tool chain".
358 In the following example, ``${SUFFIX}`` is the suffix that distinguishes the
359 particular build of GROMACS you want to target (refer to GROMACS installation
360 instructions for more information.) ``${SUFFIX}`` may simply be empty, or ``''``.
364 GMXTOOLCHAINDIR=/path/to/gromacs/share/cmake/gromacs${SUFFIX} pip install gmxapi
369 You can also install the ``gmxapi`` Python package from within a local copy of
370 the GROMACS source repository. Assuming you have already obtained the GROMACS
371 source code and you are in the root directory of the source tree, you will find
372 the ``gmxapi`` Python package sources in the ``python_packaging/src`` directory.
376 cd python_packaging/src
377 pip install -r requirements.txt
383 If the required dependencies are already installed, you can do a quick installation
384 without internet access, either from the source directory or from a source archive.
386 For example, the last line of the previous example could be replaced with::
388 pip install --no-cache-dir --no-deps --no-index --no-build-isolation .
390 Refer to ``pip`` documentation for descriptions of these options.
392 If you have built or downloaded a source distribution archive, you can provide
393 the archive file to ``pip`` instead of the ``.`` argument::
395 pip install gmxapi-0.1.0.tar.gz
397 In this example, the archive file name is as was downloaded from
398 `PyPI <https://pypi.org/project/gmxapi/#history>`_ or as built locally,
399 according to the following instructions.
401 Building a source archive
402 -------------------------
404 A source archive for the gmxapi python package can be built from the GROMACS
405 source repository using Python ``setuptools`` and ``scikit-build``.
409 pip install --upgrade setuptools scikit-build
410 cd python_packaging/src
411 python setup.py sdist
413 This command will create a ``dist`` directory containing a source distribution
414 archive file. The file name has the form *gmxapi-<version>.<suffix>*, where
415 *<version>* is the version from the ``setup.py`` file, and *<suffix>* is
416 determined by the local environment or by additional arguments to ``setup.py``.
420 Python documentation for
421 `creating a source distribution
422 <https://docs.python.org/3/distutils/sourcedist.html#creating-a-source-distribution>`_
424 Package maintainers may update the the online respository by uploading a freshly
425 built ``sdist`` with ``python -m twine upload dist/*``
427 .. _gmxapi_package_documentation:
429 Accessing gmxapi documentation
430 ==============================
432 Documentation for the Python classes and functions in the gmx module can
433 be accessed in the usual ways, using ``pydoc`` from the command line or
434 ``help()`` in an interactive Python session.
436 The complete documentation (which you are currently reading)
437 can be browsed `online <http://manual.gromacs.org/current/gmxapi/>`__
438 or built from a copy of the GROMACS source repository.
440 Documentation is built from a combination of Python module documentation and
441 static content, and requires a local copy of the GROMACS source repository.
446 To build the full gmxapi documentation with GROMACS, configure GROMACS with
447 ``-DGMX_PYTHON_PACKAGE=ON`` and build the GROMACS documentation normally.
452 Developers wishing to build just the ``gmxapi`` Python package documentation
453 may do so from the ``python_packaging`` subdirectory in the GROMACS repo::
456 pip install -r requirements-docs.txt
457 sphinx-build -b html documentation docs
462 Alternatively, build the ``docs`` Docker image from ``python_packaging/docker/docs.dockerfile``.
466 Document sample_restraint package. Reference issue
467 `2893 <https://redmine.gromacs.org/issues/2893>`_ and change
468 `11483 <https://gerrit.gromacs.org/c/gromacs/+/11483>`_
473 Couldn't find the ``gmxapi`` support library?
474 If you don't want to "source" your ``GMXRC`` file, you
475 can tell the package where to find a gmxapi compatible GROMACS installation with
476 ``gmxapi_DIR``. E.g. ``gmxapi_DIR=/path/to/gromacs pip install .``
478 Before updating the ``gmxapi`` package it is generally a good idea to remove the
479 previous installation and to start with a fresh build directory. You should be
480 able to just ``pip uninstall gmxapi``.
482 Do you see something like the following?
486 CMake Error at gmx/core/CMakeLists.txt:45 (find_package):
487 Could not find a package configuration file provided by "gmxapi" with any
488 of the following names:
493 Add the installation prefix of "gmxapi" to CMAKE_PREFIX_PATH or set
494 "gmxapi_DIR" to a directory containing one of the above files. If "gmxapi"
495 provides a separate development package or SDK, be sure it has been
498 This could be because
500 * GROMACS is not already installed
501 * GROMACS was built without the CMake variable ``GMXAPI=ON``
502 * or if ``gmxapi_DIR`` (or ``GROMACS_DIR``) is not a path containing directories
503 like ``bin`` and ``share``.
505 If you are not a system administrator you are encouraged to install in a Python
506 virtual environment, created with virtualenv or Conda.
507 Otherwise, you will need to specify the ``--user`` flag to ``pip``.
509 Two of the easiest problems to run into are incompatible compilers and
510 incompatible Python. Try to make sure that you use the same C and C++
511 compilers for GROMACS, for the Python package, and for the sample
512 plugin. These compilers should also correspond to the ``mpicc`` compiler
513 wrapper used to compile ``mpi4py``. In order to build the Python
514 package, you will need the Python headers or development installation,
515 which might not already be installed on the machine you are using. (If
516 not, then you will get an error about missing ``Python.h`` at some
517 point.) If you have multiple Python installations (or modules available
518 on an HPC system), you could try one of the other Python installations,
519 or you or a system administrator could install an appropriate Python dev
520 package. Alternatively, you might try installing your own Anaconda or
521 MiniConda in your home directory.
523 If an attempted installation fails with CMake errors about missing
524 “gmxapi”, make sure that Gromacs is installed and can be found during
525 installation. For instance,
529 gmxapi_DIR=/Users/eric/gromacs python setup.py install --verbose
531 Pip and related Python package management tools can be a little too
532 flexible and ambiguous sometimes. If things get really messed up, try
533 explicitly uninstalling the ``gmxapi`` module and its dependencies, then do
534 it again and repeat until ``pip`` can no longer find any version of any
543 Successfully running the test suite is not essential to having a working
544 ``gmxapi`` package. We are working to make the testing more robust, but
545 right now the test suite is a bit delicate and may not work right, even
546 though you have a successfully built ``gmxapi`` package. If you want to
547 troubleshoot, though, the main problems seem to be that automatic
548 installation of required python packages may not work (requiring manual
549 installations, such as with ``pip install somepackage``) and ambiguities
550 between python versions.
552 If you are working in a development branch of the repository, note that
553 the upstream branch may be reset to ``master`` after a new release is
554 tagged. In general, but particularly on the ``devel`` branch, when you
555 do a ``git pull``, you should use the ``--rebase`` flag.
557 If you fetch this repository and then see a git status like this::
561 Your branch and 'origin/devel' have diverged,
562 and have 31 and 29 different commits each, respectively.
564 then ``gmxapi`` has probably entered a new development cycle. You can
565 do ``git pull --rebase`` to update to the latest development branch.
567 If you do a ``git pull`` while in ``devel`` and get a bunch of unexpected
568 merge conflicts, do ``git merge --abort; git pull --rebase`` and you should
571 If you are developing code for gmxapi, this should be an indication to
572 rebase your feature branches for the new development cycle.