From: Szilárd Páll Date: Thu, 19 Dec 2019 19:44:35 +0000 (+0100) Subject: Enable buffer ops when GMX_GPU_DD_COMMS is set X-Git-Url: http://biod.pnpi.spb.ru/gitweb/?a=commitdiff_plain;h=06bbf801ce202aaf94f381a9515cbcd621f625ea;p=alexxy%2Fgromacs.git Enable buffer ops when GMX_GPU_DD_COMMS is set Buffer ops is always a hard dependency for DD with GPU direct comm, so enable it instead of requiring an extra env var to be set on mdrun invocation. Change-Id: Iea354f8ce636a115e0ed866228746e91f34df228 --- diff --git a/docs/release-notes/2020/major/performance.rst b/docs/release-notes/2020/major/performance.rst index 03f7c4714f..a9ab8d75e2 100644 --- a/docs/release-notes/2020/major/performance.rst +++ b/docs/release-notes/2020/major/performance.rst @@ -19,14 +19,14 @@ the PME-mesh calculation now always takes the most CPU time. Proper dihedrals of Fourier type and improper dihedrals of preriodic type are SIMD accelerated """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Avoid configuring the own-FFTW with AVX512 enabled when GROMACS does not use AVX512 -""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +Avoid configuring the own-FFTW with AVX512 enabled when |Gromacs| does not use AVX512 +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -Previously if GROMACS was configured to use any AVX flavor, the internally built FFTW +Previously if |Gromacs| was configured to use any AVX flavor, the internally built FFTW would be configured to also contain AVX512 kernels. This could cause performance loss if the (often noisy) FFTW auto-tuner picks an AVX512 kernel in a run that otherwise only uses AVX/AVX2 which could run at higher CPU clocks without AVX512 clock speed limitation. -Now AVX512 is only used for the internal FFTW if GROMACS is also configured with +Now AVX512 is only used for the internal FFTW if |Gromacs| is also configured with the same SIMD flavor. Update and constraints can run on a GPU @@ -45,18 +45,16 @@ When running on multiple GPUs with CUDA, communication operations can now be performed directly between GPU memory spaces (automatically routed, including via NVLink where available). This behaviour is not yet enabled by default: the new codepaths have been verified by the -standard Gromacs regression tests, but (at the time of release) still +standard |Gromacs| regression tests, but (at the time of release) still lack substantial "real-world" testing. They can be enabled by setting the following environment variables to any non-NULL value in your shell: GMX_GPU_DD_COMMS (for halo exchange communications between PP tasks); GMX_GPU_PME_PP_COMMS (for communications between PME and PP -tasks); and GMX_USE_GPU_BUFFER_OPS (for GPU enablement of internal -data format transformation operations, required by the GPU direct -communications). GMX_FORCE_UPDATE_DEFAULT_GPU should also be set in +tasks); GMX_FORCE_UPDATE_DEFAULT_GPU can also be set in order to combine with the new GPU update feature (above). The combination of these will (for many common simulations) keep data resident on the GPU across most timesteps, avoiding expensive data -transfers. Note that these currently require Gromacs to be built +transfers. Note that these currently require |Gromacs| to be built with its internal thread-MPI library rather than any external MPI library, and are limited to a single compute node. We stress that users should carefully verify results against the default path, and diff --git a/src/gromacs/mdrun/runner.cpp b/src/gromacs/mdrun/runner.cpp index 1b699c4a4d..3909523e14 100644 --- a/src/gromacs/mdrun/runner.cpp +++ b/src/gromacs/mdrun/runner.cpp @@ -224,7 +224,7 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md GMX_LOG(mdlog.warning) .asParagraph() .appendTextFormatted( - "NOTE: This run uses the 'GPU buffer ops' feature, enabled by the " + "This run uses the 'GPU buffer ops' feature, enabled by the " "GMX_USE_GPU_BUFFER_OPS environment variable."); } @@ -233,7 +233,7 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md GMX_LOG(mdlog.warning) .asParagraph() .appendTextFormatted( - "NOTE: This run will default to '-update gpu' as requested by the " + "This run will default to '-update gpu' as requested by the " "GMX_FORCE_UPDATE_DEFAULT_GPU environment variable."); } @@ -243,14 +243,17 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md { if (!devFlags.enableGpuBufferOps) { - gmx_fatal(FARGS, - "Cannot enable GPU halo exchange without GPU buffer operations, set " - "GMX_USE_GPU_BUFFER_OPS=1\n"); + GMX_LOG(mdlog.warning) + .asParagraph() + .appendTextFormatted( + "Enabling GPU buffer operations required by GMX_GPU_DD_COMMS " + "(equivalent with GMX_USE_GPU_BUFFER_OPS=1)."); + devFlags.enableGpuBufferOps = true; } GMX_LOG(mdlog.warning) .asParagraph() .appendTextFormatted( - "NOTE: This run uses the 'GPU halo exchange' feature, enabled by the " + "This run uses the 'GPU halo exchange' feature, enabled by the " "GMX_GPU_DD_COMMS environment variable."); } else @@ -258,7 +261,7 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md GMX_LOG(mdlog.warning) .asParagraph() .appendTextFormatted( - "NOTE: GMX_GPU_DD_COMMS environment variable detected, but the 'GPU " + "GMX_GPU_DD_COMMS environment variable detected, but the 'GPU " "halo exchange' feature will not be enabled as nonbonded interactions " "are not offloaded."); devFlags.enableGpuHaloExchange = false; @@ -272,7 +275,7 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md GMX_LOG(mdlog.warning) .asParagraph() .appendTextFormatted( - "NOTE: This run uses the 'GPU PME-PP communications' feature, enabled " + "This run uses the 'GPU PME-PP communications' feature, enabled " "by the GMX_GPU_PME_PP_COMMS environment variable."); } else @@ -291,7 +294,7 @@ static DevelopmentFeatureFlags manageDevelopmentFeatures(const gmx::MDLogger& md GMX_LOG(mdlog.warning) .asParagraph() .appendText( - "NOTE: GMX_GPU_PME_PP_COMMS environment variable detected, but the " + "GMX_GPU_PME_PP_COMMS environment variable detected, but the " "'GPU PME-PP communications' feature was not enabled as " + clarification); devFlags.enableGpuPmePPComm = false;