Hello VASP Community,
I am currently working on compiling VASP using OpenACC with the intention of leveraging GPU acceleration, and I have some questions regarding the compatibility of the compiled binaries across different GPU models.
1. Specific GPU Targeting: When compiling VASP, if a specific GPU model is targeted (e.g., nvc -acc -ta=tesla:cc70 for NVIDIA's Volta), will the resulting binary only deliver optimal performance on that specified model? Additionally, are there risks of compatibility issues if the same binary is used on a different GPU architecture (e.g., from Volta to Turing or Kepler)?
2. Generalization Approach: Is there a recommended strategy to compile a more generally applicable VASP binary that could perform reasonably well across different NVIDIA GPU architectures? Would targeting a lower compute capability during compilation (e.g., nvc -acc -ta=tesla:cc35 for Kepler or nvc -acc -ta=tesla:cc50 for Maxwell ) ensure broader compatibility, or would this significantly compromise the performance on newer architectures?
3. Multiple Binaries: Would it be advisable to compile multiple versions of VASP, each optimized for a specific GPU architecture, and then select the appropriate version based on the hardware available for execution? What are the best practices for managing such an environment efficiently?
I am looking to balance performance gains with broad usability, and any recommendations or insights based on your experiences would be greatly appreciated. Thank you in advance for your assistance in navigating these complexities.
Best regards,
Zhao
Optimizing VASP Compilation for Multi-GPU Models Compatibility Using OpenACC.
Moderators: Moderator, Global Moderator
-
- Full Member
- Posts: 181
- Joined: Tue Oct 13, 2020 11:32 pm
-
- Global Moderator
- Posts: 89
- Joined: Tue Oct 17, 2023 10:17 am
Re: Optimizing VASP Compilation for Multi-GPU Models Compatibility Using OpenACC.
Dear Zhao,
in our provided makefile.include.nvhpc_*acc we specify several compute capabilities (cc). E.g. for the Fortran compiler:
This means that the compiler will generate code that is optimized for all cards with compute capability between 6.0 and 8.0. You can check compute capabilities in the respective NVIDIA table. E.g. Tespa P100 : 6.0, Tesla V100: 7.0 , or Tesla A100: 8.0. If you already have access to a H100, you should add
to the list.
Thus, you should compile one version of VASP, which encompasses all compute capabilities of available GPUs. At runtime, the optimal instructions for the specific GPU will be selected. Multiple binaries are not needed.
Note that you might get a compiler error if you specify some compute capabilities with minor versions other than zero. Look at this thread about a not recognized M60 for example. Just drop down to e.g. 5.0 from 5.2 for example.
Let me know if this fully answers your question, so I can lock the topic.
Cheers, Michael
in our provided makefile.include.nvhpc_*acc we specify several compute capabilities (cc). E.g. for the Fortran compiler:
Code: Select all
FC = mpif90 -acc -gpu=cc60,cc70,cc80,cuda11.8 -mp
Code: Select all
cc90
Thus, you should compile one version of VASP, which encompasses all compute capabilities of available GPUs. At runtime, the optimal instructions for the specific GPU will be selected. Multiple binaries are not needed.
Note that you might get a compiler error if you specify some compute capabilities with minor versions other than zero. Look at this thread about a not recognized M60 for example. Just drop down to e.g. 5.0 from 5.2 for example.
Let me know if this fully answers your question, so I can lock the topic.
Cheers, Michael
-
- Full Member
- Posts: 181
- Joined: Tue Oct 13, 2020 11:32 pm
Re: Optimizing VASP Compilation for Multi-GPU Models Compatibility Using OpenACC.
Dear Michael,
Got it. Thank you very much for your valuable comments and explanations.
Regards,
Zhao
Got it. Thank you very much for your valuable comments and explanations.
Regards,
Zhao