vasp_gam [VASP 5.x and 6.x] with Intel got hung for large supercells
Posted: Mon Mar 20, 2023 8:32 pm
Hi All,
I have been having this for a long time, and am wondering if anyone else encounters the same issue.
1. The issue occurs with the vasp_gam version of VASP 5.x compiled with Intel Parallel Studio (e.g., 2020) and VASP 6.x (including 6.4.0) compiled with Intel oneAPI (including the latest version, 2023.0.0). I have tried many different versions/combinations.
2. Supercell calculations (e.g., 96-atom GaN supercell; either PBE or HSE) using vasp_gam got hung at the first electronic step:
running on 16 total cores
distrk: each k-point on 16 cores, 1 groups
distr: one band on 2 cores, 8 groups
using from now: INCAR
vasp.5.4.4.18Apr17-6-g9f103f2a35 (build Jun 18 2021 15:11:44) gamma-only
POSCAR found type information on POSCAR Ga N
POSCAR found : 2 types and 96 ions
scaLAPACK will be used
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ...
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
[... and stays forever here...]
running 8 mpi-ranks, with 4 threads/rank, on 1 nodes
distrk: each k-point on 8 cores, 1 groups
distr: one band on 1 cores, 8 groups
vasp.6.4.0 14Feb23 (build Mar 20 2023 14:38:27) gamma-only
POSCAR found type information on POSCAR GaN
POSCAR found : 2 types and 96 ions
Reading from existing POTCAR
scaLAPACK will be used
Reading from existing POTCAR
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ... GRIDC
FFT: planning ... GRID_SOFT
FFT: planning ... GRID
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
[... and stays forever here...]
3. The same vasp_gam executables work fine for small cell calculations (e.g., 2-atom SIC cells). So, the above issue appears to occur only for medium/large supercells.
4. The issue with vasp_gam does NOT occur when VASP is compiled with NVIDIA HPC SDK (although this has other performance issues); e.g.,
running 8 mpi-ranks, with 4 threads/rank, on 1 nodes
distrk: each k-point on 8 cores, 1 groups
distr: one band on 1 cores, 8 groups
vasp.6.4.0 14Feb23 (build Mar 18 2023 13:29:23) gamma-only
POSCAR found type information on POSCAR GaN
POSCAR found : 2 types and 96 ions
Reading from existing POTCAR
scaLAPACK will be used selectively (only on CPU)
Reading from existing POTCAR
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ... GRIDC
FFT: planning ... GRID_SOFT
FFT: planning ... GRID
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.250782203599E+04 0.25078E+04 -0.17229E+05 1192 0.101E+03
DAV: 2 -0.369381138270E+03 -0.28772E+04 -0.26693E+04 1480 0.255E+02
DAV: 3 -0.642432618445E+03 -0.27305E+03 -0.26881E+03 1504 0.799E+01
DAV: 4 -0.648335273151E+03 -0.59027E+01 -0.58373E+01 1504 0.123E+01
DAV: 5 -0.648495137422E+03 -0.15986E+00 -0.15845E+00 1528 0.181E+00 0.762E+01
DAV: 6 -0.558711986211E+03 0.89783E+02 -0.35601E+02 1440 0.275E+01 0.324E+01
DAV: 7 -0.573486239834E+03 -0.14774E+02 -0.29068E+01 1384 0.784E+00 0.187E+01
DAV: 8 -0.578620056558E+03 -0.51338E+01 -0.32284E+00 1416 0.291E+00 0.748E+00
DAV: 9 -0.579416938305E+03 -0.79688E+00 -0.11702E+00 1448 0.176E+00 0.202E+00
DAV: 10 -0.579488224920E+03 -0.71287E-01 -0.20028E-01 1472 0.604E-01 0.780E-01
DAV: 11 -0.579502899887E+03 -0.14675E-01 -0.16029E-02 1520 0.194E-01 0.381E-01
DAV: 12 -0.579506778534E+03 -0.38786E-02 -0.39974E-03 1360 0.120E-01 0.174E-01
DAV: 13 -0.579507633129E+03 -0.85459E-03 -0.71188E-04 1368 0.499E-02 0.371E-02
DAV: 14 -0.579507678529E+03 -0.45400E-04 -0.52162E-05 976 0.142E-02
1 F= -.57950768E+03 E0= -.57950768E+03 d E =-.973631E-13 mag= -0.0084
*
I appreciate any comments and/or suggestions to fix this issue.
Thank you.
I have been having this for a long time, and am wondering if anyone else encounters the same issue.
1. The issue occurs with the vasp_gam version of VASP 5.x compiled with Intel Parallel Studio (e.g., 2020) and VASP 6.x (including 6.4.0) compiled with Intel oneAPI (including the latest version, 2023.0.0). I have tried many different versions/combinations.
2. Supercell calculations (e.g., 96-atom GaN supercell; either PBE or HSE) using vasp_gam got hung at the first electronic step:
running on 16 total cores
distrk: each k-point on 16 cores, 1 groups
distr: one band on 2 cores, 8 groups
using from now: INCAR
vasp.5.4.4.18Apr17-6-g9f103f2a35 (build Jun 18 2021 15:11:44) gamma-only
POSCAR found type information on POSCAR Ga N
POSCAR found : 2 types and 96 ions
scaLAPACK will be used
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ...
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
[... and stays forever here...]
running 8 mpi-ranks, with 4 threads/rank, on 1 nodes
distrk: each k-point on 8 cores, 1 groups
distr: one band on 1 cores, 8 groups
vasp.6.4.0 14Feb23 (build Mar 20 2023 14:38:27) gamma-only
POSCAR found type information on POSCAR GaN
POSCAR found : 2 types and 96 ions
Reading from existing POTCAR
scaLAPACK will be used
Reading from existing POTCAR
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ... GRIDC
FFT: planning ... GRID_SOFT
FFT: planning ... GRID
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
[... and stays forever here...]
3. The same vasp_gam executables work fine for small cell calculations (e.g., 2-atom SIC cells). So, the above issue appears to occur only for medium/large supercells.
4. The issue with vasp_gam does NOT occur when VASP is compiled with NVIDIA HPC SDK (although this has other performance issues); e.g.,
running 8 mpi-ranks, with 4 threads/rank, on 1 nodes
distrk: each k-point on 8 cores, 1 groups
distr: one band on 1 cores, 8 groups
vasp.6.4.0 14Feb23 (build Mar 18 2023 13:29:23) gamma-only
POSCAR found type information on POSCAR GaN
POSCAR found : 2 types and 96 ions
Reading from existing POTCAR
scaLAPACK will be used selectively (only on CPU)
Reading from existing POTCAR
LDA part: xc-table for Pade appr. of Perdew
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ... GRIDC
FFT: planning ... GRID_SOFT
FFT: planning ... GRID
WAVECAR not read
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.250782203599E+04 0.25078E+04 -0.17229E+05 1192 0.101E+03
DAV: 2 -0.369381138270E+03 -0.28772E+04 -0.26693E+04 1480 0.255E+02
DAV: 3 -0.642432618445E+03 -0.27305E+03 -0.26881E+03 1504 0.799E+01
DAV: 4 -0.648335273151E+03 -0.59027E+01 -0.58373E+01 1504 0.123E+01
DAV: 5 -0.648495137422E+03 -0.15986E+00 -0.15845E+00 1528 0.181E+00 0.762E+01
DAV: 6 -0.558711986211E+03 0.89783E+02 -0.35601E+02 1440 0.275E+01 0.324E+01
DAV: 7 -0.573486239834E+03 -0.14774E+02 -0.29068E+01 1384 0.784E+00 0.187E+01
DAV: 8 -0.578620056558E+03 -0.51338E+01 -0.32284E+00 1416 0.291E+00 0.748E+00
DAV: 9 -0.579416938305E+03 -0.79688E+00 -0.11702E+00 1448 0.176E+00 0.202E+00
DAV: 10 -0.579488224920E+03 -0.71287E-01 -0.20028E-01 1472 0.604E-01 0.780E-01
DAV: 11 -0.579502899887E+03 -0.14675E-01 -0.16029E-02 1520 0.194E-01 0.381E-01
DAV: 12 -0.579506778534E+03 -0.38786E-02 -0.39974E-03 1360 0.120E-01 0.174E-01
DAV: 13 -0.579507633129E+03 -0.85459E-03 -0.71188E-04 1368 0.499E-02 0.371E-02
DAV: 14 -0.579507678529E+03 -0.45400E-04 -0.52162E-05 976 0.142E-02
1 F= -.57950768E+03 E0= -.57950768E+03 d E =-.973631E-13 mag= -0.0084
*
I appreciate any comments and/or suggestions to fix this issue.
Thank you.