My Community

Posted: **Thu Jan 06, 2011 8:04 am**

Hi, all. Here I try to explain my problem clearly.
I get two nodes, each nodes have 12 cores, and the two nodes have been connected by a Giga Switch.
Now, I test the vasp examples on nodes.
First, I boot the mpd on the single node, and all the examples run well on each of the single nodes.
then I try to run the examples on the both nodes using 24 cores, after booting the mpd on the two nodes, I got the following error.

Code: Select all

FatalÂ errorÂ inÂ MPI_Waitall:Â OtherÂ MPIÂ error,Â errorÂ stack:
MPI_Waitall(261)..................:Â MPI_Waitall(count=46,Â req_array=0x7fffeeca46a0,Â status_array=0x7fffeeca4760)Â failed
MPIDI_CH3I_Progress(150)..........:Â 
MPID_nem_mpich2_blocking_recv(948):Â 
MPID_nem_tcp_connpoll(1709).......:Â CommunicationÂ error
rankÂ 23Â inÂ jobÂ 1Â Â node0_55860Â Â Â causedÂ collectiveÂ abortÂ ofÂ allÂ ranks
Â Â exitÂ statusÂ ofÂ rankÂ 23:Â killedÂ byÂ signalÂ 9Â

As I am thinking, whether this error is from the low speed of the switch or the limitation of memory, althrough I have set the stack and memeory to unlimited .
Thanks for your attention.

Posted: **Mon Jan 31, 2011 12:58 pm**

sorry but this error is not related to vasp itself, but to an error in your MPI

Posted: **Sun Feb 06, 2011 6:54 am**

Hi, Sir, yes, today I find where the problem is.
As I mentioned, I have two nodes, and if the job run on one nodes, there should be no problem. However, if the job was assigned to run on both of the two nodes, no matter how many cores I use, so long as there is data exchange between the two nodes, the problem will come, and most important is that it depends on the size of the super cell.
let's take the handson1 as an example(1_1_O_atom), the original POSCAR is as follows:

Code: Select all

OÂ atomÂ inÂ aÂ box
Â 1.0Â Â Â Â Â Â Â Â Â Â !Â universalÂ scalingÂ parameters
Â 8.0Â 0.0Â 0.0Â Â !Â latticeÂ vectorÂ Â a(1)
Â 0.0Â 8.0Â 0.0Â Â !Â latticeÂ vectorÂ Â a(2)
Â 0.0Â 0.0Â 8.0Â Â !Â latticeÂ vectorÂ Â a(3)
1Â Â Â Â Â Â Â Â Â Â Â Â Â !Â numberÂ ofÂ atoms
cartÂ Â Â Â Â Â Â Â Â Â !Â positionsÂ inÂ cartesianÂ coordinates
Â 0Â 0Â 0

and this job can not run if two nodes are used at the same tome, however, if I change the size of the super cell to 4X4X4, means the new POSCAR will as follows:

Code: Select all

OÂ atomÂ inÂ aÂ box
Â [color=red]0.5Â Â Â Â Â Â Â Â Â Â !Â universalÂ scalingÂ parameters[/color]Â 8.0Â 0.0Â 0.0Â Â !Â latticeÂ vectorÂ Â a(1)
Â 0.0Â 8.0Â 0.0Â Â !Â latticeÂ vectorÂ Â a(2)
Â 0.0Â 0.0Â 8.0Â Â !Â latticeÂ vectorÂ Â a(3)
1Â Â Â Â Â Â Â Â Â Â Â Â Â !Â numberÂ ofÂ atoms
cartÂ Â Â Â Â Â Â Â Â Â !Â positionsÂ inÂ cartesianÂ coordinates
Â 0Â 0Â 0

now, it run very well.
so now I am thinking this should be my compiling problem, maybe from the FFT lib or the hard ware limitation, such as the limit of the CPU stack or memory.
hope you can give me soem suggestions.

Code: Select all

HPÂ BL460c:
CPUÂ 2Â X5670Â forÂ eachÂ node
MemoryÂ 24GÂ forÂ eachÂ node
NICÂ 2x10G
usingÂ BewulfÂ structure

Thanks a lot..

My Community

VASP run error under mpich2

VASP run error under mpich2

VASP run error under mpich2

VASP run error under mpich2