NaN in Machine Learning of liquid surfacs

[julien_steffen]

Hello,

I am running simulations of liquid metal surfaces (e.g., Gallium with other metals solved in it) and generating Machine Learning Force Fields for those systems as well. In most cases, this works well (both training and application); sometimes, however, a single atom detaches from the surface to the gas phase during the training. When the distance of that atom is (presumably) larger than the given cutoff values, the next energy and gradient of the system is calculated to be NaN and the calculation holds (but the process is not aborted until the walltime is exceeded).
I assume that this error occurs, because the environment of the cannot be defined and referenced?
Since the OUTCAR and vasp.out (stdout) files are too large to add them fully (almost 1 GB for the OUTCAR), I have cut a large part of the intermediate lines, leaving the start and the sections before the ending.

Best wishes,
Julien

[jonathan_lahnsteiner2]

Dear Julien,

Could you please also send me the OSZICAR and the ML_LOGFILE. This will make it
much easier to figure out the problem.

All the best
Jonathan

[julien_steffen]

Dear Jonathan,
I have now added the missing files to the zip-archive. Thank you in advance!
Best wishes,
Julien

[jonathan_lahnsteiner2]

Dear Julien,

I was looking at your data. You are completely right that there
is an issue when the distance between the atoms is larger than the cutoff.
To debug this issue it would be very helpful if you could also
supply us with the ML_ABN file of this simulation.

All the best

Jonathan

[julien_steffen]

Dear Jonathan,
I have now attached the ML_ABN file.
Another observation was that in most cases when a single atom left the surface, a "cluster" of two to three nearly merged atoms (distances less than 0.3 Angstrom) had been built near the surface. I don't know if one of the observations is a "reason" for the other, but at least there seems to be some correlation.
Best wishes,
Julien

[ferenc_karsai]

Thank you very much for reporting the problem.

There was a bug in the code, so that normalizations are calculated, even when there are no neighbors. This leads to division by zero, which results in some NaNs in some arrays.

I have fixed it now.
I can't post the fix here, because it's too much, but it will be in the next release.

Thanks again for posting the issue.