Skip to content

[WIP] Speed up python calculator - #90

Open
bernstei wants to merge 2 commits into
mainfrom
optimize_python_calc
Open

[WIP] Speed up python calculator #90
bernstei wants to merge 2 commits into
mainfrom
optimize_python_calc

Conversation

@bernstei

@bernstei bernstei commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Avoid needlessly slow operations

  • replace list.index() calls for node_types and neigh_types with pure numpy array syntax
  • replace pair_forces np.asarray with faster np.fromiter(..., dtype=...) [maybe not faster?]

TODO

closes #92

…x() calls and replacing pair_forces asarray with fromiter
@bernstei

Copy link
Copy Markdown
Collaborator Author

@wcwitt do you think there's enough test coverage that if I were to have broken something and messed up the output numbers it would be detected?

@bernstei bernstei changed the title Speed up python calculator [WIP] Speed up python calculator Aug 21, 2026
@bernstei

bernstei commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

Main remaining places for plausible speedup are conversion of self.evaluator.node_forces list into an array pair_forces, and calculating neighbor lists less often.

The neighbor lists we can do with skin, but I'm a bit worried about the implementations linked to above: could whatever agent you used to find the issue with the lammps nn list write a good test? The logic that the D3 implementations are using are different enough from lammps that I'm not sure they're getting that right

I'm less sure about node_forces, and I don't really understand why converting 5e5 floats takes an appreciable amount of time anyway. Is it possible to pass that list back as an array directly, through some numpy/ctypes magic?

@bernstei

bernstei commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator Author

I think I've convinced myself that the neighbor list skin heuristics in https://github.com/vicvaleeva/FourierD3, at least with the fixes in vicvaleeva/FourierD3#9, are correct, and could probably be easily incorporated into Symmetrix

@bernstei

Copy link
Copy Markdown
Collaborator Author

I have a version of the neighbor list skin recalculation heuristic. It's not a dramatic savings, but it's not negligible. With my 2000 atom system, after the speedups in this branch the neighbor lists are about 1/2 the remaining time. The skin heuristics drop the actual neighbor list construction down by a factor of 10 or so, but unfortunately just using the stored info to construct the current neighbor_list output (ijdD) takes almost half the original calculation time, so the savings is about 1/4 of the total time (1/2 of the 1/2 that is neighbor lists). If you think that's worth it, I can clear it and add it to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

speeding up python calculator

1 participant