JPK reader is slow for `.jpk-qi-data` files

[bicarlsen]

I tried to load a 500 MB .jpk-qi-data file. In Gwyddion it takes a few seconds, but with afmformats it took about two hours.

Here is a Google Drive link for the file.

[bicarlsen]

Here is a profile I took while loading the example file.

A major bottleneck seems to be jprops.load_properties in JPKReader._get_index_segment_properties.

[paulmueller]

Thanks for the issue. We are a bit low on developer hours, so this will probably take a long time to fix. If you have someone who could work on this issue, PRs are always very welcome!

[bicarlsen]

I created a reader in Rust with Python bindings. It loads the file I provided in about 60 seconds (compared to the 2 hours using the Python version). Let me know if it is something you feel like is worth integrating and I can issue a PR.

(I am waiting on an upsteam PR for the underlying ZIP library that would allow me to multithread it, too, taking the time down to only a few seconds.)

[paulmueller]

Are you planning to publish a Python package for your Rust reader? This would make the integration in afmformats easier to implement (dependency on other Python package instead of integrating a Rust build pipeline in CI).

It is definitely worth integrating ❤️, yes please create a PR.

[bicarlsen]

Great! Yes, I was planning to publish it as a package. I'll issue a PR then :)