newstyle Find_Python, numpy2, and xtensor 0.26

[hmaarrfk]

I'm just trying to see if a few more things can be simplified from the codebase

still very much a draft at this stage.

[hmaarrfk]

I just created a blank file here...

[hmaarrfk]

This parameter doesn't exist (anymore???)

[hmaarrfk]

Why do you specify /EHsc i'm not familiar with windows but can you remove this?

[constantinpape]

My windows knowledge is also limited... It seems like this is for suppressing some warnings during compilation. Do you think there is a problem in having it here?

[hmaarrfk]

@constantinpape It would be great to get this in so we can start to get numpy2 compatibility

[constantinpape]

Hi @hmaarrfk and thanks for looking into this!
It looks like we first need to update xtensor in the z5 upstream dependency: (see test error)

 In file included from /home/runner/micromamba/envs/nifty-build-env/include/z5/multiarray/xtensor_access.hxx:7,
                 from /home/runner/work/nifty/nifty/include/nifty/z5/z5.hxx:3,
                 from /home/runner/work/nifty/nifty/src/python/lib/z5/dataset_wrapper.cxx:5:
/home/runner/micromamba/envs/nifty-build-env/include/z5/multiarray/xtensor_util.hxx:3:10: fatal error: xtensor/xarray.hpp: No such file or directory
    3 | #include "xtensor/xarray.hpp"

I will see if I can take care of it now.

[hmaarrfk]

Do point me to your fix. Curious to learn!

[constantinpape]

The problem is/was that z5 was still pinned to an earlier xtensor version, using the previous imports. Luckily this was easy to fix, see conda-forge/z5py-feedstock#78 . So here, we just need to wait a bit till the new version is on conda-forge. Maybe should also pin the z5 version to >=2.0.20.

This should now also enable updating z5py to numpy 2 etc.

[constantinpape]

The tests pass now and the changes look good from my side.
Should I merge it @hmaarrfk or do you want to change anything else?
I would then also go ahead with a new release, so that we can get this on conda-forge.

[hmaarrfk]

I think this is great! Just wanted to help close this exploration we started a while back and see if we can remove your numpy 1 pin on micro-Sam!

Let's keep the merging and releasing momentum going ;)

[constantinpape]

Ah windows actually fails :( I will check it later.

[hmaarrfk]

I think cmake4 got more strict with path manipulation.

[hmaarrfk]

hmm, i think you and i found a similar "fix" nearly at the same time. i don't think your fix worked for me locally though.....

With my debug statements, I can see:

-- Python_SITELIB C:\Users\markh\miniforge3\envs\mcam_dev\Lib\site-packages
-- Python_SITELIB C:/Users/markh/miniforge3/envs/mcam_dev/Lib/site-packages

so it seems to me that Python_SITELIB is likely not normalized in the Find_Python package??? that would be the first time i see it...

[constantinpape]

hmm, i think you and i found a similar "fix" nearly at the same time. i don't think your fix worked for me locally though.....

Yes, my fix also didn't work in the CI.

so it seems to me that Python_SITELIB is likely not normalized in the Find_Python package??? that would be the first time i see it...

I see. Hopefully your last commit addresses this. Let's see :)

[hmaarrfk]

I see. Hopefully your last commit addresses this. Let's see :)

I was booted into windows up to 10 seconds ago. So I have high hopes.

I spent a bit of time cleaning up commit history. These build problems are annoying but I appreciate your help in unpinning dependencies!

Very excited for this!

[constantinpape]

Yes, it' fixed now! Thanks for all the efforts :) ! I will go ahead and merge this and create a new release.