Add ExtraDataFunctor for integration with pasha

[philsmt]

As it came up again in European-XFEL/pasha#14, here's a proof-of-concept on how to move the ExtraDataFunctor implementation to EXtra-data itself. This would allow it to take advantage of private APIs as well as test properly (TBD). So far it's almost exactly the same implementation only with unnecessary import checks removed.

As the code is only loaded conditionally, it does not add an actual dependency. The import of gen_split_slices could even be removed once SourceData has its own .split_trains() method.

Any ideas for the module name?

[takluyver]

Thanks. I think pasha_functor is fine for the module name - it shouldn't normally be visible in any case.

As we were just discussing, .split_trains() doesn't fully work for this, because the objects it produces don't know the index of their trains in the original object. I think it might be a useful enhancement for pasha to define a default splitting based on len(functor), so we don't need gen_split_slices everywhere. But we do also have basically the same code in read_machinery (as split_trains), so it's easy enough for this purpose.

[takluyver]

Do you want to add a test for the pasha integration, BTW?

[philsmt]

Yes. How would the tests work dependency-wise? Do we add pasha as an optional feature, or not all?

I do not much like the hack for iterating SourceData objects anymore. I've looked into supporting it properly in the class natively, but I think the proper solution for this is to move various find-data-and-chunks methods from DataCollectiontoSourceData, in particular since we now have SourceData` objects anyway at all times. So let's move this to a later release, and I will remove the support here.

[philsmt]

Do we still need this at all with the dependencies we require?

[takluyver]

Probably not really.

It's a bit tricky to be definitive because the change is in HDF5, and h5py can be built with a wide range of HDF5 versions. EXtra-data supports h5py 2.10, and the pre-built packages of 2.10 on PyPI have HDF5 1.10.4. So I might bump the required version to >= 3.0, just to be a bit cautious. I hope very few people are still stuck on h5py 2.x.

[takluyver]

I would add pasha to the tests extra in setup.py (which already includes things like cloudpickle and dask).