Cube transpose behaviour with multidimensional coordinates

[cpelley]

The context of this ticket is in utilising the capability from iris.experimental.stratify which necessarily requires us to transpose cubes to make them suitable for use.

The current implementation of Cube.transpose changes the dimension mapping of the axillary coordinates, but doesn't actually transpose the coordinates themselves. This is so confusing! (see the following gist).

Note that the altitude coordinate likely behaves in the way that I was expecting, unlike the surface altitude as it is a derived coordinate... which only adds to the confusion with the differing shapes...

I would appreciate a second opinion on this, cheers.
I would consider this a bug on the basis that I was not expecting this at all and I'm interested in hearing what others think.

Related: #2532

[cpelley]

Additionally, the above would then effect aux coord equality.

[ajdawson]

I've encountered this before and it caught me by surprise. I'd call it a bug.

[cpelley]

Thanks for your input @ajdawson. I'm glad I'm not the only one :)

[cpelley]

Looking at the recent iris.experimental.stratify implementation, it looks as though it assumed that dimension mapping for aux coords is ordered increasing. I wonder how much other iris code also assumes this to be the case...

[rcomer]

Possibly worth noting that iris.util.as_compatible_shape treats any transposed auxcoords the same way as cube.transpose (#2030). So if it's a bug, then it's a bug in at least two places!

[cpelley]

Thanks @rcomer, I didn't realise. Agreed, we should ensure the two remain consistent in their whatever we do :)

Perhaps the approach to resolving this is to list the options available to us and then can consider ranking them in order of least to greatest confusion/usefulness.

Here are the options as I see them:

1. Leave the current behaviour where aux coords associated dims are re-arranged when the cube is transposed.
2. Transpose aux coords like we do with the cube.
3. Transpose aux coords when transposing cube but ensure that this is done in such a way that the ordered mappings are ordered increasing (i.e. [0, 3, 2] becomes [0, 2, 3]).

I hope I have explained these options clearly enough :)

[cpelley]

After spending quite some time on this, I think on transposing the cube at least, it might be most useful to take the approach of pnt3. The reason being, is that the mappings of the coordinate might not originally be ordered increasing and so simply transposing this won't make things any clearer. By enforcing increasing order of the coordinate dimension mappings when requesting a transpose, we have a consistent and obvious behaviour. However I'm agnostic as to whether this makes sense in the context of as_compatible_shape as here it relates between another cube.

[cpelley]

I have put up a PR against my own fork to demonstrate what the changes might look like, so we can explore this.

[rcomer]

I think that intuitively you'd expect auxcoord dims to be increasing, so option 3 sounds good to me. It's never been a problem as such, but was quite confusing the first time I encountered it with cube.transpose.

As for as_compatible_shape, I've never used it directly (when it's called by iris.analysis.maths._binary_op_common, only the cube data is kept anyway).

[cpelley]

Thanks @rcomer. Something that just occurred to me. Perhaps this is a behavioural change worth considering across the board i.e. when attaching a coordinate to a cube as well as these situation where we are reordering a cube.

[rcomer]

I'd worry that it might be difficult to find and pin down every possible way for the auxcoords to end up with "wrong" dimension order. Maybe they are that way when you first load the cube from a file.

An alternative might be to write some sort of sanitize_auxcoords function, which could be applied to make sure they're in the expected order when it matters (e.g. the stratify function).

But we're getting into code design now, which is not my area of expertise...!

[cpelley]

Maybe they are that way when you first load the cube from a file.

Loading a cube from disk would mean the format in question going through cube construction though :)

+1 on a helper function for at least the short term though (this is what I have locally now). Great name for it too!

[cpelley]

I have just added a sanitise_auxcoords function to illustrate in my branch here. cube.transpose can call the sanitise function if we do go that route.

[pelson]

To be clear, the discussion here is about whether a cube should allow coordinates to be added where the dimensions aren't ordered, right?

NetCDF has no such constraint, but I'm uncertain about the CF conventions. Personally, I'm OK with having the order of the dimensions be non monotonic, but can see that there are some situations where that does make some code harder to implement. I'm partially tempted to suggest a normalisation function which allows non monotonically increasing coordinate dims to be made monotonic (and thus transposing the coordinates as necessary).

@cpelley - at this point, I'm looking for a good reason to justify the monotonic dimensions constraint, otherwise I would not be in favour of the change. Are you able to help?