test_transpose_forward - ValueError: axes don't match array

[NaifAlqahtani]

Hi all,

Thanks for making the course public.

I struggled to find any forums for this course online so I'm resorting to creating an issue here as a last ditch effort instead of getting in touch with the course instructors/TAs directly.

I can't for the life of me figure out why the test fails.

tests/hw1/test_autograd_hw.py::test_power_scalar_forward PASSED          [  8%]
tests/hw1/test_autograd_hw.py::test_divide_forward PASSED                [ 16%]
tests/hw1/test_autograd_hw.py::test_divide_scalar_forward PASSED         [ 25%]
tests/hw1/test_autograd_hw.py::test_matmul_forward PASSED                [ 33%]
tests/hw1/test_autograd_hw.py::test_summation_forward PASSED             [ 41%]
tests/hw1/test_autograd_hw.py::test_broadcast_to_forward PASSED          [ 50%]
tests/hw1/test_autograd_hw.py::test_reshape_forward PASSED               [ 58%]
tests/hw1/test_autograd_hw.py::test_negate_forward PASSED                [ 66%]
tests/hw1/test_autograd_hw.py::test_transpose_forward FAILED             [ 75%]
tests/hw1/test_autograd_hw.py::test_log_forward PASSED                   [ 83%]
tests/hw1/test_autograd_hw.py::test_exp_forward PASSED                   [ 91%]
tests/hw1/test_autograd_hw.py::test_ewisepow_forward PASSED              [100%]

error:

    def test_transpose_forward():
        x = ndl.Tensor([[[1.95]], [[2.7]], [[3.75]]])
>       x_T = ndl.transpose(x, axes=(1, 2))
ValueError: axes don't match array

here is the modified test case so each action is done in its own line:

def test_transpose_forward():
    x = ndl.Tensor([[[1.95]], [[2.7]], [[3.75]]])
    x_T = ndl.transpose(x, axes=(1, 2))
    x_T = x_T.numpy()
    res = np.array([[[1.95]], [[2.7]], [[3.75]]])
    np.testing.assert_allclose(x_T, res)

I have tried many solutions, a few of which:

    def compute(self, a):
        ### BEGIN YOUR SOLUTION
        return array_api.transpose(a, axes=self.axes)
        ### END YOUR SOLUTION

    def compute(self, a):
        ### BEGIN YOUR SOLUTION
        if self.axes is None:
            self.axes = (len(a.shape) - 2, len(a.shape) - 1)

        # If the tensor has fewer dimensions than the max axis in self.axes, raise an error
        if max(self.axes) >= len(a.shape):
            raise ValueError(
                "Axes out of bounds for array with shape {}".format(a.shape)
            )

        # If the specified axes are identical or the dimensions are already (1,1), return the input tensor directly
        if a.shape[self.axes[0]] == 1 and a.shape[self.axes[1]] == 1:
            return a

        # Perform the transpose
        return array_api.transpose(a, axes=self.axes)
        ### END YOUR SOLUTION

Any help please?

[NaifAlqahtani]

Found a fix,

for those running into this problem in the future.

Basically, all chatGPT's attempts failed 😂... I copied the exact same prompt to claude 3.5 sonnet and it produced the correct solutions on the second try. Claude is GOAT of programming LLMS.

I see the issue more clearly now. Let's analyze the test case:

x = ndl.Tensor([[[1.95]], [[2.7]], [[3.75]]])  # Shape is (3, 1, 1)
x_T = ndl.transpose(x, axes=(1, 2))  # Trying to transpose axes 1 and 2

The problem is that the input tensor has shape (3, 1, 1) and we're trying to transpose axes 1 and 2. 
The numpy.transpose operation expects the axes to be a valid permutation of the dimensions. 
Let's fix the implementation:

class Transpose(TensorOp):
    """Reverses the order of two axes (axis1, axis2), defaults to the last two axes
    Args:
        axes: Optional tuple specifying the axes to transpose. If None, defaults to
              transposing the last two axes.
    """
    def __init__(self, axes: Optional[tuple] = None):
        self.axes = axes

    def compute(self, a):
        ### BEGIN YOUR SOLUTION
        # If axes not specified, transpose last two dimensions
        ndim = len(a.shape)
        if self.axes is None:
            if ndim <= 1:
                return a
            self.axes = (ndim-2, ndim-1)
            
        # Create the full permutation
        # Start with identity permutation: [0, 1, 2, ..., n-1]
        perm = list(range(ndim))
        
        # Swap the specified axes
        axis1, axis2 = self.axes
        # Handle negative indices
        if axis1 < 0:
            axis1 += ndim
        if axis2 < 0:
            axis2 += ndim
            
        # Validate axes
        if not (0 <= axis1 < ndim and 0 <= axis2 < ndim):
            raise ValueError("axes out of bounds")
            
        # Swap the positions in permutation
        perm[axis1], perm[axis2] = perm[axis2], perm[axis1]
        
        return array_api.transpose(a, axes=tuple(perm))
        ### END YOUR SOLUTION

Key changes in this implementation:

1. We create a full permutation of all axes, not just the two we want to swap
   - For a 3D tensor with shape (3,1,1), the default permutation is (0,1,2)
   - When swapping axes 1 and 2, we get permutation (0,2,1)

2. We handle the axes swap by:
   - Creating a list of all dimension indices [0,1,2,...,n-1]
   - Swapping only the specified positions
   - Using this full permutation in array_api.transpose()

3. We properly handle validation:
   - Check for negative indices
   - Validate axes are within bounds
   - Convert to proper positive indices if negative

This implementation should now correctly handle the test case where:
- Input shape is (3,1,1)
- Axes to transpose are (1,2)
- Expected output shape should still be (3,1,1), just with axes 1 and 2 swapped

The key insight is that transposing axes needs to provide a full permutation to numpy.transpose(), not just the axes we want to swap. The previous implementation was trying to pass only the swapped axes, which numpy doesn't accept.

This should now pass all the test cases including the problematic one you showed. Let me know if you'd like me to explain any part in more detail!

[David-Xiang]

Hi, Naif. I'm also taking this course. I found that in numpy's manual, np.transpose only accepts that the axes param is None or a permutation of the axes list. Thus you must pass the full permutation. The following is my solution and I think it's simpler:

class Transpose(TensorOp):
    def __init__(self, axes: Optional[tuple] = None):
        self.axes = axes

    def compute(self, a):
        if self.axes:
            swap_axes = {self.axes[0]: self.axes[1], self.axes[1]: self.axes[0]}
        else:
            swap_axes = {a.ndim - 1: a.ndim - 2, a.ndim - 2: a.ndim - 1}
        permutation = [swap_axes[i] if i in swap_axes else i for i in range(a.ndim)]
        return array_api.transpose(a, axes=permutation)