Output Error in pyaxengine-AXCL Integration.

[nnn112358]

pyaxengine_axcl_test

Environment

• Hardware: Raspberry Pi 5 + Xinjian M.2 Accelerator Card (AX650N)
• Software: pyaxengine-0.1.2 (axengine-0.1.2-py3-none-any.whl)
• AXCL-SMI: V2.25.0_20250117163029

Problem Description

The issue occurs when trying to run inference with a MobileNetV2 model (torch_vision_mobilenet_v2.axmodel) using the AX650N accelerator:

1. The output name from the model is displayed as a C-style pointer (<cdata 'char *' 0x26333140>) instead of a proper Python string.

2. When executing session.run(None, input_feed={"input": img})[0], the following error occurs:

   AttributeError: cdata 'void *' has no attribute 'pOutputs'
   

Steps to reproduce

I put it here.
https://github.com/nnn112358/pyaxengine_axcl_test

Input is correctly recognized: name=input, shape=[1, 224, 224, 3]
Output shape is correct [1, 1000], but the name is problematic
The model can be opened in Netron, but there appears to be an issue with how the output layer is recognized or handled.

$ python infer_axmodel.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 3cdead5e
===== INPUT DETAILS =====
Input 0: name=input, shape=[1, 224, 224, 3]

===== OUTPUT DETAILS =====
Output 0: name=<cdata 'char *' 0x26333140>, shape=[1, 1000]

Actual output name: <cdata 'char *' 0x26333140>
Traceback (most recent call last):
  File "/home/nnn/axcl/infer_axmodel.py", line 40, in <module>
    main()
  File "/home/nnn/axcl/infer_axmodel.py", line 34, in main
    result = session.run(None, input_feed={"input": img})[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/myenv/lib/python3.11/site-packages/axengine/_session.py", line 118, in run
    return self._sess.run(output_names, input_feed, run_options, shape_group)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/myenv/lib/python3.11/site-packages/axengine/_axclrt.py", line 382, in run
    self._io[0].pOutputs[i].pVirAddr, npy_size
    ^^^^^^^^^^^^^^^^^^^^
AttributeError: cdata 'void *' has no attribute 'pOutputs'

infer_axmodel.py

import numpy as np
from axengine import InferenceSession

def main():
    model_path = "torch_vision_mobilenet_v2.axmodel"
    
    # Create a simple test image
    img = np.random.randint(0, 256, (224, 224, 3), dtype=np.uint8)
    img = img[None]  # Add batch dimension
    
    # Initialize inference session
    session = InferenceSession(
        path_or_bytes=model_path, 
        providers=['AXCLRTExecutionProvider']
    )
    
    # Debug: Print input and output details
    inputs = session.get_inputs()
    outputs = session.get_outputs()
    
    print("===== INPUT DETAILS =====")
    for idx, input_info in enumerate(inputs):
        print(f"Input {idx}: name={input_info.name}, shape={input_info.shape}")
    
    print("\n===== OUTPUT DETAILS =====")
    for idx, output_info in enumerate(outputs):
        print(f"Output {idx}: name={repr(output_info.name)}, shape={output_info.shape}")
    
    # Get the actual output name from the model
    actual_output_name = outputs[0].name
    print(f"\nActual output name: {repr(actual_output_name)}")
    
    # Run inference with the correct output name
    result = session.run(None, input_feed={"input": img})[0]
    print(f"Output shape: {result.shape}")
    
    return result

if __name__ == "__main__":
    main()

This figure shows torch_vision_mobilenet_v2.axmodel opened in nectron.

Environment-Detail

raspi5$ axcl-smi
+------------------------------------------------------------------------------------------------+
| AXCL-SMI  V2.25.0_20250117163029                                Driver  V2.25.0_20250117163029 |
+-----------------------------------------+--------------+---------------------------------------+
| Card  Name                     Firmware | Bus-Id       |                          Memory-Usage |
| Fan   Temp                Pwr:Usage/Cap | CPU      NPU |                             CMM-Usage |
|=========================================+==============+=======================================|
|    0  AX650N                    V2.25.0 | 0000:01:00.0 |                150 MiB /      945 MiB |
|   --   67C                      -- / -- | 1%        0% |                 18 MiB /     2944 MiB |
+-----------------------------------------+--------------+---------------------------------------+

+------------------------------------------------------------------------------------------------+
| Processes:                                                                                     |
| Card      PID  Process Name                                                   NPU Memory Usage |
|================================================================================================|

[nnn112358]

I tested with axengine-0.1.2 and axengine-0.1.1.

Current: axengine-0.1.2

An error occurs in session.run. The output name is displaying in a strange format.

raspi5:~/test$ pip install axengine-0.1.2-py3-none-any.whl

raspi5:~/test$ python infer_axmodel.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 3cdead5e
===== INPUT DETAILS =====
Input 0: name=input, shape=[1, 224, 224, 3]

===== OUTPUT DETAILS =====
Output 0: name=<cdata 'char *' 0x2bbac7a0>, shape=[1, 1000]

Actual output name: <cdata 'char *' 0x2bbac7a0>
Traceback (most recent call last):
  File "/home/nnn/test/infer_axmodel.py", line 40, in <module>
    main()
  File "/home/nnn/test/infer_axmodel.py", line 34, in main
    result = session.run(None, input_feed={"input": img})[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/myenv/lib/python3.11/site-packages/axengine/_session.py", line 118, in run
    return self._sess.run(output_names, input_feed, run_options, shape_group)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/myenv/lib/python3.11/site-packages/axengine/_axclrt.py", line 382, in run
    self._io[0].pOutputs[i].pVirAddr, npy_size
    ^^^^^^^^^^^^^^^^^^^^
AttributeError: cdata 'void *' has no attribute 'pOutputs'

Current: axengine-0.1.1

With this version, session.run succeeds without any errors, but the output name still displays in a strange format.

raspi5:~/test$ pip install axengine-0.1.1-py3-none-any.whl

raspi5:~/test$ python infer_axmodel.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 3cdead5e
===== INPUT DETAILS =====
Input 0: name=input, shape=[1, 224, 224, 3]

===== OUTPUT DETAILS =====
Output 0: name=<cdata 'char *' 0xd8d82b0>, shape=[1, 1000]

Actual output name: <cdata 'char *' 0xd8d82b0>
Output shape: (1, 1000)

Investigation Results

After investigating the issue, I found that the current code directly assigns the C pointer returned from the AxcrustEngineGetOutputNameByIndex function to the name variable. This causes Python to display the raw C pointer address (e.g., <cdata 'char *' 0x8820f70>) instead of the actual string value.

The proper solution involves a two-step process:

1. First, store the C pointer in a variable named cffi_name
2. Then, convert the C pointer to a string using axclrt_cffi.string() and decode it as a UTF-8 string with decode("utf-8")

This modification ensures that C string pointers are properly converted to Python strings, allowing the correct names to appear in the output results. This approach is consistent with the method already used in the _get_inputs() method.

Before change:

axengine/_axclrt.py LineNo.279

name = axclrt_lib.axclrtEngineGetOutputNameByIndex(self._info[0], index)

After change:

axengine/_axclrt.py LineNo.279

cffi_name = axclrt_lib.axclrtEngineGetOutputNameByIndex(self._info[0], index)
name = axclrt_cffi.string(cffi_name).decode("utf-8")

axengine-0.1.2 + revision

After applying the fix, the output name issue was resolved, but the same run-time error persists:

$ python infer_axmodel.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 3cdead5e
===== INPUT DETAILS =====
Input 0: name=input, shape=[1, 224, 224, 3]

===== OUTPUT DETAILS =====
Output 0: name='output', shape=[1, 1000]

Actual output name: 'output'
Traceback (most recent call last):
  File "/home/nnn/test/infer_axmodel.py", line 40, in <module>
    main()
  File "/home/nnn/test/infer_axmodel.py", line 34, in main
    result = session.run(None, input_feed={"input": img})[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/test/axengine/_session.py", line 118, in run
    return self._sess.run(output_names, input_feed, run_options, shape_group)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nnn/test/axengine/_axclrt.py", line 383, in run
    self._io[0].pOutputs[i].pVirAddr, npy_size
    ^^^^^^^^^^^^^^^^^^^^
AttributeError: cdata 'void *' has no attribute 'pOutputs'

This suggests that while we fixed the naming issue, there's a deeper structural problem in version 0.1.2 related to memory handling or object structure that needs addressing.

axengine-0.1.1 + revision

With version 0.1.1, applying the fix completely resolves the issue:

$ python infer_axmodel.py
[INFO] Available providers:  ['AXCLRTExecutionProvider']
[INFO] Using provider: AXCLRTExecutionProvider
[INFO] SOC Name: AX650N
[INFO] VNPU type: VNPUType.DISABLED
[INFO] Compiler version: 3.3 3cdead5e
===== INPUT DETAILS =====
Input 0: name=input, shape=[1, 224, 224, 3]

===== OUTPUT DETAILS =====
Output 0: name='output', shape=[1, 1000]

Actual output name: 'output'
Output shape: (1, 1000)

Conclusion

The string handling issue has been resolved by properly converting C string pointers to Python strings.
However, version 0.1.2 has an additional structural issue affecting the pOutputs attribute access that was not present in version 0.1.1.
This suggests that version 0.1.2 introduced changes to the internal data structures or memory handling that are incompatible with the current implementation.

For use, I recommend :
Use version 0.1.1 and fix axengine/_axclrt.py.