RuntimeError: Unable to cast Python instance of type <class 'numpy.ndarray'> to C++ type '?'

[ares89]

🐛 Describe the bug

cmd is:
nohup python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -m SA8295 --temperature 0 --model_mode hybrid --max_seq_len 1024 --prefill_ar_len 128 --decoder_model qwen3-0_6b --compile_only --prompt "你好啊" --checkpoint /home/xxx/.cache/meta_checkpoints/Qwen_Qwen3-0.6B.pth 1>logs/qwen3_8295.log 2>&1 &

log is like bellow, I have checked that pybinding is installed:

[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in SAVE MODE.
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
NOTE: Using slow Hadamard transform for SpinQuant. For better performance on GPU, install fast_hadamard_transform: pip install git+https://github.com/Dao-AILab/fast-hadamard-transform.git
QNN_SDK_ROOT=/home/xxx/executorch/backends/qualcomm/sdk/qnn
[QNN Partitioner Op Support]: aten.squeeze_copy.dims | True
[QNN Partitioner Op Support]: aten.permute_copy.default | True
Traceback (most recent call last):
File "/home/xxx/executorch/examples/qualcomm/oss_scripts/llama/llama.py", line 1343, in main
export_llama(args)
File "/home/xxx/executorch/examples/qualcomm/oss_scripts/llama/llama.py", line 1317, in export_llama
compile(args, decoder_model_config, pte_filename, tokenizer, chat_template)
File "/home/xxx/executorch/examples/qualcomm/oss_scripts/llama/llama.py", line 811, in compile
edge_prog_mgr = to_edge_transform_and_lower_to_qnn(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/utils/utils.py", line 445, in to_edge_transform_and_lower_to_qnn
return to_edge_transform_and_lower(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/program/_program.py", line 114, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/program/_program.py", line 1371, in to_edge_transform_and_lower
edge_manager = edge_manager.to_backend(method_to_partitioner)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/program/_program.py", line 114, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/program/_program.py", line 1672, in to_backend
new_edge_programs = to_backend(method_to_programs_and_partitioners)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/miniconda3/envs/execu2/lib/python3.12/functools.py", line 912, in wrapper
return dispatch(args[0].class)(*args, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/backend/backend_api.py", line 721, in _
partitioner_result = partitioner_instance(fake_edge_program)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/backend/partitioner.py", line 66, in call
return self.partition(exported_program)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/partition/qnn_partitioner.py", line 199, in partition
partitions = self.generate_partitions(edge_program)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/partition/qnn_partitioner.py", line 164, in generate_partitions
return generate_partitions_from_list_of_nodes(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/exir/backend/canonical_partitioners/pattern_op_partitioner.py", line 54, in generate_partitions_from_list_of_nodes
partition_list = capability_partitioner.propose_partitions()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/miniconda3/envs/execu2/lib/python3.12/site-packages/torch/fx/passes/infra/partitioner.py", line 226, in propose_partitions
if self._is_node_supported(node) and node not in assignment:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/miniconda3/envs/execu2/lib/python3.12/site-packages/torch/fx/passes/infra/partitioner.py", line 87, in _is_node_supported
return self.operator_support.is_node_supported(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/partition/qnn_partitioner.py", line 100, in is_node_supported
op_wrapper = self.node_visitors[node.target.name].define_node(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/builders/op_conv.py", line 140, in define_node
filter_tensor_wrapper = self.define_tensor(
^^^^^^^^^^^^^^^^^^^
File "/home/xxx/executorch/backends/qualcomm/builders/node_visitor.py", line 462, in define_tensor
tensor_wrapper = PyQnnWrapper.TensorWrapper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Unable to cast Python instance of type <class 'numpy.ndarray'> to C++ type '?' (#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for details)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xxx/executorch/examples/qualcomm/oss_scripts/llama/llama.py", line 1354, in
main()
File "/home/xxx/executorch/examples/qualcomm/oss_scripts/llama/llama.py", line 1349, in main
raise Exception(e)
Exception: Unable to cast Python instance of type <class 'numpy.ndarray'> to C++ type '?' (#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for details)
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters

python -c "import pybind11; print(pybind11.__version__)"

3.0.1

python -c "import numpy as np; print(np.__version__)"

2.4.1

Versions

Collecting environment information...
PyTorch version: 2.9.0+cu128
Is debug build: False
CUDA used to build PyTorch: 12.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0
Clang version: Could not collect
CMake version: version 3.29.0
Libc version: glibc-2.35

Python version: 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:16:04) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-160-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.2.91
CUDA_MODULE_LOADING set to:
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 4090
GPU 1: NVIDIA GeForce RTX 4090
GPU 2: NVIDIA GeForce RTX 4090
GPU 3: NVIDIA GeForce RTX 4090
GPU 4: NVIDIA GeForce RTX 4090
GPU 5: NVIDIA GeForce RTX 4090
GPU 6: NVIDIA GeForce RTX 4090
GPU 7: NVIDIA GeForce RTX 4090

Nvidia driver version: 550.163.01
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 256
On-line CPU(s) list: 0-255
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 64
Socket(s): 2
Stepping: 1
Frequency boost: enabled
CPU max MHz: 3530.4929
CPU min MHz: 1500.0000
BogoMIPS: 4891.30
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm
Virtualization: AMD-V
L1d cache: 4 MiB (128 instances)
L1i cache: 4 MiB (128 instances)
L2 cache: 64 MiB (128 instances)
L3 cache: 512 MiB (16 instances)
NUMA node(s): 2
NUMA node0 CPU(s): 0-63,128-191
NUMA node1 CPU(s): 64-127,192-255
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Mitigation; safe RET
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines; IBPB conditional; IBRS_FW; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsa: Vulnerable: Clear CPU buffers attempted, no microcode
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] executorch==1.0.1a0+087568a
[pip3] numpy==2.4.1
[pip3] nvidia-cublas-cu12==12.8.4.1
[pip3] nvidia-cuda-cupti-cu12==12.8.90
[pip3] nvidia-cuda-nvrtc-cu12==12.8.93
[pip3] nvidia-cuda-runtime-cu12==12.8.90
[pip3] nvidia-cudnn-cu12==9.10.2.21
[pip3] nvidia-cufft-cu12==11.3.3.83
[pip3] nvidia-curand-cu12==10.3.9.90
[pip3] nvidia-cusolver-cu12==11.7.3.90
[pip3] nvidia-cusparse-cu12==12.5.8.93
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-nccl-cu12==2.27.5
[pip3] nvidia-nvjitlink-cu12==12.8.93
[pip3] nvidia-nvtx-cu12==12.8.90
[pip3] pytorch_tokenizers==1.0.1
[pip3] torch==2.9.0
[pip3] torchao==0.14.0+git02941240f
[pip3] torchaudio==2.9.0+cpu
[pip3] torchdata==0.11.0+cpu
[pip3] torchsr==1.0.4
[pip3] torchtune==0.7.0+cpu
[pip3] torchvision==0.24.0+cpu
[pip3] triton==3.5.0
[conda] No relevant packages

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

[Gasoonjia]

@winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic mind take a look?

[ares89]

In the same env, I create a test for the methond PyQnnWrapper.TensorWrapper, and it runs ok.

"""
define_tensor 方法单元测试
测试 backends/qualcomm/builders/node_visitor.py 中的 define_tensor 方法
"""

import unittest
import sys
import os
import numpy as np
from unittest.mock import Mock, MagicMock, patch
import torch


class TestDefineTensor(unittest.TestCase):
    """测试NodeVisitor类中的define_tensor方法"""
    
    def setUp(self):
        """设置测试环境"""
        # 检查是否可以导入必要的模块
        try:
            from executorch.backends.qualcomm.python.PyQnnWrapperAdaptor import (
                Qnn_TensorType_t,
                Qnn_DataType_t,
                Qnn_QuantizationEncoding_t
            )
            self.qnn_types_available = True
            self.qnn_tensor_type = Qnn_TensorType_t
            self.qnn_data_type = Qnn_DataType_t
            self.qnn_quant_enc = Qnn_QuantizationEncoding_t
        except ImportError:
            print("警告：无法导入QNN类型，将使用Mock")
            self.qnn_types_available = False
    
    @patch('executorch.backends.qualcomm.python.PyQnnWrapperAdaptor.TensorWrapper')
    def test_define_tensor_basic(self, mock_tensor_wrapper):
        """测试define_tensor方法的基本功能"""
        print("正在测试define_tensor方法的基本功能...")
        
        if not self.qnn_types_available:
            # 如果无法导入QNN类型，使用Mock
            mock_qnn_types = {
                'QNN_TENSOR_TYPE_NATIVE': Mock(),
                'QNN_DATATYPE_FLOAT_32': Mock(),
                'QNN_QUANTIZATION_ENCODING_UNDEFINED': Mock()
            }
        else:
            mock_qnn_types = {
                'QNN_TENSOR_TYPE_NATIVE': self.qnn_tensor_type.QNN_TENSOR_TYPE_NATIVE,
                'QNN_DATATYPE_FLOAT_32': self.qnn_data_type.QNN_DATATYPE_FLOAT_32,
                'QNN_QUANTIZATION_ENCODING_UNDEFINED': self.qnn_quant_enc.QNN_QUANTIZATION_ENCODING_UNDEFINED
            }
        
        # 创建一个mock的edge_program和node
        mock_edge_program = Mock()
        mock_node = Mock()
        mock_node.name = "test_node"
        
        # 创建NodeVisitor实例（我们需要绕过初始化）
        with patch('executorch.backends.qualcomm.builders.node_visitor.NodeVisitor.__init__', 
                  return_value=None):
            visitor = object.__new__(type('MockNodeVisitor', (), {
                '__init__': lambda self: None,
                'edge_program': mock_edge_program,
                'node': mock_node,
                'enable_tensor_dump': False
            }))
            
            # 由于我们无法直接访问define_tensor方法，我们将模拟它的行为
            def mock_define_tensor(tensor, edge_node_name, node_outputs=[], output_idx=0):
                """模拟define_tensor方法的行为"""
                # 根据原始代码片段，这个方法应该处理tensor并创建TensorWrapper
                if isinstance(tensor, torch.Tensor):
                    # 如果是PyTorch张量，转换为numpy
                    np_tensor = tensor.detach().numpy()
                elif isinstance(tensor, np.ndarray):
                    np_tensor = tensor
                else:
                    np_tensor = np.array(tensor)
                    
                # 确定维度信息
                nominal_dims = list(np_tensor.shape)
                dynamic_dims = [0] * len(np_tensor.shape)  # 简化的动态维度
                
                # 设置类型参数
                tensor_name = edge_node_name
                tensor_type = mock_qnn_types['QNN_TENSOR_TYPE_NATIVE']
                dtype = mock_qnn_types['QNN_DATATYPE_FLOAT_32']
                quant_encoding = mock_qnn_types['QNN_QUANTIZATION_ENCODING_UNDEFINED']
                quant_configs = {}
                
                # 调用TensorWrapper
                tensor_wrapper = mock_tensor_wrapper(
                    tensor_name,
                    tensor_type,
                    dtype,
                    quant_encoding,
                    quant_configs,
                    len(nominal_dims),
                    nominal_dims,
                    dynamic_dims,
                    np.array([]),  # 根据代码片段，这里传递空数组
                    False,
                )
                
                return tensor_wrapper
            
            # 测试用不同类型的输入
            test_cases = [
                {
                    'name': 'torch_tensor',
                    'input': torch.randn(2, 3, 4),
                    'expected_shape': [2, 3, 4]
                },
                {
                    'name': 'numpy_array',
                    'input': np.random.rand(2, 3),
                    'expected_shape': [2, 3]
                },
                {
                    'name': 'python_list',
                    'input': [[1, 2], [3, 4]],
                    'expected_shape': [2, 2]
                }
            ]
            
            for case in test_cases:
                print(f"  测试用例: {case['name']}")
                try:
                    result = mock_define_tensor(
                        case['input'],
                        f"test_{case['name']}"
                    )
                    
                    # 验证TensorWrapper被正确调用
                    mock_tensor_wrapper.assert_called()
                    
                    # 获取最后一次调用的参数
                    args, kwargs = mock_tensor_wrapper.call_args
                    print(f"    ✓ TensorWrapper被正确调用，参数数量: {len(args)}")
                    
                    # 验证参数顺序和类型
                    self.assertEqual(args[0], f"test_{case['name']}")  # tensor_name
                    print(f"    ✓ 张量名称正确: {args[0]}")
                    
                    # 验证维度
                    self.assertEqual(args[6], case['expected_shape'])  # nominal_dims
                    print(f"    ✓ 维度信息正确: {args[6]}")
                    
                except Exception as e:
                    print(f"    ✗ 测试用例 {case['name']} 失败: {e}")
                    self.fail(f"测试用例 {case['name']} 失败: {e}")
    
    def test_define_tensor_with_real_qnn_types(self):
        """使用真实QNN类型测试define_tensor"""
        print("\n使用真实QNN类型测试define_tensor...")
        
        if not self.qnn_types_available:
            print("  跳过：无法导入真实QNN类型")
            return
        
        try:
            # 创建一个测试张量
            test_tensor = torch.randn(2, 3, 4)
            np_tensor = test_tensor.detach().numpy()
            
            # 创建维度信息
            nominal_dims = list(np_tensor.shape)
            dynamic_dims = [0] * len(np_tensor.shape)
            
            # 使用真实类型创建TensorWrapper
            from executorch.backends.qualcomm.python.PyQnnWrapperAdaptor import (
                TensorWrapper,
                Qnn_TensorType_t,
                Qnn_DataType_t,
                Qnn_QuantizationEncoding_t
            )
            
            # 这里我们只是验证类型是否正确，不实际创建TensorWrapper以避免错误
            tensor_name = "test_real_types"
            tensor_type = Qnn_TensorType_t.QNN_TENSOR_TYPE_NATIVE
            dtype = Qnn_DataType_t.QNN_DATATYPE_FLOAT_32
            quant_encoding = Qnn_QuantizationEncoding_t.QNN_QUANTIZATION_ENCODING_UNDEFINED
            quant_configs = {}
            
            print(f"  ✓ 成功获取真实QNN类型")
            print(f"    - 张量名称: {tensor_name}")
            print(f"    - 张量类型: {tensor_type}")
            print(f"    - 数据类型: {dtype}")
            print(f"    - 量化编码: {quant_encoding}")
            print(f"    - 维度: {nominal_dims}")
            
        except ImportError:
            print("  跳过：无法导入真实QNN类型")
        except Exception as e:
            print(f"  注意：真实类型测试出现问题: {e}")
    
    @unittest.skipUnless(
        hasattr(sys, 'getrefcount'), 
        "需要引用计数功能（CPython）"
    )
    def test_define_tensor_memory_handling(self):
        """测试内存处理（仅在支持引用计数的Python实现中）"""
        print("\n测试内存处理...")
        
        # 创建一个较大的张量来测试内存处理
        large_tensor = torch.randn(100, 100)
        
        # 验证张量被正确处理
        np_tensor = large_tensor.detach().numpy()
        self.assertEqual(np_tensor.shape, (100, 100))
        print(f"  ✓ 正确处理了较大张量，形状: {np_tensor.shape}")


def run_tests():
    """运行所有测试"""
    print("="*80)
    print("开始测试 define_tensor 方法")
    print("="*80)
    
    # 创建测试套件
    suite = unittest.TestLoader().loadTestsFromTestCase(TestDefineTensor)
    
    # 运行测试
    runner = unittest.TextTestRunner(verbosity=2)
    result = runner.run(suite)
    
    print("\n" + "="*80)
    if result.wasSuccessful():
        print("✓ 所有测试均通过！")
    else:
        print(f"✗ {len(result.failures)} 个测试失败，{len(result.errors)} 个测试出错")
        
        for failure in result.failures:
            print(f"失败: {failure[0]}")
            print(f"详情: {failure[1]}")
            
        for error in result.errors:
            print(f"错误: {error[0]}")
            print(f"详情: {error[1]}")
    print("="*80)
    
    return result.wasSuccessful()


if __name__ == '__main__':
    success = run_tests()
    sys.exit(0 if success else 1)

/data/miniconda3/envs/execu2/bin/python /home/lishuguang8/executorch/examples/qualcomm/test_result/test_define_tensor.py
================================================================================
开始测试 define_tensor 方法
================================================================================
test_define_tensor_basic (__main__.TestDefineTensor.test_define_tensor_basic)
测试define_tensor方法的基本功能 ... 正在测试define_tensor方法的基本功能...
  测试用例: torch_tensor
    ✓ TensorWrapper被正确调用，参数数量: 10
    ✓ 张量名称正确: test_torch_tensor
    ✓ 维度信息正确: [2, 3, 4]
  测试用例: numpy_array
    ✓ TensorWrapper被正确调用，参数数量: 10
    ✓ 张量名称正确: test_numpy_array
    ✓ 维度信息正确: [2, 3]
  测试用例: python_list
    ✓ TensorWrapper被正确调用，参数数量: 10
    ✓ 张量名称正确: test_python_list
    ✓ 维度信息正确: [2, 2]
ok
test_define_tensor_memory_handling (__main__.TestDefineTensor.test_define_tensor_memory_handling)
测试内存处理（仅在支持引用计数的Python实现中） ... 
测试内存处理...
  ✓ 正确处理了较大张量，形状: (100, 100)
ok
test_define_tensor_with_real_qnn_types (__main__.TestDefineTensor.test_define_tensor_with_real_qnn_types)
使用真实QNN类型测试define_tensor ... 
使用真实QNN类型测试define_tensor...
  ✓ 成功获取真实QNN类型
    - 张量名称: test_real_types
    - 张量类型: Qnn_TensorType_t.QNN_TENSOR_TYPE_NATIVE
    - 数据类型: Qnn_DataType_t.QNN_DATATYPE_FLOAT_32
    - 量化编码: Qnn_QuantizationEncoding_t.QNN_QUANTIZATION_ENCODING_UNDEFINED
    - 维度: [2, 3, 4]
ok

----------------------------------------------------------------------
Ran 3 tests in 3.067s

OK

================================================================================
✓ 所有测试均通过！
================================================================================

[shewu-quic]

Hi @ares89 ,
Thanks for your trying. Just a few checklist:

• Did you follow the guidance in https://docs.pytorch.org/executorch/1.0/backends-qualcomm.html to install the qnn backend?
• Are you able to run any simple model https://github.com/pytorch/executorch/tree/main/examples/qualcomm#simple-examples-to-verify-the-backend-is-working on SA8295?

If possible, could you provide full log file for debug?

[ares89]

Hi @shewu-quic ,
I succeed test a model inference on QNN HTP emulator.

python export_example.py -m add -g
./qnn_executor_runner --model_path add.pte

examples/qualcomm/executor_runner/qnn_executor_runner --model_path /home/xxx/executorch/examples/qualcomm/scripts/add.pte
I 00:00:00.000384 executorch:qnn_executor_runner.cpp:232] Model file /home/xxx/executorch/examples/qualcomm/scripts/add.pte is loaded.
I 00:00:00.000411 executorch:qnn_executor_runner.cpp:242] Using method forward
I 00:00:00.000414 executorch:qnn_executor_runner.cpp:289] Setting up planned buffer 0, size 48.
[INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol
[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1
[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE.
[INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000
[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
I 00:00:00.074134 executorch:qnn_executor_runner.cpp:313] Method loaded.
E 00:00:00.074151 executorch:method.cpp:1240] Output 0 is memory planned, or is a constant. Cannot override the existing data pointer.
I 00:00:00.074158 executorch:qnn_executor_runner.cpp:370] ignoring error from set_output_data_ptr(): 0x2
I 00:00:00.074161 executorch:qnn_executor_runner.cpp:376] Inputs prepared.
I 00:00:00.074205 executorch:qnn_executor_runner.cpp:568] Input list not provided. Inputs prepared with default values set.
I 00:00:00.075538 executorch:qnn_executor_runner.cpp:579] Model executed successfully.
I 00:00:00.075545 executorch:qnn_executor_runner.cpp:582] Perform 0 inferences for warming up
I 00:00:00.075671 executorch:qnn_executor_runner.cpp:599] 1 inferences took 0.122000 ms, avg 0.122000 ms
I 00:00:00.075688 executorch:qnn_executor_runner.cpp:611] Write etdump to etdump.etdp, Size = 576
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend

[shewu-quic]

It appears to be related to the numpy version. Errors occur when using numpy version 2.4.0 or higher. Could you try downgrading numpy to version 2.2.6?

[ares89]

Thank you very much. I tried downgrading NumPy to version 2.2.6 as suggested, and the error disappeared. The problem was indeed related to the NumPy version.