KeyError: <class 'ctypes.HRESULT'>, _ctype_to_vartype, HRESULT: VT_HRESULT

[davidschranz]

Thank you for the wonderful comtypes. I sucessfully use it to interact with a COM interface 'HeidenhainDNC.dll'

The only thing I had to modify was the '_ctype_to_vartype' dict within the 'automation.py' file:
_ctype_to_vartype: Dict[Type[_CData], int] = {
HRESULT: VT_HRESULT, # added this line for HeidenhainDNC.dll to work

How could I use the comtypes package without modifing this file?
Any hint are appreciated.

comtypes 1.4.8
python 3.13.0

[junkmd]

Thank you for your report.

There are a few things I need clarification on first, so I would appreciate it if you could provide more details.

1. What is HeidenhainDNC.dll?
   I’m not familiar with all the countless COM libraries out there, so I’d like to understand what this DLL is.
   Is it related to https://www.heidenhain.us/?
   Is it redistributable, or can it be installed in my environment without any licensing problems?

2. The situation where the error occurred and the results after fixing it
   I assume HeidenhainDNC.dll runs correctly in your environment.
   If there are no NDA or licensing problems, could you share the simplest use case for using this DLL with comtypes, as well as the script that caused the error?
   If possible, please try creating a minimal reproducer following the guidelines in "How to create a minimal, reproducible example".

3. Contents of the module generated in comtypes.gen
   When comtypes invokes GetModule or CreateObject, it generates Python modules defining the COM interfaces in your/env/packages/comtypes/gen/....
   A module generated from HeidenhainDNC.dll might exist in your environment. If there are no NDA or licensing problems, please share it with us.
   Instead of attaching files, I suggest following 'SyntaxError: invalid syntax' occurs during MS Project module generation with GetModule function  #524 (comment) or Question about comtypes.client typing for Autocad automation #516 (comment) to create a public repository and upload the content.

P.S. I’d appreciate it if you could highlight code and stack traces using the method described in "Creating and highlighting code blocks".

[davidschranz]

Thank you for your quick reply. Here the link to the git with a minimal reproducer as well as the gen folder content:
https://github.com/davidschranz/comtypes_key_error_1_4_8

The COM library is related to https://www.heidenhain.us/ and used to connect to their CNC controls.
It is redistributable and can be installed in your environment using HeidenhainDNC.msi without any licensing problems.

[junkmd]

Thank you for providing an excellent reproducer.
Your approach will surely serve as a model for the future.

The changes to _ctype_to_vartype have resolved the error in the stack trace that occurred with POINTER(_midlSAFEARRAY(HRESULT)).

To better understand the impact of this change, could you test the behavior of IJHDataAccess4 methods that take POINTER(_midlSAFEARRAY(HRESULT)) as a parameter in your environment?

Based on the methods definition lines of the IJHDataAccess4 and the interface definition lines of the IJHDataAccess4, I have created a sample code.
(Note: byref may not be required, and this code may depend on specific configurations in your environment or application. I’m not familiar with this COM interface.)

from ctypes import HRESULT, byref

from comtypes.automation import _midlSAFEARRAY
import comtypes.client


comtypes.client.GetModule((comtypes.GUID("{14B95319-AEF9-492A-A878-CA18FEB1F5BF}"), 1, 7))
import comtypes.gen.HeidenhainDNCLib as HeidenhainDNCLib


jhdaccess = comtypes.client.CreateObject(HeidenhainDNCLib.JHDataAccess, interface=HeidenhainDNCLib.IJHDataAccess4)
sa_type = _midlSAFEARRAY(HRESULT)
arr_lock = sa_type.create([])
arr_unlock = sa_type.create([])

jhdaccess.LockConfig(["Idents", "To", "Lock"], byref(arr_lock))  # first parameter is example
jhdaccess.UnlockConfig(["Idents", "To", "Unlock"], byref(arr_unlock))  # first parameter is example

print(arr_lock)
print(arr_unlock)

[davidschranz]

I updated the git with your sample code (data_access_01.py) and the result of it (data_access_01.md).
https://github.com/davidschranz/comtypes_key_error_1_4_8

I further added a sample code (data_access_02.py) which show how I can successfully use the IJHDataAccess4 interface.
Then I added part of your code (data_access_03.py) and the result of it (data_access_03.md).

Hope the result does make sense to you. Please let me know if I can test something further:

[junkmd]

Let us focus on the inherent peculiarity of HRESULT.
While HRESULT is a 32-bit integer type, it is specifically intended as the return type for COM (Windows) methods, used to indicate success or failure.

To the best of my knowledge, if simply a 32-bit integer type is required, LONG (VT_I4) is more commonly used instead.

Even if the purpose is to check the success or failure of a method, I have not encountered a case where a parameter with flags like ["in", "out"] is assigned a type related to HRESULT.

That said, the world of source code often contains many elements that don’t align with the "ideal" design.
There might be some intent or reasoning behind this type definition.

If you know of any use cases for IJHDataAccess4.LockConfig or IJHDataAccess4.UnlockConfig, in C++ or any other language, please share them.
I’d like to understand what kind of preparation is needed and what outcomes are expected.
This could provide valuable hints for resolving the issue.

Even if no such use cases are found, I believe it’s possible to derive a better solution.
Thus, instead of spending an extensive amount of time scouring every corner of the world, a more cursory investigation might help us resolve this issue more efficiently.

(Note: Please bear in mind that some COM methods, like IBindCtx::EnumObjectParam, inherently raise errors when called from client side, and as such, no practical use cases exist for them.)

[davidschranz]

I was not aware of the IJHDataAccess4.LockConfig method since it is not (yet) documented. Once it gets documented from Heidenhain, I will be able to test it. Right know I just don't know how the bstrIdentsToLock has to lock like.

I do have a working sample code (data_access_04.py) which suits my needs.

"""
This code does only work together with a heidenhain control
The IJHMachine4 needs to be connected to be able to get the IJHDataAccess4 interface

Disconnect will fail if not all interfaces queried from IJHMachine4 are released.
comtypes will call Release itself once the object is deleted

The method LockConfig is not (yet) documented.
I do not know how the bstrIdentsToLock would have to lock like.
bstrIdentsToLock = ["\\PLC\\memory\\M\\0"]
ppsafLockResults = jjhdataaccess.LockConfig(bstrIdentsToLock)
Returns _ctypes.COMError: (-2147024809, 'The parameter is incorrect.', (None, None, None, 0, None))

Wrapping the HeidenhainDNCLib will fail (comtypes.client.GetModule) if VT_HRESULT does not exit.
"""
from comtypes.automation import _ctype_to_vartype, HRESULT, VT_HRESULT
_ctype_to_vartype.update({HRESULT: VT_HRESULT})

import comtypes.client
comtypes.client.GetModule((comtypes.GUID("{14B95319-AEF9-492A-A878-CA18FEB1F5BF}"), 1, 7))
import comtypes.gen.HeidenhainDNCLib as HeidenhainDNCLib

jhmachine = comtypes.client.CreateObject("HeidenhainDnc.JHMachineInProcess")
ijhmachine = jhmachine.QueryInterface(HeidenhainDNCLib.IJHMachine4)
name = ijhmachine.ConfigureConnection(HeidenhainDNCLib.DNC_CONFIGURE_MODE_ALL)
ijhmachine.Connect(name)

jhdataaccess = ijhmachine.GetInterface(HeidenhainDNCLib.DNC_INTERFACE_JHDATAACCESS)
jjhdataaccess = jhdataaccess.QueryInterface(HeidenhainDNCLib.IJHDataAccess4)
jjhdataaccess.SetAccessMode(HeidenhainDNCLib.DNC_ACCESS_MODE_PLCDATAACCESS, '807667')

entry = r"\PLC\memory\M\0"
ijhdataentry = jjhdataaccess.GetDataEntry2(entry, HeidenhainDNCLib.DNC_DATA_UNIT_SELECT_METRIC, False)
value = ijhdataentry.GetPropertyValue(HeidenhainDNCLib.DNC_DATAENTRY_PROPKIND_DATA)
ijhdataentry.SetPropertyValue(HeidenhainDNCLib.DNC_DATAENTRY_PROPKIND_DATA, not(value), False)
print(value)

del ijhdataentry
del jjhdataaccess
del jhdataaccess

ijhmachine.Disconnect()
del ijhmachine
del jhmachine

[davidschranz]

Thank you very much for your help.
I very much appreciated it.

[junkmd]

I think using _ctype_to_vartype.update as an ad-hoc solution is the best approach.

However, since it references a private element starting with _, it is fragile.

For comtypes to permanently support this type library, a PR to modify automation.py as shown below would be welcome.

 _ctype_to_vartype: Dict[Type[_CData], int] = {
     c_byte: VT_I1,
     c_ubyte: VT_UI1,
     c_short: VT_I2,
     c_ushort: VT_UI2,
     c_long: VT_I4,
     c_ulong: VT_UI4,
     c_float: VT_R4,
     c_double: VT_R8,
     c_ulonglong: VT_UI8,
     VARIANT_BOOL: VT_BOOL,
     BSTR: VT_BSTR,
+    HRESULT: VT_HRESULT,
     VARIANT: VT_VARIANT,
     # SAFEARRAY(VARIANT *)
     #

By modifying the code in safearray.py to raise an exception when attempting to call the _midlSAFEARRAY(HRESULT).create method for this corner case, it would be an improvement over the current situation where a MemoryError is being raised without any meaningful error message.
This change would make it clearer when an unsupported use case is encountered, providing more useful feedback to the user.

 def _make_safearray_type(itemtype):
     # Create and return a subclass of tagSAFEARRAY
     from comtypes.automation import (
         _ctype_to_vartype,
         VT_RECORD,
         VT_UNKNOWN,
         IDispatch,
         VT_DISPATCH,
+        VT_HRESULT,
     )

     meta = type(_safearray.tagSAFEARRAY)

     @Patch(POINTER(sa_type))
     class _(object):
         # Should explain the ideas how SAFEARRAY is used in comtypes
         _itemtype_ = itemtype  # a ctypes type
         _vartype_ = vartype  # a VARTYPE value: VT_...
         _needsfree = False

         @classmethod
         def create(cls, value, extra=extra):
             """Create a POINTER(SAFEARRAY_...) instance of the correct
             type; value is an object containing the items to store.

             Python lists, tuples, and array.array instances containing
             compatible item types can be passed to create
             one-dimensional arrays.  To create multidimensional arrys,
             numpy arrays must be passed.
             """
+            if cls._vartype_ == VT_HRESULT:
+                raise TypeError(
+                    # There are COM type libraries that define the
+                    # `_midlSAFEARRAY(HRESULT)` type; however, creating `HRESULT`
+                    # safearray pointer instance does not work.
+                    # See also: https://github.com/enthought/comtypes/issues/668
+                    "Cannot create SAFEARRAY type VT_HRESULT."
+                )
+
             if comtypes.npsupport.isndarray(value):
                 return cls.create_from_ndarray(value, extra)

Until now, it has been physically impossible to create a SAFEARRAY pointer with HRESULT element, and the values of VT_... are standardized in COM specifications as unique values, so this modification will not break backward compatibility.

Additional Information: HeidenhainDNC.dll, Type Definitions, and Generated Python Modules

Upon further review, I found that the idlflags for LockConfig and UnlockConfig are [dispid(...), 'hidden'], indicating that these are likely private APIs.
This explains the absence of documentation and the lack of use cases.

It is possible that Python modules are being generated based on method definitions, like in #604, where the methods are not accessible from the client side.
While it's doubtful that these methods will actually function as expected, you might be able to manually modify the following part of the _14B95319_AEF9_492A_A878_CA18FEB1F5BF_0_1_7.py file to call jjhdataaccess.LockConfig(bstrIdentsToLock).

     COMMETHOD(
         [dispid(8), 'hidden'],
         HRESULT,
         'LockConfig',
         (['in'], _midlSAFEARRAY(BSTR), 'bstrIdentsToLock'),
-        (['in', 'out'], POINTER(_midlSAFEARRAY(HRESULT)), 'ppsafLockResults')
     ),
     COMMETHOD(
         [dispid(9), 'hidden'],
         HRESULT,
         'UnlockConfig',
         (['in'], _midlSAFEARRAY(BSTR), 'bstrIdentsToUnlock'),
-        (['in', 'out'], POINTER(_midlSAFEARRAY(HRESULT)), 'ppsafUnlockResults')
     ),