[WIP][Stubgen] Introduce --init-path for package-level generation

[junrushao]

This PR introduces an additional flag --init-path to the tool tvm-ffi-stubgen.

When this flag is specified, the tool will actively look for global functions and objects that are not yet exported into Python, and generate their type stubs into the path specified by --init-path.

To give an example, if a C++ TVM-FFI extension defines two items:

• A global function: my_ffi_extension.raise_error
• An object: my_ffi_extension.IntPair
  which are not yet exported to Python, the tool will generate the following two files if --init-path=/path/ is set:

/path/__init__.py
/path/_ffi_api.py

Where __init__.py contains:

"""Package my_ffi_extension."""

# tvm-ffi-stubgen(begin): export/_ffi_api
# fmt: off
# isort: off
from ._ffi_api import *  # noqa: F403
from ._ffi_api import __all__ as _ffi_api__all__
if "__all__" not in globals(): __all__ = []
__all__.extend(_ffi_api__all__)
# isort: on
# fmt: on
# tvm-ffi-stubgen(end)

and _ffi_api.py contains:

"""FFI API bindings for my_ffi_extension."""

# tvm-ffi-stubgen(begin): import
# fmt: off
# isort: off
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from tvm_ffi import Object
# isort: on
# fmt: on
# tvm-ffi-stubgen(end)

# tvm-ffi-stubgen(begin): global/my_ffi_extension
# fmt: off
# isort: off
from tvm_ffi import init_ffi_api as _INIT
_INIT("my_ffi_extension", __name__)
# isort: on
if TYPE_CHECKING:
    def raise_error(_0: str, /) -> None: ...
# fmt: on
# tvm-ffi-stubgen(end)

__all__ = [
    # tvm-ffi-stubgen(begin): __all__
    "IntPair",
    "raise_error",
    # tvm-ffi-stubgen(end)
]

# isort: off
import tvm_ffi
# isort: on

@tvm_ffi.register_object("my_ffi_extension.IntPair")
class IntPair(tvm_ffi.Object):
    """FFI binding for `my_ffi_extension.IntPair`."""

    # tvm-ffi-stubgen(begin): object/my_ffi_extension.IntPair
    # fmt: off
    a: int
    b: int
    if TYPE_CHECKING:
        @staticmethod
        def __c_ffi_init__(_0: int, _1: int, /) -> Object: ...
        @staticmethod
        def static_get_second(_0: IntPair, /) -> int: ...
        def get_first(self, /) -> int: ...
    # fmt: on
    # tvm-ffi-stubgen(end)

---

Outstanding items:

• Generate __init__.py
• Generate _ffi_api.py
• Support # tvm-ffi-stubgen(begin): export
• Support importing parent type in class generation
• Support package-level DLL loading
• CMake integration

[junrushao]

/gemini review

[gemini-code-assist]

Code Review

This PR introduces a significant new feature to tvm-ffi-stubgen with the --init-path flag, enabling package-level stub generation. The changes are extensive, touching the CLI, analysis, and code generation parts of the tool. The implementation correctly handles the generation of __init__.py and _ffi_api.py files for modules, and the logic to avoid duplicating content on subsequent runs is sound. I've identified a couple of areas where error handling can be made more robust by avoiding bare except clauses, and a suggestion to improve code readability. Overall, this is a great enhancement to the stub generation tool.