Skip to content

build: Add Cython compilation support for wheel obfuscation #979

@devin-ai-integration

Description

@devin-ai-integration

Summary

Add Cython compilation to the PyAirbyte build pipeline so that PyPI-published wheels contain compiled .so/.pyd extensions instead of readable .py source files, providing basic code obfuscation for distribution from the private repo.

Context

This pattern has been prototyped in the airbyte-ops-mcp and connector-builder-mcp repos:

The approach uses:

  • hatch-cythonize build hook (maintained fork of hatch-cython) to compile .py.c.so during wheel builds
  • Build-time injection of Cython config (not stored in pyproject.toml) to avoid breaking uv sync / editable installs
  • Post-build source stripping to remove .py files from wheels where compiled .so counterparts exist
  • Exclusion of modules that break under Cython compilation (e.g., MCP/Pydantic presentation layers)

Key Learnings from Ops MCP Implementation

  1. Cython config must NOT be in pyproject.toml — it runs during editable installs (uv sync), breaking CI workflows. Solution: inject at build time via a script.
  2. Pydantic/FastMCP modules must be excluded — Cython-compiled functions (cyfunction) don't expose type annotations the same way, breaking Pydantic schema generation.
  3. __init__.py files must be excluded — required for Python package structure.
  4. Platform-specific wheels — must build per OS (Linux, macOS) and Python version (3.10, 3.11, 3.12).

Action Items

  • Evaluate whether this pattern should be applied to PyAirbyte
  • If yes, adapt the enable_cython_build.py and strip_source_from_wheel.py scripts for PyAirbyte's package structure
  • Identify which modules (if any) need to be excluded from compilation
  • Integrate with existing publish/release workflows

Devin session: https://app.devin.ai/sessions/7c8622f9452141d1bbad93f74079278f
Requested by: Aaron ("AJ") Steers (@aaronsteers)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions