From 252ea295cc0b8adc2eecb872809f52165945bfd1 Mon Sep 17 00:00:00 2001 From: bbopen Date: Sat, 14 Feb 2026 07:05:11 -0800 Subject: [PATCH 1/2] docs: address RFC review follow-ups --- .../2026-01-22-tywrapped-ecosystem-vision.md | 105 ++++++++++++++++-- 1 file changed, 98 insertions(+), 7 deletions(-) diff --git a/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md index 5e76ab8..1efebf9 100644 --- a/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md +++ b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md @@ -110,12 +110,26 @@ Examples: | Tywrap fix for same numpy | `@tywrapped/numpy@1.26.0-tywrap.1` | | New numpy release | `@tywrapped/numpy@1.26.1` (resets suffix) | -**Republish policy**: Every tywrap core release triggers regeneration and republish of all active packages with compound version. Users on any upstream version benefit from improvements. +**Republish policy**: Every tywrap core release can trigger regeneration and republish of all active packages with a compound version so users on supported upstream versions benefit from improvements. + +Operational scalability (suggested defaults): +- Canary wave: 5-10% of packages first; require green CI and no runtime regressions before expanding. +- Publish gating: require unit + integration smoke tests; fail-closed on flaky/timeout tests. +- Batching: cap concurrent publishes (e.g., 3-5) to avoid CI bottlenecks and npm rate limits. +- Rate limits: retry with exponential backoff (e.g., 5 retries, 30s → 5m); pause the queue on repeated 429s. +- Rollback: move dist-tags back to the last known-good build; open an incident issue and stop the wave. **Version support breadth**: Separate indicator from quality tiers. Packages aim to match upstream's official support matrix (e.g., if numpy supports 1.24+, so does `@tywrapped/numpy`). **Breaking changes**: Mirror upstream directly. No buffering, no parallel support periods. The wrapper is a transparent pass-through - breaking changes in numpy mean breaking changes in `@tywrapped/numpy`. Document this clearly so users understand they're accepting upstream's upgrade path. +### Enterprise stability options + +- Optional channels: `stable`/`lts` dist-tags for org-blessed "slow lane" packages when demand justifies it; explicitly time-box support windows. +- Migration guides: require a short migration note for any breaking change (template + checklist) before publish. +- Pinning guidance (enterprise): prefer exact versions + lockfiles for production; treat `^` upgrades as opt-in. +- Trade-off: strict mirroring maximizes transparency and reduces maintainer load; stability channels increase operational overhead but improve adoption. + --- ## Template Repository Structure @@ -168,6 +182,12 @@ Releases triggered by **either**: Library maintainer must approve before publish - automation does the heavy lifting, humans ensure quality. +Fallbacks when maintainers are unavailable (suggested defaults): +- Minimum 2 maintainers per package (primary + backup). +- Patch releases: auto-approve after 72 hours with no response (logged), unless a maintainer vetoes. +- Security fixes: emergency override by technical committee (2-of-3) with mandatory postmortem. +- All overrides require a public audit trail (issue + PR link + rationale). + --- ## Package Health Indicators @@ -188,9 +208,51 @@ Two separate badge systems for orthogonal concerns: | Level | Indicator | Meaning | |-------|-----------|---------| | Basic | `latest` | Only current upstream version | -| Standard | `latest+1` | Latest + previous major | +| Standard | `latest-1` | Latest + previous major | | Full | `upstream` | Matches upstream's support matrix | -| Extended | `upstream+LTS` | Upstream support + extended LTS | +| Extended | `upstream-lts` | Upstream support + extended LTS | + +#### Implementation notes + +Recommended approach: single package name + npm dist-tags. + +- One package per library (e.g., `@tywrapped/numpy`) publishes multiple tested wrapper versions (mirroring upstream + `-tywrap.N` patches). +- Use dist-tags to make supported tracks easy to install: + - `latest` → newest supported upstream release line + - `latest-1` → previous major (or previous supported line) + - `upstream` → "full matrix" track (only if you actually publish/test it) + - `upstream-lts` → extended support line (explicitly time-boxed) +- Wrapper repo `package.json` stays straightforward: name is stable, version mirrors upstream with optional `-tywrap.N`, and `publishConfig.access` is public. + +Publish/tag flow (sketch): + +```bash +npm publish --tag latest +npm dist-tag add @tywrapped/numpy@1.26.0-tywrap.3 latest +npm dist-tag add @tywrapped/numpy@1.25.4-tywrap.2 latest-1 +``` + +CI matrix outline (sketch): + +```yaml +strategy: + matrix: + python: ["3.10", "3.11", "3.12"] + upstream: ["numpy==1.24.*", "numpy==1.25.*", "numpy==1.26.*"] +``` + +Publish script logic (sketch): +1. Install target upstream version(s) into an isolated env (per matrix entry). +2. Generate wrappers (tywrap) and run the test tier for that package. +3. Compute package version (`-tywrap.N`) and publish. +4. Apply/update dist-tags for the supported tracks. + +User-facing installs: +- Default: `npm i @tywrapped/numpy` (uses `latest`) +- Specific track: `npm i @tywrapped/numpy@latest-1` +- Exact pin: `npm i @tywrapped/numpy@1.26.0-tywrap.3` + +Alternative (if tags become unwieldy): separate packages per track (e.g., `@tywrapped/numpy-1.24`), at the cost of more maintenance and user confusion. ### Display Format @@ -223,15 +285,17 @@ Quality: 🥇 Gold | Support: upstream (1.24+) | Runtimes: node, browser **Override for power users:** ```typescript -import { setRuntime, InProcessBridge } from '@tywrapped/numpy'; +import { setRuntime } from '@tywrapped/numpy'; +import { InProcessBridge } from '@tywrap/runtime-inprocess'; + setRuntime(new InProcessBridge()); // Use node-calls-python for speed ``` ### Runtime Packages Scope policy: -- `@tywrapped/*` are public library wrappers (e.g., `@tywrapped/numpy`, exports `setRuntime` and `InProcessBridge`) -- `@tywrap/*` are runtime and infrastructure packages (e.g., `@tywrap/runtime-subprocess`, `@tywrap/runtime-inprocess`, `@tywrap/runtime-pyodide`) +- `@tywrapped/*` are public library wrappers (e.g., `@tywrapped/numpy`, exports `setRuntime` and wrapper APIs) +- `@tywrap/*` are runtime and infrastructure packages (e.g., `@tywrap/runtime-subprocess`, `@tywrap/runtime-inprocess`, `@tywrap/runtime-pyodide`) and export runtime implementations like `InProcessBridge` - `@tywrap/runtime-subprocess` - Bundled by default, no native dependencies - `@tywrap/runtime-inprocess` - Optional, requires node-calls-python native addon @@ -247,6 +311,15 @@ Packages declare supported runtimes in metadata: Visible in README badge, npm package metadata, and package documentation. +#### Validation & enforcement (template repo) + +- Automated validation: add a `browser-compat` CI job that runs bundler builds (esbuild/rollup/webpack) and Playwright smoke tests for packages that declare `runtimes: ["browser"]` (or `["node", "browser"]`). +- Enforcement at package boundary: + - Build-time: fail CI if declared runtimes don't have matching artifacts/entry points. + - Runtime: add a guard in Node-only entry points that throws a clear error when imported in the browser. +- Publish-time checks: lint `package.json` `runtimes`, generate README badges, and fail publish if declared runtime artifacts are missing. +- Pyodide handling: for browser-enabled packages, lazy-load Pyodide/WASM and keep heavy Python deps optional; include a headless-browser smoke test that imports the package and initializes the runtime. + --- ## Initial Launch Libraries @@ -286,10 +359,18 @@ Criteria for selection: - 15% - Bounties for new packages, test improvements, documentation - 5% - Reserve for legal/unexpected expenses +#### Governance and Fund Allocation + +- Decision body: technical committee + maintainers (define this in a governance charter). +- Maintainer stipends (60%): split using a simple, auditable formula (e.g., base stipend per maintained package + a variable component tied to monthly activity); publish the formula. +- Bounties (15%): bounties are proposed via issues, approved by the committee within a fixed SLA (e.g., 7 days), and paid on merge + release of the deliverable. +- Reserve (5%): only tapped for legal/incidents; require explicit approval (e.g., 2-of-3 committee vote) and a public rationale. +- Transparency: publish quarterly reports + a public ledger of disbursements; define a dispute-resolution path (mediation → committee decision → community escalation). + ### Trademark: Register Early - File trademark application before public launch -- Estimated cost: $250-400 per class +- Estimated cost (fees-only): ~$250-400 per class (USPTO filing fees); budget ~$1,200-3,000+ per class including search + attorney filing/prosecution and possible office actions - Protects brand from squatting - Establishes legitimacy for enterprise adoption - Required classes: software development tools, software distribution @@ -332,6 +413,16 @@ interface Transport extends Disposable { `BridgeProtocol` is already transport-agnostic - it accepts any `Transport` implementation. +#### Large-data handling notes + +The current `Transport` interface is string-based, which is great for simplicity but has practical limits for multi-GB tensors/DataFrames. + +High-level strategies (future work): +- Binary channel: add optional `sendBinary(...)` support (or extend `send(...)` to accept `ArrayBuffer`) and use Transferables where available. +- Out-of-band transfer: for very large payloads, send references (file paths, HTTP URLs, object-store URIs) instead of inlining data in the message. +- Streaming/chunking: add `streamStart`/`streamChunk`/`streamEnd` semantics with backpressure and `AbortSignal`. +- Capability negotiation: have transports advertise features (binary/streaming) via a meta call (e.g., `getBridgeInfo()` capabilities). + ### Adding node-calls-python Support **Recommended approach: JSON passthrough** From 0a361a99d19f10470abcd2a767c6929c2d17812f Mon Sep 17 00:00:00 2001 From: bbopen Date: Sat, 14 Feb 2026 07:13:56 -0800 Subject: [PATCH 2/2] docs: align RFC snippet with tywrap API --- docs/plans/2026-01-22-tywrapped-ecosystem-vision.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md index 1efebf9..ced24ae 100644 --- a/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md +++ b/docs/plans/2026-01-22-tywrapped-ecosystem-vision.md @@ -282,13 +282,12 @@ Quality: 🥇 Gold | Support: upstream (1.24+) | Runtimes: node, browser - Browser environment → Pyodide runtime - Node.js environment → Subprocess runtime (safest, no native deps) -**Override for power users:** +**Override for power users (current tywrap API):** ```typescript -import { setRuntime } from '@tywrapped/numpy'; -import { InProcessBridge } from '@tywrap/runtime-inprocess'; +import { setRuntimeBridge, NodeBridge } from 'tywrap'; -setRuntime(new InProcessBridge()); // Use node-calls-python for speed +setRuntimeBridge(new NodeBridge()); // Explicit Node.js subprocess runtime ``` ### Runtime Packages @@ -418,8 +417,8 @@ interface Transport extends Disposable { The current `Transport` interface is string-based, which is great for simplicity but has practical limits for multi-GB tensors/DataFrames. High-level strategies (future work): -- Binary channel: add optional `sendBinary(...)` support (or extend `send(...)` to accept `ArrayBuffer`) and use Transferables where available. -- Out-of-band transfer: for very large payloads, send references (file paths, HTTP URLs, object-store URIs) instead of inlining data in the message. +- Binary channel: add optional `sendBinary(...)` support (or extend `send(...)` to accept `ArrayBuffer`) and use `Transferable` objects where available. +- Out-of-band transfer: for large payloads, send references (file paths, HTTP URLs, object-store URIs) instead of inlining data. - Streaming/chunking: add `streamStart`/`streamChunk`/`streamEnd` semantics with backpressure and `AbortSignal`. - Capability negotiation: have transports advertise features (binary/streaming) via a meta call (e.g., `getBridgeInfo()` capabilities).