Skip to content
This repository was archived by the owner on Apr 7, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions src/gateway/proxy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,11 @@ import type { GatewayAuthEnv } from "./service-key-auth.js";
import { proxySSEStream } from "./streaming.js";
import type { FetchFn, GatewayConfig, ProviderConfig } from "./types.js";

const DEFAULT_MARGIN = 1.3;
/**
* Fallback only used when resolveMargin is not provided (tests only).
* Production MUST provide resolveMargin — mountRoutes enforces this.
*/
const TEST_ONLY_MARGIN = 1.3;

/** Max call duration cap: 4 hours = 240 minutes. */
const MAX_CALL_DURATION_MINUTES = 240;
Expand Down Expand Up @@ -96,7 +100,10 @@ export function buildProxyDeps(config: GatewayConfig): ProxyDeps {
providers: config.providers,
defaultModel: config.defaultModel,
resolveDefaultModel: config.resolveDefaultModel,
defaultMargin: config.defaultMargin ?? DEFAULT_MARGIN,
get defaultMargin() {
if (config.resolveMargin) return config.resolveMargin();
return config.defaultMargin ?? TEST_ONLY_MARGIN;
},
fetchFn: config.fetchFn ?? fetch,
arbitrageRouter: config.arbitrageRouter,
rateLookupFn: config.rateLookupFn,
Expand Down
4 changes: 3 additions & 1 deletion src/gateway/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,10 @@ export interface GatewayConfig {
graceBufferCents?: number;
/** Upstream provider credentials */
providers: ProviderConfig;
/** Default margin multiplier (default: 1.3 = 30%) */
/** Static margin (for tests only). Production should use resolveMargin. */
defaultMargin?: number;
/** Live margin resolver — called per-request, reads from DB. Takes priority over defaultMargin. */
resolveMargin?: () => number;
/** Optional arbitrage router for multi-provider cost optimization (WOP-463) */
arbitrageRouter?: import("../monetization/arbitrage/router.js").ArbitrageRouter;
/** Injectable fetch for testing */
Expand Down
4 changes: 4 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,7 @@ export * from "./tenancy/index.js";

// tRPC
export * from "./trpc/index.js";
// monorepo e2e cutover test
// hybrid dockerfile e2e
// sequential build test
// lockfile build
Comment on lines +51 to +54
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Leftover CI breadcrumb comments in public barrel

These four lines look like temporary markers left over from debugging the build pipeline and should not be committed to the public export barrel:

Suggested change
// monorepo e2e cutover test
// hybrid dockerfile e2e
// sequential build test
// lockfile build
// tRPC
export * from "./trpc/index.js";
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/index.ts
Line: 51-54

Comment:
**Leftover CI breadcrumb comments in public barrel**

These four lines look like temporary markers left over from debugging the build pipeline and should not be committed to the public export barrel:

```suggestion
// tRPC
export * from "./trpc/index.js";
```

How can I resolve this? If you propose a fix, please make it concise.

6 changes: 5 additions & 1 deletion src/server/__tests__/container.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ describe("createTestContainer", () => {

const gateway: GatewayServices = {
serviceKeyRepo: {} as never,
meter: {} as never,
budgetChecker: {} as never,
};

const hotPool: HotPoolServices = {
Expand All @@ -189,7 +191,9 @@ describe("createTestContainer", () => {
});

it("overrides merge without affecting other defaults", () => {
const c = createTestContainer({ gateway: { serviceKeyRepo: {} as never } });
const c = createTestContainer({
gateway: { serviceKeyRepo: {} as never, meter: {} as never, budgetChecker: {} as never },
});

// Overridden field
expect(c.gateway).not.toBeNull();
Expand Down
10 changes: 9 additions & 1 deletion src/server/container.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ export interface StripeServices {

export interface GatewayServices {
serviceKeyRepo: IServiceKeyRepository;
meter: import("../metering/emitter.js").MeterEmitter;
budgetChecker: import("../monetization/budget/budget-checker.js").IBudgetChecker;
Comment on lines +54 to +55
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 meter typed as concrete class instead of its interface

IMeterEmitter is exported from src/metering/index.ts and matches the convention that sub-container fields take repository/emitter interfaces, not implementations. budgetChecker correctly uses IBudgetChecker; meter should be consistent:

Suggested change
meter: import("../metering/emitter.js").MeterEmitter;
budgetChecker: import("../monetization/budget/budget-checker.js").IBudgetChecker;
meter: import("../metering/emitter.js").IMeterEmitter;
budgetChecker: import("../monetization/budget/budget-checker.js").IBudgetChecker;

Rule Used: WOPR codebase conventions:

  • Repository pattern is... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/server/container.ts
Line: 54-55

Comment:
**`meter` typed as concrete class instead of its interface**

`IMeterEmitter` is exported from `src/metering/index.ts` and matches the convention that sub-container fields take repository/emitter *interfaces*, not implementations. `budgetChecker` correctly uses `IBudgetChecker`; `meter` should be consistent:

```suggestion
  meter: import("../metering/emitter.js").IMeterEmitter;
  budgetChecker: import("../monetization/budget/budget-checker.js").IBudgetChecker;
```

**Rule Used:** WOPR codebase conventions:
- Repository pattern is... ([source](https://app.greptile.com/review/custom-context?memory=158cf84c-c9a3-4390-b104-b5df2c2423d6))

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

}

export interface HotPoolServices {
Expand Down Expand Up @@ -239,8 +241,14 @@ export async function buildContainer(bootConfig: BootConfig): Promise<PlatformCo
let gateway: GatewayServices | null = null;
if (bootConfig.features.gateway) {
const { DrizzleServiceKeyRepository } = await import("../gateway/service-key-repository.js");
const { MeterEmitter } = await import("../metering/emitter.js");
const { DrizzleMeterEventRepository } = await import("../metering/meter-event-repository.js");
const { DrizzleBudgetChecker } = await import("../monetization/budget/budget-checker.js");

const serviceKeyRepo: IServiceKeyRepository = new DrizzleServiceKeyRepository(db as never);
gateway = { serviceKeyRepo };
const meter = new MeterEmitter(new DrizzleMeterEventRepository(db as never), { flushIntervalMs: 5_000 });
const budgetChecker = new DrizzleBudgetChecker(db as never, { cacheTtlMs: 30_000 });
gateway = { serviceKeyRepo, meter, budgetChecker };
}

// 12. Build the container (hotPool bound after construction)
Expand Down
2 changes: 1 addition & 1 deletion src/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ export async function bootPlatformServer(config: BootConfig): Promise<BootResult
const container = await buildContainer(config);
const app = new Hono();

mountRoutes(
await mountRoutes(
app,
container,
{
Expand Down
44 changes: 41 additions & 3 deletions src/server/mount-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,12 @@ export interface MountConfig {
* 6. Product-specific route plugins
* 7. Tenant proxy middleware (catch-all — must be last)
*/
export function mountRoutes(
export async function mountRoutes(
app: Hono,
container: PlatformContainer,
config: MountConfig,
plugins: RoutePlugin[] = [],
): void {
): Promise<void> {
// 1. CORS middleware
const origins = deriveCorsOrigins(container.productConfig.product, container.productConfig.domains);
app.use(
Expand Down Expand Up @@ -95,7 +95,45 @@ export function mountRoutes(
);
}

// 6. Product-specific route plugins
// 6. Metered inference gateway (when gateway is enabled)
if (container.gateway) {
// Validate billing config exists in DB — fail hard, no silent defaults
const billingConfig = container.productConfig.billing;
const marginConfig = billingConfig?.marginConfig as { default?: number } | null;
if (!marginConfig?.default) {
Comment on lines +102 to +103
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Low server/mount-routes.ts:102

The check !marginConfig?.default at line 103 treats a margin of 0 as "not set" and throws an error, even though 0 is a valid configuration value (e.g., for testing or promotional pricing). Consider using typeof marginConfig?.default !== 'number' to properly distinguish between unset and zero values.

    const marginConfig = billingConfig?.marginConfig as { default?: number } | null;
-    if (!marginConfig?.default) {
+    if (typeof marginConfig?.default !== 'number') {
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file src/server/mount-routes.ts around lines 102-103:

The check `!marginConfig?.default` at line 103 treats a margin of `0` as "not set" and throws an error, even though `0` is a valid configuration value (e.g., for testing or promotional pricing). Consider using `typeof marginConfig?.default !== 'number'` to properly distinguish between unset and zero values.

Evidence trail:
src/server/mount-routes.ts lines 102-107 at REVIEWED_COMMIT: The check `if (!marginConfig?.default)` on line 103 uses JavaScript falsy check which treats 0 as falsy, causing a valid margin value of 0 to trigger the error. The type annotation `{ default?: number }` on line 102 confirms that `default` is expected to be a number, and 0 is a valid number value.

throw new Error(
"Gateway enabled but product_billing_config.margin_config.default is not set. " +
"Seed the DB: INSERT INTO product_billing_config (product_id, margin_config) VALUES ('<id>', '{\"default\": 4.0}')",
);
}
Comment on lines +103 to +108
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Margin config convention violated — must use MARGIN_CONFIG_JSON env var

Every WOPR codebase convention rule explicitly states: "Margin config is stored in MARGIN_CONFIG_JSON env var, never in source" (with loadMarginConfig() in src/monetization/adapters/margin-config.ts as the canonical loader). This PR moves the margin to product_billing_config.margin_config.default in the database instead, which is an intentional but undiscussed departure from the team-wide convention.

If the intent is to permanently migrate to DB-driven margin, the custom instruction rules and loadMarginConfig() path should be updated in the same PR so no future author is misled. If this is a product-specific override on top of the env-var baseline, the env-var path should still be respected as a fallback.

Rule Used: WOPR codebase conventions:

  • Repository pattern is... (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/server/mount-routes.ts
Line: 103-108

Comment:
**Margin config convention violated — must use `MARGIN_CONFIG_JSON` env var**

Every WOPR codebase convention rule explicitly states: *"Margin config is stored in `MARGIN_CONFIG_JSON` env var, never in source"* (with `loadMarginConfig()` in `src/monetization/adapters/margin-config.ts` as the canonical loader). This PR moves the margin to `product_billing_config.margin_config.default` in the database instead, which is an intentional but undiscussed departure from the team-wide convention.

If the intent is to permanently migrate to DB-driven margin, the custom instruction rules and `loadMarginConfig()` path should be updated in the same PR so no future author is misled. If this is a product-specific override on top of the env-var baseline, the env-var path should still be respected as a fallback.

**Rule Used:** WOPR codebase conventions:
- Repository pattern is... ([source](https://app.greptile.com/review/custom-context?memory=b986c674-3c9f-4f81-9c16-19366bd8a16f))

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +103 to +108
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Falsy guard rejects a 0 margin

!marginConfig?.default is true when default is 0, which would throw even though 0 is a valid (if degenerate) numeric value. Prefer an explicit null/undefined check:

Suggested change
if (!marginConfig?.default) {
throw new Error(
"Gateway enabled but product_billing_config.margin_config.default is not set. " +
"Seed the DB: INSERT INTO product_billing_config (product_id, margin_config) VALUES ('<id>', '{\"default\": 4.0}')",
);
}
if (marginConfig?.default == null) {
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/server/mount-routes.ts
Line: 103-108

Comment:
**Falsy guard rejects a `0` margin**

`!marginConfig?.default` is `true` when `default` is `0`, which would throw even though `0` is a valid (if degenerate) numeric value. Prefer an explicit null/undefined check:

```suggestion
    if (marginConfig?.default == null) {
```

How can I resolve this? If you propose a fix, please make it concise.


// Live margin — reads from productConfig per-request (DB-cached with TTL)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium server/mount-routes.ts:110

The resolveMargin closure reads from container.productConfig.billing?.marginConfig, which is a static snapshot captured at startup. Because the comment explicitly claims "Live margin — reads from productConfig per-request (DB-cached with TTL)", the implementation contradicts its own documentation: margin changes in the database will never be reflected during the process lifetime. Consider reading from container.productConfigService.getBySlug() instead to actually leverage the TTL-cached service.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file src/server/mount-routes.ts around line 110:

The `resolveMargin` closure reads from `container.productConfig.billing?.marginConfig`, which is a static snapshot captured at startup. Because the comment explicitly claims "Live margin — reads from productConfig per-request (DB-cached with TTL)", the implementation contradicts its own documentation: margin changes in the database will never be reflected during the process lifetime. Consider reading from `container.productConfigService.getBySlug()` instead to actually leverage the TTL-cached service.

Evidence trail:
src/server/mount-routes.ts lines 110-115 (REVIEWED_COMMIT): comment claims 'Live margin — reads from productConfig per-request (DB-cached with TTL)' but resolveMargin reads `container.productConfig.billing?.marginConfig`. src/product-config/boot.ts lines 39-45: platformBoot returns `{ service, config }` where config is `await service.getBySlug(slug)` - a one-time fetch. src/server/container.ts line 144: `const { config: productConfig, service: productConfigService } = await platformBoot(...)`. src/product-config/service.ts lines 37-49: ProductConfigService.getBySlug() has TTL-cached DB reads (default 60s). container.productConfig is static; container.productConfigService.getBySlug() is TTL-cached.

const initialMargin = marginConfig.default;
const resolveMargin = (): number => {
const cfg = container.productConfig.billing?.marginConfig as { default?: number } | null;
return cfg?.default ?? initialMargin;
};
Comment on lines +110 to +115
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 resolveMargin reads a stale boot-time snapshot, not live DB values

container.productConfig is a plain ProductConfig object set once when buildContainer runs and is never mutated or reassigned afterward. The closure therefore always returns the margin captured at startup — identical to initialMargin. The ProductConfigService (with its TTL cache) is available as container.productConfigService, but it is not used here.

The PR description claims "resolveMargin getter for runtime DB changes without restart", but since container.productConfig is a frozen snapshot, this does not work. An operator who updates margin_config.default in the DB mid-run will see no effect until the server restarts.

To make the margin truly live the resolver must call container.productConfigService.getBySlug(slug) and await it; but since resolveMargin is typed as () => number (sync), this requires either making the type () => Promise<number> throughout the gateway, or adding a separate background-refresh loop that pushes the cached value into a mutable local variable. As-is, the "live margin" feature is a no-op.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/server/mount-routes.ts
Line: 110-115

Comment:
**`resolveMargin` reads a stale boot-time snapshot, not live DB values**

`container.productConfig` is a plain `ProductConfig` object set once when `buildContainer` runs and is **never mutated or reassigned** afterward. The closure therefore always returns the margin captured at startup — identical to `initialMargin`. The `ProductConfigService` (with its TTL cache) is available as `container.productConfigService`, but it is not used here.

The PR description claims "resolveMargin getter for runtime DB changes without restart", but since `container.productConfig` is a frozen snapshot, this does not work. An operator who updates `margin_config.default` in the DB mid-run will see no effect until the server restarts.

To make the margin truly live the resolver must call `container.productConfigService.getBySlug(slug)` and await it; but since `resolveMargin` is typed as `() => number` (sync), this requires either making the type `() => Promise<number>` throughout the gateway, or adding a separate background-refresh loop that pushes the cached value into a mutable local variable. As-is, the "live margin" feature is a no-op.

How can I resolve this? If you propose a fix, please make it concise.


const gw = container.gateway;
const { mountGateway } = await import("../gateway/index.js");
mountGateway(app, {
meter: gw.meter,
budgetChecker: gw.budgetChecker,
creditLedger: container.creditLedger,
resolveMargin,
providers: {
openrouter: process.env.OPENROUTER_API_KEY
? { apiKey: process.env.OPENROUTER_API_KEY, baseUrl: process.env.OPENROUTER_BASE_URL || undefined }
: undefined,
},
resolveServiceKey: async (key: string) => {
const tenant = await gw.serviceKeyRepo.resolve(key);
return tenant ?? null;
},
});
}

// 7. Product-specific route plugins
for (const plugin of plugins) {
app.route(plugin.path, plugin.handler(container));
}
Expand Down
6 changes: 3 additions & 3 deletions src/server/routes/__tests__/admin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ describe("createAdminRouter", () => {
},
serviceKeyRepo: {} as never,
},
gateway: { serviceKeyRepo: {} as never, meter: {} as never, budgetChecker: {} as never },
});

const caller = makeCaller(container);
Expand Down Expand Up @@ -220,6 +221,7 @@ describe("createAdminRouter", () => {
},
serviceKeyRepo: {} as never,
},
gateway: { serviceKeyRepo: {} as never, meter: {} as never, budgetChecker: {} as never },
});

const caller = makeCaller(container);
Expand Down Expand Up @@ -273,9 +275,7 @@ describe("createAdminRouter", () => {

const container = createTestContainer({
pool: mockPool as never,
gateway: {
serviceKeyRepo: {} as never,
},
gateway: { serviceKeyRepo: {} as never, meter: {} as never, budgetChecker: {} as never },
});

const caller = makeCaller(container);
Expand Down
Loading