Unified Type Architecture for API Contracts and Internal Types

[cqnykamp]

Proposal: Unified Type Architecture for API Contracts and Internal Types

Summary

This proposal establishes a three-layer type architecture that eliminates duplicate type definitions while cleanly separating API contracts (wire format) from server internal types (database/Prisma format). This provides a single source of truth for API types while preserving type safety for server-internal operations.

Problem Statement

Current Issues

1. Type Duplication: UserInfo, ContentType, and related types are defined identically in both client/src/types.ts and server/src/types.ts, violating DRY principles
2. Unclear Boundaries: Server code mixes internal types (Uuid = Uint8Array) with API-level code, with runtime conversion via convertUUID() but no type-level distinction
3. Maintenance Burden: Changes to API contracts require updating multiple files and keeping them manually synchronized
4. Type Safety Gaps: The convertUUID() function transforms Uint8Array → string at runtime, but TypeScript doesn't track this transformation

Current Architecture

┌─────────────────────────┐    ┌─────────────────────────┐
│  client/src/types.ts    │    │  server/src/types.ts    │
│  ─────────────────────  │    │  ─────────────────────  │
│  Uuid = string          │    │  Uuid = Uint8Array      │
│  UserInfo { userId }    │    │  UserInfo { userId }    │
│  ContentType = ...      │    │  ContentType = ...      │
└─────────────────────────┘    └─────────────────────────┘
         ▲                                ▲
         │                                │
         │                                │ convertUUID()
         └──── DUPLICATED TYPES ──────────┘

Proposed Solution

Three-Layer Type Architecture

                    ┌──────────────────────────────┐
                    │  Shared API Types            │
                    │  (shared/src/apiTypes.ts)    │
                    │  ─────────────────────────── │
                    │  Wire format contracts       │
                    │  Uuid = string               │
                    │  DoenetDateTime = string     │
                    │  UserInfo, ContentType, etc. │
                    └──────────────────────────────┘
                               ▲       ▲
                    imports    │       │    imports
               ┌───────────────┘       └───────────────┐
               │                                       │
   ┌───────────────────────┐           ┌──────────────────────────┐
   │  Client Types         │           │  Server API Layer        │
   │  (client/src/types)   │           │  (routes/middleware)     │
   │  ──────────────────── │           │  ──────────────────────  │
   │  Re-exports API types │           │  Uses API types          │
   │  (same as wire)       │           │  for responses           │
   └───────────────────────┘           └──────────────────────────┘
                                                      │
                                         Converts    │
                                                     ▼
                                       ┌──────────────────────────┐
                                       │  Server Internal Types   │
                                       │  (server/src/internal)   │
                                       │  ──────────────────────  │
                                       │  Prisma/DB format        │
                                       │  Uuid = Uint8Array       │
                                       │  DoenetDateTime = Date   │
                                       │  UserInfoInternal, etc.  │
                                       └──────────────────────────┘

Implementation Structure

1. Shared API Types (shared/src/apiTypes.ts)

Purpose: Single source of truth for all API contracts
Audience: Both client and server (at API boundaries)

/**
 * API Types - Wire Format Contracts
 * 
 * These types represent the data format used in HTTP requests/responses.
 * All modules import these types when working with API data.
 */

/** UUID as string (RFC 4122 format) */
export type Uuid = string;

/** ISO 8601 date-time string */
export type DoenetDateTime = string;

/** User information as sent over the wire */
export type UserInfo = {
  userId: Uuid;
  firstNames: string | null;
  lastNames: string;
  isAnonymous?: boolean;
  numLibrary?: number;
  numCommunity?: number;
  isMaskForLibrary?: boolean;
};

export function isUserInfo(obj: unknown): obj is UserInfo {
  const typedObj = obj as UserInfo;
  return (
    typedObj !== null &&
    typeof typedObj === "object" &&
    typeof typedObj.userId === "string" &&
    (typedObj.firstNames === null || typeof typedObj.firstNames === "string") &&
    typeof typedObj.lastNames === "string"
  );
}

export type ContentType = "singleDoc" | "select" | "sequence" | "folder";

export function isContentType(type: unknown): type is ContentType {
  return (
    typeof type === "string" &&
    ["singleDoc", "select", "sequence", "folder"].includes(type)
  );
}

// ... other API types

2. Client Types (client/src/types.ts)

Purpose: Re-export shared API types, add client-only types
Change: Minimal - just import and re-export

/**
 * Client Types
 * 
 * For client code, API types are sufficient since all data comes from the server API.
 * This module re-exports shared API types and defines client-specific extensions.
 */

// Re-export all API types
export {
  type ContentType,
  type DoenetDateTime,
  type UserInfo,
  type Uuid,
  isContentType,
  isUserInfo,
} from "@doenet-tools/shared";

// Client-specific types remain here
export type UserInfoWithEmail = UserInfo & {
  email: string | null;
  isAuthor?: boolean;
  isEditor?: boolean;
};

// ... other client-specific types

3. Server Internal Types (server/src/typesInternal.ts) - NEW

Purpose: Define server-internal types that match Prisma query results
Audience: Server query functions, internal business logic

/**
 * Server Internal Types
 * 
 * These types match the Prisma query results and internal database representations.
 * - UUIDs are stored as Uint8Array (Binary(16) in MySQL)
 * - Dates are Date objects
 * 
 * Query functions work with these types internally and convert to API types
 * at the boundary (routes/middleware).
 */

/** Internal UUID representation (binary) */
export type InternalUuid = Uint8Array;

/** Internal date-time representation */
export type InternalDoenetDateTime = Date;

/** User information as stored in database */
export type UserInfoInternal = {
  userId: InternalUuid;
  firstNames: string | null;
  lastNames: string;
  isAnonymous?: boolean;
};

export type UserInfoWithEmailInternal = UserInfoInternal & {
  email: string | null;
  isAuthor?: boolean;
  isEditor?: boolean;
};

// Helper type to transform API types to internal types
export type ToInternal<T> = T extends string & { __uuidBrand: true }
  ? InternalUuid
  : T extends string & { __dateTimeBrand: true }
  ? InternalDoenetDateTime
  : T extends Array<infer U>
  ? Array<ToInternal<U>>
  : T extends object
  ? { [K in keyof T]: ToInternal<T[K]> }
  : T;

// Alternative: explicitly define internal versions of all compound types
export type ContentInfoInternal = {
  contentId: InternalUuid;
  ownerId: InternalUuid;
  createdAt: InternalDoenetDateTime;
  // ... other fields
};

4. Type-Safe Conversion Layer (server/src/utils/typeConverters.ts) - NEW

Purpose: Provide type-safe conversions between internal and API types

import {
  UserInfo,
  type Uuid as ApiUuid,
  type DoenetDateTime as ApiDoenetDateTime,
} from "@doenet-tools/shared";
import {
  UserInfoInternal,
  type InternalUuid,
  type InternalDoenetDateTime,
} from "../typesInternal";
import { fromBinaryUUID, toBinaryUUID } from "./binary-uuid";

/**
 * Convert internal UUID to API UUID
 */
export function toApiUuid(internal: InternalUuid): ApiUuid {
  return fromBinaryUUID(internal);
}

/**
 * Convert API UUID to internal UUID
 */
export function toInternalUuid(api: ApiUuid): InternalUuid {
  return toBinaryUUID(api);
}

/**
 * Convert internal DateTime to API DateTime
 */
export function toApiDateTime(internal: InternalDoenetDateTime): ApiDoenetDateTime {
  return internal.toISOString();
}

/**
 * Convert API DateTime to internal DateTime
 */
export function toInternalDateTime(api: ApiDoenetDateTime): InternalDoenetDateTime {
  return new Date(api);
}

/**
 * Convert internal UserInfo to API UserInfo
 */
export function toApiUserInfo(internal: UserInfoInternal): UserInfo {
  return {
    userId: toApiUuid(internal.userId),
    firstNames: internal.firstNames,
    lastNames: internal.lastNames,
    isAnonymous: internal.isAnonymous,
  };
}

/**
 * Generic deep conversion for any object (maintains backward compatibility)
 * This is the enhanced version of the current convertUUID function.
 */
export function toApiFormat(obj: unknown): unknown {
  if (obj instanceof Uint8Array) {
    return toApiUuid(obj);
  }
  if (obj instanceof Date) {
    return toApiDateTime(obj);
  }
  if (Array.isArray(obj)) {
    return obj.map(toApiFormat);
  }
  if (obj && typeof obj === "object") {
    return Object.fromEntries(
      Object.entries(obj).map(([key, val]) => [key, toApiFormat(val)])
    );
  }
  return obj;
}

5. Server Query Functions

Change: Use *Internal types, return internal format

import { UserInfoInternal } from '../typesInternal';
import { InternalUuid } from '../typesInternal';

/**
 * Query functions work with internal types
 */
export async function getAuthorInfo(userId: InternalUuid): Promise<UserInfoInternal> {
  return await prisma.users.findUniqueOrThrow({
    where: { userId },
    select: {
      userId: true,
      firstNames: true,
      lastNames: true,
    },
  });
}

export async function getMyUserInfo({
  loggedInUserId,
}: {
  loggedInUserId: InternalUuid;
}): Promise<{ user: UserInfoInternal }> {
  const user = await prisma.users.findUniqueOrThrow({
    where: { userId: loggedInUserId },
    select: {
      userId: true,
      firstNames: true,
      lastNames: true,
      isAnonymous: true,
    },
  });
  return { user };
}

6. Server Routes/Middleware

Change: Import API types from shared, convert at boundary

import { UserInfo } from '@doenet-tools/shared'; // API type
import { toApiFormat } from '../utils/typeConverters';

type LoggedInUser = {
  loggedInUserId: InternalUuid; // Internal type for middleware
};

export function queryLoggedIn<T extends z.ZodTypeAny>(
  query: (params: z.infer<T> & LoggedInUser) => unknown,
  schema: T,
) {
  return async (req: Request, res: Response) => {
    if (!req.user) {
      res.status(StatusCodes.FORBIDDEN).json({ error: "Must be logged in" });
    } else {
      try {
        const loggedInUserId = req.user.userId; // Uint8Array from session
        const params = schema.parse({ ...req.body, ...req.query, ...req.params });
        
        // Query returns internal types
        const internalResult = await query({ loggedInUserId, ...params });
        
        // Convert to API types before sending
        const apiResult = toApiFormat(internalResult);
        res.send(apiResult);
      } catch (e) {
        handleErrors(res, e);
      }
    }
  };
}

Benefits

1. Single Source of Truth

• API types defined once in shared/src/apiTypes.ts
• No more duplicate definitions across modules
• Changes propagate automatically to all consumers

2. Type Safety

• Explicit distinction between internal and API types
• Conversion functions provide type-safe boundaries
• TypeScript catches mismatches at compile time

3. Clear Architecture

• Three distinct layers with clear responsibilities
• Obvious where conversions happen (middleware/routes)
• Internal implementation details isolated from API contracts

4. Maintainability

• Adding new API types requires updating only one location
• Server refactoring doesn't affect API contracts
• Easier to reason about data flow

5. Documentation

• Types themselves document the data formats
• Clear distinction between "what's sent over the wire" vs "what's in the database"

Migration Path

Phase 1: Setup (Low Risk)

1. Create shared/src/apiTypes.ts with API types
2. Update client/src/types.ts to import from shared
3. Update tests-cypress to import from shared

Phase 2: Server Internal Types (Medium Risk)

4. Create server/src/typesInternal.ts with internal type definitions
5. Create server/src/utils/typeConverters.ts with conversion functions
6. Update a single query function + route as proof of concept
7. Validate that conversion works correctly

Phase 3: Gradual Migration (Incremental)

8. Create migration checklist of all query functions
9. Update query functions module-by-module:
   • User queries
   • Content queries
   • Assignment queries
   • etc.
10. Update routes to use API types for responses
11. Run tests after each module migration

Phase 4: Cleanup (Final)

12. Remove duplicate definitions from server/src/types.ts
13. Update documentation
14. Remove old convertUUID in favor of toApiFormat

Risks and Mitigations

Risk: Large Migration Scope

Mitigation: Incremental migration by module, validate after each step

Risk: Breaking Changes

Mitigation: Comprehensive test coverage, backward compatibility layer during migration

Risk: Performance Impact

Mitigation: Conversion functions are lightweight, minimal overhead

Risk: Developer Confusion

Mitigation: Clear documentation, naming conventions (*Internal suffix), examples

Alternative Considered: Status Quo

Keep current architecture: Server and client maintain separate type definitions

Pros: No migration needed, works today
Cons: Duplicate definitions, manual synchronization, type safety gaps

Decision: Rejected - technical debt outweighs migration cost

Success Criteria

1. ✅ Zero duplicate type definitions across modules
2. ✅ All API types imported from shared/src/apiTypes.ts
3. ✅ Server internal types explicitly separate from API types
4. ✅ Type-safe conversions at all boundaries
5. ✅ All existing tests pass
6. ✅ No performance regression

Open Questions

1. Should we migrate all types at once or start with high-value types (UserInfo, Content)?
2. Do we need backward compatibility during migration (e.g., both old and new imports work)?
3. Should internal types live in typesInternal.ts or co-located with query modules?
4. Do we want branded types (nominal typing) for Uuid/DateTime to prevent mixing?