Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
469 changes: 469 additions & 0 deletions _bmad-output/implementation-artifacts/tech-spec-audit-trail-mvp.md

Large diffs are not rendered by default.

117 changes: 117 additions & 0 deletions docs/architecture/audit-trail.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Audit Trail

The `AuditModule` provides an append-only audit log for security-sensitive actions. It captures who did what, when, and from where — for compliance, incident investigation, and operational accountability.

## Architecture

```mermaid
flowchart LR
subgraph Interceptor Path
A["@Audited() decorator"] --> B[AuditInterceptor]
B -->|post-response tap| C[AuditService.Emit]
end

subgraph Direct Emit Path
D[AuthService] -->|fire-and-forget| C
end

C -->|enqueue| E[AUDIT queue]
E --> F[AuditProcessor]
F -->|em.fork + create + flush| G[(audit_log table)]
```

### Two Emission Paths

| Path | When | Context Source |
| --------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
| **Interceptor** | Standard authenticated endpoints (logout, sync, submissions, pipelines) | CLS (`CurrentUserService`, `RequestMetadataService`) with JWT payload fallback |
| **Direct emit** | Auth events where CLS context is unavailable (login success/failure, token refresh) | Explicit params from `AuthService` |

Both paths feed the same `AuditService.Emit()` method, which enqueues a job to the `AUDIT` BullMQ queue.

## AuditLog Entity

Append-only, immutable. Does **not** extend `CustomBaseEntity` (no `updatedAt`, no `deletedAt`). Follows the `SyncLog` precedent.

| Column | Type | Notes |
| --------------- | ------------------ | ------------------------------------------------------------------------------------------------- |
| `id` | `varchar` PK | UUID v4, auto-generated |
| `action` | `varchar` | Indexed. Dot-notation action code (e.g., `auth.login.success`) |
| `actorId` | `varchar` nullable | Indexed. Plain string, **not** a FK — survives user deletion |
| `actorUsername` | `varchar` nullable | Denormalized for historical accuracy |
| `resourceType` | `varchar` nullable | Entity name (e.g., `User`, `AnalysisPipeline`) |
| `resourceId` | `varchar` nullable | UUID of affected resource |
| `metadata` | `jsonb` nullable | Action-specific details (capped at 4KB from interceptor) |
| `browserName` | `varchar` nullable | From `MetaDataInterceptor` via CLS |
| `os` | `varchar` nullable | From `MetaDataInterceptor` via CLS |
| `ipAddress` | `varchar` nullable | From `x-forwarded-for` or socket |
| `occurredAt` | `timestamptz` | Indexed. Set from job payload (event time, not processing time). DB default `now()` as safety net |

Queries must use `filters: { softDelete: false }` to bypass the global soft-delete filter.

## MVP Actions

```typescript
export const AuditAction = {
AUTH_LOGIN_SUCCESS: 'auth.login.success',
AUTH_LOGIN_FAILURE: 'auth.login.failure',
AUTH_LOGOUT: 'auth.logout',
AUTH_TOKEN_REFRESH: 'auth.token.refresh',
ADMIN_SYNC_TRIGGER: 'admin.sync.trigger',
ADMIN_SYNC_SCHEDULE_UPDATE: 'admin.sync-schedule.update',
QUESTIONNAIRE_SUBMIT: 'questionnaire.submit',
QUESTIONNAIRE_INGEST: 'questionnaire.ingest',
QUESTIONNAIRE_SUBMISSIONS_WIPE: 'questionnaire.submissions.wipe',
ANALYSIS_PIPELINE_CREATE: 'analysis.pipeline.create',
ANALYSIS_PIPELINE_CONFIRM: 'analysis.pipeline.confirm',
ANALYSIS_PIPELINE_CANCEL: 'analysis.pipeline.cancel',
} as const;
```

## Interceptor Path Detail

Endpoints are tagged with the `@Audited({ action, resource? })` decorator, which sets Reflector metadata. The `AuditInterceptor` reads this metadata and, on successful response (RxJS `tap`, not `finalize`), enqueues an audit event.

Interceptor ordering matters: `MetaDataInterceptor` (IP/browser/OS) must run before `AuditInterceptor`. When `CurrentUserInterceptor` is present, it runs between them to populate the CLS user.

```typescript
@UseInterceptors(MetaDataInterceptor, CurrentUserInterceptor, AuditInterceptor)
```

The interceptor extracts `resourceId` from route params using a UUID v4 regex heuristic. Metadata captures route params and query params (not request body), capped at 4KB.

## Direct Emit Path Detail

Used in `AuthService` for login success, login failure, and token refresh. These events occur before JWT authentication is established, so CLS user context is unavailable.

- **Login success**: Emitted after the transaction returns, with `actorId`, `actorUsername`, and `strategyUsed` metadata.
- **Login failure**: Emitted after the transaction rejects, with `username` and a sanitized `reason` code (`no_matching_strategy` or `strategy_execution_failed`). Raw error messages are never persisted.
- **Token refresh**: Emitted after the transaction returns, with `actorId` and `actorUsername`.

All direct emits use `void this.auditService?.Emit(...)` — fire-and-forget, never inside a transaction.

## Queue & Processor

| Property | Value |
| ------------------ | --------------------------------- |
| Queue name | `audit` |
| Concurrency | 1 |
| Retry attempts | 1 (no retries) |
| `removeOnComplete` | `true` |
| `removeOnFail` | 100 (keep last 100 for debugging) |

The `AuditProcessor` extends `WorkerHost` directly (no HTTP dispatch). It forks the `EntityManager`, creates an `AuditLog` entity, and flushes. The `@OnWorkerEvent('failed')` handler logs non-PII fields only (no `metadata`).

## Module Design

`AuditModule` is `@Global()` — the only application module using this decorator. This makes `AuditService` and `AuditInterceptor` injectable everywhere without explicit imports. Justified because audit is a cross-cutting concern consumed by many modules.

`AuditService` is injected with `@Optional()` in `AuthService` to avoid making audit a hard dependency of authentication. All `Emit()` calls use optional chaining.

## Error Handling

Audit failures never break the request:

1. `AuditService.Emit()` wraps `queue.add()` in try/catch — logs a warning, returns void.
2. `AuditInterceptor` wraps the entire `tap` callback in try/catch — errors are logged, never propagated.
3. The `.catch()` on the `Emit()` promise handles async rejections.
13 changes: 12 additions & 1 deletion docs/architecture/core-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ classDiagram
DimensionsModule
FacultyModule
CurriculumModule
AuditModule
}

AppModule --> InfrastructureModules : "imports"
Expand All @@ -68,6 +69,8 @@ classDiagram
QuestionnaireModule --> CommonModule : "uses UnitOfWork"
AnalysisModule --> BullModule : "uses BullMQ queues"
AnalyticsModule --> BullModule : "uses BullMQ queue"
AuditModule --> BullModule : "uses BullMQ queue"
AuthModule --> AuditModule : "uses AuditService"
AnalyticsModule --> CommonModule : "uses ScopeResolverService"
FacultyModule --> CommonModule : "uses ScopeResolverService"
CurriculumModule --> CommonModule : "uses ScopeResolverService"
Expand Down Expand Up @@ -118,6 +121,13 @@ classDiagram
+IngestionEngine
+IngestionMapperService
}

class AuditModule {
<<Global>>
+AuditService
+AuditProcessor
+AuditInterceptor
}
```

## 4. Login Strategy Pattern
Expand Down Expand Up @@ -205,7 +215,7 @@ Each stage has a corresponding `RunStatus` (`PENDING` → `PROCESSING` → `COMP

### Queue Architecture

Six BullMQ queues with independent concurrency. Queue names are centralized in `src/configurations/common/queue-names.ts`.
Seven BullMQ queues with independent concurrency. Queue names are centralized in `src/configurations/common/queue-names.ts`.

| Queue | Processor | Concurrency Default | Module |
| ------------------- | --------------------------- | ------------------- | --------------- |
Expand All @@ -215,6 +225,7 @@ Six BullMQ queues with independent concurrency. Queue names are centralized in `
| `topic-model` | `TopicModelProcessor` | 1 | AnalysisModule |
| `recommendations` | `RecommendationsProcessor` | 1 | AnalysisModule |
| `analytics-refresh` | `AnalyticsRefreshProcessor` | 1 | AnalyticsModule |
| `audit` | `AuditProcessor` | 1 | AuditModule |

### REST Endpoints

Expand Down
30 changes: 30 additions & 0 deletions docs/decisions/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,36 @@ The `MoodleSyncScheduler` was rewritten from a static `@Cron(CronExpression.EVER
- **Soft-delete filter bypass:** Since `SyncLog` has no `deletedAt` column, MikroORM's global `softDelete` filter would fail at query time. Queries must use `filters: { softDelete: false }`. The `@Filter` decorator approach (`cond: {}, default: false`) was found to be insufficient at runtime.
- **Trade-off:** Admin schedule changes don't survive process restarts unless persisted to the database (which they are, via `SystemConfig`). The scheduler reads from DB on init, so restarts pick up the latest admin-configured interval.

## 34. Append-Only Audit Entity (No CustomBaseEntity)

The `AuditLog` entity does not extend `CustomBaseEntity`. It has no `updatedAt` or `deletedAt` — records are immutable and never soft-deleted. The `actorId` column is a plain string, not a `@ManyToOne` FK, so audit records survive user deletion.

- **Rationale:** Audit logs must be tamper-evident and permanent. Soft delete semantics would allow "hiding" audit records. FK constraints would cause cascade failures when users are deleted, creating a perverse incentive to retain user data solely for audit integrity.
- **Precedent:** Follows the `SyncLog` entity pattern. Queries must use `filters: { softDelete: false }` to bypass the global filter.
- **Trade-off:** No ORM-level relationship to `User` — joins require manual `actorId` matching. Acceptable because audit query endpoints (future) will use raw SQL or query builder, not entity relationships.

## 35. Global AuditModule with @Global() Decorator

`AuditModule` uses the `@Global()` class decorator — the only application module to do so. Infrastructure modules achieve global scope via config options (`isGlobal: true`), but `@Global()` is appropriate here because audit is a cross-cutting concern consumed by many modules.

- **Rationale:** Without `@Global()`, every module that uses `@Audited()` endpoints would need to explicitly import `AuditModule`. Since the interceptor is applied per-endpoint (not per-module), this friction discourages adoption with no compensating benefit.
- **Trade-off:** `AuditService` is injectable everywhere, which could lead to misuse. Mitigated by the fire-and-forget API — `Emit()` has no return value and catches all errors internally.

## 36. Dual Audit Emission Paths (Interceptor + Direct)

Audit events are captured through two paths: an interceptor for standard authenticated endpoints, and direct `AuditService.Emit()` calls for auth events.

- **Rationale:** The interceptor path requires CLS context (`CurrentUserService`, `RequestMetadataService`), which is unavailable during login (no JWT yet) and inconsistently available during token refresh. Rather than forcing all audit events through one path, two paths allow each context to use the most natural capture mechanism.
- **Convergence:** Both paths feed the same `AuditService.Emit()` → AUDIT queue → `AuditProcessor` → `audit_log` table pipeline. The entity schema is identical regardless of emission path.
- **Trade-off:** Two integration patterns to understand. Mitigated by clear separation — interceptor path is decorator-driven (declarative), direct path is explicit method calls in `AuthService` only.

## 37. Sanitized Audit Metadata (No Raw Error Messages)

Login failure audit events store a fixed reason code (`no_matching_strategy`, `strategy_execution_failed`) instead of the raw `error.message`.

- **Rationale:** Raw error messages may contain connection strings, hostnames, SQL fragments, or stack traces — especially from Moodle connectivity errors or database driver failures. Persisting these in an immutable, append-only table creates a permanent information disclosure risk.
- **Trade-off:** Less diagnostic detail in audit logs. Full error details are still available in application logs (which are rotatable and not permanent).

## 30. Semester Code Parsing for Display Labels

The Moodle category sync now parses semester codes (e.g., `S22526`) into human-readable `label` ("Semester 2") and `academicYear` ("2025-2026") fields on the `Semester` entity.
Expand Down
22 changes: 21 additions & 1 deletion docs/workflows/auth-hydration.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,11 @@ flowchart TD

G --> N[Issue JWT + RefreshToken]
L --> N
N --> O[200 OK Tokens]
N --> P[Audit: auth.login.success]
P --> O[200 OK Tokens]

J --> Q[Audit: auth.login.failure]
M --> Q
```

## Moodle Login Flow (Detail)
Expand Down Expand Up @@ -72,6 +76,22 @@ sequenceDiagram
AuthController-->>Client: 200 OK (Tokens)
```

## Audit Events

Auth events are captured via the direct emit path (not the interceptor) because CLS user context is unavailable during login. All emits are fire-and-forget (`void`) and occur **outside** the database transaction.

| Event | Action Code | When | Metadata |
| ------------------------------ | -------------------- | ---------------------------- | --------------------------------------------------- |
| Login success | `auth.login.success` | After transaction commits | `{ strategyUsed }` |
| Login failure (no strategy) | `auth.login.failure` | After transaction rejects | `{ username, reason: 'no_matching_strategy' }` |
| Login failure (strategy threw) | `auth.login.failure` | After transaction rejects | `{ username, reason: 'strategy_execution_failed' }` |
| Token refresh | `auth.token.refresh` | After transaction commits | _(none)_ |
| Logout | `auth.logout` | Via `@Audited()` interceptor | _(route params)_ |

`AuditService` is injected with `@Optional()` — auth works even if the audit module fails to bootstrap.

See [Audit Trail Architecture](../architecture/audit-trail.md) for the full audit system design.

## Institutional Role Resolution

The system detects institutional management roles from Moodle category capabilities. Roles have a `source` field (`auto` or `manual`) that determines whether hydration can manage them.
Expand Down
1 change: 1 addition & 0 deletions src/configurations/common/queue-names.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ export const QueueName = {
RECOMMENDATIONS: 'recommendations',
MOODLE_SYNC: 'moodle-sync',
ANALYTICS_REFRESH: 'analytics-refresh',
AUDIT: 'audit',
} as const;

export type QueueName = (typeof QueueName)[keyof typeof QueueName];
44 changes: 44 additions & 0 deletions src/entities/audit-log.entity.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import { Entity, Index, Opt, PrimaryKey, Property } from '@mikro-orm/core';
import { v4 } from 'uuid';

// Audit records are never soft-deleted. Queries must use
// `filters: { softDelete: false }` to bypass the global filter.
// See SyncLog for precedent.
@Entity()
export class AuditLog {
@PrimaryKey()
id: string & Opt = v4();

@Index()
@Property()
action!: string;

@Index()
@Property({ nullable: true })
actorId?: string;

@Property({ nullable: true })
actorUsername?: string;

@Property({ nullable: true })
resourceType?: string;

@Property({ nullable: true })
resourceId?: string;

@Property({ type: 'jsonb', nullable: true })
metadata?: Record<string, unknown>;

@Property({ nullable: true })
browserName?: string;

@Property({ nullable: true })
os?: string;

@Property({ nullable: true })
ipAddress?: string;

@Index()
@Property()
occurredAt!: Date;
}
3 changes: 3 additions & 0 deletions src/entities/index.entity.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import { TopicAssignment } from './topic-assignment.entity';
import { Section } from './section.entity';
import { TopicModelRun } from './topic-model-run.entity';
import { SyncLog } from './sync-log.entity';
import { AuditLog } from './audit-log.entity';

export {
ChatKitThread,
Expand Down Expand Up @@ -64,6 +65,7 @@ export {
TopicAssignment,
TopicModelRun,
SyncLog,
AuditLog,
};

export const entities = [
Expand Down Expand Up @@ -99,4 +101,5 @@ export const entities = [
TopicAssignment,
TopicModelRun,
SyncLog,
AuditLog,
];
Loading
Loading