Skip to content

feat: production-ready refactor — Neon PostgreSQL + Drizzle + multi-tenant RLS#1

Open
Alemusica wants to merge 26 commits intomainfrom
feat/production-ready-refactor
Open

feat: production-ready refactor — Neon PostgreSQL + Drizzle + multi-tenant RLS#1
Alemusica wants to merge 26 commits intomainfrom
feat/production-ready-refactor

Conversation

@Alemusica
Copy link
Copy Markdown
Collaborator

@Alemusica Alemusica commented Apr 2, 2026

Summary

Full production-ready refactor of flutur-ops MCP server:

  • Database: SurrealDB → Neon PostgreSQL (pgvector/pg16), Drizzle ORM with 27 tables + 3 junction tables, pgvector HNSW indexes (384d)
  • Multi-tenancy: Row-Level Security with FORCE ROW LEVEL SECURITY on all 24 tenant-scoped tables, withTenant() session-based isolation
  • Architecture: Monolithic mcp-server.ts → modular src/server/tools/*.ts (7 domain files) + src/services/ (MCP-independent, testable)
  • Infrastructure: Docker Compose dev, daily pg_dump backup (launchd), Sentry monitoring (opt-in), multi-stage Dockerfile

What changed

Layer Before After
DB SurrealDB (surreal cloud) Neon PostgreSQL + pgvector (Docker dev, Neon prod)
ORM Raw SurrealQL strings Drizzle ORM (type-safe, auto-migrations)
Schema .surql files src/db/schema.ts (27 tables, 5 enums, 3 HNSW indexes)
Multi-tenant None RLS policies + withTenant() session var
MCP tools 1 giant file (2000+ lines) 7 domain files (43 tools total)
Business logic Mixed in MCP handlers src/services/ (6 domains, reusable)
Logging console.log Structured JSON, stderr-only (stdio-safe)
Credentials dotenv macOS Keychain via security CLI
Email guard SurrealDB queries Drizzle + fail-closed, 55/day limit
Telegram bot Direct API calls Thin client → services layer

Verification

  • 17 tests passing (6 test files)
  • 0 TypeScript errors in new code
  • MCP server starts, lists 43 tools
  • RLS tenant isolation verified (flutur_app non-superuser)
  • Backup script works (pg_dump via Docker)
  • Docker Compose healthy (pgvector:pg16 on :5433)

Test plan

  • npx vitest run — 17/17 pass
  • npx tsc --noEmit — 0 errors in src/{server,services,lib,config}/
  • MCP server stdio init + tools/list — 43 tools
  • RLS isolation via psql with non-superuser role
  • pg_dump backup round-trip
  • Post-merge: run npx tsx scripts/migrate-from-surreal.ts to migrate live data

🤖 Generated with Claude Code

alelab and others added 26 commits April 2, 2026 09:49
@alelab @claude
feat: 10-card newspaper component system + SILO-inspired CSS
3148070 
- 3 shared primitives (NpHeadline, NpFigure, NpBar) enforce uniform anatomy
- 10 typed cards: Article, Video, Quote, Credential, Press, Tech, Show, Gallery, CTA, Footer
- Grid uses gap+background pattern for perfectly equal column widths
- Single font (Space Grotesk) — hierarchy via size only, weight 500 max
- Credential/Tech/Press cards with SILO-derived list patterns
- EPKGrid rewritten as pure card composition — zero inline markup
- Performance modes split into individual ArticleCards

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: add Docker Compose (pgvector/pg16) + Drizzle ORM dependencies
8bbe48b 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: complete Drizzle schema with all tables, enums, indexes, vector…
768c30c 
… columns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: Drizzle DB client with tenant context (node-postgres pool + wit…
2d3bf3a 
…hTenant)

Replaces SurrealDB client with node-postgres pool backed Drizzle client.
withTenant() acquires a dedicated pool connection, sets app.tenant_id via
set_config() for RLS, then resets and releases. docker-compose and drizzle.config
updated to port 5433 to avoid conflict with existing local postgres on 5432.
Both tests pass (tenant_id set / reset after execution).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: Drizzle migrations with RLS policies for all tenant-scoped tables
04c911f 
Generates full initial schema migration (0000) from drizzle-kit and adds
custom RLS migration (0001) that enables + forces row-level security with
tenant_id-scoped policies on all 24 tenant-scoped tables. Junction tables
and the tenant table are correctly excluded. Verified isolation: non-superuser
sees 0 rows from other tenant, 1 from own. Existing DB client tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: seed script (flutur + lago_maggiore tenants) + Vitest config
3d87ca3 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: structured JSON logger (stderr-only, safe for stdio transport)
281383b 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: email guard migrated to Drizzle (fail-closed, 3-layer dedup)
ab25029 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: credentials manager (Keychain-based, tenant-aware structure)
9be8e44 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: tenant config loader with cache (1min TTL)
be8f395 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: MCP server entry point (SDK v1.x, stdio) + system tools (status…
b06084a 
…, connections, preflight)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: platform clients migration to src/services/platform/
2b85668 
Move Twitter, Instagram, YouTube, and Gmail clients from src/clients/
to src/services/platform/ with three structural changes:
- console.log/error replaced with createLogger() (stderr-safe, JSON)
- getDb() and all SurrealDB writes removed — clients are now pure API
- gmail.ts merges gmail-reader.ts + email.ts into one module;
  scanOutreachReplies() now takes domainToVenue as a parameter
  instead of loading it from DB, keeping the client dependency-free

Barrel export at src/services/platform/index.ts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: migrate outreach services (pipeline, conversation, intelligence…
9a05630 
…, preflight, logistics, scheduler)

6 outreach modules migrated from SurrealDB to Drizzle ORM:
- conversation.ts: thread assembly from email + outreach_reply tables
- pipeline.ts: batch lifecycle (generate -> preview -> approve -> send)
- intelligence.ts: event -> action dispatcher with logistics/cluster/conversation departments
- logistics.ts: cost estimates and cluster analysis (pure functions + DB for cluster)
- scheduler.ts: tour-to-pipeline bridge, uncontacted venue surveys
- preflight.ts: fail-closed pre-send checks (daily limit, dedup, thread state)

All SurrealDB getDb() calls replaced with Drizzle queries. tenantId parameter
added to all functions. Logger replaces console.log/warn.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: migrate content + analytics services to Drizzle
945a2c4 
- src/services/content/drafter.ts — already existed; re-exported via barrel
- src/services/content/duplicate-checker.ts — kept JSON-file index, replaced console.log with createLogger()
- src/services/content/editorial-planner.ts — SurrealDB raw queries → Drizzle (post_draft, storyFragment, post tables); tenantId param; static fallback library retained
- src/services/content/index.ts — barrel
- src/services/analytics/instagram.ts — saveAudienceSnapshot / getLatestAudience / getAudienceHistory / getAudienceEvolution / getCorridorAnalysis
- src/services/analytics/youtube.ts — saveYouTubeSnapshot / saveVideoCountryData / getLatestYouTubeSnapshot / getVideoCountriesForSnapshot
- src/services/analytics/correlator.ts — 3-day email→views→visit→reply pattern; correlations stored as memoryLink rows (signalType='correlation')
- src/services/analytics/index.ts — barrel

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: migrate calendar + memory services (session log, agent memory, …
a515c53 
…story/research stores)

Replaces SurrealDB with Drizzle ORM across all calendar and memory modules.
Adds tenantId to every function signature per multi-tenant architecture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: register all MCP tools across 7 domain files (posting, outreach…
09419fd 
…, analytics, content, calendar, intelligence, system)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: SurrealDB to Neon migration script (one-shot, with validation)
4c3372f 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: automated daily PostgreSQL backup with 30-day rotation + launch…
205e434 
…d plist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: Sentry MCP monitoring (opt-in via SENTRY_DSN)
d6dfa8b 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat: refactor Telegram bot as thin client calling services
ae26b24 
New src/server/telegram.ts replaces inline DB queries and direct platform
instantiation with imports from src/services/. Uses createLogger("telegram"),
getDefaultTenantId() + withTenant() pattern, and proper tenantId threading
through all outreach service calls (runPipeline, getBatchStatus, getDailyLimits,
approveBatch, sendApprovedBatch, getConversationDashboard). Original
src/telegram-bot.ts left untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@alelab @claude
feat: production Dockerfile (multi-stage, node:22-slim) + .mcp.json u…
9225727 
…pdate for new server

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
chore: remove one-off debugging/testing scripts
7ce56c6 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
docs: update CLAUDE.md FONTI CANONICHE to reflect new architecture
1d090a4 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
fix: resolve transitive import chain breaking MCP server startup
62331b9 
- Extract pure pillar helpers to src/services/content/pillars.ts
  (no DB dependency, fixes old src/core/pillar-helpers.ts → getDb chain)
- Fix z.record() to z.record(z.string(), z.string()) in intelligence tools
- Update platform clients to import from migrated duplicate-checker

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
feat(epk): newspaper grid rewrite — SILO gap+background pattern, font…
ece33b4 
… toggle, YouTube player

Replace pseudo-element column rules with SILO's native CSS Grid pattern:
grid background bleeds through 1px gaps creating structural borders in
both directions. Add font toggle (6 presets with x-height compensation),
YouTube hover-to-unmute player with global audio unlock, Lenis smooth
scroll, and scroll-reveal animations. Remove legacy section components
in favor of card-based newspaper composition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@alelab @claude
refactor: remove ~39K lines of old code, complete architecture cleanup
28a814b 
Old monoliths removed: mcp-server.ts (2083 lines), telegram-bot.ts (1338),
core/ (2919), agents/ (6564), outreach/ (3863), analytics/ (1369),
calendar/ (766), clients/ (2944), digital-department/ (905), content/ (279),
old db helpers (3540), root files (2281), utils/ (603).
86 one-off scripts deleted (113 → 18).

New architecture is self-contained: server/tools/ → services/ → db/ + lib/ + types/
Zero TypeScript errors. No circular dependencies.

Types extracted to src/types/index.ts + src/types/analytics.ts.
Email preview module spec added (planned, not implemented).
CLAUDE.md updated to v9.0. TODO.md rewritten for current state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Alemusica @alelab