From ed538587e60eb6959e51ec5b722146fef3e69ea0 Mon Sep 17 00:00:00 2001 From: Nathan Clevenger <4130910+nathanclevenger@users.noreply.github.com> Date: Wed, 7 Jan 2026 18:30:53 -0600 Subject: [PATCH 01/60] docs: Transform 51 README files to gold standard natural language API - Replace verbose sdk.method({ config }) patterns with sdk`natural language` - Add promise pipelining with .map() chains for efficient operations - Make all examples pass the "dictation test" - code that can be spoken naturally - Add tree-shakable imports (sdk.do, sdk.do/tiny, sdk.do/feature) - Restructure to match gold standard sections (Problem, Solution, Features, Architecture) READMEs transformed: airbyte, airtable, algolia, amplitude, asana, bamboohr, composio, confluence, coupa, customerio, databricks, datadog, docusign, dynamics, fivetran, fsx, hubspot, inngest, intercom, jira, launchdarkly, linear, looker, make, mixpanel, monday, n8n, netsuite, notion, orb, pipedrive, posthog, powerbi, rippling, salesforce, sap, segment, sentry, servicenow, servicetitan, shopify, snowflake, splunk, studio, tableau, temporal, trigger, workday, zapier, zendesk, zoho Co-Authored-By: Claude Opus 4.5 --- rewrites/airbyte/README.md | 616 ++++++++-------- rewrites/airtable/README.md | 705 +++++------------- rewrites/algolia/README.md | 422 +++++++---- rewrites/amplitude/README.md | 783 ++++++++------------ rewrites/asana/README.md | 484 ++++-------- rewrites/bamboohr/README.md | 542 +++++--------- rewrites/composio/README.md | 417 +++++++---- rewrites/confluence/README.md | 454 ++++-------- rewrites/coupa/README.md | 808 +++++--------------- rewrites/customerio/README.md | 543 ++++++++------ rewrites/databricks/README.md | 1218 +++++++------------------------ rewrites/datadog/README.md | 688 ++++++----------- rewrites/docusign/README.md | 727 +++++------------- rewrites/dynamics/README.md | 1127 +++++++++------------------- rewrites/fivetran/README.md | 600 ++++++++------- rewrites/fsx/README.md | 460 +++++++----- rewrites/hubspot/README.md | 899 +++++++++-------------- rewrites/inngest/README.md | 539 ++++++++------ rewrites/intercom/README.md | 668 ++++++----------- rewrites/jira/README.md | 562 +++++--------- rewrites/launchdarkly/README.md | 482 +++++++----- rewrites/linear/README.md | 597 ++++++--------- rewrites/looker/README.md | 691 ++++++------------ rewrites/make/README.md | 627 ++++++++-------- rewrites/mixpanel/README.md | 659 +++++------------ rewrites/monday/README.md | 623 ++++++---------- rewrites/n8n/README.md | 584 +++++++-------- rewrites/netsuite/README.md | 518 +++++-------- rewrites/notion/README.md | 520 +++++-------- rewrites/orb/README.md | 547 ++++++++------ rewrites/pipedrive/README.md | 847 ++++++--------------- rewrites/posthog/README.md | 451 ++++++------ rewrites/powerbi/README.md | 486 ++++++------ rewrites/rippling/README.md | 462 ++++-------- rewrites/salesforce/README.md | 1118 ++++++---------------------- rewrites/sap/README.md | 1154 +++++++++-------------------- rewrites/segment/README.md | 655 ++++++++++------- rewrites/sentry/README.md | 487 ++++++++---- rewrites/servicenow/README.md | 503 +++++++------ rewrites/servicetitan/README.md | 454 ++++++------ rewrites/shopify/README.md | 605 ++++----------- rewrites/snowflake/README.md | 544 ++++---------- rewrites/splunk/README.md | 976 +++++++++++-------------- rewrites/studio/README.md | 451 +++++++----- rewrites/tableau/README.md | 673 +++++++---------- rewrites/temporal/README.md | 698 ++++++++---------- rewrites/trigger/README.md | 229 +++--- rewrites/workday/README.md | 619 +++++++++------- rewrites/zapier/README.md | 306 ++++---- rewrites/zendesk/README.md | 420 ++++++----- rewrites/zoho/README.md | 753 +++++-------------- 51 files changed, 12782 insertions(+), 19219 deletions(-) diff --git a/rewrites/airbyte/README.md b/rewrites/airbyte/README.md index 8564baa5..5b8d84c2 100644 --- a/rewrites/airbyte/README.md +++ b/rewrites/airbyte/README.md @@ -1,374 +1,369 @@ # airbyte.do -Data integration that speaks your language. +> Data Integration. Edge-Native. Natural Language First. -## The Hero +Airbyte raised $181M to build "the open-source standard for data integration." Now they run a cloud service charging $1.50/credit, require Kubernetes for self-hosting, and make data engineers babysit YAML files and Docker containers. Moving data shouldn't require a DevOps team. -**For data engineers tired of babysitting Kubernetes just to move data.** +**airbyte.do** is the serverless alternative. No Kubernetes. No Docker. No YAML. Natural language pipelines that deploy in seconds. -You know the drill: 3-node K8s cluster, 2 hours to deploy a connector, $500/month in compute, and a DevOps team on speed dial. All because you need to sync Salesforce to Snowflake. - -What if you could just... ask? +## AI-Native API ```typescript -import { airbyte } from '@dotdo/airbyte' +import { airbyte } from 'airbyte.do' // Full SDK +import { airbyte } from 'airbyte.do/tiny' // Minimal client +import { airbyte } from 'airbyte.do/streaming' // Streaming ops +``` -airbyte`sync GitHub commits to Snowflake hourly` -airbyte`extract Stripe payments since January into BigQuery` -airbyte`why is the Salesforce sync failing?` +Natural language for data pipelines: + +```typescript +import { airbyte } from 'airbyte.do' + +// Talk to it like a colleague +const syncs = await airbyte`failing syncs this week` +const slow = await airbyte`connections taking > 1 hour` +const stale = await airbyte`sources not synced in 3 days` + +// Chain like sentences +await airbyte`stripe charges` + .sync(`to snowflake`) + +// Pipelines that build themselves +await airbyte`connect salesforce` + .discover() // find all objects + .sync(`bigquery`) // sync everything + .schedule(`hourly`) // keep it fresh ``` -No Kubernetes. No Docker. No YAML. Just data pipelines that work. +## The Problem -## Promise Pipelining +Self-hosted Airbyte dominates open-source ELT: -Chain operations without waiting. One network round trip. +| What Airbyte Requires | The Reality | +|-----------------------|-------------| +| **Infrastructure** | 3-node Kubernetes cluster minimum | +| **Deployment** | 2+ hours to configure, test, deploy | +| **Monthly Compute** | $500-2000/month for mid-size workloads | +| **Expertise** | DevOps team or K8s knowledge required | +| **Configuration** | YAML files, Docker images, Helm charts | +| **Debugging** | kubectl logs, pod restarts, OOM errors | -```typescript -const synced = await airbyte`discover all postgres tables` - .map(table => airbyte`sync ${table} to bigquery incrementally`) - .map(sync => airbyte`verify ${sync} completed successfully`) -// One network round trip! -``` +### The Kubernetes Tax -Build complex pipelines that feel like talking to a teammate: +Self-hosting Airbyte means: -```typescript -const pipeline = await airbyte`list all stripe payment sources` - .map(source => airbyte`sync ${source} to snowflake with deduplication`) - .map(sync => airbyte`add transformation: convert cents to dollars for ${sync}`) - .map(result => airbyte`notify #data-team when ${result} completes`) -``` +- Multi-node K8s cluster (EKS, GKE, or roll your own) +- Temporal for workflow orchestration +- PostgreSQL for metadata +- MinIO or S3 for staging +- Monitoring stack (Prometheus, Grafana) +- On-call rotation for connector failures -## Agent Integration +Data engineers spend more time on infrastructure than data. -airbyte.do integrates seamlessly with the workers.do agent ecosystem: +### The Cloud Alternative Isn't Cheap -```typescript -import { tom, priya } from 'agents.do' +Airbyte Cloud charges per credit: -// Tom sets up the infrastructure -tom`set up a pipeline from Salesforce to Snowflake, syncing hourly` +- $1.50/credit (rows synced) +- High-volume = high bills +- Unpredictable costs at scale +- Still debugging connector configs -// Priya monitors and troubleshoots -priya`why did the nightly sync fail? fix it and set up alerting` +### The Configuration Complexity -// Chain agents and tools -const ready = await priya`design a data pipeline for customer analytics` - .map(spec => tom`implement ${spec} using airbyte.do`) - .map(pipeline => airbyte`validate and deploy ${pipeline}`) +Every connector needs: + +```yaml +# This is what you're escaping +sourceDefinitionId: 778daa7c-feaf-4db6-96f3-70fd645acc77 +connectionConfiguration: + credentials: + auth_type: OAuth + client_id: ${SALESFORCE_CLIENT_ID} + client_secret: ${SALESFORCE_CLIENT_SECRET} + refresh_token: ${SALESFORCE_REFRESH_TOKEN} + start_date: "2024-01-01T00:00:00Z" + streams_criteria: + - criteria: starts with + value: Account ``` -## The Transformation +## The Solution -| Before (Self-Hosted Airbyte) | After (airbyte.do) | -|------------------------------|-------------------| -| 3-node Kubernetes cluster | Zero infrastructure | -| 2 hours to deploy a connector | `airbyte\`add stripe source\`` | -| $500+/month compute costs | Pay per sync | -| DevOps team required | Natural language | -| YAML configuration files | Tagged template literals | -| Docker image management | Managed connectors | -| Manual scaling | Edge-native auto-scale | -| Self-managed updates | Always current | -| Debug Kubernetes pods | `airbyte\`why did it fail?\`` | -| Monitoring stack setup | Built-in observability | +**airbyte.do** reimagines data integration: -## When You Need Control +``` +Self-Hosted Airbyte airbyte.do +----------------------------------------------------------------- +3-node K8s cluster Zero infrastructure +2 hours to deploy Deploy in seconds +$500/month compute Pay per sync +YAML configuration Natural language +Docker image management Managed connectors +kubectl debug \`airbyte\`why did it fail?\`\` +Temporal orchestration Durable Objects +Manual scaling Edge-native auto-scale +``` -For programmatic access and fine-grained configuration: +## One-Click Deploy -```typescript -import { Airbyte } from '@dotdo/airbyte' - -const airbyte = new Airbyte({ workspace: 'my-workspace' }) - -// Define a source (GitHub) -const github = await airbyte.sources.create({ - name: 'github-source', - type: 'github', - config: { - credentials: { personal_access_token: env.GITHUB_TOKEN }, - repositories: ['myorg/myrepo'], - start_date: '2024-01-01' - } -}) +```bash +npx create-dotdo airbyte +``` -// Define a destination (Snowflake) -const snowflake = await airbyte.destinations.create({ - name: 'snowflake-dest', - type: 'snowflake', - config: { - host: 'account.snowflakecomputing.com', - database: 'analytics', - schema: 'raw', - credentials: { password: env.SNOWFLAKE_PASSWORD } - } -}) +A full ELT platform. Running on your Cloudflare account. 300+ connectors ready. -// Create a connection (sync job) -const connection = await airbyte.connections.create({ - name: 'github-to-snowflake', - source: github.id, - destination: snowflake.id, - streams: [ - { name: 'commits', syncMode: 'incremental', cursorField: 'date' }, - { name: 'pull_requests', syncMode: 'incremental', cursorField: 'updated_at' }, - { name: 'issues', syncMode: 'full_refresh' } - ], - schedule: { cron: '0 */6 * * *' } // Every 6 hours -}) +```typescript +import { Airbyte } from 'airbyte.do' -// Trigger a manual sync -await airbyte.connections.sync(connection.id) +export default Airbyte({ + name: 'my-data-platform', + domain: 'data.mycompany.com', +}) ``` ## Features -- **300+ Connectors** - Sources and destinations via MCP tools -- **Incremental Sync** - Only sync changed data with cursor-based tracking -- **Schema Discovery** - Automatically detect and map source schemas -- **CDC Support** - Change Data Capture for database sources -- **Normalization** - Optional transformation to analytics-ready schemas -- **TypeScript First** - Full type safety for configurations -- **Edge Native** - Runs on Cloudflare's global network +### Sources -## Architecture +```typescript +// Connect sources naturally +await airbyte`connect to postgres at db.example.com` +await airbyte`add stripe as a source` +await airbyte`connect salesforce with oauth` + +// AI infers what you need +await airbyte`postgres tables` // lists available tables +await airbyte`stripe schema` // shows data structure +await airbyte`test salesforce connection` // verifies connectivity +``` + +### Destinations +```typescript +// Warehouses are one line +await airbyte`send to snowflake analytics.raw` +await airbyte`connect bigquery my-project.raw_data` +await airbyte`add databricks lakehouse` + +// Or dictate directly +await airbyte`sync stripe to snowflake, bigquery, and redshift` ``` - +----------------------+ - | airbyte.do | - | (Cloudflare Worker) | - +----------------------+ - | - +---------------+---------------+---------------+ - | | | | - +------------------+ +------------------+ +------------------+ +------------------+ - | SourceDO | | DestinationDO | | ConnectionDO | | SyncDO | - | (connectors) | | (connectors) | | (orchestration) | | (job execution) | - +------------------+ +------------------+ +------------------+ +------------------+ - | | | | - +---------------+-------+-------+---------------+ - | - +-------------------+-------------------+ - | | - +-------------------+ +-------------------+ - | Cloudflare Queues | | MCP Tools | - | (job scheduling) | | (fsx.do, gitx.do) | - +-------------------+ +-------------------+ + +### Connections + +```typescript +// Just say it +await airbyte`sync postgres users to bigquery hourly` +await airbyte`github commits to snowflake every 6 hours` +await airbyte`stripe to databricks, incremental on updated_at` + +// AI configures sync modes automatically +await airbyte`what should I sync from salesforce?` + .sync(`to snowflake`) // submits with optimal settings + +// Batch connections read like a pipeline manifest +await airbyte` + postgres production: + - users incremental + - orders incremental + - products full refresh + all to bigquery hourly +` ``` -**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each connection gets its own ConnectionDO for orchestration, and each sync job gets a SyncDO for execution tracking. +### Sync Status -## Installation +```typescript +// View status naturally +await airbyte`sync status` +await airbyte`failed syncs today` +await airbyte`slowest connections this week` -```bash -npm install @dotdo/airbyte +// AI surfaces what needs attention ``` -## Quick Start +### Schema Discovery + +```typescript +// Discover sources naturally +await airbyte`what tables are in postgres?` +await airbyte`salesforce objects with data` +await airbyte`stripe schema changes since last week` +``` -### Natural Language First +### Troubleshooting ```typescript -import { airbyte } from '@dotdo/airbyte' +// Diagnose issues naturally +await airbyte`why did stripe sync fail?` +await airbyte`postgres connection timing out` +await airbyte`fix the salesforce oauth error` + +// AI chains diagnosis and fixes +await airbyte`connectors with errors` + .map(c => airbyte`diagnose ${c}`) + .map(fix => airbyte`apply ${fix}`) +``` -// Discover what's available -await airbyte`what sources can you connect to?` +## Promise Pipelining -// Set up a pipeline in one line -await airbyte`sync all tables from postgres://prod.db.com to bigquery, hourly` +Chain operations without waiting. One network round trip. -// Monitor your pipelines -await airbyte`show me all failed syncs from the last 24 hours` +```typescript +// Discover and sync in one chain +await airbyte`postgres tables` + .map(table => airbyte`sync ${table} to bigquery incrementally`) + .map(sync => airbyte`verify ${sync}`) -// Troubleshoot issues -await airbyte`the stripe sync is slow, diagnose and optimize it` +// Build complex pipelines that feel like talking +await airbyte`all stripe payment tables` + .map(table => airbyte`sync ${table} to snowflake with dedup`) + .map(sync => airbyte`transform cents to dollars in ${sync}`) + .map(result => airbyte`notify #data-team when ${result} completes`) ``` -### Define Sources +## Agent Integration ```typescript -import { Airbyte } from '@dotdo/airbyte' - -const airbyte = new Airbyte({ workspace: 'my-workspace' }) - -// Database source (Postgres) -const postgres = await airbyte.sources.create({ - name: 'postgres-prod', - type: 'postgres', - config: { - host: 'db.example.com', - port: 5432, - database: 'production', - username: 'airbyte', - password: env.POSTGRES_PASSWORD, - replication_method: { method: 'CDC' } - } -}) +import { tom, priya } from 'agents.do' -// API source (Stripe) -const stripe = await airbyte.sources.create({ - name: 'stripe-source', - type: 'stripe', - config: { - client_secret: env.STRIPE_SECRET_KEY, - account_id: 'acct_xxx', - start_date: '2024-01-01' - } -}) +// Chain agents and tools +await priya`design customer analytics pipeline` + .map(spec => tom`implement ${spec} with airbyte.do`) + .map(pipeline => airbyte`deploy ${pipeline}`) +``` + +## Architecture + +### Durable Object per Connection -// File source (S3) -const s3 = await airbyte.sources.create({ - name: 's3-events', - type: 's3', - config: { - bucket: 'my-events-bucket', - aws_access_key_id: env.AWS_ACCESS_KEY, - aws_secret_access_key: env.AWS_SECRET_KEY, - path_pattern: 'events/**/*.parquet' - } -}) +``` +AirbyteDO (config, connectors, catalog) + | + +-- SourcesDO (source connectors) + | |-- SQLite: Source configs (encrypted) + | +-- Schema cache + | + +-- DestinationsDO (destination connectors) + | |-- SQLite: Destination configs (encrypted) + | + +-- ConnectionsDO (sync orchestration) + | |-- SQLite: Connection state + | +-- Sync history + | + +-- SyncsDO (job execution) + |-- SQLite: Job state, metrics + +-- R2: Staging data ``` -### Define Destinations +### Storage Tiers -```typescript -// Data warehouse (BigQuery) -const bigquery = await airbyte.destinations.create({ - name: 'bigquery-analytics', - type: 'bigquery', - config: { - project_id: 'my-project', - dataset_id: 'raw_data', - credentials_json: env.BIGQUERY_CREDENTIALS - } -}) +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active connections, recent syncs | <10ms | +| **Warm** | R2 + Index | Sync history (30 days) | <100ms | +| **Cold** | R2 Archive | Audit logs (1+ years) | <1s | -// Data lake (Databricks) -const databricks = await airbyte.destinations.create({ - name: 'databricks-lakehouse', - type: 'databricks', - config: { - host: 'my-workspace.databricks.com', - http_path: '/sql/1.0/warehouses/xxx', - token: env.DATABRICKS_TOKEN, - catalog: 'main', - schema: 'raw' - } -}) +## vs Airbyte -// Vector store (Pinecone) -const pinecone = await airbyte.destinations.create({ - name: 'pinecone-embeddings', - type: 'pinecone', - config: { - api_key: env.PINECONE_API_KEY, - index: 'documents', - embedding_model: 'text-embedding-3-small' - } -}) -``` +| Feature | Airbyte (Self-Hosted) | Airbyte Cloud | airbyte.do | +|---------|----------------------|---------------|------------| +| **Infrastructure** | 3-node K8s cluster | None | None | +| **Deployment** | 2+ hours | Minutes | Seconds | +| **Monthly Cost** | $500-2000 compute | $1.50/credit | Pay per sync | +| **Configuration** | YAML files | UI forms | Natural language | +| **Debugging** | kubectl logs | Logs UI | `airbyte\`why did it fail?\`` | +| **Scaling** | Manual | Managed | Edge-native auto-scale | +| **Data Location** | Your K8s cluster | Airbyte's cloud | Your Cloudflare account | +| **Lock-in** | Open source | Proprietary cloud | MIT licensed | -### Create Connections +## Use Cases -```typescript -// Full pipeline with incremental sync -const pipeline = await airbyte.connections.create({ - name: 'postgres-to-bigquery', - source: postgres.id, - destination: bigquery.id, - streams: [ - { name: 'users', syncMode: 'incremental', cursorField: 'updated_at' }, - { name: 'orders', syncMode: 'incremental', cursorField: 'created_at' }, - { name: 'products', syncMode: 'full_refresh' } - ], - schedule: { cron: '0 * * * *' }, - normalization: 'basic' -}) +### Data Warehouse Loading -// On-demand sync -await airbyte.connections.sync(pipeline.id) +Just sync and schedule. No infrastructure. -// Check sync status -const status = await airbyte.connections.status(pipeline.id) -``` +### Real-Time Analytics -### Schema Discovery +CDC from databases, streaming to warehouses. + +### AI/ML Pipelines ```typescript -// Discover available streams from a source -const schema = await airbyte.sources.discover(postgres.id) +// Sync to vector stores for RAG +await airbyte`sync notion pages to pinecone for search` -// Test source connectivity -const check = await airbyte.sources.check(postgres.id) -// { status: 'succeeded', message: 'Successfully connected to database' } +// Embeddings included +await airbyte`zendesk tickets to weaviate with embeddings` ``` -### Sync Modes +### Multi-Destination Fan-Out ```typescript -// Full Refresh - Replace all data each sync -{ name: 'dim_products', syncMode: 'full_refresh', destinationSyncMode: 'overwrite' } +// One source, many destinations +await airbyte`stripe to snowflake, bigquery, and redshift` +``` -// Full Refresh + Append - Append all data each sync -{ name: 'event_log', syncMode: 'full_refresh', destinationSyncMode: 'append' } +## Connectors -// Incremental - Only new/updated records -{ name: 'fact_orders', syncMode: 'incremental', cursorField: 'updated_at' } +### 300+ Sources Supported -// Incremental + Dedup - Deduplicate by primary key -{ name: 'users', syncMode: 'incremental', cursorField: 'updated_at', primaryKey: [['id']] } -``` +| Category | Examples | +|----------|----------| +| **Databases** | PostgreSQL, MySQL, MongoDB, SQL Server, Oracle | +| **Data Warehouses** | Snowflake, BigQuery, Redshift, Databricks | +| **SaaS** | Salesforce, HubSpot, Stripe, Shopify, Zendesk | +| **Files** | S3, GCS, SFTP, local files | +| **APIs** | REST, GraphQL, webhooks | -## MCP Tools +### 50+ Destinations Supported -Register as MCP tools for AI agents: +| Category | Examples | +|----------|----------| +| **Warehouses** | Snowflake, BigQuery, Redshift, Databricks | +| **Lakes** | S3, GCS, Delta Lake, Iceberg | +| **Vector Stores** | Pinecone, Weaviate, Qdrant, Milvus | +| **Databases** | PostgreSQL, MySQL, MongoDB | -```typescript -export const mcpTools = { - 'airbyte.sources.create': airbyte.sources.create, - 'airbyte.sources.discover': airbyte.sources.discover, - 'airbyte.sources.check': airbyte.sources.check, - 'airbyte.destinations.create': airbyte.destinations.create, - 'airbyte.destinations.check': airbyte.destinations.check, - 'airbyte.connections.create': airbyte.connections.create, - 'airbyte.connections.sync': airbyte.connections.sync, - 'airbyte.connections.status': airbyte.connections.status, - 'airbyte.catalog.sources.list': airbyte.catalog.sources.list, - 'airbyte.catalog.destinations.list': airbyte.catalog.destinations.list -} -``` +## Deployment Options -## The Rewrites Ecosystem +### Cloudflare Workers (Recommended) -airbyte.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare: +```bash +npx create-dotdo airbyte +# Deploys to your Cloudflare account +``` -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| [inngest.do](https://inngest.do) | Inngest | Workflows/Jobs for AI | -| **airbyte.do** | Airbyte | Data integration for AI | -| kafka.do | Kafka | Event streaming for AI | -| nats.do | NATS | Messaging for AI | +### Self-Hosted -Each rewrite follows the same pattern: -- Durable Objects for state -- SQLite for persistence -- Cloudflare Queues for messaging -- Compatible API with the original -- Natural language interface via tagged templates +```bash +# Deploy to your infrastructure +docker run -p 8787:8787 dotdo/airbyte +``` ## Why Cloudflare? -1. **Global Edge** - Sync jobs run close to data sources -2. **No Cold Starts** - Durable Objects stay warm -3. **Unlimited Duration** - Long-running syncs work naturally -4. **Built-in Queues** - Reliable job scheduling -5. **R2 Storage** - Staging area for large syncs -6. **Workers AI** - Embeddings and transformations +### 1. Global Edge + +Sync jobs run close to data sources. Lower latency, faster syncs. + +### 2. No Cold Starts + +Durable Objects stay warm. No waiting for containers. + +### 3. Unlimited Duration + +Long-running syncs work naturally. No 30-second timeouts. + +### 4. Built-in Queues + +Reliable job scheduling. No external Temporal cluster. + +### 5. R2 Storage + +Staging area for large syncs. No S3 or MinIO setup. ## Related Domains @@ -377,6 +372,55 @@ Each rewrite follows the same pattern: - **connectors.do** - Connector marketplace - **catalog.do** - Data catalog and discovery +## Roadmap + +### Core ELT +- [x] Source connectors (300+) +- [x] Destination connectors (50+) +- [x] Incremental sync +- [x] Full refresh +- [x] Schema discovery +- [ ] CDC support +- [ ] Custom transformations +- [ ] dbt integration + +### AI +- [x] Natural language configuration +- [x] Auto-troubleshooting +- [ ] Schema mapping suggestions +- [ ] Sync optimization +- [ ] Anomaly detection + +### Enterprise +- [x] Multi-workspace +- [ ] RBAC +- [ ] Audit logs +- [ ] SSO + +## Contributing + +airbyte.do is open source under the MIT license. + +```bash +git clone https://github.com/dotdo/airbyte.do +cd airbyte.do +pnpm install +pnpm test +``` + ## License -MIT +MIT License - Move data freely. + +--- + +

+ No Kubernetes. No Docker. No YAML. +
+ Natural language pipelines. Edge-native. Serverless. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/airtable/README.md b/rewrites/airtable/README.md index 5096d322..cf5f4962 100644 --- a/rewrites/airtable/README.md +++ b/rewrites/airtable/README.md @@ -1,11 +1,39 @@ # airtable.do -> The spreadsheet-database. Open source. AI-supercharged. +> The spreadsheet-database. Edge-Native. Open by Default. AI-First. -Airtable bridged the gap between spreadsheets and databases. Anyone could build apps without code. But at $20-45/user/month with row limits, record caps, and AI locked behind enterprise pricing, it's become expensive for what it is. +Airtable bridged the gap between spreadsheets and databases. But at $20-45/user/month with row limits, record caps, and AI locked behind enterprise pricing, it's become expensive for what it is. **airtable.do** is the spreadsheet-database reimagined. No row limits. No per-seat pricing. AI that builds apps for you. Own your data infrastructure. +## AI-Native API + +```typescript +import { airtable } from 'airtable.do' // Full SDK +import { airtable } from 'airtable.do/tiny' // Minimal client +import { airtable } from 'airtable.do/sync' // Real-time sync +``` + +Natural language for data workflows: + +```typescript +import { airtable } from 'airtable.do' + +// Talk to it like a colleague +const deals = await airtable`deals over $50k closing this month` +const forecast = await airtable`projected revenue for Q2` +const churn = await airtable`customers at risk of churning` + +// Chain like sentences +await airtable`leads needing followup` + .notify(`Time to reach out`) + +// Apps that build themselves +await airtable`create project tracker with tasks, owners, and deadlines` + .addView('kanban by status') + .addView('calendar by deadline') +``` + ## The Problem Airtable's pricing creates artificial scarcity: @@ -51,638 +79,228 @@ npx create-dotdo airtable Your own Airtable. Running on Cloudflare. No limits. -```bash -# Or add to existing workers.do project -npx dotdo add airtable -``` - -## The workers.do Way - -You're building a product. Your team needs a flexible database. Airtable wants $27k/year with row limits and 5 requests/second API throttling. AI locked behind enterprise. There's a better way. - -**Natural language. Tagged templates. AI agents that work.** - -```typescript -import { airtable } from 'airtable.do' -import { priya, ralph, mark } from 'agents.do' - -// Talk to your database like a human -const deals = await airtable`show deals over $50k closing this month` -const forecast = await airtable`projected revenue for Q2?` -const churn = await airtable`which customers are at risk?` -``` - -**Promise pipelining - chain work without Promise.all:** - ```typescript -// CRM automation pipeline -const processed = await airtable`get new leads` - .map(lead => priya`qualify ${lead}`) - .map(lead => sally`draft outreach for ${lead}`) - .map(lead => airtable`update ${lead} status`) +import { Airtable } from 'airtable.do' -// Data cleanup pipeline -const cleaned = await airtable`find records with missing data` - .map(record => ralph`enrich ${record} from sources`) - .map(record => airtable`update ${record}`) +export default Airtable({ + name: 'my-workspace', + domain: 'data.my-company.com', +}) ``` -One network round trip. Record-replay pipelining. Workers working for you. - ## Features ### Bases & Tables -The familiar structure: +```typescript +// Create bases naturally +await airtable`create Product Development base` +await airtable`add Features table with name, status, priority, owner, and due date` +await airtable`add Sprints table linked to Features` + +// AI infers the schema from your description +await airtable` + create CRM with: + - Companies with name, industry, size, and website + - Contacts linked to companies with name, email, role + - Deals linked to contacts with amount, stage, close date +` +``` + +### Records ```typescript -import { Base, Table, Field } from 'airtable.do' - -// Create a base -const productBase = await Base.create({ - name: 'Product Development', - tables: [ - Table.create({ - name: 'Features', - fields: { - Name: Field.text({ primary: true }), - Description: Field.longText(), - Status: Field.singleSelect(['Planned', 'In Progress', 'Shipped']), - Priority: Field.singleSelect(['Low', 'Medium', 'High', 'Critical']), - Owner: Field.user(), - Team: Field.multipleSelect(['Frontend', 'Backend', 'Mobile', 'Platform']), - Sprint: Field.linkedRecord('Sprints'), - StartDate: Field.date(), - DueDate: Field.date(), - Progress: Field.percent(), - Attachments: Field.attachment(), - Created: Field.createdTime(), - Modified: Field.lastModifiedTime(), - }, - }), - Table.create({ - name: 'Sprints', - fields: { - Name: Field.text({ primary: true }), - StartDate: Field.date(), - EndDate: Field.date(), - Features: Field.linkedRecord('Features', { bidirectional: true }), - TotalPoints: Field.rollup({ - linkedField: 'Features', - rollupField: 'Points', - aggregation: 'SUM', - }), - }, - }), - ], -}) +// Create records naturally +await airtable`add task "Design homepage" for Sarah due Friday` +await airtable`new deal Acme Corp $50k in negotiation with John` +await airtable` + add contacts: + - Jane Smith, CEO at TechCo, jane@techco.com + - Bob Wilson, CTO at StartupX, bob@startupx.com +` + +// Query naturally +const tasks = await airtable`tasks assigned to Sarah` +const overdue = await airtable`overdue items` +const pipeline = await airtable`deals in negotiation over $25k` + +// Update naturally +await airtable`mark "Design homepage" complete` +await airtable`move Acme deal to closed won` +await airtable`assign all unassigned tasks to Mike` ``` ### Field Types -All the field types you need: - -| Type | Description | -|------|-------------| -| **Text** | Single line text | -| **Long Text** | Rich text with formatting | -| **Number** | Integers and decimals with formatting | -| **Currency** | Money with currency symbol | -| **Percent** | Percentage values | -| **Checkbox** | Boolean values | -| **Date** | Date with optional time | -| **Duration** | Time duration | -| **Single Select** | Dropdown with one choice | -| **Multiple Select** | Tags, multiple choices | -| **User** | Collaborators | -| **Linked Record** | Relations to other tables | -| **Lookup** | Values from linked records | -| **Rollup** | Aggregations across links | -| **Count** | Count of linked records | -| **Formula** | Calculated values | -| **Attachment** | Files, images | -| **URL** | Links with preview | -| **Email** | Email addresses | -| **Phone** | Phone numbers | -| **Rating** | Star ratings | -| **Barcode** | Barcode/QR data | -| **Auto Number** | Auto-incrementing IDs | -| **Created Time** | When record was created | -| **Last Modified** | When record was updated | -| **Created By** | Who created it | -| **Last Modified By** | Who last updated it | +All the field types you'd expect - text, numbers, dates, dropdowns, linked records, formulas, attachments, and more. AI infers the right type from context. ### Formulas -Powerful calculations: - ```typescript -// Simple formulas -const fullName = Field.formula('FirstName & " " & LastName') -const daysUntilDue = Field.formula('DATETIME_DIFF(DueDate, TODAY(), "days")') -const isOverdue = Field.formula('AND(Status != "Done", DueDate < TODAY())') - -// Complex formulas -const priorityScore = Field.formula(` - IF(Priority = "Critical", 100, - IF(Priority = "High", 75, - IF(Priority = "Medium", 50, 25))) - * IF(DueDate < TODAY(), 1.5, 1) -`) - -// Rollup with filter -const completedPoints = Field.rollup({ - linkedField: 'Tasks', - rollupField: 'Points', - aggregation: 'SUM', - filter: '{Status} = "Done"', -}) +// Describe the calculation, AI writes the formula +await airtable`add formula "days until due" to Tasks` +await airtable`add field showing whether tasks are overdue` +await airtable`calculate total deal value per company` + +// Or express complex logic naturally +await airtable` + add priority score that's higher for: + - critical items (100 points) + - overdue items (1.5x multiplier) + - high value deals (extra 25 points if over $50k) +` ``` ### Views -Same data, infinite perspectives: - ```typescript -// Grid View (default) -const allFeatures = View.grid({ - fields: ['Name', 'Status', 'Priority', 'Owner', 'DueDate'], - sort: [{ field: 'Priority', direction: 'desc' }], - filter: { Status: { neq: 'Shipped' } }, -}) - -// Kanban View -const featureBoard = View.kanban({ - groupBy: 'Status', - cardFields: ['Name', 'Owner', 'Priority', 'DueDate'], - coverField: 'Attachments', -}) - -// Calendar View -const roadmapCalendar = View.calendar({ - dateField: 'DueDate', - endDateField: 'EndDate', // Optional, for ranges - color: 'Priority', -}) - -// Timeline View (Gantt) -const projectTimeline = View.timeline({ - startField: 'StartDate', - endField: 'DueDate', - groupBy: 'Team', - color: 'Status', -}) - -// Gallery View -const designGallery = View.gallery({ - coverField: 'Mockups', - titleField: 'Name', - descriptionField: 'Description', -}) - -// Form View -const featureRequest = View.form({ - title: 'Submit Feature Request', - fields: ['Name', 'Description', 'Priority'], - submitMessage: 'Thanks! We\'ll review your request.', - allowAnonymous: true, -}) +// Create views naturally +await airtable`show tasks as kanban by status` +await airtable`show deals as calendar by close date` +await airtable`show projects as timeline from start to due date` +await airtable`show designs as gallery with mockups` + +// Filtered views +await airtable`show my tasks sorted by priority` +await airtable`show overdue items grouped by owner` +await airtable`show Q1 deals over $50k as funnel by stage` ``` ### Forms -Collect data from anyone: - ```typescript -const feedbackForm = Form.create({ - table: 'Feedback', - title: 'Product Feedback', - description: 'Help us improve our product', - fields: { - Name: { required: true }, - Email: { required: true }, - Category: { - required: true, - options: ['Bug', 'Feature Request', 'General'] - }, - Description: { required: true }, - Priority: { required: false }, - Attachments: { required: false }, - }, - branding: { - logo: '/logo.png', - color: '#0066FF', - }, - notifications: { - slack: '#feedback', - email: 'product@company.com', - }, -}) +// Create forms from tables +await airtable`create feedback form from Feedback table` +await airtable`create job application form with name, email, resume, and cover letter` -// Public URL: https://your-org.airtable.do/forms/feedbackForm +// Configure with natural language +await airtable`make feedback form public with logo and custom thank you message` +await airtable`notify #product-feedback on Slack when form submitted` ``` ## AI-Native Data Management AI doesn't just assist - it builds with you. -### AI Schema Design - -Describe your data, AI builds the schema: +### Schema Design ```typescript -import { ai } from 'airtable.do' - -const schema = await ai.designSchema(` - I need to track our content marketing. - We have blog posts, authors, and topics. - Posts go through draft, review, published stages. - Need to track SEO metrics and social engagement. -`) - -// AI creates: -{ - tables: [ - { - name: 'Posts', - fields: { - Title: Field.text({ primary: true }), - Slug: Field.formula('LOWER(SUBSTITUTE(Title, " ", "-"))'), - Status: Field.singleSelect(['Draft', 'In Review', 'Published']), - Author: Field.linkedRecord('Authors'), - Topics: Field.linkedRecord('Topics', { multiple: true }), - PublishDate: Field.date(), - Content: Field.longText(), - FeaturedImage: Field.attachment(), - SEOTitle: Field.text(), - SEODescription: Field.text(), - PageViews: Field.number(), - TimeOnPage: Field.duration(), - SocialShares: Field.number(), - // ... - }, - }, - { - name: 'Authors', - fields: { /* ... */ }, - }, - { - name: 'Topics', - fields: { /* ... */ }, - }, - ], - relationships: [/* ... */], - suggestedViews: [/* ... */], -} +// Describe what you need, AI builds the schema +await airtable` + I need to track content marketing: + - blog posts with authors and topics + - draft, review, published workflow + - SEO metrics and social engagement +` +// AI creates Posts, Authors, Topics tables with proper relationships ``` -### AI Data Entry - -Enter data in natural language: +### Data Entry ```typescript -// Create records from natural language -await ai.createRecord('Posts', ` - New blog post about AI in project management by Sarah, - topics: AI, Productivity. Ready for review. -`) -// Creates record with fields populated - -// Bulk create from text -await ai.createRecords('Contacts', ` - John Smith, CEO at Acme Corp, john@acme.com, met at conference - Jane Doe, CTO at TechCo, jane@techco.com, inbound lead - Bob Wilson, PM at StartupX, bob@startupx.com, referral from John -`) +// Enter data naturally +await airtable`add post "AI in Project Management" by Sarah, topics AI and Productivity` +await airtable` + add contacts: + - John Smith, CEO at Acme, john@acme.com, met at conference + - Jane Doe, CTO at TechCo, jane@techco.com, inbound lead +` ``` -### AI Data Cleanup - -Fix messy data automatically: +### Data Cleanup ```typescript -// Clean and standardize data -const cleanup = await ai.cleanData({ - table: 'Contacts', - operations: [ - { type: 'normalize', field: 'Phone', format: 'E.164' }, - { type: 'deduplicate', fields: ['Email'], action: 'merge' }, - { type: 'categorize', field: 'Company', into: 'Industry' }, - { type: 'fix-typos', field: 'Country' }, - { type: 'parse-names', source: 'FullName', into: ['FirstName', 'LastName'] }, - ], -}) - -// Preview changes before applying -console.log(`${cleanup.recordsAffected} records will be updated`) -cleanup.preview.forEach(change => console.log(change)) - -// Apply changes -await cleanup.apply() +// Fix messy data with one command +await airtable`clean up Contacts: fix phone formats, merge duplicates, categorize companies` +await airtable`standardize country names in Leads table` +await airtable`split full names into first and last name` ``` -### AI Formula Generation - -Describe calculations, AI writes formulas: +### Insights ```typescript -// Generate formula from description -const formula = await ai.formula(` - Calculate the health score based on: - - Days since last activity (more recent = better) - - Number of completed tasks (more = better) - - Whether payment is overdue (bad) - Scale should be 0-100 -`) - -// Returns: -'100 - (IF(DaysSinceActivity > 30, 30, DaysSinceActivity)) + (CompletedTasks * 2) - (IF(PaymentOverdue, 50, 0))' -``` - -### AI Insights - -Get insights from your data: +// Ask questions about your data +await airtable`which products are performing best this quarter?` +await airtable`who's exceeding quota?` +await airtable`show me churn risk by customer segment` -```typescript -const insights = await ai.analyze({ - table: 'Sales', - questions: [ - 'What products are performing best this quarter?', - 'Which sales reps are exceeding quota?', - 'What\'s the trend in deal size over time?', - ], -}) - -// Returns: -[ - { - question: 'What products are performing best?', - insight: 'Enterprise Plan leads with $1.2M in Q4, up 45% from Q3. Growth is driven by new security features.', - visualization: { type: 'bar', data: [/* ... */] }, - recommendation: 'Consider bundling security add-ons with Team plan to increase average deal size.', - }, - // ... -] -``` - -### Natural Language Queries - -Query your data conversationally: - -```typescript -const results = await ai.query('Sales', ` - show me all deals over $50k that closed this month - with the enterprise plan, sorted by size -`) - -const forecast = await ai.query('Pipeline', ` - what's our projected revenue for Q2 based on current pipeline? -`) - -const analysis = await ai.query('Customers', ` - which customers are at risk of churning? -`) +// Get actionable recommendations +await airtable`what should we focus on to hit Q2 targets?` ``` ## Automations -Powerful automations without limits: - ```typescript -import { Automation, Trigger, Action } from 'airtable.do' - -// Record-based trigger -const welcomeEmail = Automation.create({ - name: 'Welcome new signup', - trigger: Trigger.recordCreated('Users'), - actions: [ - Action.sendEmail({ - to: '{Email}', - template: 'welcome', - data: { name: '{Name}' }, - }), - Action.createRecord('Onboarding', { - User: '{Record ID}', - Status: 'Started', - StartDate: 'TODAY()', - }), - Action.slack({ - channel: '#new-signups', - message: 'New signup: {Name} ({Email})', - }), - ], -}) +// Create automations naturally +await airtable`when new user signs up, send welcome email and notify #signups` +await airtable`when deal over $100k changes, notify sales director` +await airtable`every Monday at 9am, email pipeline report to sales team` -// Conditional automation -const escalateHighValue = Automation.create({ - name: 'Escalate high-value deals', - trigger: Trigger.fieldChanged('Deals', 'Amount'), - conditions: [ - Condition.field('Amount', '>', 100000), - Condition.field('Status', '!=', 'Won'), - ], - actions: [ - Action.updateRecord({ - Priority: 'Critical', - Owner: '@sales-director', - }), - Action.notify({ - user: '@sales-director', - message: 'High-value deal needs attention: {Name} - ${Amount}', - }), - ], -}) +// Chain automations with agents +await airtable`new leads` + .map(lead => priya`qualify ${lead}`) + .map(lead => airtable`update ${lead} with qualification score`) + .filter(lead => lead.score > 80) + .map(lead => sally`draft outreach for ${lead}`) -// Scheduled automation -const weeklyReport = Automation.create({ - name: 'Weekly pipeline report', - trigger: Trigger.schedule({ day: 'Monday', time: '09:00' }), - actions: [ - Action.runScript(async (base) => { - const deals = await base.table('Deals').records({ - filter: { Status: { in: ['Negotiation', 'Proposal'] } }, - }) - const total = deals.reduce((sum, d) => sum + d.Amount, 0) - return { deals: deals.length, total } - }), - Action.sendEmail({ - to: 'sales-team@company.com', - subject: 'Weekly Pipeline Report', - template: 'pipeline-report', - data: '{{script.output}}', - }), - ], -}) +// Bulk operations as pipelines +await airtable`overdue tasks` + .map(task => airtable`notify owner of ${task}`) + .map(task => airtable`escalate ${task} if over 7 days`) ``` ## Interfaces (Apps) -Build custom apps from your data: - ```typescript -import { Interface, Page, Component } from 'airtable.do' - -const salesDashboard = Interface.create({ - name: 'Sales Dashboard', - pages: [ - Page.create({ - name: 'Overview', - components: [ - Component.number({ - title: 'Pipeline Value', - table: 'Deals', - aggregation: 'SUM', - field: 'Amount', - filter: { Status: { neq: 'Lost' } }, - format: 'currency', - }), - Component.chart({ - title: 'Deals by Stage', - table: 'Deals', - type: 'funnel', - groupBy: 'Status', - value: { field: 'Amount', aggregation: 'SUM' }, - }), - Component.grid({ - title: 'Recent Deals', - table: 'Deals', - fields: ['Name', 'Company', 'Amount', 'Status', 'Owner'], - sort: [{ field: 'Created', direction: 'desc' }], - limit: 10, - editable: true, - }), - Component.kanban({ - title: 'Pipeline', - table: 'Deals', - groupBy: 'Status', - cardFields: ['Name', 'Amount', 'Owner'], - }), - ], - }), - Page.create({ - name: 'Team Performance', - components: [/* ... */], - }), - ], -}) - -// Deploy as standalone app -const appUrl = await salesDashboard.deploy({ - subdomain: 'sales', - auth: 'sso', // or 'public', 'password' -}) -// https://sales.your-org.airtable.do +// Build dashboards naturally +await airtable`create sales dashboard with pipeline value, deals by stage, and recent activity` +await airtable`add funnel chart showing deals by status` +await airtable`add team performance page with quotas and leaderboard` + +// Deploy as standalone apps +await airtable`publish sales dashboard at sales.my-company.com with SSO` +await airtable`create public status page from Projects table` ``` ## API Compatible -Full Airtable API compatibility: - -```typescript -// REST API -GET /v0/{baseId}/{tableName} -POST /v0/{baseId}/{tableName} -PATCH /v0/{baseId}/{tableName} -DELETE /v0/{baseId}/{tableName} - -GET /v0/{baseId}/{tableName}/{recordId} -PATCH /v0/{baseId}/{tableName}/{recordId} -DELETE /v0/{baseId}/{tableName}/{recordId} - -// With standard parameters -?filterByFormula=... -?sort[0][field]=... -?sort[0][direction]=... -?maxRecords=... -?pageSize=... -?offset=... -?view=... -``` - -Existing Airtable SDK code works: +Full Airtable REST API compatibility. Existing Airtable SDK code works - just change the URL: ```typescript import Airtable from 'airtable' const base = new Airtable({ apiKey: process.env.AIRTABLE_TOKEN, - endpointUrl: 'https://your-org.airtable.do', // Just change the URL + endpointUrl: 'https://your-org.airtable.do', // Just change this }).base('appXXXXXXXX') - -const records = await base('Features').select({ - filterByFormula: '{Status} = "In Progress"', - sort: [{ field: 'Priority', direction: 'desc' }], -}).all() ``` ## Architecture ### Durable Object per Base -Each base is fully isolated: - ``` WorkspaceDO (bases, permissions) | +-- BaseDO:product-base - | +-- SQLite: all tables, records, relations - | +-- Views, filters, sorts - | +-- Formulas computed on read + | +-- SQLite: tables, records, relations + | +-- Views, filters, formulas | +-- WebSocket: real-time sync | +-- BaseDO:crm-base +-- BaseDO:content-base +-- AutomationDO (automation engine) - +-- InterfaceDO (custom apps) ``` -### Efficient Storage - -Records stored efficiently: - -```typescript -interface RecordRow { - id: string - table_id: string - fields: object // JSON of field values - created_time: string - modified_time: string - created_by: string - modified_by: string -} - -// Indexes on common query patterns -// Linked records resolved efficiently via SQLite joins -// Formulas computed on read, cached -``` - -### No Row Limits - -```typescript -// SQLite handles millions of rows efficiently -// Pagination for API responses -// Indexed queries stay fast -// R2 for cold storage if needed -``` +Each base is fully isolated. SQLite handles millions of rows efficiently. No artificial limits. ## Migration from Airtable -Import your existing bases: - ```bash -npx airtable-do migrate \ - --token=your_airtable_pat \ - --base=appXXXXXXXX +npx airtable-do migrate --token=your_pat --base=appXXXXXXXX ``` -Imports: -- All tables and fields -- All records and data -- Views and view configurations -- Linked records and relations -- Formulas (converted) -- Automations -- Interfaces (basic conversion) +Imports everything: tables, records, views, linked records, formulas, automations. ## Roadmap @@ -724,12 +342,17 @@ Key areas: ## License -MIT License - Use it however you want. Build your business on it. Fork it. Make it your own. +MIT License - Build your business on it. ---

- airtable.do is part of the dotdo platform. + The row limits end here.
- Website | Docs | Discord + No caps. No per-seat pricing. AI-native. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/algolia/README.md b/rewrites/algolia/README.md index 2002503b..bc033cc9 100644 --- a/rewrites/algolia/README.md +++ b/rewrites/algolia/README.md @@ -1,58 +1,211 @@ # algolia.do -Algolia on Cloudflare Durable Objects - Search-as-a-Service for AI agents. +> Search-as-a-Service. Edge-Native. AI-First. Natural Language. + +Algolia charges $290/month for 100K documents. Typesense Cloud charges $60. algolia.do costs $1. Same features. 100x cheaper. Because search belongs at the edge. + +**algolia.do** is search reimagined for AI agents. Natural language queries. No configuration objects. Just say what you want to find. + +## AI-Native API + +```typescript +import { algolia } from 'algolia.do' // Full SDK +import { algolia } from 'algolia.do/tiny' // Minimal client +import { algolia } from 'algolia.do/instant' // InstantSearch compatible +``` + +Natural language for search: + +```typescript +import { algolia } from 'algolia.do' + +// Talk to it like a colleague +const headphones = await algolia`wireless headphones under $200` +const trending = await algolia`best selling products this week` +const similar = await algolia`products similar to AirPods Pro` + +// Chain like sentences +await algolia`products needing reviews` + .notify(`Your review would help other customers`) + +// Index without boilerplate +await algolia`index products from catalog` +await algolia`add SKU-12345 to products: Wireless Headphones $99` +await algolia`remove discontinued items from inventory` +``` ## The Problem -AI agents need fast, typo-tolerant search. Millions of indexes. Each isolated. Each with their own ranking. +Algolia dominates search: + +| What Algolia Charges | The Reality | +|----------------------|-------------| +| **Search Operations** | $1.50 per 1,000 requests | +| **Records** | $0.40 per 1,000 records/month | +| **Index Replicas** | Each replica costs extra | +| **AI Features** | Premium tier only | +| **Support** | Enterprise pricing for priority | +| **Overages** | Surprise bills when you scale | + +### The Hidden Costs + +Beyond the pricing page: +- Vendor lock-in (proprietary query syntax) +- Limited semantic search +- AI features gated behind enterprise +- Cold start latency on shared infrastructure +- Per-index pricing kills multi-tenant apps -Traditional search services were built for humans: -- One shared cluster for many users -- Centralized infrastructure -- Manual scaling -- Expensive per-index +### What AI Agents Need + +AI agents search differently: +- Natural language, not query DSL +- One index per agent or project +- Semantic understanding built-in +- Infinite indexes without infinite cost +- Edge latency, not datacenter latency + +## The Solution + +**algolia.do** reimagines search for AI: + +``` +Algolia algolia.do +----------------------------------------------------------------- +$290/month for 100K docs ~$1/month +Proprietary query syntax Natural language +Shared infrastructure Your Durable Object +Manual index management Just describe what you want +AI features = enterprise AI-first by default +Cold starts Sub-10ms edge latency +Per-index pricing Pay for compute, not indexes +``` -AI agents need the opposite: -- One index per agent (or per project) -- Distributed by default -- Infinite automatic scaling -- Free at the index level, pay for usage +## One-Click Deploy -## The Vision +```bash +npx create-dotdo algolia +``` -Every AI agent gets their own Algolia. +Search infrastructure on your Cloudflare account. Your data. Your control. ```typescript -import { tom, ralph, priya } from 'agents.do' import { Algolia } from 'algolia.do' -// Each agent has their own isolated search index -const tomSearch = Algolia.for(tom) -const ralphSearch = Algolia.for(ralph) -const priyaSearch = Algolia.for(priya) +export default Algolia({ + name: 'my-search', + domain: 'search.myapp.com', + semantic: true, +}) +``` -// Full Algolia API -await tomSearch.initIndex('reviews').saveObjects([ - { objectID: 'pr-123', title: 'Auth refactor', status: 'approved' } -]) +## Features + +### Searching -const { hits } = await tomSearch.initIndex('reviews').search('auth') +```typescript +// Just say what you want +const results = await algolia`wireless headphones` +const filtered = await algolia`Sony headphones under $150` +const semantic = await algolia`comfortable audio for long flights` + +// AI infers what you need +await algolia`headphones` // returns products +await algolia`headphones by brand` // returns faceted results +await algolia`headphones trending` // returns sorted by popularity ``` -Not a shared index with API keys. Not a multi-tenant nightmare. Each agent has their own complete Algolia instance. +### Indexing -## Features +```typescript +// Index naturally +await algolia`index products from catalog` +await algolia`add to products: Wireless Earbuds $79 category:audio` +await algolia`update SKU-12345 price to $89` +await algolia`remove out-of-stock items` + +// Bulk operations read like instructions +await algolia` + products index: + - SKU-001: Wireless Headphones $99 + - SKU-002: Bluetooth Speaker $49 + - SKU-003: USB-C Cable $15 +` +``` + +### Faceting + +```typescript +// Natural facet queries +const brands = await algolia`headphones by brand` +const priceRanges = await algolia`headphones grouped by price range` +const categories = await algolia`all product categories with counts` + +// Filter like you'd say it +await algolia`Sony or Bose headphones under $200` +await algolia`4+ star products in electronics` +``` -- **Hybrid Search** - FTS5 keyword + Vectorize semantic search -- **Typo Tolerance** - Fuzzy matching out of the box -- **Faceting** - Counts, refinement, hierarchical facets -- **Custom Ranking** - Configure ranking formulas per index -- **InstantSearch Compatible** - Works with Algolia's React/JS libraries -- **Sub-10ms Latency** - Edge-deployed with KV caching -- **MCP Tools** - Model Context Protocol for AI-native search +### Synonyms and Rules + +```typescript +// Configure with natural language +await algolia`synonyms: headphones = earbuds = earphones` +await algolia`synonyms: tv = television = flatscreen` +await algolia`boost: promoted products should rank higher` +await algolia`rule: searches for "cheap" should filter under $50` +``` + +### Hybrid Search + +```typescript +// Semantic search is natural +await algolia`good headphones for noisy offices` +await algolia`something to block out airplane noise` +await algolia`gift for someone who loves music` + +// AI understands intent, not just keywords +``` + +### Real-time Updates + +```typescript +// Live search that updates +const search = await algolia`wireless headphones`.live() + +search.on('update', (results) => { + // New products matching query +}) + +// Or with React +await algolia`trending products` + .subscribe(products => setProducts(products)) +``` + +## Pipeline Chains + +Chain operations without Promise.all: + +```typescript +// Find, analyze, act +await algolia`low stock products` + .map(product => algolia`suppliers for ${product.sku}`) + .map(suppliers => suppliers.notify(`Restock needed`)) + +// Search across multiple indexes +await algolia`search all indexes for "wireless"` + .map(result => result.boost()) + +// Batch operations +await algolia`products without images` + .map(product => product.generateImage()) + .map(product => product.save()) +``` ## Architecture +### Durable Object per Index + ``` +-----------------------+ | algolia.do | @@ -75,70 +228,72 @@ Not a shared index with API keys. Not a multi-tenant nightmare. Each agent has t +-------------------+ ``` -**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each agent's index is a Durable Object. SQLite FTS5 handles keyword search. Vectorize handles semantic search. +Each agent, project, or tenant gets their own Durable Object. SQLite FTS5 for keyword search. Vectorize for semantic search. No shared infrastructure. -## Installation +### Storage Tiers -```bash -npm install algolia.do -``` +| Tier | Storage | Use Case | Latency | +|------|---------|----------|---------| +| **Hot** | SQLite FTS5 | Active indexes | <10ms | +| **Warm** | KV Cache | Frequent queries | <5ms | +| **Cold** | R2 | Index snapshots | <100ms | -## Quick Start +## vs Algolia -### Basic Search +| Feature | Algolia | algolia.do | +|---------|---------|------------| +| **100K docs** | ~$290/month | ~$1/month | +| **Query syntax** | Proprietary DSL | Natural language | +| **Semantic search** | Enterprise only | Built-in | +| **Latency** | 50-100ms | <10ms edge | +| **Multi-tenant** | Expensive | Free (per-DO) | +| **AI features** | Premium tier | Native | +| **Lock-in** | Proprietary | Open source | +| **InstantSearch** | Official | Compatible | -```typescript -import { algoliasearch } from 'algolia.do' +## Use Cases -const client = algoliasearch('your-app-id', 'your-api-key') -const index = client.initIndex('products') +### E-commerce -// Index documents -await index.saveObjects([ - { objectID: '1', title: 'Wireless Headphones', price: 99 }, - { objectID: '2', title: 'Bluetooth Speaker', price: 49 } -]) +```typescript +// Product search +await algolia`red running shoes size 10` +await algolia`gifts under $50 for runners` -// Search -const { hits } = await index.search('wireless', { - filters: 'price < 100', - hitsPerPage: 20 -}) +// Inventory management +await algolia`low stock items needing reorder` + .notify(`Restock alert`) ``` -### Faceted Search +### Documentation ```typescript -const { hits, facets } = await index.search('headphones', { - facets: ['brand', 'category'], - facetFilters: [['brand:Sony', 'brand:Bose']] -}) - -// facets = { brand: { Sony: 15, Bose: 12 }, category: { audio: 27 } } +// Search docs naturally +await algolia`how to configure authentication` +await algolia`examples of rate limiting` +await algolia`errors related to permissions` ``` -### Hybrid Search +### AI Agents ```typescript -const { hits } = await index.search('comfortable travel audio', { - semantic: true, // Enable vector search - hybrid: { - alpha: 0.7 // 70% semantic, 30% keyword - } -}) +import { tom, ralph, priya } from 'agents.do' + +// Each agent has isolated search +await algolia.for(tom)`my reviewed PRs` +await algolia.for(ralph)`components I built` +await algolia.for(priya)`features in the roadmap` ``` -### With InstantSearch +## InstantSearch Compatible ```typescript import { InstantSearch, SearchBox, Hits } from 'react-instantsearch' -import { algoliasearch } from 'algolia.do' - -const client = algoliasearch('your-app-id', 'your-api-key') +import { algolia } from 'algolia.do' function App() { return ( - + @@ -146,51 +301,22 @@ function App() { } ``` -## API Reference +Works with Algolia's React, Vue, Angular, and vanilla JS libraries. -### Client - -```typescript -algoliasearch(appId: string, apiKey: string): AlgoliaClient -``` +## MCP Tools -### Index Operations +AI-native search through Model Context Protocol: ```typescript -// Initialize index -const index = client.initIndex('products') - -// Indexing -await index.saveObjects(objects) -await index.saveObject(object) -await index.partialUpdateObjects(objects) -await index.deleteObjects(objectIDs) -await index.clearObjects() - -// Search -await index.search(query, params) -await index.searchForFacetValues(facetName, facetQuery, params) - -// Settings -await index.setSettings(settings) -await index.getSettings() -``` - -### Search Parameters - -| Parameter | Type | Description | -|-----------|------|-------------| -| `query` | string | Search query | -| `filters` | string | Filter expression | -| `facetFilters` | array | Facet filter array | -| `numericFilters` | array | Numeric filter array | -| `hitsPerPage` | number | Results per page (default: 20) | -| `page` | number | Page number (0-indexed) | -| `facets` | array | Facets to retrieve | -| `attributesToRetrieve` | array | Fields to return | -| `attributesToHighlight` | array | Fields to highlight | -| `semantic` | boolean | Enable semantic search | -| `hybrid.alpha` | number | Semantic vs keyword weight (0-1) | +// Available as MCP tool +await mcp.search({ + index: 'products', + query: 'wireless headphones under $200' +}) + +// Or natural language +await mcp.algolia`find me noise cancelling headphones` +``` ## The Rewrites Ecosystem @@ -205,26 +331,58 @@ algolia.do is part of the rewrites family: | [mongo.do](https://mongo.do) | MongoDB | Document database for AI | | [kafka.do](https://kafka.do) | Kafka | Event streaming for AI | -## Cost Comparison - -### Scenario: 100K documents, 500K searches/month +## Roadmap + +### Search +- [x] Full-text search (FTS5) +- [x] Semantic search (Vectorize) +- [x] Hybrid search +- [x] Faceting +- [x] Typo tolerance +- [x] Synonyms +- [ ] Geo search +- [ ] Personalization + +### Features +- [x] Natural language queries +- [x] Real-time updates +- [x] InstantSearch compatible +- [x] MCP tools +- [ ] Query suggestions +- [ ] A/B testing +- [ ] Analytics + +### AI +- [x] Semantic understanding +- [x] Intent detection +- [ ] Query expansion +- [ ] Result re-ranking +- [ ] Conversational search + +## Contributing + +algolia.do is open source under the MIT license. -| Platform | Monthly Cost | -|----------|--------------| -| Algolia | ~$290 | -| Typesense Cloud | ~$60 | -| **algolia.do** | **~$1** | - -100x+ cost reduction at scale. +```bash +git clone https://github.com/dotdo/algolia.do +cd algolia.do +pnpm install +pnpm test +``` -## Why Durable Objects? +## License -1. **Single-threaded consistency** - No race conditions in ranking -2. **Per-index isolation** - Each agent's data is separate -3. **Automatic scaling** - Millions of indexes, zero configuration -4. **Global distribution** - Search at the edge -5. **SQLite FTS5** - Real full-text search, real performance +MIT License - Search belongs at the edge. -## License +--- -MIT +

+ 100x cheaper. Natural language. Edge-native. +
+ Search reimagined for AI. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/amplitude/README.md b/rewrites/amplitude/README.md index 5585d117..db5325b0 100644 --- a/rewrites/amplitude/README.md +++ b/rewrites/amplitude/README.md @@ -1,54 +1,82 @@ # amplitude.do -> The product analytics platform. Now open source. AI-native. +> Product Analytics. Edge-Native. Open by Default. AI-First. -Amplitude powers product-led growth for thousands of companies. But at $50k+/year for Growth plans, with per-MTU pricing that explodes at scale, and critical features locked behind enterprise tiers, it's time for a new approach. +Amplitude built a $50k+/year product analytics empire. Per-MTU pricing that explodes at scale. Behavioral cohorts locked behind Growth tier. Experiments gated to Enterprise. Data retention capped. Your own data held hostage. -**amplitude.do** reimagines product analytics for the AI era. Every event captured. Every funnel analyzed. Every cohort defined. Zero MTU pricing. +**amplitude.do** is the open-source alternative. Zero MTU pricing. Unlimited events. AI-powered insights. Deploys in minutes, not sales calls. -## The Problem +## AI-Native API -Amplitude built a product analytics empire on: +```typescript +import { amplitude } from 'amplitude.do' // Full SDK +import { amplitude } from 'amplitude.do/tiny' // Minimal client +import { amplitude } from 'amplitude.do/analytics' // Analytics only +``` -- **Per-MTU pricing** - Monthly Tracked Users that scale costs with success -- **Feature gating** - Behavioral cohorts, predictions, experiments on Growth+ -- **Data retention limits** - Historical data access requires higher tiers -- **Governance lock** - Data taxonomy and governance tools are enterprise-only -- **Identity resolution** - Cross-device user stitching costs extra -- **Raw data access** - Exporting your own data requires premium plans +Natural language for product analytics: -10 million MTUs? You're looking at **$100k+/year**. And that's before experiments. +```typescript +import { amplitude } from 'amplitude.do' -## The workers.do Way +// Talk to it like a PM +const funnel = await amplitude`signup to purchase funnel this month` +const cohort = await amplitude`power users: 10+ sessions this week` +const drop = await amplitude`why did signups drop last Tuesday?` -You're scaling fast. Your analytics bill just hit $100k/year. Every tracked user costs money. Every funnel you analyze, every cohort you build, every experiment you run - another line item on a bill that grows with your success. +// Chain like sentences +await amplitude`users who signed up but never purchased` + .notify(`Complete your first purchase for 20% off`) -What if analytics was just a conversation? +// Events that capture themselves +await amplitude`user-123 clicked signup button` +await amplitude`sarah@acme.com upgraded to pro plan` +await amplitude`track pageview /dashboard for user-456` +``` -```typescript -import { amplitude, priya } from 'workers.do' +## The Problem -// Natural language analytics -const funnel = await amplitude`show signup funnel for ${cohort}` -const retention = await amplitude`analyze retention for users who ${action}` -const insights = await amplitude`why did signups drop on ${date}?` +Amplitude dominates product analytics with aggressive pricing: -// Chain analytics into product decisions -const roadmap = await amplitude`analyze user activation patterns` - .map(patterns => amplitude`identify friction points in ${patterns}`) - .map(friction => priya`recommend product changes for ${friction}`) -``` +| What Amplitude Charges | The Reality | +|------------------------|-------------| +| **Per-MTU Pricing** | Costs explode with success | +| **Growth Plan** | $50k+/year minimum | +| **Enterprise Plan** | $100k+/year+ | +| **Behavioral Cohorts** | Locked behind Growth tier | +| **Predictions** | Enterprise only | +| **Experiments** | Enterprise only | +| **Data Retention** | Capped by tier | + +### The MTU Tax + +10 million Monthly Tracked Users? You're paying $100k+/year. And that's before: + +- Behavioral cohorts +- Churn predictions +- A/B experiments +- Raw data export +- SSO/SAML + +Every successful feature launch increases your analytics bill. + +### The Feature Lock + +The most valuable analytics capabilities are paywalled: -One import. Natural language. AI-powered insights that chain into action. +- **Behavioral cohorts** - Define users by what they do, not who they are +- **Predictions** - Churn risk, conversion likelihood, LTV +- **Experiments** - A/B testing with statistical rigor +- **Root cause** - AI explaining why metrics changed -That's analytics that works for you. +Growth companies need these most. Amplitude charges them extra. ## The Solution -**amplitude.do** is Amplitude reimagined: +**amplitude.do** reimagines product analytics: ``` -Traditional Amplitude amplitude.do +Amplitude amplitude.do ----------------------------------------------------------------- Per-MTU pricing Flat: pay for storage, not users $50k/year Growth $0 - run your own @@ -66,38 +94,12 @@ npx create-dotdo amplitude Your own Amplitude instance. Running on Cloudflare. Zero MTU fees. -## Product Analytics That Scale - -Track every user action without worrying about costs: - ```typescript -import { amplitude } from 'amplitude.do' - -// Track events (unlimited) -amplitude.track('Button Clicked', { - button_name: 'Sign Up', - page: 'landing', - variant: 'blue', -}) - -// Identify users -amplitude.identify('user-123', { - email: 'user@example.com', - plan: 'pro', - company: 'Acme Corp', -}) - -// Track revenue -amplitude.revenue({ - productId: 'pro-plan', - price: 99, - quantity: 1, -}) +import { Amplitude } from 'amplitude.do' -// Group users (accounts/companies) -amplitude.group('company', 'acme-corp', { - industry: 'Technology', - employees: 500, +export default Amplitude({ + name: 'my-product', + domain: 'analytics.my-product.com', }) ``` @@ -105,366 +107,202 @@ amplitude.group('company', 'acme-corp', { ### Event Tracking -Comprehensive event capture: - ```typescript -import { Amplitude } from 'amplitude.do' - -const amplitude = new Amplitude({ - apiKey: env.AMPLITUDE_KEY, // Your instance -}) +// Just say it +await amplitude`user-123 clicked signup button` +await amplitude`sarah@acme.com viewed pricing page` +await amplitude`user-456 completed checkout $99` + +// AI infers the event structure +await amplitude`user-123 clicked signup button` +// → { event: 'Button Clicked', properties: { button: 'signup' }, user_id: 'user-123' } + +// Batch events read like a log +await amplitude` + user-123: + - viewed landing page + - clicked signup + - completed registration + - started onboarding +` +``` -// Standard events -amplitude.track('Page Viewed', { - page: '/dashboard', - referrer: document.referrer, -}) +### User Identification -// Custom events -amplitude.track('Feature Used', { - feature_name: 'export', - format: 'csv', - row_count: 1000, -}) - -// Event with timestamp -amplitude.track('Purchase Completed', { - product_id: 'widget-pro', - amount: 49.99, -}, { - time: Date.parse('2024-01-15T10:30:00Z'), -}) +```typescript +// Identify naturally +await amplitude`user-123 is sarah@acme.com on pro plan at Acme Corp` +await amplitude`sarah works at Acme, 500 employees, Technology` -// Batch events -amplitude.trackBatch([ - { event: 'Step 1 Completed', properties: { ... } }, - { event: 'Step 2 Completed', properties: { ... } }, - { event: 'Step 3 Completed', properties: { ... } }, -]) +// AI links identities +await amplitude`link user-123 to device-abc` +await amplitude`merge anonymous-456 into user-123` ``` ### Funnels -Analyze conversion through any sequence: - ```typescript -import { Funnel } from 'amplitude.do/analytics' - -// Define a funnel -const signupFunnel = Funnel({ - name: 'Signup Flow', - steps: [ - { event: 'Landing Page Viewed' }, - { event: 'Sign Up Started' }, - { event: 'Email Verified' }, - { event: 'Profile Completed' }, - { event: 'First Action Taken' }, - ], - timeWindow: '7 days', -}) - -// Query funnel metrics -const results = await signupFunnel.query({ - dateRange: { start: '2024-01-01', end: '2024-03-31' }, - segmentBy: 'platform', -}) - -// Returns conversion at each step -console.log(results) -// { -// steps: [ -// { name: 'Landing Page Viewed', count: 100000, rate: 1.0 }, -// { name: 'Sign Up Started', count: 35000, rate: 0.35 }, -// { name: 'Email Verified', count: 28000, rate: 0.80 }, -// { name: 'Profile Completed', count: 21000, rate: 0.75 }, -// { name: 'First Action Taken', count: 15000, rate: 0.71 }, -// ], -// overallConversion: 0.15, -// medianTimeToConvert: '2.3 days', -// segments: { -// 'ios': { overallConversion: 0.18 }, -// 'android': { overallConversion: 0.14 }, -// 'web': { overallConversion: 0.12 }, -// } -// } +// Funnels are one line +const funnel = await amplitude`signup to purchase funnel this month` +const mobile = await amplitude`iOS signup funnel last week` +const checkout = await amplitude`cart to checkout to payment funnel` + +// AI infers the steps +await amplitude`signup to purchase funnel` +// → Landing → Signup → Email Verified → First Purchase + +// Segment naturally +await amplitude`signup funnel by platform` +await amplitude`checkout funnel iOS vs Android` +await amplitude`onboarding funnel for enterprise users` + +// Find the leaks +await amplitude`where do users drop in checkout?` +await amplitude`biggest drop in signup funnel` ``` ### Retention -Understand user stickiness: - ```typescript -import { Retention } from 'amplitude.do/analytics' - -// N-day retention -const retention = await Retention.nDay({ - startEvent: 'Signed Up', - returnEvent: 'Any Active Event', - dateRange: { start: '2024-01-01', end: '2024-01-31' }, - days: [1, 3, 7, 14, 30], -}) +// Retention curves naturally +await amplitude`day 1 7 30 retention this quarter` +await amplitude`retention for users who completed onboarding` +await amplitude`retention iOS vs Android` -// Returns retention curve -console.log(retention) -// { -// cohortSize: 5000, -// retention: { -// day1: 0.45, -// day3: 0.32, -// day7: 0.25, -// day14: 0.20, -// day30: 0.15, -// } -// } - -// Unbounded retention (return on or after day N) -const unbounded = await Retention.unbounded({ - startEvent: 'Signed Up', - returnEvent: 'Purchase Completed', - dateRange: { start: '2024-01-01', end: '2024-03-31' }, -}) +// Compare cohorts +await amplitude`retention: Jan signups vs Feb signups` +await amplitude`do power users retain better?` -// Bracket retention (custom time windows) -const brackets = await Retention.bracket({ - startEvent: 'Signed Up', - returnEvent: 'Any Active Event', - brackets: ['0-1 days', '2-7 days', '8-30 days', '31-90 days'], -}) +// Unbounded retention +await amplitude`users who ever purchased after signup` ``` ### Cohorts -Define and analyze user segments: - ```typescript -import { Cohort } from 'amplitude.do/analytics' - -// Behavioral cohort -const powerUsers = Cohort({ - name: 'Power Users', - definition: { - all: [ - { event: 'Any Active Event', count: { gte: 10 }, within: '7 days' }, - { property: 'plan', operator: 'is', value: 'pro' }, - ], - }, -}) - -// Query cohort size over time -const trend = await powerUsers.sizeTrend({ - dateRange: { start: '2024-01-01', end: '2024-03-31' }, - granularity: 'week', -}) - -// Export cohort for targeting -const users = await powerUsers.export({ limit: 10000 }) - -// Use cohort in analysis -const funnel = await signupFunnel.query({ - cohort: powerUsers, -}) - -// Lifecycle cohorts (built-in) -const newUsers = Cohort.new({ within: '7 days' }) -const activeUsers = Cohort.active({ within: '7 days' }) -const dormantUsers = Cohort.dormant({ after: '30 days' }) -const resurrectedUsers = Cohort.resurrected({ after: '30 days', active: '7 days' }) +// Behavioral cohorts in plain English +const power = await amplitude`power users: 10+ sessions this week` +const dormant = await amplitude`users inactive for 30 days` +const churning = await amplitude`users likely to churn` + +// Combine conditions naturally +await amplitude`pro users who haven't used feature X` +await amplitude`signed up last month but never purchased` +await amplitude`mobile users with 5+ sessions this week` + +// Lifecycle cohorts just work +await amplitude`new users this week` +await amplitude`resurrected users` +await amplitude`users at risk of churning` + +// Act on cohorts +await amplitude`users who signed up but never purchased` + .notify(`Complete your first purchase for 20% off`) + .each(user => amplitude`flag ${user} for sales outreach`) ``` ### User Journeys -Visualize paths through your product: - ```typescript -import { Journey } from 'amplitude.do/analytics' - -// Most common paths -const paths = await Journey.paths({ - startEvent: 'App Opened', - endEvent: 'Purchase Completed', - steps: 5, - dateRange: { start: '2024-01-01', end: '2024-03-31' }, -}) - -// Pathfinder (Sankey diagram) -const pathfinder = await Journey.pathfinder({ - startEvent: 'Landing Page Viewed', - depth: 4, - minPercentage: 0.01, // Show paths with >1% of users -}) - -// Session replay (if enabled) -const sessions = await Journey.sessions({ - userId: 'user-123', - dateRange: { start: '2024-03-01', end: '2024-03-31' }, -}) +// See how users flow +await amplitude`paths from signup to purchase` +await amplitude`what do users do after checkout?` +await amplitude`how do power users navigate?` + +// Find friction +await amplitude`where do users get stuck?` +await amplitude`dead ends in the product` +await amplitude`rage clicks this week` ``` -### Experiments (A/B Testing) - -Run and analyze experiments: +### Experiments ```typescript -import { Experiment } from 'amplitude.do/experiments' - -// Create experiment -const checkoutExperiment = Experiment({ - name: 'New Checkout Flow', - variants: [ - { name: 'control', weight: 50 }, - { name: 'new_flow', weight: 50 }, - ], - targetCohort: 'all_users', - primaryMetric: { - event: 'Purchase Completed', - type: 'conversion', - }, - secondaryMetrics: [ - { event: 'Purchase Completed', property: 'amount', type: 'sum' }, - { event: 'Checkout Abandoned', type: 'conversion' }, - ], -}) - -// Assign variant -const variant = await checkoutExperiment.getVariant('user-123') -// 'control' or 'new_flow' - -// Track exposure -amplitude.track('$exposure', { - experiment: 'new_checkout_flow', - variant: variant, -}) - -// Query results -const results = await checkoutExperiment.results({ - dateRange: { start: '2024-03-01', end: '2024-03-31' }, -}) - -// Returns statistical analysis -console.log(results) -// { -// variants: { -// control: { -// users: 5000, -// conversions: 500, -// rate: 0.10 -// }, -// new_flow: { -// users: 5000, -// conversions: 600, -// rate: 0.12 -// }, -// }, -// lift: 0.20, // 20% improvement -// confidence: 0.95, -// winner: 'new_flow', -// sampleSizeReached: true, -// } +// A/B tests are questions +await amplitude`new checkout vs control` +await amplitude`which pricing page converts better?` +await amplitude`is dark mode increasing engagement?` + +// Run experiments naturally +await amplitude`run experiment: blue button vs green button` + .on(`signup page visitors`) + .measure(`signups`) + +// Get results +await amplitude`checkout experiment results` +// → { winner: 'new_flow', lift: '+20%', confidence: 95% } + +// Feature flags +await amplitude`should user-123 see new feature?` +await amplitude`roll out dark mode to 10% of users` ``` -## AI-Native Features - -### Natural Language Analytics +## AI-Native Analytics -Ask questions about your product: +### Root Cause Analysis ```typescript -import { ask } from 'amplitude.do' - -// Simple questions -const answer1 = await ask('how many users signed up last week?') -// Returns: { value: 3500, trend: '+12%', visualization: } - -// Funnel questions -const answer2 = await ask('what is our checkout funnel conversion?') -// Returns: { funnel: [...], overallConversion: 0.15, visualization: } - -// Comparative questions -const answer3 = await ask('how does iOS retention compare to Android?') -// Returns: { comparison: {...}, visualization: } +// Ask why, get answers +await amplitude`why did signups drop last Tuesday?` +// → { factors: ['ad campaign ended', 'homepage A/B test'], confidence: 0.85 } -// Root cause questions -const answer4 = await ask('why did signups drop last Tuesday?') -// Returns: { factors: [...], narrative: "...", visualization: } +await amplitude`what changed when conversion improved?` +await amplitude`why is iOS retention higher than Android?` +await amplitude`what do power users do differently?` ``` ### Predictive Analytics -AI-powered predictions: - ```typescript -import { predict } from 'amplitude.do' - -// Churn prediction -const churnRisk = await predict.churn({ - userId: 'user-123', - timeframe: '30 days', -}) -// { probability: 0.72, factors: ['no login in 14 days', 'support ticket open'] } +// Predictions as questions +await amplitude`will user-123 churn?` +// → { probability: 0.72, factors: ['no login in 14 days', 'support ticket open'] } -// Conversion prediction -const conversionLikelihood = await predict.conversion({ - userId: 'user-123', - event: 'Purchase Completed', - timeframe: '7 days', -}) -// { probability: 0.35, factors: ['viewed pricing 3x', 'pro plan interest'] } +await amplitude`likelihood user-123 purchases this week` +await amplitude`predicted LTV for user-123` -// LTV prediction -const ltv = await predict.ltv({ - userId: 'user-123', - timeframe: '12 months', -}) -// { predicted: 450, confidence: 0.8, cohortAverage: 380 } +// Batch predictions +await amplitude`users likely to churn this month` + .each(user => amplitude`flag ${user} for retention outreach`) ``` ### Auto-Instrumentation -AI captures events automatically: - ```typescript -import { autoTrack } from 'amplitude.do' - -// Auto-track all user interactions -autoTrack({ - clicks: true, // Button clicks, links - forms: true, // Form submissions - pageViews: true, // Page navigation - scrollDepth: true, // Scroll percentage - rage: true, // Rage clicks (frustration) - errors: true, // JavaScript errors -}) +// AI captures everything automatically +await amplitude`auto-track my-app.com` // AI names events semantically -// "Submit Button Clicked" instead of "click_btn_abc123" +// "Submit Button Clicked" not "click_btn_abc123" +// "Pricing Page Viewed" not "pageview_/pricing" + +// Rage click detection automatic +// Dead click detection automatic +// Session recording optional ``` ### AI Agents as Analysts -AI agents can analyze your product: - ```typescript import { priya, quinn } from 'agents.do' -import { amplitude } from 'amplitude.do' -// Product manager analyzes user behavior -const analysis = await priya` - analyze our user activation funnel and identify - the biggest drop-off points with recommendations -` +// PM analyzes user behavior +await priya`analyze our activation funnel and recommend changes` + .map(recommendations => amplitude`create issues for ${recommendations}`) // QA finds issues through event patterns -const issues = await quinn` - look for anomalies in our event data that might - indicate bugs or UX problems -` +await quinn`find anomalies in our event data` + .map(anomalies => amplitude`alert on-call for ${anomalies}`) + +// Chain insights into action +await amplitude`weekly product insights` + .map(insights => priya`prioritize these for Q2`) + .map(priorities => amplitude`create dashboard for ${priorities}`) ``` ## Architecture -### Event Ingestion - -High-throughput event pipeline: +### Event Pipeline ``` Client SDK --> Edge Worker --> Event DO --> Storage Tiers @@ -475,116 +313,92 @@ Client SDK --> Edge Worker --> Event DO --> Storage Tiers (Schema) (GeoIP, UA) ``` -### Durable Objects +### Durable Object per Product ``` - +------------------------+ - | amplitude.do Worker | - | (API + Ingestion) | - +------------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | UserDO | | EventStoreDO | | AnalyticsDO | - | (Identity Graph) | | (Event Buffer) | | (Query Engine) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +--------------------+--------------------+ - | | | - +----------+ +-----------+ +------------+ - | D1 | | R2 | | Analytics | - | (Users) | | (Events) | | Engine | - +----------+ +-----------+ +------------+ +ProductDO (config, users, events) + | + +-- UsersDO (identity graph) + | |-- SQLite: User profiles + | +-- Identity resolution + | + +-- EventsDO (event buffer) + | |-- SQLite: Recent events (7 days) + | +-- R2: Historical events (Parquet) + | + +-- AnalyticsDO (query engine) + | |-- Analytics Engine: Real-time aggregations + | +-- Cohort definitions + | + +-- ExperimentsDO (A/B testing) + |-- SQLite: Experiment configs + +-- Variant assignments ``` ### Storage Tiers -- **Hot (SQLite/D1)** - Recent events (7 days), user profiles, cohort definitions -- **Warm (R2 Parquet)** - Historical events, queryable -- **Cold (R2 Archive)** - Long-term retention, compressed +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Recent events (7 days) | <10ms | +| **Warm** | R2 Parquet | Historical events (months) | <100ms | +| **Cold** | R2 Archive | Long-term retention (years) | <1s | ### Query Engine -Built on Cloudflare Analytics Engine for real-time aggregations: +Real-time aggregations on Analytics Engine. Billions of events. Sub-second queries. -```typescript -// Analytics Engine handles high-cardinality aggregations -AnalyticsEngine.write({ - indexes: [userId, eventName, timestamp], - blobs: [JSON.stringify(properties)], -}) +## vs Amplitude -// Query across billions of events -AnalyticsEngine.query({ - timeRange: { start: '-30d' }, - metrics: ['count', 'uniqueUsers'], - dimensions: ['eventName', 'platform'], - filters: [{ dimension: 'country', value: 'US' }], -}) -``` +| Feature | Amplitude | amplitude.do | +|---------|-----------|--------------| +| **Pricing** | Per-MTU ($50k+/year) | Flat storage cost | +| **Cohorts** | Growth tier only | Free | +| **Predictions** | Enterprise only | Free | +| **Experiments** | Enterprise only | Free | +| **Data Retention** | Tier-limited | Unlimited | +| **Raw Data Export** | Premium only | Your data, always | +| **Architecture** | Centralized | Edge-native, global | +| **Data Location** | Amplitude's cloud | Your Cloudflare account | +| **Lock-in** | Years of data trapped | MIT licensed | -## SDKs +## Use Cases -### Browser SDK +### Product Analytics ```typescript -import { Amplitude } from 'amplitude.do/browser' - -const amplitude = new Amplitude({ - apiKey: 'your-api-key', - serverUrl: 'https://your-org.amplitude.do', -}) - -amplitude.init() -amplitude.track('Event Name', { property: 'value' }) +// Complete product analytics in natural language +await amplitude`daily active users this month` +await amplitude`feature adoption for new feature X` +await amplitude`power users vs casual users comparison` ``` -### React SDK - -```tsx -import { AmplitudeProvider, useTrack, useIdentify } from 'amplitude.do/react' - -function App() { - return ( - - - - ) -} - -function MyComponent() { - const track = useTrack() - const identify = useIdentify() - - return ( - - ) -} +### Growth Experiments + +```typescript +// Run experiments, ship winners +await amplitude`onboarding A vs B experiment results` + .map(result => amplitude`roll out ${result.winner} to 100%`) ``` -### Node.js SDK +### Churn Prevention ```typescript -import { Amplitude } from 'amplitude.do/node' +// Find and save at-risk users +await amplitude`users likely to churn this month` + .map(users => amplitude`create cohort at-risk from ${users}`) + .map(cohort => amplitude`notify ${cohort} with retention offer`) +``` -const amplitude = new Amplitude({ - apiKey: 'your-api-key', - serverUrl: 'https://your-org.amplitude.do', -}) +### Product-Led Growth -// Server-side tracking -amplitude.track('Server Event', { userId: 'user-123' }) +```typescript +// Activation insights +await amplitude`users who activated vs didn't - what's different?` + .map(insights => priya`recommend onboarding changes`) ``` ## Migration from Amplitude -### Export Your Data - ```bash # Export from Amplitude amplitude export --project YOUR_PROJECT --start 2024-01-01 --end 2024-03-31 @@ -595,43 +409,42 @@ npx amplitude-migrate import ./export.json ### API Compatibility -Drop-in replacement for Amplitude HTTP API: - -``` -Endpoint Status ------------------------------------------------------------------ -POST /2/httpapi Supported -POST /batch Supported -POST /identify Supported -POST /groupidentify Supported -GET /userprofile Supported -POST /export Supported -``` - -### Taxonomy Migration - -```bash -# Export event taxonomy -npx amplitude-migrate export-taxonomy --source amplitude - -# Import to amplitude.do -npx amplitude-migrate import-taxonomy ./taxonomy.json -``` +Drop-in replacement for Amplitude HTTP API. All endpoints supported. ## Roadmap -- [x] Event tracking (browser, Node.js, React) -- [x] Funnel analysis -- [x] Retention analysis -- [x] Cohort builder -- [x] User journeys (paths) -- [x] Natural language queries -- [ ] A/B testing (full) -- [ ] Predictions (churn, LTV) -- [ ] Session replay -- [ ] Feature flags integration -- [ ] Data governance -- [ ] Real-time alerts +### Core Analytics +- [x] Event Tracking +- [x] User Identification +- [x] Identity Resolution +- [x] Funnels +- [x] Retention +- [x] Cohorts +- [x] User Journeys +- [ ] Session Replay +- [ ] Heatmaps + +### AI +- [x] Natural Language Queries +- [x] Root Cause Analysis +- [x] Predictive Analytics +- [x] Auto-Instrumentation +- [ ] Anomaly Detection +- [ ] AI Insights Digest + +### Experiments +- [x] A/B Testing +- [x] Statistical Analysis +- [x] Feature Flags +- [ ] Holdout Groups +- [ ] Multi-Armed Bandits + +### Platform +- [x] Real-time Streaming +- [x] Batch Import +- [x] Data Export +- [ ] Warehouse Sync +- [ ] Reverse ETL ## Why Open Source? @@ -644,14 +457,30 @@ Product analytics shouldn't cost more as you succeed: Amplitude showed the world what product analytics could be. **amplitude.do** makes it accessible to everyone. +## Contributing + +amplitude.do is open source under the MIT license. + +```bash +git clone https://github.com/dotdo/amplitude.do +cd amplitude.do +pnpm install +pnpm test +``` + ## License -MIT License - Use it however you want. Track all the events. Run all the experiments. +MIT License - Track all the events. Run all the experiments. ---

- amplitude.do is part of the dotdo platform. + The $50k/year tax ends here.
- Website | Docs | Discord + Zero MTU. Unlimited events. AI-first. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/asana/README.md b/rewrites/asana/README.md index 36ca79f0..e65cf93c 100644 --- a/rewrites/asana/README.md +++ b/rewrites/asana/README.md @@ -55,36 +55,49 @@ Your own Asana. Running on Cloudflare. AI that actually coordinates work. npx dotdo add asana ``` -## The workers.do Way +## AI-Native API -You're building a product. Your team needs work coordination. Asana wants $60k/year with goals and portfolios locked behind premium tiers. AI is an afterthought. There's a better way. +```typescript +import { asana } from 'asana.do' // Full SDK +import { asana } from 'asana.do/tiny' // Minimal client +import { asana } from 'asana.do/goals' // Goals-focused operations +``` -**Natural language. Tagged templates. AI agents that work.** +Natural language for work management: ```typescript import { asana } from 'asana.do' -import { priya, ralph, quinn } from 'agents.do' -// Talk to your work manager like a human +// Talk to it like a colleague const blocked = await asana`what's blocking the mobile app?` -const capacity = await asana`does the team have bandwidth?` +const capacity = await asana`does engineering have bandwidth?` const priorities = await asana`what should I focus on today?` + +// Chain like sentences +await asana`overdue tasks in Q1 Launch` + .map(task => asana`notify assignee of ${task}`) + +// Tasks that delegate themselves +await asana`new feature requests this week` + .analyze() // AI categorizes and estimates + .assign() // best-fit team members + .schedule() // realistic timelines ``` **Promise pipelining - chain work without Promise.all:** ```typescript // Goal tracking pipeline -const tracked = await asana`get Q1 goals` - .map(goal => priya`analyze progress on ${goal}`) +await asana`Q1 goals` + .map(goal => asana`analyze progress on ${goal}`) .map(goal => asana`update status for ${goal}`) - .map(status => mark`write executive summary for ${status}`) + .notify(`Q1 goal status updated`) // Task delegation pipeline -const delegated = await asana`get new tasks` - .map(task => priya`analyze requirements for ${task}`) - .map(task => ralph`estimate ${task}`) +await asana`new tasks in backlog` + .map(task => asana`estimate ${task}`) .map(task => asana`assign ${task} to best fit`) + .map(task => asana`add ${task} to current sprint`) ``` One network round trip. Record-replay pipelining. Workers working for you. @@ -96,39 +109,17 @@ One network round trip. Record-replay pipelining. Workers working for you. The atomic unit of work: ```typescript -import { Task, Project, Section } from 'asana.do' - -// Create a task -const task = await Task.create({ - name: 'Implement user authentication', - assignee: '@alice', - dueDate: '2025-01-20', - priority: 'high', - description: ` - Implement OAuth2 authentication flow: - - Google SSO - - GitHub SSO - - Email/password fallback - `, - projects: ['backend-q1'], - tags: ['security', 'authentication'], -}) - -// Add subtasks -await task.addSubtask({ - name: 'Set up OAuth providers', - assignee: '@alice', - dueDate: '2025-01-15', -}) - -await task.addSubtask({ - name: 'Implement session management', - assignee: '@alice', - dueDate: '2025-01-18', -}) - -// Add dependencies -await task.addDependency(otherTask) +// Create tasks naturally +await asana`create "Implement user authentication" for Alice due Jan 20` +await asana`add OAuth2 flow task - Google SSO, GitHub SSO, email fallback` + +// Subtasks just work +await asana`add subtask "Set up OAuth providers" due Jan 15 to auth task` +await asana`add subtask "Implement session management" due Jan 18 to auth task` + +// Dependencies read like speech +await asana`session management depends on OAuth setup` +await asana`payment integration blocked by legal review` ``` ### Projects @@ -136,26 +127,14 @@ await task.addDependency(otherTask) Organize tasks into projects: ```typescript -const project = await Project.create({ - name: 'Backend Q1 2025', - team: 'engineering', - owner: '@alice', - dueDate: '2025-03-31', - template: 'software-development', - defaultView: 'board', - sections: [ - Section.create({ name: 'Backlog' }), - Section.create({ name: 'Ready' }), - Section.create({ name: 'In Progress' }), - Section.create({ name: 'In Review' }), - Section.create({ name: 'Done' }), - ], - customFields: { - Priority: Field.dropdown(['Low', 'Medium', 'High', 'Critical']), - Points: Field.number(), - Sprint: Field.dropdown(['Sprint 1', 'Sprint 2', 'Sprint 3']), - }, -}) +// Create projects naturally +await asana`create project "Backend Q1 2025" for engineering owned by Alice` +await asana`add sections Backlog, Ready, In Progress, In Review, Done to Backend Q1` + +// Set up custom fields +await asana`add priority field to Backend Q1 with Low, Medium, High, Critical` +await asana`add points field to Backend Q1` +await asana`add sprint field to Backend Q1 with Sprint 1, Sprint 2, Sprint 3` ``` ### Views @@ -163,39 +142,12 @@ const project = await Project.create({ Multiple ways to see your work: ```typescript -// List View (default) -const listView = View.list({ - groupBy: 'section', - sort: [{ field: 'dueDate', direction: 'asc' }], - filter: { assignee: '@me', completed: false }, -}) - -// Board View (Kanban) -const boardView = View.board({ - columns: 'section', - cardFields: ['assignee', 'dueDate', 'priority'], - swimlanes: 'assignee', -}) - -// Timeline View (Gantt) -const timelineView = View.timeline({ - dateField: 'dueDate', - color: 'priority', - showDependencies: true, -}) - -// Calendar View -const calendarView = View.calendar({ - dateField: 'dueDate', - color: 'project', -}) - -// Workload View -const workloadView = View.workload({ - team: 'engineering', - capacity: { type: 'points', perWeek: 40 }, - dateRange: { start: 'now', end: '+4 weeks' }, -}) +// Query views naturally +await asana`my incomplete tasks sorted by due date` +await asana`Backend Q1 board view` +await asana`Backend Q1 timeline with dependencies` +await asana`my calendar for this month` +await asana`engineering workload next 4 weeks` ``` ### Portfolios @@ -203,32 +155,14 @@ const workloadView = View.workload({ See across all projects: ```typescript -import { Portfolio } from 'asana.do' - -const q1Portfolio = await Portfolio.create({ - name: 'Q1 2025 Initiatives', - owner: '@cto', - projects: ['backend-q1', 'frontend-q1', 'mobile-q1', 'infrastructure-q1'], - fields: [ - Field.status(), // Project health - Field.progress(), // Completion percentage - Field.date('dueDate'), - Field.person('owner'), - Field.custom('priority'), - Field.custom('budget'), - ], -}) - -// Portfolio views -const statusView = portfolio.view('status', { - groupBy: 'status', - sort: 'dueDate', -}) - -const timelineView = portfolio.view('timeline', { - showMilestones: true, - color: 'status', -}) +// Create portfolios naturally +await asana`create portfolio "Q1 2025 Initiatives" for CTO` +await asana`add Backend Q1, Frontend Q1, Mobile Q1, Infrastructure Q1 to Q1 portfolio` + +// Query portfolio views +await asana`Q1 portfolio by status` +await asana`Q1 portfolio timeline with milestones` +await asana`projects at risk in Q1 portfolio` ``` ### Goals @@ -236,41 +170,18 @@ const timelineView = portfolio.view('timeline', { Align work to outcomes: ```typescript -import { Goal } from 'asana.do' - -// Company goal -const companyGoal = await Goal.create({ - name: 'Reach $10M ARR by end of 2025', - owner: '@ceo', - timeframe: '2025', - metric: { - type: 'currency', - current: 5200000, - target: 10000000, - }, -}) - -// Team goal supporting company goal -const teamGoal = await Goal.create({ - name: 'Launch enterprise features', - owner: '@pm', - timeframe: 'Q1 2025', - parent: companyGoal, - supportingWork: [ - { type: 'project', id: 'enterprise-features' }, - { type: 'portfolio', id: 'q1-initiatives' }, - ], - metric: { - type: 'percentage', - current: 0, - target: 100, - autoCalculate: 'from-projects', // Progress from linked projects - }, -}) - -// Goals cascade automatically -// When projects complete, goal progress updates -// When goals update, parent goals recalculate +// Create goals naturally +await asana`create goal "Reach $10M ARR" for 2025 owned by CEO` +await asana`current ARR is $5.2M target is $10M` + +// Team goals cascade automatically +await asana`create goal "Launch enterprise features" for Q1 supporting $10M ARR` +await asana`link Backend Q1 to enterprise features goal` + +// Check goal progress +await asana`Q1 goals progress` +await asana`goals at risk` +await asana`how are we tracking to $10M ARR?` ``` ## AI-Native Work Management @@ -282,22 +193,16 @@ AI doesn't just help - it coordinates. Describe work naturally: ```typescript -import { ai } from 'asana.do' - -// Create tasks from natural language -const tasks = await ai.createTasks(` +// Just describe the work - AI creates the tasks +await asana` We need to launch the new pricing page. Alice should design the page by Friday. Bob needs to implement it, depends on design. Carol will write the copy, can start now. Dave does QA before launch. -`) +` -// AI creates structured tasks with: -// - Assignees extracted -// - Dependencies mapped -// - Due dates inferred -// - Subtasks where appropriate +// AI extracts assignees, maps dependencies, infers due dates ``` ### AI Project Planning @@ -305,40 +210,11 @@ const tasks = await ai.createTasks(` Let AI structure your project: ```typescript -const projectPlan = await ai.planProject({ - description: 'Build a customer feedback portal', - team: ['@alice', '@bob', '@carol'], - deadline: '2025-03-01', - style: 'agile', -}) - -// Returns: -{ - phases: [ - { - name: 'Discovery', - duration: '1 week', - tasks: [ - { name: 'User interviews', assignee: '@carol', points: 3 }, - { name: 'Competitive analysis', assignee: '@alice', points: 2 }, - ], - }, - { - name: 'Design', - duration: '2 weeks', - tasks: [/* ... */], - }, - // ... - ], - milestones: [ - { name: 'Design Review', date: '2025-01-24' }, - { name: 'Beta Launch', date: '2025-02-14' }, - { name: 'Production Launch', date: '2025-03-01' }, - ], - risks: [ - { risk: 'Third-party API integration', mitigation: 'Start early, have fallback' }, - ], -} +// Describe what you need - get a full plan +await asana`plan customer feedback portal for Alice Bob Carol by March 1` + .breakdown() // phases, milestones, dependencies + .assign() // best-fit team members + .schedule() // realistic timelines ``` ### AI Task Assignment @@ -346,31 +222,14 @@ const projectPlan = await ai.planProject({ Smart work distribution: ```typescript -// When new task comes in -const assignment = await ai.assignTask(task, { - team: 'engineering', - factors: { - expertise: true, // Match skills to task - workload: true, // Current capacity - availability: true, // Calendar, PTO - preference: true, // Historical preferences - development: true, // Growth opportunities - }, -}) - -// Returns: -{ - recommended: '@alice', - confidence: 0.91, - reasoning: 'Best expertise match for auth work. Current load at 70%. No conflicts in timeline.', - alternatives: [ - { person: '@bob', score: 0.78, note: 'Good skills, but already on 2 auth tasks' }, - ], - considerations: [ - 'Alice completed 8 similar tasks with 95% on-time rate', - 'Task aligns with her Q1 growth goal: security expertise', - ], -} +// AI assigns based on expertise, workload, availability +await asana`assign new auth task to best fit on engineering` + +// Or let AI rebalance the team +await asana`rebalance engineering workload` + +// Check capacity before assigning +await asana`who can take the payment integration?` ``` ### AI Goal Tracking @@ -378,33 +237,13 @@ const assignment = await ai.assignTask(task, { AI monitors and reports on goals: ```typescript -// AI analyzes goal progress -const goalAnalysis = await ai.analyzeGoal(revenueGoal, { - depth: 'full', - forecast: true, - recommendations: true, -}) - -// Returns: -{ - status: 'at_risk', - progress: 0.52, // 52% to target - timeElapsed: 0.75, // 75% through timeframe - forecast: { - predicted: 8500000, // Predicted end value - probability: 0.65, // 65% chance of hitting target - trend: 'slowing', - }, - contributing: [ - { project: 'enterprise-sales', impact: 'high', status: 'on_track' }, - { project: 'pricing-update', impact: 'medium', status: 'delayed' }, - ], - recommendations: [ - 'Pricing update is 2 weeks behind - consider adding resources', - 'Enterprise sales pipeline strong - accelerate onboarding', - ], - narrative: 'Revenue goal is at risk. While enterprise sales are performing well, the pricing page delay impacts our ability to capture mid-market customers. Recommend prioritizing pricing page launch.', -} +// Check goal health +await asana`how is $10M ARR goal tracking?` +await asana`goals at risk this quarter` +await asana`what's impacting enterprise features goal?` + +// Get recommendations +await asana`recommendations for Q1 goals` ``` ### AI Status Updates @@ -412,42 +251,12 @@ const goalAnalysis = await ai.analyzeGoal(revenueGoal, { Never write another status update: ```typescript -// Generate status update from work activity -const update = await ai.generateUpdate({ - scope: 'project', // or 'portfolio', 'goal', 'team' - project: 'backend-q1', - period: 'this week', - audience: 'stakeholders', -}) - -// Returns: -` -## Backend Q1 - Weekly Update (Jan 13-17) - -### Summary -Strong progress this week with 12 tasks completed. Authentication module shipped to staging. - -### Completed -- User authentication flow (shipped to staging) -- Database migration scripts -- API rate limiting implementation - -### In Progress -- Payment integration (60% complete, on track) -- Admin dashboard backend - -### Blockers -- Waiting on legal review for payment terms - -### Next Week -- Complete payment integration -- Start caching layer implementation +// Generate updates from work activity +await asana`Backend Q1 status update for stakeholders` +await asana`weekly summary for engineering` +await asana`Q1 portfolio executive summary` -### Metrics -- 12 tasks completed (up from 8 last week) -- 3 days average cycle time -- On track for Q1 deadline -` +// AI writes the narrative from actual work data ``` ### Natural Language Queries @@ -455,10 +264,10 @@ Strong progress this week with 12 tasks completed. Authentication module shipped Query your work naturally: ```typescript -const blocked = await ai.query`what's blocking the mobile app?` -const capacity = await ai.query`does the engineering team have bandwidth?` -const deadline = await ai.query`will we hit the Q1 launch date?` -const priorities = await ai.query`what should I focus on today?` +await asana`what's blocking the mobile app?` +await asana`does engineering have bandwidth?` +await asana`will we hit the Q1 launch date?` +await asana`what should I focus on today?` ``` ## Inbox & My Tasks @@ -466,25 +275,16 @@ const priorities = await ai.query`what should I focus on today?` Personal work management: ```typescript -import { Inbox, MyTasks } from 'asana.do' - -// Inbox - all notifications -const inbox = await Inbox.get({ - unread: true, - types: ['assigned', 'mentioned', 'commented', 'liked'], -}) - -// My Tasks - personal organization -const myTasks = await MyTasks.get({ - sections: ['Recently Assigned', 'Today', 'Upcoming', 'Later'], - sort: 'dueDate', -}) - -// Move between sections -await task.moveToSection('Today') - -// Set personal due date (different from task due date) -await task.setMyDueDate('2025-01-15') +// Your inbox +await asana`my unread notifications` +await asana`tasks assigned to me this week` +await asana`mentions and comments` + +// My Tasks sections +await asana`my tasks for today` +await asana`upcoming tasks` +await asana`move auth task to Today` +await asana`snooze payment task until Monday` ``` ## Rules (Automations) @@ -492,43 +292,15 @@ await task.setMyDueDate('2025-01-15') Automate repetitive work: ```typescript -import { Rule, Trigger, Action } from 'asana.do' - -// Move to section based on status -const moveWhenDone = Rule.create({ - name: 'Move completed tasks', - project: 'backend-q1', - trigger: Trigger.fieldChange('completed', true), - actions: [ - Action.moveToSection('Done'), - Action.addComment('Moved to Done'), - ], -}) - -// Notify on due date approaching -const dueDateReminder = Rule.create({ - name: 'Due date reminder', - project: 'backend-q1', - trigger: Trigger.dueDateApproaching({ days: 1 }), - conditions: [ - Condition.field('completed', false), - ], - actions: [ - Action.notifyAssignee('Task due tomorrow: {task_name}'), - ], -}) - -// Auto-assign based on section -const autoAssign = Rule.create({ - name: 'Auto-assign by section', - project: 'support-queue', - trigger: Trigger.taskAddedToSection('Urgent'), - actions: [ - Action.setField('priority', 'Critical'), - Action.addFollower('@oncall'), - Action.notify({ channel: 'slack', to: '#support-urgent' }), - ], -}) +// Set up rules naturally +await asana`when task completed move to Done section` +await asana`when due date is tomorrow notify assignee` +await asana`when task added to Urgent set priority Critical and notify #support-urgent` + +// Chain automations with pipelining +await asana`overdue tasks in support-queue` + .map(task => asana`escalate ${task} to manager`) + .map(task => asana`notify customer about ${task}`) ``` ## API Compatible @@ -591,15 +363,9 @@ All changes sync instantly: ```typescript // Subscribe to project changes -project.subscribe((event) => { - switch (event.type) { - case 'task:created': - case 'task:updated': - case 'task:moved': - case 'task:completed': - // Update UI - } -}) +await asana`watch Backend Q1 for changes` + .on('task:created', task => console.log('New task:', task)) + .on('task:completed', task => console.log('Done:', task)) ``` ### Storage diff --git a/rewrites/bamboohr/README.md b/rewrites/bamboohr/README.md index ac3195ff..0246c143 100644 --- a/rewrites/bamboohr/README.md +++ b/rewrites/bamboohr/README.md @@ -4,29 +4,37 @@ You're a startup founder. Your team just hit 50 people. HR software wants $6/employee/month - that's $3,600/year just to track PTO and store employee records. For what is essentially a spreadsheet with a nice UI. Your growing team shouldn't be penalized for growing. -## The workers.do Way +**bamboohr.do** is the open-source alternative. Run it yourself or let us host it. No per-employee fees. Ever. + +## AI-Native API ```typescript -import { bamboohr, olive } from 'bamboohr.do' - -// Natural language HR operations -const balance = await bamboohr`get ${employee} PTO balance` -const directory = await bamboohr`who reports to ${manager}?` -const policy = await olive`what's our remote work policy?` - -// Promise pipelining for onboarding workflows -const onboarded = await bamboohr`hire ${candidate}` - .map(emp => bamboohr`provision ${emp} with ${apps}`) - .map(emp => bamboohr`assign ${emp} to ${team}`) - .map(emp => olive`guide ${emp} through day one`) - -// AI-assisted performance reviews -const review = await olive`prepare ${employee}'s quarterly review` - .map(draft => manager`review and approve ${draft}`) - .map(final => bamboohr`submit ${final} to HR`) +import { bamboohr } from 'bamboohr.do' // Full SDK +import { bamboohr } from 'bamboohr.do/tiny' // Minimal client +import { olive } from 'bamboohr.do/agents' // AI HR assistant ``` -One API call. Natural language. AI handles the complexity. +Natural language for HR workflows: + +```typescript +import { bamboohr } from 'bamboohr.do' + +// Talk to it like a colleague +await bamboohr`hire Sarah Chen as Engineer starting Jan 15` +await bamboohr`who's on PTO next week?` +await bamboohr`employees in Engineering with anniversaries this month` + +// Chain like sentences +await bamboohr`new hires this month` + .map(emp => bamboohr`assign ${emp} onboarding workflow`) + .map(emp => bamboohr`schedule ${emp} orientation`) + +// Onboarding that flows naturally +await bamboohr`onboard Sarah Chen` + .provision(`laptop, Slack, GitHub`) + .assign(`Engineering`) + .buddy(`Alex Kim`) +``` ## The Problem @@ -60,16 +68,11 @@ npx create-dotdo bamboohr Your SMB-grade HR system is live. No per-employee fees. Ever. ```typescript -import { hr } from 'bamboohr.do' - -// Add your first employee -await hr.employees.add({ - firstName: 'Sarah', - lastName: 'Chen', - email: 'sarah@startup.com', - department: 'Engineering', - startDate: '2025-01-15', - manager: 'alex-kim' +import { BambooHR } from 'bamboohr.do' + +export default BambooHR({ + name: 'acme-startup', + domain: 'hr.acme.com', }) ``` @@ -80,22 +83,16 @@ await hr.employees.add({ The heart of any HR system. Everyone in one place. ```typescript -// Full employee profiles -const sarah = await hr.employees.get('sarah-chen') - -sarah.firstName // Sarah -sarah.department // Engineering -sarah.manager // Alex Kim -sarah.location // San Francisco -sarah.startDate // 2025-01-15 -sarah.tenure // "2 months" (computed) - -// Org chart built-in -const engineering = await hr.directory.team('engineering') -// Returns hierarchy: manager -> reports -> their reports - -// Search across all fields -await hr.directory.search('san francisco engineering') +// Find anyone +const sarah = await bamboohr`Sarah Chen` +const engineering = await bamboohr`everyone in Engineering` +const managers = await bamboohr`all managers in San Francisco` + +// AI infers what you need +await bamboohr`Sarah Chen` // returns employee +await bamboohr`Sarah's manager` // returns Alex Kim +await bamboohr`who reports to Alex?` // returns team list +await bamboohr`org chart Engineering` // returns hierarchy ``` ### Time Off @@ -103,34 +100,21 @@ await hr.directory.search('san francisco engineering') Request, approve, track. No spreadsheets. ```typescript -// Check balances -const balance = await hr.timeOff.balance('sarah-chen') -// { -// vacation: { available: 80, accrued: 96, used: 16, unit: 'hours' }, -// sick: { available: 40, accrued: 40, used: 0, unit: 'hours' }, -// personal: { available: 16, accrued: 16, used: 0, unit: 'hours' } -// } - -// Request time off -await hr.timeOff.request({ - employee: 'sarah-chen', - type: 'vacation', - start: '2025-03-17', - end: '2025-03-21', - hours: 40, - notes: 'Spring break trip' -}) +// Natural as asking a coworker +await bamboohr`how much PTO does Sarah have?` +await bamboohr`who's out this week?` +await bamboohr`vacation calendar for March` + +// Request time off in one line +await bamboohr`Sarah taking March 17-21 off for spring break` // Auto-routes to manager for approval -// Manager approves -await hr.timeOff.approve('request-123', { - approver: 'alex-kim', - notes: 'Enjoy your trip!' -}) +// Manager approves naturally +await bamboohr`approve Sarah's PTO request` -// Calendar view -await hr.timeOff.calendar('2025-03') -// Shows who's out when +// Bulk queries just work +await bamboohr`employees with more than 80 hours unused PTO` +await bamboohr`team utilization this quarter` ``` ### Onboarding @@ -138,36 +122,21 @@ await hr.timeOff.calendar('2025-03') New hire checklists that actually get completed. ```typescript -// Create onboarding workflow -await hr.onboarding.createWorkflow('engineering-new-hire', { - tasks: [ - // Before day 1 - { task: 'Send welcome email', due: -7, assignee: 'hr' }, - { task: 'Order laptop', due: -7, assignee: 'it' }, - { task: 'Set up accounts', due: -3, assignee: 'it' }, - { task: 'Assign buddy', due: -3, assignee: 'manager' }, - - // Day 1 - { task: 'Complete I-9', due: 0, assignee: 'employee' }, - { task: 'Review handbook', due: 0, assignee: 'employee' }, - { task: 'Team introductions', due: 0, assignee: 'manager' }, - - // First week - { task: 'Set up dev environment', due: 3, assignee: 'employee' }, - { task: '1:1 with manager', due: 5, assignee: 'manager' }, - { task: 'First project assignment', due: 5, assignee: 'manager' }, - - // First month - { task: '30-day check-in', due: 30, assignee: 'hr' }, - ] -}) - -// Start onboarding for new hire -await hr.onboarding.start('sarah-chen', 'engineering-new-hire') - -// Track progress -const status = await hr.onboarding.status('sarah-chen') -// { completed: 5, total: 11, nextTask: 'Set up dev environment', dueIn: '2 days' } +// Onboard in one line +await bamboohr`onboard Sarah Chen as Engineer on Alex's team` + .provision(`laptop, Slack, GitHub, Figma`) + .buddy(`Jamie Wong`) + +// Or step by step +await bamboohr`hire Sarah Chen as Engineer starting Jan 15` +await bamboohr`assign Sarah to Engineering onboarding` +await bamboohr`order laptop for Sarah` +await bamboohr`set up Sarah's accounts` + +// Track progress naturally +await bamboohr`Sarah's onboarding status` +await bamboohr`incomplete onboarding tasks this week` +await bamboohr`new hires without assigned buddies` ``` ### Performance Management @@ -175,65 +144,23 @@ const status = await hr.onboarding.status('sarah-chen') Goal setting, reviews, feedback. Lightweight but effective. ```typescript -// Set goals -await hr.performance.setGoals('sarah-chen', { - period: '2025-Q1', - goals: [ - { - title: 'Ship authentication service', - description: 'Design and implement OAuth2 + SAML support', - weight: 40, - keyResults: [ - 'OAuth2 provider integration complete', - 'SAML SSO working with 3+ IdPs', - '< 100ms auth latency' - ] - }, - { - title: 'Reduce API errors by 50%', - description: 'Improve error handling and monitoring', - weight: 30, - keyResults: [ - 'Error rate < 0.1%', - 'All errors properly categorized', - 'Alerting in place for anomalies' - ] - }, - { - title: 'Mentor junior developer', - description: 'Help Jamie ramp up on the codebase', - weight: 30, - keyResults: [ - 'Weekly 1:1s scheduled', - 'Code review feedback provided', - 'Jamie shipping independently by end of Q1' - ] - } - ] -}) +// Set goals naturally +await bamboohr`Sarah's Q1 goal: ship auth service with OAuth2 and SAML` +await bamboohr`Sarah's Q1 goal: reduce API errors by 50%` +await bamboohr`Sarah's Q1 goal: mentor Jamie on the codebase` // Request feedback -await hr.performance.requestFeedback('sarah-chen', { - from: ['alex-kim', 'jamie-wong', 'chris-taylor'], - questions: [ - 'What should Sarah keep doing?', - 'What could Sarah improve?', - 'Any other feedback?' - ], - due: '2025-03-15' -}) +await bamboohr`request feedback on Sarah from her teammates` +await bamboohr`360 review for Sarah due March 15` -// Conduct review -await hr.performance.createReview('sarah-chen', { - reviewer: 'alex-kim', - period: '2025-Q1', - rating: 'exceeds-expectations', - summary: 'Sarah has been exceptional...', - goalAssessments: [ - { goal: 'Ship authentication service', rating: 'met', notes: '...' }, - // ... - ] -}) +// Reviews flow naturally +await bamboohr`start Sarah's Q1 review` +await bamboohr`Sarah exceeded expectations this quarter` + +// Bulk operations +await bamboohr`employees without Q1 goals` +await bamboohr`pending reviews this week` +await bamboohr`team feedback completion rates` ``` ### Document Storage @@ -241,27 +168,18 @@ await hr.performance.createReview('sarah-chen', { Employee documents in one place. ```typescript -// Store documents -await hr.documents.upload('sarah-chen', { - type: 'offer-letter', - file: offerLetterPdf, - visibility: 'employee-and-hr' -}) - -// Organize by type -await hr.documents.list('sarah-chen') -// [ -// { type: 'offer-letter', uploaded: '2025-01-01' }, -// { type: 'i9', uploaded: '2025-01-15' }, -// { type: 'w4', uploaded: '2025-01-15' }, -// { type: 'direct-deposit', uploaded: '2025-01-15' } -// ] - -// E-signature integration -await hr.documents.requestSignature('sarah-chen', { - document: 'policy-update-2025', - due: '2025-02-01' -}) +// Store documents naturally +await bamboohr`upload Sarah's offer letter` +await bamboohr`add I-9 for Sarah` + +// Find documents +await bamboohr`Sarah's documents` +await bamboohr`unsigned policy acknowledgments` +await bamboohr`expiring certifications this month` + +// E-signatures +await bamboohr`send handbook acknowledgment to Sarah for signature` +await bamboohr`policy update needs signature from all employees by Feb 1` ``` ### Reporting @@ -269,36 +187,16 @@ await hr.documents.requestSignature('sarah-chen', { See your workforce data clearly. ```typescript -// Headcount over time -await hr.reports.headcount({ - from: '2024-01-01', - to: '2025-01-01', - groupBy: 'department' -}) - -// Turnover analysis -await hr.reports.turnover({ - period: '2024', - groupBy: 'department' -}) -// { engineering: 8%, sales: 22%, support: 15% } - -// Time off utilization -await hr.reports.timeOffUtilization({ - period: '2024', - type: 'vacation' -}) - -// Tenure distribution -await hr.reports.tenure() -// { '<1 year': 35%, '1-2 years': 25%, '2-5 years': 30%, '5+ years': 10% } - -// Custom reports -await hr.reports.custom({ - select: ['department', 'location', 'count(*)'], - groupBy: ['department', 'location'], - where: { status: 'active' } -}) +// Ask for what you need +await bamboohr`headcount by department this year` +await bamboohr`turnover rate by department 2024` +await bamboohr`PTO utilization across Engineering` +await bamboohr`tenure distribution company-wide` + +// Complex queries, natural language +await bamboohr`employees by department and location` +await bamboohr`hiring trend last 12 months` +await bamboohr`average tenure in Sales vs Engineering` ``` ## AI Assistant @@ -308,26 +206,18 @@ await hr.reports.custom({ ```typescript import { olive } from 'bamboohr.do/agents' -// Employees get instant answers +// Employees talk to Olive naturally await olive`How much PTO do I have?` -// "You have 80 hours of vacation, 40 hours of sick leave, and 16 hours of personal time available." - -await olive`I need to take next Friday off` -// "I've created a vacation request for Friday, March 21st (8 hours). -// I've sent this to Alex Kim for approval. You'll be notified when approved." - -await olive`What's the policy on remote work?` -// "According to our Employee Handbook (last updated January 2025): -// - Engineering can work remotely up to 3 days per week -// - Remote work requires manager approval for more than 3 consecutive days -// - International remote work requires HR approval -// Would you like me to submit a remote work request?" - -await olive`Who's out this week?` -// "This week in Engineering: -// - Jamie Wong: Out Monday (sick) -// - Chris Taylor: Out Thursday-Friday (vacation) -// - Maria Garcia: Working remotely Tuesday-Thursday" +await olive`I need next Friday off` +await olive`what's our remote work policy?` +await olive`who's out this week?` + +// Olive handles the complexity +// - Checks balances +// - Creates requests +// - Routes for approval +// - Answers policy questions +// - All from natural language ``` ### AI-Powered Onboarding @@ -335,88 +225,46 @@ await olive`Who's out this week?` Olive guides new hires through their first days: ```typescript -// New hire asks Olive for help -await olive`Hi, I'm new here. What should I do first?` -// "Welcome to the team, Sarah! I'm Olive, your HR assistant. -// -// For today, here's what you need to complete: -// 1. Complete your I-9 form (I can help you with this) -// 2. Review the employee handbook -// 3. Your team intro meeting is at 2pm in Conf Room A -// -// Would you like to start with the I-9? I can walk you through it." - -await olive`Yes, let's do the I-9` -// "Great! The I-9 verifies your identity and work authorization. -// -// I've pulled up the form. For Section 1, I'll need: -// - Full legal name (I have Sarah Chen from your profile) -// - Address (I have 123 Main St, San Francisco) -// - Date of birth -// - Social Security Number -// - Citizenship status -// -// Does 'Sarah Chen' at '123 Main St' look correct?" +// New hires talk to Olive +await olive`I'm new here, what should I do first?` +await olive`help me with my I-9` +await olive`where do I find the employee handbook?` +await olive`who's my buddy?` + +// Olive walks them through everything +// - Forms and paperwork +// - Account setup +// - Team introductions +// - First week tasks ``` ### AI-Assisted Performance Reviews ```typescript -// Manager asks for help writing reviews -await olive`Help me write Sarah's performance review` -// "Based on Sarah's goals and activity this quarter: -// -// GOALS ACHIEVED: -// - Authentication service shipped 2 weeks early -// - API errors reduced by 62% (exceeded 50% target) -// - Weekly mentoring sessions with Jamie documented -// -// FEEDBACK RECEIVED: -// - Peers praised her code review thoroughness -// - Jamie mentioned her patience and clear explanations -// -// DRAFT SUMMARY: -// 'Sarah exceeded expectations in Q1. She delivered the critical -// authentication service ahead of schedule while significantly -// improving system reliability. Her mentorship of Jamie has -// accelerated his onboarding and demonstrates leadership potential.' -// -// Would you like me to draft the full review based on this?" +// Managers get help writing reviews +await olive`help me write Sarah's performance review` +await olive`summarize Sarah's Q1 accomplishments` +await olive`what feedback did Sarah receive this quarter?` + +// Olive drafts, you approve +// - Pulls goals and achievements +// - Incorporates peer feedback +// - Drafts review summary +// - You review and finalize ``` ### AI for HR Operations ```typescript -// HR asks for insights -await olive`Show me onboarding completion rates by department` -// "Onboarding completion rates (last 12 months): -// -// Engineering: 94% completed within 30 days -// Sales: 87% completed within 30 days -// Marketing: 91% completed within 30 days -// Support: 78% completed within 30 days (below target) -// -// Support has the most 'buddy assignment' task failures. -// Recommend: Review Support buddy program capacity." - -await olive`Draft an announcement for the new PTO policy` -// "Here's a draft announcement: -// -// Subject: Updated PTO Policy - More Flexibility in 2025 -// -// Team, -// -// We're excited to announce updates to our PTO policy, effective March 1st: -// -// - Vacation increases from 15 to 20 days annually -// - Unused PTO now rolls over (up to 5 days) -// - New 'floating holidays' - 3 days for personal observances -// -// Full details in the updated handbook. Questions? Ask me or Olive. -// -// - HR Team -// -// Want me to adjust the tone or add anything?" +// HR gets insights +await olive`onboarding completion rates by department` +await olive`which teams have low buddy assignment rates?` +await olive`draft announcement for new PTO policy` + +// Olive surfaces actionable insights +// - Identifies bottlenecks +// - Drafts communications +// - Suggests improvements ``` ## Self-Service @@ -424,33 +272,14 @@ await olive`Draft an announcement for the new PTO policy` Employees handle their own HR tasks. No tickets required. ```typescript -// Employee updates their own info -await hr.self.updateAddress('sarah-chen', { - street: '456 Oak Ave', - city: 'San Francisco', - state: 'CA', - zip: '94102' -}) - -// Emergency contact -await hr.self.updateEmergencyContact('sarah-chen', { - name: 'John Chen', - relationship: 'Spouse', - phone: '415-555-1234' -}) - -// Direct deposit -await hr.self.updateBankAccount('sarah-chen', { - routingNumber: '******456', - accountNumber: '******7890', - accountType: 'checking' -}) - -// View pay stubs (if using payroll integration) -await hr.self.payStubs('sarah-chen') - -// Update profile photo -await hr.self.updatePhoto('sarah-chen', photoBlob) +// Employees update their own info naturally +await bamboohr`update my address to 456 Oak Ave, San Francisco` +await bamboohr`my emergency contact is John Chen, spouse, 415-555-1234` +await bamboohr`update my direct deposit` +await bamboohr`show my pay stubs` +await bamboohr`update my profile photo` + +// All self-service, no HR tickets needed ``` ## Architecture @@ -484,54 +313,23 @@ DocumentDO - Employee documents ## Integrations -### Payroll (Optional) - -Connect to your payroll provider: +Connect naturally: ```typescript -await hr.integrations.connect('gusto', { - apiKey: process.env.GUSTO_API_KEY -}) - -// Or use gusto.do for fully integrated payroll -await hr.integrations.connect('gusto.do', { - // Same account - seamless integration -}) -``` +// Payroll +await bamboohr`connect to Gusto` +await bamboohr`sync payroll with gusto.do` -### SSO - -```typescript -await hr.integrations.connect('okta', { - clientId: process.env.OKTA_CLIENT_ID, - clientSecret: process.env.OKTA_CLIENT_SECRET -}) +// SSO +await bamboohr`enable Okta SSO` -// Employees sign in with company SSO -``` - -### Slack - -```typescript -await hr.integrations.connect('slack', { - botToken: process.env.SLACK_BOT_TOKEN -}) - -// Notifications in Slack -// - "Sarah Chen joined the team!" -// - "Time off request approved" -// - "New company announcement" - -// Ask Olive in Slack +// Slack +await bamboohr`connect Slack` +// Then employees ask Olive in Slack: // @Olive how much PTO do I have? -``` - -### Calendar -```typescript -await hr.integrations.connect('google-calendar', { - // Syncs time-off to team calendars -}) +// Calendar +await bamboohr`sync time-off to Google Calendar` ``` ## Pricing @@ -552,14 +350,10 @@ Same company on bamboohr.do Managed: $1,188/year Import from BambooHR: ```typescript -import { migrate } from 'bamboohr.do/migrate' - -await migrate.fromBambooHR({ - apiKey: process.env.BAMBOOHR_API_KEY, - subdomain: 'your-company' -}) +// One line migration +await bamboohr`import from BambooHR` -// Imports: +// Imports everything: // - All employee records // - Time off balances and history // - Documents diff --git a/rewrites/composio/README.md b/rewrites/composio/README.md index ad2eb531..02384526 100644 --- a/rewrites/composio/README.md +++ b/rewrites/composio/README.md @@ -1,170 +1,230 @@ # composio.do -**Hands for AI Agents** - 150+ tool integrations with managed auth on Cloudflare. +> Hands for AI Agents. 150+ Tools. Zero Auth Headaches. -## The Hero +Your agent can think. It can reason. It can plan. But it needs **hands** to act. -You're building AI agents. They can think, reason, plan. But they need **hands** to act. +"Fix the bug and open a PR" means 5 APIs: GitHub branches, commits, PRs, Slack webhooks, Linear tickets. 40+ hours of OAuth nightmares. 3am token refresh bugs. Rate limit hell. -Your agent wants to "fix the bug and open a PR" - but that means: -- Create a branch on GitHub -- Write the code changes -- Commit with the right message -- Open a PR with context -- Notify the team on Slack +**composio.do** gives your agents hands. 150+ tools. One line per action. -**40+ hours** per integration. OAuth nightmares. 3am token refresh bugs. Rate limit hell. +## AI-Native API -**composio.do** gives your agents hands. 150+ tools. Zero auth headaches. +```typescript +import { composio } from 'composio.do' // Full SDK +import { composio } from 'composio.do/tiny' // Minimal client +import { composio } from 'composio.do/mcp' // MCP tools only +``` + +Natural language for tool integrations: ```typescript -import { composio } from '@dotdo/composio' +import { composio } from 'composio.do' + +// Talk to it like a colleague +await composio`connect GitHub for user-123` +await composio`available actions for Slack` +await composio`execute send-message on Slack #general "Hello team"` + +// Chain like sentences +await composio`create GitHub issue ${bug}` + .then(issue => composio`notify Slack #bugs about ${issue}`) -composio`connect user-123 to GitHub` -composio`create issue: Login bug on mobile in acme/webapp` -composio.github`open PR for feature branch` -composio.slack`notify #engineering about deployment` +// Agents get tools automatically +await ralph.with(composio)`fix the login bug and open a PR` ``` -Natural language. Tagged templates. Your agent just talks. +## The Problem -## The Real Power: Agent Integration +Composio (the original) charges for every action: -The real magic is when agents get tools automatically: +| What They Charge | The Reality | +|------------------|-------------| +| **Per-Action Pricing** | $0.001-0.01 per tool call adds up fast | +| **Vendor Lock-in** | Your agent's hands belong to them | +| **Latency** | Round trip to their servers per action | +| **Tool Limits** | Quota overages kill your agent mid-task | +| **No Self-Hosting** | Your credentials, their servers | -```typescript -import { ralph } from 'agents.do' +### The Real Cost + +At scale, tool calls explode: +- Simple bug fix: 10-20 tool calls +- Feature implementation: 50-100 tool calls +- Daily agent operations: 1,000+ tool calls + +**$100-1,000/month** just for your agent to have hands. -// Ralph gets 150+ composio tools automatically -ralph.with(composio)`fix the login bug and open a PR` -// Uses github_create_branch, github_commit, github_create_pr, slack_send_message +## The Solution + +**composio.do** is the open-source alternative: + +``` +Composio (Original) composio.do +----------------------------------------------------------------- +Per-action pricing $0 - run your own +Their servers Your Cloudflare account +Latency per action Edge-native, global +Quota limits Unlimited +Vendor lock-in Open source, MIT licensed +``` + +## One-Click Deploy + +```bash +npx create-dotdo composio ``` -Ralph doesn't need to know about OAuth, tokens, or API schemas. He just acts. +150+ tool integrations. Running on infrastructure you control. Zero per-action costs. ```typescript -import { priya, ralph, tom, quinn } from 'agents.do' +import { Composio } from 'composio.do' -// Plan to deploy -const deployment = await priya`plan deployment for v2.0` - .map(plan => ralph.with(composio)`implement ${plan}`) - .map(code => tom.with(composio.github)`review and approve ${code}`) - .map(approved => composio.slack`notify #releases: ${approved}`) -// One network round trip! +export default Composio({ + name: 'my-agent-tools', + domain: 'tools.myagent.ai', + apps: ['github', 'slack', 'notion', 'linear'], +}) ``` -## Promise Pipelining +## Features -Chain actions without `Promise.all`. CapnWeb pipelining: +### Connections ```typescript -const logged = await composio.github`create issue ${bug}` - .map(issue => composio.slack`notify #bugs: ${issue}`) - .map(notification => composio.notion`log ${notification}`) -// One network round trip! +// Connect anyone to anything +await composio`connect GitHub for user-123` +await composio`connect Slack for team-acme` +await composio`connect Salesforce for org-enterprise` + +// AI infers what you need +await composio`user-123 connections` // returns all connections +await composio`GitHub status for user-123` // returns connection health +await composio`refresh tokens for user-123` // refreshes expired tokens ``` -Cross-platform workflows: +### Tool Discovery ```typescript -const deployed = await composio.github`merge PR #${prNumber}` - .map(merge => composio.vercel`deploy production`) - .map(deploy => composio.datadog`create deployment marker`) - .map(marker => composio.slack`announce deployment in #releases`) +// Find tools naturally +await composio`what can I do with GitHub?` +await composio`Slack actions for messaging` +await composio`search tools for file management` + +// Get tool schemas +await composio`schema for github_create_issue` +await composio`required params for slack_send_message` ``` -## When You Need Control +### Tool Execution -For structured access, the API is still there: +```typescript +// Just say it +await composio`create GitHub issue: Login bug on mobile in acme/webapp` +await composio`send Slack message to #engineering: Deployment complete` +await composio`create Notion page: Sprint 23 Retro in Engineering space` + +// AI parses intent, executes the right action +await composio`add label "urgent" to issue #42 on acme/webapp` +await composio`assign @tom to PR #123` +await composio`close Linear ticket ENG-456 as done` +``` + +### Agent Integration ```typescript -import { Composio } from '@dotdo/composio' +import { ralph } from 'agents.do' -const composio = new Composio({ apiKey: env.COMPOSIO_API_KEY }) +// Ralph gets 150+ tools automatically +await ralph.with(composio)`fix the login bug and open a PR` +// Uses: github_create_branch, github_commit, github_create_pr, slack_send_message -// Connect a user to GitHub via OAuth -const connection = await composio.connect({ - userId: 'user-123', - app: 'github', - redirectUrl: 'https://myapp.com/callback' -}) +// Ralph doesn't know about OAuth, tokens, or API schemas +// He just acts +``` -// Get tools for an agent framework -const githubTools = await composio.getTools({ - apps: ['github'], - actions: ['create_issue', 'create_pr', 'list_repos'] -}) +### Promise Pipelining -// Execute an action directly -const result = await composio.execute({ - action: 'github_create_issue', - params: { - repo: 'my-org/my-repo', - title: 'Bug: Login fails on mobile', - body: 'Steps to reproduce...' - }, - entityId: 'user-123' -}) +```typescript +import { priya, ralph, tom, quinn } from 'agents.do' + +// Plan to deploy - one network round trip +const deployment = await priya`plan deployment for v2.0` + .map(plan => ralph.with(composio)`implement ${plan}`) + .map(code => tom.with(composio)`review and approve ${code}`) + .map(approved => composio`notify Slack #releases: ${approved}`) + +// Cross-platform workflows +await composio`merge GitHub PR #${prNumber}` + .map(merge => composio`deploy Vercel production`) + .map(deploy => composio`create Datadog deployment marker`) + .map(marker => composio`announce in Slack #releases: ${marker}`) ``` ## 150+ Tools, Zero Config -| Category | Apps | What Your Agent Can Do | -|----------|------|------------------------| -| **Developer** | GitHub, GitLab, Linear, Jira | Create issues, branches, PRs, manage repos | -| **Communication** | Slack, Discord, Teams, Email | Send messages, manage channels, notify teams | -| **Productivity** | Notion, Asana, Monday, Trello | Create tasks, update pages, manage boards | -| **CRM** | Salesforce, HubSpot, Pipedrive | Manage contacts, deals, log activities | -| **Storage** | Google Drive, Dropbox, Box | Upload, download, share files | -| **Analytics** | Datadog, Mixpanel, Amplitude | Track events, create dashboards | -| **AI/ML** | OpenAI, Anthropic, Cohere | Generate text, embeddings, analysis | +| Category | Apps | Example Commands | +|----------|------|------------------| +| **Developer** | GitHub, GitLab, Linear, Jira | `create issue`, `open PR`, `merge branch` | +| **Communication** | Slack, Discord, Teams, Email | `send message`, `create channel`, `notify team` | +| **Productivity** | Notion, Asana, Monday, Trello | `create task`, `update page`, `move card` | +| **CRM** | Salesforce, HubSpot, Pipedrive | `create contact`, `log activity`, `update deal` | +| **Storage** | Google Drive, Dropbox, Box | `upload file`, `share folder`, `create doc` | +| **Analytics** | Datadog, Mixpanel, Amplitude | `track event`, `create dashboard`, `query metric` | +| **AI/ML** | OpenAI, Anthropic, Cohere | `generate text`, `create embedding`, `analyze` | -Your agent gets all of these. Just `.with(composio)`. +```typescript +// All tools, one interface +await composio`create GitHub issue: ${bug}` +await composio`send Slack message: ${notification}` +await composio`create Notion page: ${doc}` +await composio`log Salesforce activity: ${call}` +await composio`upload to Drive: ${file}` +await composio`track Mixpanel event: ${action}` +``` ## MCP Native -Every tool is an MCP tool. Claude can use them directly: +Every tool is an MCP tool. Claude uses them directly: ```typescript -import { createMCPServer } from '@dotdo/composio/mcp' - -const mcpServer = createMCPServer({ - apps: ['github', 'slack', 'notion'], - entityId: 'user-123' -}) - -// Tools are automatically exposed -// github_create_issue, slack_send_message, notion_create_page... +// Expose tools to Claude +await composio`serve MCP for user-123 with github slack notion` + +// Claude sees: +// - github_create_issue +// - github_create_pr +// - slack_send_message +// - notion_create_page +// ... 150+ tools ``` ## Framework Adapters -Works with every agent framework: - ```typescript -// LangChain -import { composioTools } from '@dotdo/composio/langchain' -const tools = await composioTools.getTools({ apps: ['github'], entityId: 'user-123' }) - -// CrewAI -import { composioTools } from '@dotdo/composio/crewai' -const tools = await composioTools.getTools({ apps: ['notion'], entityId: 'user-123' }) - +// Works with every framework // But really, just use agents.do + import { ralph } from 'agents.do' -ralph.with(composio)`do the thing` // This is the way +await ralph.with(composio)`do the thing` // This is the way ``` ## Auth Made Invisible -The hardest part of integrations is auth. Composio handles it all: +The hardest part of integrations is auth. Just say it: ```typescript -// OAuth - handled -composio`connect user-123 to Salesforce` +// OAuth - one line +await composio`connect Salesforce for user-123` // API Keys - stored securely -composio`connect user-123 to OpenAI with ${apiKey}` +await composio`connect OpenAI for user-123 with ${apiKey}` + +// Check connection status +await composio`status for user-123 connections` + +// Disconnect +await composio`disconnect GitHub for user-123` // Token refresh - automatic // Rate limits - managed @@ -175,73 +235,130 @@ Each user gets isolated credentials. Per-entity Durable Objects. No cross-contam ## Architecture +### Durable Object per Entity + ``` - +----------------------+ - | composio.do | - | (Cloudflare Worker) | - +----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | ToolsDO | | ConnectionDO | | ExecutionDO | - | (tool registry) | | (OAuth/API keys) | | (action sandbox) | - +------------------+ +------------------+ +------------------+ +ComposioDO (config, apps, schemas) + | + +-- ConnectionDO (per user - OAuth tokens, API keys) + | |-- SQLite: Credentials (encrypted) + | +-- R2: Connection metadata + | + +-- ExecutionDO (sandboxed tool execution) + | |-- Rate limiting per entity + | +-- Audit logging + | + +-- WebhookDO (inbound events from apps) + |-- GitHub webhooks + +-- Slack events ``` **Key insight**: Each user entity gets its own ConnectionDO for credential isolation. Tool execution happens in sandboxed ExecutionDO instances with rate limiting per entity. -## Installation +### Storage Tiers -```bash -npm install @dotdo/composio -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active connections, recent executions | <10ms | +| **Warm** | R2 + Index | Historical logs, audit trails | <100ms | +| **Cold** | R2 Archive | Compliance retention | <1s | + +### Encryption + +Per-entity encryption for credentials. AES-256-GCM. Automatic key rotation. Immutable audit logs. + +## vs Composio (Original) + +| Feature | Composio (Original) | composio.do | +|---------|---------------------|-------------| +| **Pricing** | Per-action fees | $0 - run your own | +| **Data Location** | Their servers | Your Cloudflare account | +| **Latency** | Centralized | Edge-native, global | +| **Quotas** | Monthly limits | Unlimited | +| **Customization** | Limited | Full source access | +| **Lock-in** | Vendor dependency | MIT licensed | +| **MCP** | Partial support | Native | -## Quick Start +## Use Cases + +### AI Coding Agents ```typescript -import { composio } from '@dotdo/composio' +// Agent that ships code +await ralph.with(composio)` + fix the login bug in auth.ts, + create a branch, + commit the fix, + open a PR, + notify #engineering on Slack +` +``` -// Connect (one-time OAuth) -composio`connect user-123 to GitHub` +### Workflow Automation -// Act -composio.github`create issue: Fix login bug in acme/webapp` -composio.slack`message #dev: Issue created` +```typescript +// When issue created, assign and notify +await composio`watch GitHub issues on acme/webapp` + .on('created', issue => composio` + assign @tom to ${issue}, + add label "triage", + notify #support: New issue ${issue.title} + `) +``` -// Or let an agent do it all -import { ralph } from 'agents.do' -ralph.with(composio)`fix the login bug, create a PR, notify the team` +### Multi-App Orchestration + +```typescript +// Sales flow across apps +await composio`new contact ${lead} in HubSpot` + .map(contact => composio`create task in Asana: Follow up with ${contact}`) + .map(task => composio`schedule calendar meeting for ${task.due}`) + .map(meeting => composio`send Slack reminder: ${meeting}`) ``` -## The Rewrites Ecosystem +## Deployment Options -composio.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare: +### Cloudflare Workers -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [inngest.do](https://inngest.do) | Inngest | Workflows/Jobs for AI | -| **composio.do** | Composio | Tool integrations for AI | -| kafka.do | Kafka | Event streaming for AI | -| nats.do | NATS | Messaging for AI | +```bash +npx create-dotdo composio +# Your account, your data +``` -## Why Cloudflare? +### Private Cloud -1. **Global Edge** - Tool execution close to your agents -2. **Durable Objects** - Per-entity credential isolation -3. **Workers KV** - Fast tool schema caching -4. **Queues** - Reliable webhook delivery -5. **Zero Cold Starts** - Your agent never waits +```bash +# Docker deployment +docker run -p 8787:8787 dotdo/composio + +# Kubernetes +kubectl apply -f composio-do.yaml +``` + +## Contributing -## Related Domains +composio.do is open source under the MIT license. -- **agents.do** - AI agents that use these tools -- **tools.do** - Generic tool registry -- **auth.do** - Authentication platform -- **workflows.do** - Workflow orchestration +```bash +git clone https://github.com/dotdo/composio.do +cd composio.do +pnpm install +pnpm test +``` ## License -MIT +MIT License - Give your agents hands. + +--- + +

+ Per-action pricing ends here. +
+ 150+ tools. Zero cost. Your infrastructure. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/confluence/README.md b/rewrites/confluence/README.md index 14d7a188..4824d86b 100644 --- a/rewrites/confluence/README.md +++ b/rewrites/confluence/README.md @@ -6,6 +6,61 @@ Confluence became the default for team documentation. It also became the place w **confluence.do** reimagines team knowledge for the AI era. AI writes docs from your code. AI keeps docs in sync. AI answers questions. Your wiki finally works. +## AI-Native API + +```typescript +import { confluence } from 'confluence.do' // Full SDK +import { confluence } from 'confluence.do/tiny' // Minimal client +import { confluence } from 'confluence.do/search' // Search-only operations +``` + +Natural language for knowledge workflows: + +```typescript +import { confluence } from 'confluence.do' + +// Talk to it like a colleague +const answer = await confluence`how do we handle authentication?` +const stale = await confluence`outdated docs in Engineering` +const api = await confluence`pages mentioning API redesign` + +// Chain like sentences +await confluence`docs about payments` + .notify(`Please review for accuracy`) + +// Documentation that writes itself +await confluence`document the auth system from code` + .review() // AI checks accuracy + .publish() // your approval +``` + +### Promise Pipelining with Agents + +Chain work without Promise.all: + +```typescript +import { confluence } from 'confluence.do' +import { mark, tom, priya } from 'agents.do' + +// Keep docs in sync with code +await confluence`docs about authentication` + .map(doc => tom`verify ${doc} against codebase`) + .map(doc => mark`update ${doc} if stale`) + .map(doc => priya`review ${doc}`) + +// Generate docs from code changes +await git`recent commits to ${repo}` + .map(commit => mark`document ${commit}`) + .map(doc => confluence`publish ${doc} to Engineering`) + +// Close documentation gaps at scale +await confluence`stale docs in Engineering` + .map(doc => mark`update ${doc} from current code`) + .each(doc => doc.publish()) +``` + +One network round trip. Record-replay pipelining. Workers working for you. + ## The Problem Atlassian bundled Confluence with Jira, and companies got stuck: @@ -50,174 +105,77 @@ npx create-dotdo confluence Your team wiki. Running on Cloudflare. AI-native. -```bash -# Or add to existing workers.do project -npx dotdo add confluence -``` - -## The workers.do Way - -You're building a product. Your team needs documentation. Confluence wants $70k/year for a wiki where knowledge goes to die. Your engineers can't find anything. Docs drift from reality. There's a better way. - -**Natural language. Tagged templates. AI agents that work.** - -```typescript -import { confluence } from 'confluence.do' -import { mark, tom, priya } from 'agents.do' - -// Talk to your wiki like a human -const docs = await confluence`search ${query} in ${space}` -const answer = await confluence`how do we handle authentication?` -const stale = await confluence`find outdated documentation` -``` - -**Promise pipelining - chain work without Promise.all:** - ```typescript -// Keep docs in sync with code -const synced = await confluence`find docs about ${feature}` - .map(doc => tom`verify ${doc} against codebase`) - .map(doc => mark`update ${doc} if stale`) - .map(doc => priya`review ${doc}`) +import { Confluence } from 'confluence.do' -// Generate docs from code changes -const documented = await git`recent commits to ${repo}` - .map(commit => mark`document ${commit}`) - .map(doc => confluence`publish ${doc} to ${space}`) +export default Confluence({ + name: 'acme-wiki', + domain: 'wiki.acme.com', + spaces: ['Engineering', 'Product', 'Design'], +}) ``` -One network round trip. Record-replay pipelining. Workers working for you. - ## Features ### Spaces & Pages -Organize knowledge naturally: - ```typescript -import { Space, Page } from 'confluence.do' +// Create naturally +await confluence`create space "Engineering" for technical docs` +await confluence`create page "Auth Architecture" in Engineering under Architecture` -// Create a space -const engineeringSpace = Space.create({ - key: 'ENG', - name: 'Engineering', - description: 'Technical documentation and decisions', -}) +// Or just write content +await confluence` + page "API Guidelines" in Engineering: -// Create pages -const page = Page.create({ - space: 'ENG', - title: 'Authentication Architecture', - parent: 'Architecture', - content: ` -# Authentication Architecture + # API Guidelines -Our auth system uses JWT tokens with refresh rotation... - -## Components - -\`\`\`mermaid -graph TD - A[Client] --> B[API Gateway] - B --> C[Auth Service] - C --> D[User Database] -\`\`\` + All APIs must be RESTful with JSON responses... +` +``` -## Decision Record +### Search & Discovery -| Date | Decision | Rationale | -|------|----------|-----------| -| 2024-01-15 | JWT over sessions | Stateless scaling | -| 2024-02-20 | Refresh rotation | Security hardening | - `, -}) +```typescript +// Find anything +const authDocs = await confluence`pages about authentication` +const stale = await confluence`docs not updated in 6 months` +const byAlice = await confluence`pages Alice wrote last month` + +// AI infers what you need +await confluence`authentication` // returns relevant pages +await confluence`how do we deploy?` // returns answer with sources +await confluence`what did we decide about caching?` // finds ADRs ``` ### Real-Time Collaboration -Multiple editors, zero conflicts: +Multiple editors, zero conflicts. CRDT-based editing syncs instantly. ```typescript -// Subscribe to page changes -page.subscribe((event) => { - if (event.type === 'content.changed') { - console.log(`${event.user} edited at ${event.position}`) - } -}) - -// Presence awareness -page.onPresence((users) => { - console.log(`${users.length} people viewing this page`) - users.forEach(u => console.log(`${u.name} - cursor at ${u.position}`)) -}) - -// Collaborative editing with CRDT -const doc = page.getCollaborativeDoc() -doc.on('change', (delta) => { - // Real-time sync across all editors -}) +// Presence just works +// See who's editing, where their cursor is +// Changes merge automatically ``` ### Page Trees & Navigation -Hierarchical organization with instant access: - ```typescript -// Get the full page tree -const tree = await Space.getPageTree('ENG') - -// Smart breadcrumbs -const breadcrumbs = await page.getBreadcrumbs() -// ['Engineering', 'Architecture', 'Authentication Architecture'] - -// Related pages -const related = await page.getRelated() -// AI finds semantically similar pages +// Navigate naturally +await confluence`Engineering page tree` +await confluence`pages under Architecture` +await confluence`related to Auth Architecture` ``` ### Templates -Define reusable templates: - ```typescript -import { Template } from 'confluence.do' - -export const adrTemplate = Template({ - name: 'Architecture Decision Record', - labels: ['adr', 'architecture'], - schema: { - status: { type: 'select', options: ['Proposed', 'Accepted', 'Deprecated'] }, - deciders: { type: 'users' }, - date: { type: 'date' }, - }, - content: ` -# {title} - -**Status:** {status} -**Deciders:** {deciders} -**Date:** {date} +// Use templates by name +await confluence`new ADR "Use PostgreSQL for User Data"` +await confluence`new runbook for payment-service` +await confluence`new meeting notes from standup` -## Context - -What is the issue that we're seeing that is motivating this decision? - -## Decision - -What is the change that we're proposing? - -## Consequences - -What becomes easier or more difficult because of this change? - `, -}) - -// Create page from template -await Page.createFromTemplate(adrTemplate, { - title: 'Use PostgreSQL for User Data', - status: 'Proposed', - deciders: ['@alice', '@bob'], - date: '2025-01-15', -}) +// AI fills in context automatically ``` ## AI-Native Documentation @@ -229,19 +187,16 @@ The real magic. AI that keeps your documentation alive. Search that actually understands: ```typescript -import { ai, search } from 'confluence.do' - // Semantic search - not just keywords -const results = await search`how do we handle authentication?` +await confluence`how do we handle authentication?` // Finds pages about auth, JWT, login, sessions - even without those exact words -// Question answering -const answer = await ai.ask`What database do we use for user data?` +// Question answering with sources +await confluence`what database do we use for user data?` // Returns: "PostgreSQL, as decided in ADR-042. The users table schema is defined in..." -// With source citations -const { answer, sources } = await ai.askWithSources`How do I deploy to production?` -// sources: [{ page: 'Deployment Guide', section: 'Production', relevance: 0.94 }] +// Complex queries +await confluence`how do I deploy to production? show sources` ``` ### AI Writes Documentation @@ -250,28 +205,16 @@ Stop staring at blank pages: ```typescript // Generate docs from code -const apiDocs = await ai.documentCode({ - repo: 'github.com/company/api', - path: 'src/routes/', - style: 'technical', -}) +await confluence`document the API routes in src/routes/` +await confluence`write technical docs for payment-service` -// Generate from template + context -const runbook = await ai.generate({ - template: 'incident-runbook', - context: { - service: 'payment-service', - oncall: '@backend-team', - dependencies: ['stripe', 'postgres', 'redis'], - }, -}) +// Generate from context +await confluence`write runbook for payment-service with @backend-team oncall` -// Generate meeting notes from transcript -const notes = await ai.meetingNotes({ - transcript: meetingTranscript, - attendees: ['alice', 'bob', 'carol'], - template: 'decision-meeting', -}) +// Generate meeting notes +await confluence`meeting notes from standup recording` + .review() // you approve + .publish() // done ``` ### AI Freshness Verification @@ -280,32 +223,12 @@ Know if your docs are current: ```typescript // AI checks documentation freshness -const freshness = await ai.checkFreshness(page, { - compareToCode: true, // Diff against source code - checkLinks: true, // Find broken links - detectContradictions: true, // Find conflicting information -}) +await confluence`check freshness of API docs against code` +await confluence`find broken links in Engineering` +await confluence`docs that contradict each other` -// Returns: -{ - status: 'stale', - confidence: 0.87, - issues: [ - { - type: 'code_drift', - section: 'API Endpoints', - detail: 'Documentation shows POST /users but code has POST /api/v2/users', - suggestion: 'Update endpoint path to /api/v2/users' - }, - { - type: 'broken_link', - section: 'References', - detail: 'Link to old-architecture.md returns 404' - } - ], - lastVerified: '2025-01-15T10:30:00Z', - suggestedUpdate: '...' // AI-generated fix -} +// Or just ask +await confluence`is the deployment guide current?` ``` ### AI Doc Sync @@ -313,29 +236,14 @@ const freshness = await ai.checkFreshness(page, { Keep docs and code in sync automatically: ```typescript -import { DocSync } from 'confluence.do' - // Sync API docs with OpenAPI spec -DocSync.register({ - source: 'github.com/company/api/openapi.yaml', - target: 'ENG/API Reference', - transform: 'openapi-to-markdown', - schedule: 'on-push', // or 'daily', 'weekly' -}) +await confluence`sync API Reference from openapi.yaml on every push` // Sync README with wiki -DocSync.register({ - source: 'github.com/company/service/README.md', - target: 'ENG/Services/Payment Service', - bidirectional: true, // Changes sync both ways -}) +await confluence`sync payment-service README bidirectionally` -// Auto-generate changelog from commits -DocSync.register({ - source: 'github.com/company/app/commits', - target: 'PRODUCT/Changelog', - transform: 'conventional-commits-to-changelog', -}) +// Auto-generate changelog +await confluence`generate changelog from commits weekly` ``` ### AI Q&A Bot @@ -343,13 +251,8 @@ DocSync.register({ Answer questions from your wiki: ```typescript -import { QABot } from 'confluence.do' - -// Deploy a Q&A bot for your wiki -const bot = QABot.create({ - spaces: ['ENG', 'PRODUCT', 'DESIGN'], - channels: ['slack:#engineering', 'discord:#help'], -}) +// Deploy a Q&A bot - one line +await confluence`deploy bot to #engineering and #help` // In Slack: "@wiki-bot how do we deploy to production?" // Bot responds with synthesized answer + source links @@ -363,16 +266,13 @@ Seamless connection with your issue tracker: // Auto-link issues mentioned in docs // Writing "BACKEND-123" in a page automatically links to jira.do -// Create doc from issue -const issue = await jira.getIssue('BACKEND-500') -const designDoc = await ai.generateDesignDoc(issue) -await Page.create({ - space: 'ENG', - parent: 'Design Documents', - title: designDoc.title, - content: designDoc.content, - linkedIssues: ['BACKEND-500'], -}) +// Create design doc from issue +await confluence`design doc for BACKEND-500` + +// Or chain from jira +await jira`BACKEND-500` + .map(issue => confluence`write design doc for ${issue}`) + .map(doc => confluence`publish to Design Documents`) // Embed issue status in docs // {% jira BACKEND-500 %} shows live status @@ -385,58 +285,30 @@ Rich components for modern documentation: ### Diagrams ```typescript -// Mermaid diagrams -\`\`\`mermaid -sequenceDiagram - Client->>API: POST /login - API->>Auth: validate(credentials) - Auth-->>API: token - API-->>Client: 200 OK + JWT -\`\`\` - -// Excalidraw embeds -\`\`\`excalidraw -{ - "type": "excalidraw", - "id": "arch-diagram-001" -} -\`\`\` +// Generate diagrams naturally +await confluence`diagram the auth flow` +await confluence`architecture diagram for payment-service` + +// Or embed Mermaid/Excalidraw directly in content ``` ### Code Blocks ```typescript -// Syntax highlighted code -\`\`\`typescript -import { User } from './models' - -export async function createUser(data: UserInput): Promise { - return db.users.insert(data) -} -\`\`\` - -// Live code from GitHub -\`\`\`github -repo: company/api -path: src/models/user.ts -lines: 10-25 -\`\`\` +// Live code from GitHub - always current +await confluence`embed src/models/user.ts lines 10-25` + +// Code stays in sync automatically ``` ### Tables & Databases ```typescript -// Dynamic tables -{% table %} - {% query project = "BACKEND" AND type = "Bug" AND status = "Open" %} -{% /table %} +// Dynamic tables from queries +await confluence`table of open bugs in BACKEND` // Embedded databases (Notion-style) -{% database id="features-db" %} - columns: [Name, Status, Owner, Priority, Sprint] - filter: Status != Done - sort: Priority DESC -{% /database %} +await confluence`database of features with status, owner, priority` ``` ### Callouts & Panels @@ -453,10 +325,6 @@ Be careful when modifying production data. {% danger %} This action cannot be undone. {% /danger %} - -{% success %} -Your deployment completed successfully. -{% /success %} ``` ## API Compatible @@ -475,17 +343,7 @@ GET /wiki/rest/api/content/{id}/child/page GET /wiki/rest/api/content/search?cql={query} ``` -Existing Confluence integrations work: - -```typescript -// Your existing confluence client -const client = new ConfluenceClient({ - host: 'your-org.confluence.do', // Just change the host - // ... rest stays the same -}) - -const page = await client.getPage('123456') -``` +Existing Confluence integrations work. Just change the host. ## Architecture @@ -522,16 +380,23 @@ Both operations merge automatically: ### Storage Tiers -```typescript -// Hot: SQLite in Durable Object -// Current pages, recent edits, page tree +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Current pages, recent edits | <10ms | +| **Warm** | R2 + Index | Attachments, page history | <100ms | +| **Cold** | R2 Archive | Old versions, deleted content | <1s | -// Warm: R2 object storage -// Attachments, images, page history +## vs Atlassian Confluence -// Cold: R2 archive -// Old versions, deleted content (retention) -``` +| Feature | Atlassian Confluence | confluence.do | +|---------|---------------------|---------------| +| **Annual Cost** | $69,300 (500 users) | ~$50/month | +| **Search** | Keyword matching | AI semantic search | +| **Freshness** | Manual verification | AI-verified | +| **Documentation** | Write everything | AI generates from code | +| **Architecture** | Centralized cloud | Edge-native, global | +| **Data Location** | Atlassian's servers | Your Cloudflare account | +| **Lock-in** | Years of migration | MIT licensed | ## Migration from Confluence @@ -594,12 +459,17 @@ Key areas: ## License -MIT License - Use it however you want. Build your business on it. Fork it. Make it your own. +MIT License - For teams who deserve better documentation. ---

- confluence.do is part of the dotdo platform. + The $70k wiki ends here.
- Website | Docs | Discord + AI-native. Always fresh. Knowledge that works. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/coupa/README.md b/rewrites/coupa/README.md index aed655fe..6c7434df 100644 --- a/rewrites/coupa/README.md +++ b/rewrites/coupa/README.md @@ -6,41 +6,34 @@ Coupa built a $8B company (acquired by Thoma Bravo) managing enterprise procurem **coupa.do** is the open-source alternative. One-click deploy your own procurement platform. AI that actually finds savings. No per-seat ransomware. -## The workers.do Way - -You're a CFO trying to control spend across a growing company. But your "spend management" platform costs more than some of the things you're buying. And half your team doesn't have seats because of per-user pricing. - -**workers.do** gives you AI that finds real savings: - -```typescript -import { coupa, mark } from 'workers.do' - -// Natural language for procurement -const savings = await coupa`find contracts up for renewal with negotiation leverage` -const maverick = await coupa`show off-contract spend by department this quarter` -const suppliers = await coupa`which suppliers have declining performance scores` -``` - -Promise pipelining for procure-to-pay - one network round trip: +## AI-Native API ```typescript -// Requisition to payment -const paid = await coupa`create requisition for ${items} from ${supplier}` - .map(req => coupa`route for approval based on amount and category`) - .map(approved => coupa`generate PO and send to supplier portal`) - .map(received => coupa`three-way match and schedule payment`) - .map(invoice => mark`notify AP that ${invoice} is ready for payment`) +import { coupa } from 'coupa.do' // Full SDK +import { coupa } from 'coupa.do/tiny' // Minimal client +import { coupa } from 'coupa.do/p2p' // Procure-to-pay only ``` -AI agents that optimize your spend: +Natural language for procurement workflows: ```typescript -import { priya, tom, sally } from 'agents.do' - -// Procurement intelligence -await priya`analyze supplier concentration risk across categories` -await tom`benchmark our SaaS spend against industry rates` -await sally`prepare negotiation brief for ${vendor} renewal` +import { coupa } from 'coupa.do' + +// Talk to it like a procurement manager +const pending = await coupa`POs pending approval over $10k` +const maverick = await coupa`off-contract spend by department this quarter` +const savings = await coupa`find me $500k in savings this year` + +// Chain like sentences +await coupa`contracts expiring Q2` + .map(contract => coupa`prepare renewal brief for ${contract}`) + +// Procure-to-pay in one flow +await coupa`requisition 5 laptops for Engineering` + .approve() // routes based on amount and policy + .order() // generates PO to preferred vendor + .receive() // confirms delivery + .pay() // three-way match and payment ``` ## The Problem @@ -77,7 +70,16 @@ Proprietary workflows Open, customizable npx create-dotdo coupa ``` -Your own procurement platform. Every user. Every supplier. No per-seat pricing. +A full procurement platform. Running on infrastructure you control. Every user, every supplier, no per-seat pricing. + +```typescript +import { Coupa } from 'coupa.do' + +export default Coupa({ + name: 'acme-procurement', + domain: 'procurement.acme.com', +}) +``` ## Features @@ -86,579 +88,227 @@ Your own procurement platform. Every user. Every supplier. No per-seat pricing. Request what you need: ```typescript -import { procure } from 'coupa.do' - -// Create requisition -const req = await procure.requisitions.create({ - requestor: 'user-001', - department: 'Engineering', - type: 'goods', - justification: 'New laptops for Q1 hires', - lines: [ - { - description: 'MacBook Pro 16" M3 Max', - category: 'IT Equipment', - quantity: 5, - unitPrice: 3499, - currency: 'USD', - supplier: 'Apple', - deliverTo: 'HQ - IT Closet', - needBy: '2025-02-15', - }, - { - description: 'Apple Care+', - category: 'IT Equipment', - quantity: 5, - unitPrice: 399, - currency: 'USD', - supplier: 'Apple', - }, - ], - attachments: ['equipment-justification.pdf'], - coding: { - costCenter: 'CC-001', - glAccount: '6410', - project: 'Q1-HIRING', - }, -}) +import { coupa } from 'coupa.do' + +// Just ask for it +await coupa`5 MacBook Pro M3 Max for Q1 hires, deliver to HQ IT closet by Feb 15` + +// AI figures out supplier, pricing, coding, routing +await coupa`office supplies for marketing, about $500` -// Submit for approval -await req.submit() +// Bulk requests read like a shopping list +await coupa` + Engineering needs: + - 10 monitors for new hires + - standing desks for the Austin office + - ergonomic keyboards, same as last order +` ``` ### Approvals -Flexible, rule-based workflows: +Approvals flow naturally: ```typescript -// Define approval rules -await procure.approvals.configure({ - rules: [ - { - name: 'Manager Approval', - condition: 'amount > 0', - approvers: ['requestor.manager'], - required: true, - }, - { - name: 'Director Approval', - condition: 'amount > 5000', - approvers: ['department.director'], - required: true, - }, - { - name: 'VP Approval', - condition: 'amount > 25000', - approvers: ['department.vp'], - required: true, - }, - { - name: 'Finance Review', - condition: 'amount > 50000 OR category IN ("Capital", "Professional Services")', - approvers: ['finance-team'], - required: true, - }, - { - name: 'Executive Approval', - condition: 'amount > 100000', - approvers: ['cfo'], - required: true, - }, - ], - - delegation: { - enabled: true, - maxDays: 30, - notifyDelegator: true, - }, - - escalation: { - afterDays: 3, - escalateTo: 'approver.manager', - notifyRequestor: true, - }, -}) +// Check what needs your attention +await coupa`my pending approvals` +await coupa`urgent approvals over $50k` -// Approve/reject with comments -await procure.approvals.decide({ - request: 'REQ-001', - decision: 'approve', // or 'reject', 'return' - comments: 'Approved. Please consolidate with Q2 order if possible.', - approver: 'user-002', -}) +// Approve with context +await coupa`approve REQ-001, consolidate with Q2 order if possible` +await coupa`reject REQ-002, use preferred vendor instead` +await coupa`return REQ-003 to Sarah for more detail` + +// Delegation when you're out +await coupa`delegate my approvals to Mike for two weeks` ``` ### Purchase Orders -Professional PO management: +POs in plain English: ```typescript // Create PO from approved requisition -const po = await procure.purchaseOrders.create({ - requisition: 'REQ-001', - supplier: 'SUP-APPLE', - terms: { - paymentTerms: 'Net 30', - incoterms: 'DDP', - currency: 'USD', - }, - shippingAddress: { - attention: 'IT Department', - line1: '100 Main Street', - city: 'San Francisco', - state: 'CA', - postalCode: '94102', - }, - instructions: 'Deliver to loading dock. Call (555) 123-4567 on arrival.', -}) +await coupa`create PO for REQ-001 to Apple, Net 30, ship to SF office` -// Send to supplier -await po.send({ - method: 'email', // or 'supplier-portal', 'edi', 'cxml' - contacts: ['orders@apple.com'], -}) +// Or generate directly +await coupa`PO to Dell for 20 laptops from the IT equipment contract` -// Acknowledge receipt (by supplier) -await po.acknowledge({ - confirmedBy: 'apple-rep', - estimatedDelivery: '2025-02-10', - notes: 'All items in stock, shipping next week', -}) +// Check status +await coupa`open POs for Engineering this month` +await coupa`POs pending supplier acknowledgment` -// Receive goods -await procure.receiving.create({ - purchaseOrder: po.id, - receivedBy: 'user-003', - lines: [ - { line: 1, quantityReceived: 5, condition: 'good' }, - { line: 2, quantityReceived: 5, condition: 'good' }, - ], - packing: 'SN12345678', -}) +// Receiving is one line +await coupa`received all items on PO-001` +await coupa`received 5 of 10 monitors on PO-002, rest backordered` ``` ### Invoices -Three-way matching and payment: +Three-way matching in natural language: ```typescript -// Receive invoice -const invoice = await procure.invoices.create({ - supplier: 'SUP-APPLE', - invoiceNumber: 'INV-2025-001234', - invoiceDate: '2025-02-10', - dueDate: '2025-03-12', - lines: [ - { - purchaseOrderLine: 'PO-001-1', - description: 'MacBook Pro 16" M3 Max', - quantity: 5, - unitPrice: 3499, - total: 17495, - }, - { - purchaseOrderLine: 'PO-001-2', - description: 'Apple Care+', - quantity: 5, - unitPrice: 399, - total: 1995, - }, - ], - tax: 1558.32, - total: 21048.32, - attachments: ['invoice-scan.pdf'], -}) +// Process incoming invoices +await coupa`match invoice from Apple INV-2025-001234` + +// Query invoice status +await coupa`invoices not matched to POs` +await coupa`invoices pending approval over $10k` +await coupa`overdue invoices by supplier` -// Three-way match (PO, Receipt, Invoice) -const match = await invoice.match() -// { -// status: 'matched', -// poMatch: true, -// receiptMatch: true, -// priceMatch: true, -// quantityMatch: true, -// exceptions: [] -// } - -// Auto-approve if matched -if (match.status === 'matched') { - await invoice.approve({ auto: true }) -} +// Handle exceptions naturally +await coupa`why won't INV-001 match?` +await coupa`approve INV-001 despite price variance, vendor raised prices` // Payment scheduling -await procure.payments.schedule({ - invoice: invoice.id, - paymentDate: '2025-03-10', // 2 days early for 2% discount - paymentMethod: 'ACH', - earlyPayDiscount: { - percentage: 2, - savings: 420.97, - }, -}) +await coupa`schedule INV-001 for early pay discount` +await coupa`hold payment on INV-002 until quality issue resolved` ``` ### Supplier Management -Your supplier network: +Know your suppliers: ```typescript -// Onboard supplier -const supplier = await procure.suppliers.create({ - name: 'Apple Inc.', - type: 'vendor', - categories: ['IT Equipment', 'Software'], - contacts: [ - { - name: 'Enterprise Sales', - email: 'enterprise@apple.com', - phone: '1-800-800-2775', - type: 'sales', - }, - { - name: 'Accounts Receivable', - email: 'ar@apple.com', - type: 'billing', - }, - ], - addresses: [ - { - type: 'remit', - line1: 'One Apple Park Way', - city: 'Cupertino', - state: 'CA', - postalCode: '95014', - }, - ], - payment: { - terms: 'Net 30', - method: 'ACH', - bankAccount: { - // Encrypted - bankName: 'Bank of America', - accountNumber: '****1234', - routingNumber: '****5678', - }, - }, - tax: { - id: '94-2404110', - w9OnFile: true, - type: '1099-NEC', - }, -}) - -// Supplier qualification -await procure.suppliers.qualify({ - supplier: supplier.id, - questionnaire: 'standard-vendor', - responses: { - yearsInBusiness: 48, - annualRevenue: 394000000000, - publicCompany: true, - diversityCertifications: [], - insuranceCoverage: true, - socCompliance: true, - }, - documents: ['w9.pdf', 'insurance-certificate.pdf'], -}) - -// Supplier performance -const performance = await procure.suppliers.scorecard('SUP-APPLE') -// { -// overall: 4.5, -// metrics: { -// onTimeDelivery: 0.96, -// qualityScore: 0.99, -// responsiveness: 4.2, -// priceCompetitiveness: 3.8, -// invoiceAccuracy: 0.98, -// }, -// trend: 'stable', -// spendTTM: 250000, -// } +// Find and query suppliers +await coupa`our top 10 suppliers by spend` +await coupa`suppliers in IT Equipment category` +await coupa`which suppliers have declining scores?` + +// Onboard naturally +await coupa`add Apple as IT Equipment vendor, enterprise@apple.com` +await coupa`request W-9 and insurance certificate from Apple` + +// Performance tracking +await coupa`Apple scorecard` +await coupa`suppliers with late delivery issues this quarter` +await coupa`suppliers with expiring certifications` ``` ### Contracts -Procurement contract management: +Contract management without the forms: ```typescript -// Create contract -await procure.contracts.create({ - supplier: 'SUP-APPLE', - type: 'master-agreement', - title: 'Apple Enterprise Agreement 2025', - effectiveDate: '2025-01-01', - expirationDate: '2027-12-31', - value: { - type: 'estimated', - amount: 500000, - period: 'annual', - }, - terms: { - paymentTerms: 'Net 30 with 2% 10-day discount', - priceProtection: '12 months', - volumeDiscounts: [ - { threshold: 50, discount: 5 }, - { threshold: 100, discount: 10 }, - { threshold: 250, discount: 15 }, - ], - }, - pricingSchedule: 'pricing-schedule-2025.xlsx', - renewalType: 'auto', - noticePeriod: 90, -}) - -// Contract compliance check on PO -await procure.purchaseOrders.checkContract('PO-001') -// Validates pricing, terms, supplier status against contract +// Query contracts +await coupa`contracts expiring in 90 days` +await coupa`our agreement with Apple` +await coupa`contracts with volume discounts we're not hitting` + +// Contract intelligence +await coupa`which contracts have renewal leverage?` +await coupa`off-contract spend by category this quarter` +await coupa`are we getting the contracted price on PO-001?` + +// Renewal prep +await coupa`prepare negotiation brief for Apple renewal` ``` ### Catalogs -Guided buying from approved sources: +Shopping from approved sources: ```typescript -// Configure punch-out catalog -await procure.catalogs.punchout({ - supplier: 'SUP-AMAZON-BUSINESS', - name: 'Amazon Business', - protocol: 'cxml', // or 'oci' - credentials: { - identity: '...', - sharedSecret: '...', - }, - url: 'https://www.amazon.com/punchout', - categories: ['Office Supplies', 'IT Accessories'], - maxOrderValue: 5000, -}) +// Browse what's available +await coupa`show me approved laptops` +await coupa`office supplies catalog` +await coupa`what can I buy from Amazon Business?` -// Static catalog -await procure.catalogs.create({ - name: 'Preferred IT Equipment', - supplier: 'SUP-APPLE', - items: [ - { - sku: 'MBP16-M3MAX', - name: 'MacBook Pro 16" M3 Max', - description: 'Laptop with M3 Max chip, 36GB RAM, 1TB SSD', - price: 3499, - category: 'IT Equipment', - image: 'https://...', - }, - // ... more items - ], - contract: 'CTR-APPLE-2025', - validThrough: '2025-12-31', -}) +// Smart recommendations +await coupa`I need a laptop for a new developer` +// AI suggests: "MacBook Pro 16" M3 Max from Apple contract at $3,499 +// - 15% discount vs list +// - 2-day delivery available +// - 47 others ordered this quarter" ``` ## Spend Analytics -Understand where the money goes: +Ask questions about your spend: ```typescript -// Spend by category -const categorySpend = await procure.analytics.spendByCategory({ - period: '2024', - groupBy: 'category', -}) -// [ -// { category: 'IT Equipment', spend: 2400000, percentage: 24 }, -// { category: 'Professional Services', spend: 1800000, percentage: 18 }, -// { category: 'Marketing', spend: 1500000, percentage: 15 }, -// ... -// ] - -// Spend by supplier -const supplierSpend = await procure.analytics.spendBySupplier({ - period: '2024', - topN: 20, -}) - -// Maverick spend (off-contract) -const maverickSpend = await procure.analytics.maverickSpend({ - period: '2024-Q4', -}) -// { -// total: 450000, -// percentage: 12, -// categories: [...], -// departments: [...], -// recommendations: [...] -// } - -// Contract utilization -const utilization = await procure.analytics.contractUtilization({ - period: '2024', -}) -// Shows spend vs contract commitments +// Understand where money goes +await coupa`spend by category this year` +await coupa`top 20 suppliers by spend` +await coupa`IT spend trend last 12 months` + +// Find savings opportunities +await coupa`maverick spend by department this quarter` +await coupa`contracts we're underutilizing` +await coupa`duplicate purchases across departments` + +// Executive dashboards +await coupa`spend summary for the board` +await coupa`year-over-year savings from procurement` ``` ## AI-Native Procurement -### AI Spend Analysis +### Procurement Intelligence ```typescript -import { ada } from 'coupa.do/agents' - -// Identify savings opportunities -await ada` - Analyze our 2024 spend data and identify: - 1. Categories with supplier consolidation opportunity - 2. Contracts up for renewal with negotiation leverage - 3. Price variance for same items across suppliers - 4. Maverick spend that should go through contracts - 5. Early payment discount opportunities not being captured - - Quantify potential savings for each opportunity. -` - -// Ada analyzes and returns: -// "Total identified savings opportunities: $847,000 -// -// 1. SUPPLIER CONSOLIDATION: $320,000 -// - IT Accessories: 12 suppliers, recommend 3 preferred -// - Office Supplies: Consolidate to Amazon Business -// -// 2. CONTRACT RENEWALS: $180,000 -// - AWS contract expires Q2 - usage up 40%, negotiate volume discount -// - Salesforce - paying list price, competitors offering 25% off -// -// 3. PRICE VARIANCE: $127,000 -// - Same Dell monitors: $499 vs $449 depending on buyer -// - Professional services: Rate card not enforced -// -// 4. MAVERICK SPEND: $145,000 -// - Marketing buying software outside IT agreements -// - Regional offices purchasing office supplies locally -// -// 5. EARLY PAY DISCOUNTS: $75,000 -// - $3.2M eligible for 2/10 Net 30, only capturing 40%" +// One question unlocks insights +await coupa`find me $500K in savings this year` +// AI analyzes spend patterns, contract leverage, pricing variance, +// maverick spend, and early pay discounts - returns actionable plan + +// Continuous monitoring +await coupa`supplier risk alerts` +await coupa`contracts expiring without coverage` +await coupa`price increases above inflation` ``` -### AI Invoice Processing +### Invoice Automation ```typescript -import { ralph } from 'agents.do' - -// Auto-extract invoice data -await ralph` - Process the attached invoice image: - 1. Extract supplier, invoice number, date, amounts - 2. Match line items to open POs - 3. Identify any discrepancies - 4. Flag unusual items or pricing - 5. Code to appropriate GL accounts -` - -// Ralph processes: -// "Invoice processed: INV-2025-001234 -// -// Match Results: -// - Line 1: Matched to PO-001 Line 1 (5x MacBook Pro) ✓ -// - Line 2: Matched to PO-001 Line 2 (5x AppleCare) ✓ -// - Tax: Matches expected rate (8%) ✓ -// -// No exceptions. Ready for auto-approval." +// Drop in an invoice, AI handles the rest +await coupa`process invoice from Apple` + .match() // three-way match to PO and receipt + .code() // GL coding from patterns + .approve() // auto-approve if clean + +// Exception handling +await coupa`invoices with matching exceptions` + .map(inv => coupa`explain why ${inv} won't match`) ``` -### AI Contract Negotiation +### Negotiation Support ```typescript import { sally } from 'agents.do' -// Prepare for supplier negotiation -await sally` - We're renewing our Salesforce contract: - - Current annual spend: $450,000 - - Contract expires: March 31, 2025 - - Current discount: 15% off list - - Research: - 1. What are competitors paying? (benchmark data) - 2. What leverage do we have? - 3. What additional services should we negotiate? - 4. What's our BATNA? +// Prep for renewals in one line +await sally`prepare Salesforce renewal brief, current spend $450K` +// Returns: market benchmarks, leverage points, target pricing, BATNA - Provide negotiation strategy and target pricing. -` - -// Sally provides: -// "NEGOTIATION BRIEF: Salesforce Renewal -// -// MARKET INTELLIGENCE: -// - Similar-sized companies averaging 25-30% discount -// - New competitors (HubSpot, Dynamics) actively pursuing accounts -// - Salesforce pushing multi-year deals for better rates -// -// LEVERAGE POINTS: -// - Contract expiring in 60 days - time pressure on both sides -// - 85% license utilization - room to right-size -// - Competitor quote in hand (HubSpot -40%) -// -// TARGET: -// - 30% discount (from 15%) -// - Right-size to 90% current licenses -// - Include Premier Support (currently paying extra) -// - 2-year term max (preserve flexibility) -// -// EXPECTED OUTCOME: $130,000 annual savings" +// Bulk renewal prep +await coupa`contracts expiring Q2` + .map(contract => sally`negotiation brief for ${contract}`) ``` -### AI Supplier Risk +### Supplier Risk Monitoring ```typescript import { tom } from 'agents.do' -// Monitor supplier risk -await tom` - Assess risk across our top 50 suppliers: - 1. Financial stability (public filings, credit ratings) - 2. Concentration risk (% of our spend, alternatives) - 3. Geographic risk (single source, political stability) - 4. Compliance risk (certifications, audits) - 5. Cyber risk (security posture, breaches) - - Flag any suppliers requiring immediate action. -` +// Continuous risk assessment +await tom`assess risk across our top 50 suppliers` -// Tom monitors continuously: -// "SUPPLIER RISK ALERT -// -// HIGH RISK (Immediate Action Required): -// - SUP-047 (ChipTech Inc): Credit rating downgraded B- to C+ -// Action: Accelerate second-source qualification -// Exposure: $2.1M annual spend, 90-day lead time -// -// MEDIUM RISK (Monitor): -// - SUP-012 (GlobalWidgets): 100% of Category X spend -// Recommendation: Qualify alternative supplier -// -// - SUP-089 (DataCorp): SOC 2 certification expired -// Action: Request updated certification" +// Alert on changes +await coupa`suppliers with credit downgrades` +await coupa`single-source categories` +await coupa`suppliers with expiring SOC 2 certs` ``` -### AI Purchase Recommendations +### Smart Recommendations ```typescript import { priya } from 'agents.do' -// Smart recommendations -await priya` - User wants to purchase 10 laptops for new hires. - Based on: - - Our preferred vendors and contracts - - Historical purchases - - Budget constraints - - Delivery requirements - - Recommend: - 1. Best value option - 2. Best performance option - 3. Most compliant option - - Include pricing, availability, and approval requirements. -` +// AI suggests based on context +await priya`recommend laptops for 10 new hires` +// Returns options ranked by value, performance, compliance +// with pricing, availability, and approval requirements ``` ## Architecture @@ -689,90 +339,33 @@ CompanyDO (config, users, approval rules) +-- R2: Data warehouse ``` -### Integration Architecture +### Integrations -```typescript -// ERP Integration -await procure.integrations.erp({ - system: 'netsuite', // or 'sap', 'oracle', 'dynamics' - sync: { - vendors: 'bidirectional', - glAccounts: 'pull', - purchaseOrders: 'push', - invoices: 'push', - payments: 'pull', - }, - schedule: 'real-time', // or 'hourly', 'daily' -}) +Connect to your systems naturally: -// Banking Integration -await procure.integrations.banking({ - provider: 'plaid', // or direct bank API - accounts: ['operating', 'payables'], - capabilities: ['balance', 'payments', 'reconciliation'], -}) +```typescript +// ERP sync just works +await coupa`sync vendors and GL accounts from NetSuite` +await coupa`push approved POs to SAP` -// Supplier Portal -await procure.supplierPortal.configure({ - domain: 'suppliers.yourcompany.com', - capabilities: [ - 'view-pos', - 'submit-invoices', - 'update-profile', - 'view-payments', - 'upload-documents', - ], -}) +// Supplier portal included +await coupa`enable supplier portal for our vendors` +// Suppliers can view POs, submit invoices, check payment status ``` -### Workflow Engine +## vs Coupa -```typescript -// Complex approval workflows -await procure.workflows.create({ - name: 'Capital Equipment Approval', - trigger: { - type: 'requisition', - conditions: ['category = Capital', 'amount > 10000'], - }, - steps: [ - { - name: 'Manager Approval', - type: 'approval', - assignee: 'requestor.manager', - sla: '2 business days', - }, - { - name: 'Budget Check', - type: 'automatic', - action: 'checkBudget', - onFail: 'routeToFinance', - }, - { - name: 'IT Review', - type: 'approval', - assignee: 'it-team', - condition: 'category = IT Equipment', - }, - { - name: 'Finance Approval', - type: 'approval', - assignee: 'finance-team', - parallel: true, - }, - { - name: 'Executive Approval', - type: 'approval', - assignee: 'cfo', - condition: 'amount > 50000', - }, - ], - escalation: { - after: '5 business days', - action: 'notifyProcurement', - }, -}) -``` +| Feature | Coupa | coupa.do | +|---------|-------|----------| +| **Per-User Cost** | $150-300/month | $0 - run your own | +| **Implementation** | $500K-2M | Deploy in hours | +| **Supplier Network** | Locked to their network | Your suppliers, direct | +| **AI/Analytics** | Premium upsell | AI-native from day one | +| **Customization** | Consultant required | Configure yourself | +| **Data Location** | Their cloud | Your Cloudflare account | +| **Workflow Changes** | Change request process | Code it yourself | +| **API Access** | Limited, extra cost | Full access, open | +| **Lock-in** | Years of migration | MIT licensed | ## Why Open Source for Procurement? @@ -846,11 +439,9 @@ kubectl apply -f coupa-do-deployment.yaml ### Hybrid ```typescript -// Edge for requisitions, origin for analytics -await procure.config.hybrid({ - edge: ['requisitions', 'approvals', 'catalogs'], - origin: ['invoices', 'payments', 'analytics'], -}) +// Edge for fast user experience, origin for heavy analytics +await coupa`run requisitions and approvals at the edge` +await coupa`run analytics on origin for heavy queries` ``` ## Roadmap @@ -918,7 +509,12 @@ MIT License - Procure freely. ---

- coupa.do is part of the dotdo platform. + The $8B acquisition ends here.
- Website | Docs | Discord + AI-native. Every user. No per-seat pricing. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/customerio/README.md b/rewrites/customerio/README.md index 9a8a8d42..642e74f8 100644 --- a/rewrites/customerio/README.md +++ b/rewrites/customerio/README.md @@ -1,95 +1,66 @@ # customerio.do -Customer.io on Cloudflare Durable Objects - Marketing automation for every AI agent. +> Marketing Automation. Edge-Native. Natural Language. AI-First. -## The Problem - -AI agents need marketing automation. Millions of them. Running in parallel. Each with their own customer relationships. - -Traditional marketing platforms were built for humans: -- One shared platform for many marketers -- Centralized campaign management -- Manual audience building -- Expensive per-send pricing - -AI agents need the opposite: -- One automation engine per agent -- Distributed by default -- Dynamic audiences computed in real-time -- Free at the instance level, pay for delivery +Customer.io charges $150/month for 12,000 profiles. Their API requires configuration objects, event schemas, and workflow builders. Every integration is a developer task. -## The Vision +**customerio.do** is the open-source alternative. Natural language. Deploy in seconds. Talk to it like a marketer. -Every AI agent gets their own Customer.io. +## AI-Native API ```typescript -import { tom, ralph, priya } from 'agents.do' -import { CustomerIO } from 'customerio.do' - -// Each agent has their own isolated marketing platform -const tomCRM = CustomerIO.for(tom) -const ralphCRM = CustomerIO.for(ralph) -const priyaCRM = CustomerIO.for(priya) - -// Full Customer.io API -await tomCRM.identify('user_123', { email: 'alice@example.com', plan: 'pro' }) -await ralphCRM.track('user_123', 'feature_used', { feature: 'export' }) -await priyaCRM.workflows.trigger('onboarding', { recipients: ['user_123'] }) +import { customerio } from 'customerio.do' // Full SDK +import { customerio } from 'customerio.do/tiny' // Minimal client +import { customerio } from 'customerio.do/segments' // Segments-only ``` -Not a shared marketing platform. Not a multi-tenant nightmare. Each agent has their own complete Customer.io instance. +Natural language for marketing automation: -## Features +```typescript +import { customerio } from 'customerio.do' -- **Track API** - identify() and track() for user behavior -- **Campaigns/Journeys** - Visual workflow builder with CF Workflows -- **Dynamic Segments** - Real-time audience computation -- **Multi-Channel** - Email, push, SMS, in-app, webhooks -- **Liquid Templates** - Personalized content rendering -- **Preference Management** - User opt-in/opt-out handling -- **MCP Tools** - Model Context Protocol for AI-native marketing +// Talk to it like a colleague +await customerio`user-123 signed up from Google Ads` +await customerio`alice@example.com upgraded to Pro` +await customerio`send welcome sequence to users signed up today` -## Architecture +// Chain like sentences +await customerio`users who abandoned cart` + .notify('Complete your purchase - 10% off!') -``` - +-----------------------+ - | customerio.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | CustomerDO (Tom) | | CustomerDO (Rae) | | CustomerDO (...) | - | SQLite + R2 | | SQLite + R2 | | SQLite + R2 | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +-------------------+ - | CF Workflows | - | (Journey Engine) | - +-------------------+ +// Campaigns that write themselves +await customerio`users inactive for 30 days` + .map(user => customerio`win back ${user}`) ``` -**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each agent's customer data is a Durable Object. SQLite handles queries. Cloudflare Workflows handle multi-step journeys. +## The Problem -## Installation +Customer.io dominates marketing automation alongside Braze and Iterable: -```bash -npm install @dotdo/customerio -``` +| What Customer.io Charges | The Reality | +|--------------------------|-------------| +| **Essentials** | $100/month for 5,000 profiles | +| **Premium** | $1,000+/month | +| **Per-Message** | Overage charges on email/SMS | +| **Enterprise** | Custom pricing (read: expensive) | +| **Integration** | Developers write event tracking | +| **Vendor Lock-in** | Data trapped in their platform | -## Quick Start +### The Integration Tax -### Event Tracking +Every marketing tool requires: -```typescript -import { CustomerIO } from 'customerio.do' +- Schema definitions for events +- Code changes for tracking +- Developer time for campaigns +- Platform-specific syntax -const cio = new CustomerIO(env.CUSTOMERIO) +Marketers can't move without engineering help. -// Identify user with attributes +### The Configuration Hell + +```typescript +// OLD: Customer.io SDK (verbose, developer-only) await cio.identify('user_123', { email: 'alice@example.com', name: 'Alice', @@ -97,173 +68,299 @@ await cio.identify('user_123', { created_at: Date.now() }) -// Track events await cio.track('user_123', 'order_placed', { order_id: 'order_456', amount: 99.99, items: [{ sku: 'widget-1', qty: 2 }] }) +``` -// Anonymous tracking with later resolution -await cio.track(null, 'page_viewed', { url: '/pricing' }, { - anonymousId: 'anon_789' -}) +Nobody talks like that. A marketer would say: "Alice placed a $100 order." -// Later, resolve anonymous to user -await cio.identify('user_123', {}, { - anonymousId: 'anon_789' // Merges anonymous history -}) +## The Solution + +**customerio.do** reimagines marketing automation: + +``` +Customer.io customerio.do +----------------------------------------------------------------- +$100-1000+/month $0 - run your own +Configuration objects Natural language +Developer-required Marketer-friendly +Proprietary workflows Cloudflare Workflows +Data trapped Your Cloudflare account +Weeks to integrate Deploy in seconds ``` -### Workflow Triggers +## One-Click Deploy + +```bash +npx create-dotdo customerio +``` + +A complete marketing automation platform. Running on infrastructure you control. ```typescript import { CustomerIO } from 'customerio.do' -const cio = new CustomerIO(env.CUSTOMERIO) - -// Trigger a journey -const { runId } = await cio.workflows.trigger('onboarding', { - recipients: ['user_123'], - data: { - trial_days: 14, - features: ['export', 'api'] - } +export default CustomerIO({ + name: 'my-startup', + domain: 'marketing.my-startup.com', }) +``` + +## Features + +### User Tracking -// Check run status -const status = await cio.workflows.getStatus(runId) -console.log(status) -// { status: 'running', currentStep: 'welcome_email', progress: 2/5 } +```typescript +// Just describe what happened +await customerio`user-123 signed up from Google Ads` +await customerio`alice@example.com is on the Pro plan` +await customerio`bob viewed pricing page 3 times today` + +// AI infers what you need +await customerio`user-123` // returns user profile +await customerio`user-123 activity` // returns event history +await customerio`users from Twitter` // returns segment ``` -### Dynamic Segments +### Events ```typescript -import { CustomerIO } from 'customerio.do' +// Natural event tracking +await customerio`alice placed a $99 order` +await customerio`bob abandoned cart with 3 items` +await customerio`user-123 used the export feature` + +// Batch events read like a log +await customerio` + alice upgraded to Pro + bob canceled subscription + charlie started trial +` +``` -const cio = new CustomerIO(env.CUSTOMERIO) - -// Create a segment -await cio.segments.create({ - name: 'Active Pro Users', - rules: { - and: [ - { attribute: 'plan', operator: 'eq', value: 'pro' }, - { event: 'login', operator: 'did', timeframe: '7d' } - ] - } -}) +### Segments -// Query segment membership -const members = await cio.segments.members('active-pro-users') -console.log(`${members.count} users in segment`) +```typescript +// Query your audience like a database +await customerio`users who signed up this week` +await customerio`Pro users who haven't logged in for 30 days` +await customerio`users from Google Ads with no purchase` + +// Segments that update themselves +await customerio`active Pro users` + .count() // real-time membership + +await customerio`cart abandoners from yesterday` + .each(user => customerio`remind ${user} about cart`) +``` + +### Campaigns + +```typescript +// Trigger campaigns naturally +await customerio`send welcome email to alice@example.com` +await customerio`start onboarding sequence for new signups` +await customerio`notify Pro users about the new feature` + +// Multi-step journeys +await customerio`users who signed up today` + .notify('Welcome! Here is how to get started...') + .wait('3 days') + .notify('Have you tried our export feature?') + .wait('7 days') + .notify('Ready to upgrade to Pro?') ``` ### Multi-Channel Delivery ```typescript -import { CustomerIO } from 'customerio.do' +// Just say what you want +await customerio`email alice about the sale` +await customerio`push notification to mobile users` +await customerio`SMS bob about order shipped` + +// Automatic fallback +await customerio`notify alice about new feature` + // Tries push, falls back to email, then SMS +``` -const cio = new CustomerIO(env.CUSTOMERIO) +### Templates -// Send directly to a channel -await cio.send('user_123', { - channel: 'email', - template: 'welcome', - data: { name: 'Alice' } -}) +```typescript +// AI writes personalized content +await customerio`welcome email for Pro users` + // Generates context-aware template -// Send with fallback -await cio.send('user_123', { - channel: 'push', - fallback: 'email', // If push fails or user opted out - template: 'notification', - data: { message: 'New feature available!' } -}) +// Or preview with data +await customerio`preview welcome email for alice` ``` -### Liquid Templates +## Promise Pipelining + +Chain operations without waiting: ```typescript -import { CustomerIO } from 'customerio.do' +// One network round trip +const campaigns = await customerio`users from Google Ads this week` + .map(user => customerio`send welcome sequence to ${user}`) + .map(result => customerio`track ${result} delivery status`) + +// Parallel outreach +await customerio`inactive Pro users` + .map(user => [ + customerio`email ${user} win-back offer`, + customerio`push ${user} reminder`, + ]) +``` -const cio = new CustomerIO(env.CUSTOMERIO) - -// Create a template -await cio.templates.create({ - id: 'welcome', - channel: 'email', - subject: 'Welcome, {{ user.name }}!', - body: ` -

Welcome to {{ company.name }}

- - {% if user.plan == 'pro' %} -

Thank you for choosing Pro!

- {% else %} -

Upgrade to Pro for more features.

- {% endif %} - -
    - {% for feature in user.features %} -
  • {{ feature | capitalize }}
    • - {% endfor %} -
- ` -}) +## Architecture -// Preview a template -const preview = await cio.templates.preview('welcome', { - user: { name: 'Alice', plan: 'pro', features: ['export', 'api'] }, - company: { name: 'Acme Inc' } -}) +``` +Internet --> Cloudflare Worker --> Durable Object --> SQLite + | | | + Edge Routing Customer Data User Profiles + Event History Segments + Campaign State Preferences ``` -## API Overview +### Durable Object per Workspace -### Track API (`customerio.do`) +``` +CustomerIODO (config, channels, templates) + | + +-- UsersDO (profiles, attributes) + | |-- SQLite: User records + | +-- R2: Profile data + | + +-- EventsDO (tracking, history) + | |-- SQLite: Event stream + | +-- R2: Event archive + | + +-- SegmentsDO (audiences, rules) + | |-- SQLite: Segment definitions + | + +-- CampaignsDO (journeys, workflows) + |-- SQLite: Campaign state + +-- CF Workflows: Journey engine +``` -- `identify(userId, traits, options)` - Create/update user profile -- `track(userId, event, properties, options)` - Record event -- `batch(events)` - Bulk event ingestion +### Storage Tiers -### Workflows (`customerio.do/workflows`) +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active users, recent events | <10ms | +| **Warm** | R2 + SQLite Index | Historical events (30-90 days) | <100ms | +| **Cold** | R2 Archive | Compliance retention (1+ years) | <1s | -- `trigger(workflowId, options)` - Start a journey -- `getStatus(runId)` - Check run status -- `cancel(runId)` - Cancel in-progress run +## vs Customer.io -### Segments (`customerio.do/segments`) +| Feature | Customer.io | customerio.do | +|---------|-------------|---------------| +| **Pricing** | $100-1000+/month | ~$10/month | +| **API Style** | Configuration objects | Natural language | +| **Setup** | Developer required | Marketer-friendly | +| **Workflows** | Proprietary builder | Cloudflare Workflows | +| **Data Location** | Their cloud | Your Cloudflare account | +| **Customization** | Limited | Code it yourself | +| **Lock-in** | Data export fees | MIT licensed | -- `create(definition)` - Define a segment -- `update(segmentId, definition)` - Modify rules -- `members(segmentId)` - Query membership -- `check(segmentId, userId)` - Check single user +## Use Cases -### Delivery (`customerio.do/delivery`) +### Product-Led Growth -- `send(userId, options)` - Send message -- `status(messageId)` - Delivery status +```typescript +// Trial conversion automation +await customerio`users on day 7 of trial who haven't activated` + .notify('Need help getting started?') + .wait('3 days') + .notify('Your trial ends soon - upgrade now for 20% off') + +// Feature adoption +await customerio`Pro users who never used export` + .notify('Did you know you can export your data?') +``` -### Templates (`customerio.do/templates`) +### E-Commerce -- `create(template)` - Create template -- `update(templateId, template)` - Modify template -- `preview(templateId, context)` - Render preview +```typescript +// Cart recovery +await customerio`cart abandoners in the last hour` + .wait('1 hour') + .notify('You left something behind!') + .wait('24 hours') + .notify('Still thinking about it? Here is 10% off') + +// Post-purchase +await customerio`customers who ordered today` + .notify('Your order is confirmed!') + .wait('3 days') + .notify('How was your experience? Leave a review') +``` -## The Rewrites Ecosystem +### SaaS Onboarding -customerio.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare Durable Objects: +```typescript +// Welcome sequence +await customerio`new signups` + .notify('Welcome! Here is how to get started') + .wait('1 day') + .notify('Have you completed your first project?') + .wait('3 days') + .notify('Meet the features that power users love') + +// Re-engagement +await customerio`users inactive for 30 days` + .notify('We miss you! Here is what is new') +``` + +## AI-Native Marketing + +### Audience Discovery + +```typescript +// AI finds your best audiences +await customerio`users most likely to convert` +await customerio`customers at risk of churning` +await customerio`users who would benefit from Pro` +``` + +### Content Generation + +```typescript +// AI writes the copy +await customerio`write win-back email for churned Pro users` +await customerio`generate subject lines for cart abandonment` +await customerio`personalize welcome email for alice` +``` -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| **customerio.do** | Customer.io | Marketing automation for AI | -| [notify.do](https://notify.do) | Novu/Knock | Notification infrastructure | -| kafka.do | Kafka | Event streaming for AI | -| nats.do | NATS | Messaging for AI | +### Campaign Optimization + +```typescript +// AI optimizes timing and content +await customerio`best time to email alice` +await customerio`which subject line performs better for Pro users` +await customerio`optimize cart abandonment sequence` +``` + +## Multi-Agent Marketing + +Every AI agent gets their own marketing instance: + +```typescript +import { mark, sally } from 'agents.do' +import { customerio } from 'customerio.do' + +// Mark handles marketing campaigns +await mark`announce the new feature to Pro users` + // Uses customerio under the hood + +// Sally handles sales outreach +await sally`reach out to trial users about to expire` + // Different campaigns, same platform +``` ## The workers.do Platform @@ -271,27 +368,65 @@ customerio.do is a core service of [workers.do](https://workers.do) - the platfo ```typescript import { priya, ralph, tom, mark } from 'agents.do' -import { CustomerIO } from 'customerio.do' - -// AI agents with marketing automation -const startup = { - product: priya, - engineering: ralph, - tech: tom, - marketing: mark, -} - -// Mark runs marketing campaigns -const markCIO = CustomerIO.for(mark) -await markCIO.workflows.trigger('launch-announcement', { - segment: 'active-users', - data: { feature: 'AI Assistant', launch_date: '2024-01-15' } -}) +// AI agents with marketing built in +await mark`launch announcement for AI Assistant` +await sally`follow up with leads from the webinar` +await priya`notify beta users about the new feature` ``` Both kinds of workers. Working for you. +## Roadmap + +### Core Features +- [x] User Profiles +- [x] Event Tracking +- [x] Dynamic Segments +- [x] Multi-Channel Delivery +- [x] Campaign Workflows +- [ ] A/B Testing +- [ ] Predictive Analytics + +### Channels +- [x] Email (via Resend/Postmark) +- [x] Push Notifications +- [x] SMS (via Twilio) +- [x] In-App Messages +- [ ] WhatsApp +- [ ] Slack + +### AI +- [x] Natural Language Queries +- [x] Content Generation +- [ ] Send Time Optimization +- [ ] Churn Prediction +- [ ] Audience Discovery + +## Contributing + +customerio.do is open source under the MIT license. + +```bash +git clone https://github.com/dotdo/customerio.do +cd customerio.do +pnpm install +pnpm test +``` + ## License -MIT +MIT License - Marketing automation for everyone. + +--- + +

+ Talk to your marketing platform. +
+ Natural language. Edge-native. AI-first. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/databricks/README.md b/rewrites/databricks/README.md index ab255bfa..6410e15d 100644 --- a/rewrites/databricks/README.md +++ b/rewrites/databricks/README.md @@ -2,1083 +2,365 @@ > The $62B Data Lakehouse. Now open source. AI-native. Zero complexity. -[![npm version](https://img.shields.io/npm/v/databricks.do.svg)](https://www.npmjs.com/package/databricks.do) -[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) - ---- - Databricks built a $62B empire on Apache Spark. Unity Catalog costs $6/DBU. Serverless SQL warehouses start at $0.22/DBU. MLflow "Enterprise" requires premium tiers. A 50-person data team easily spends $500K-$1M/year. **databricks.do** is the open-source alternative. Lakehouse architecture on Cloudflare. Delta tables backed by R2. SQL warehouses at the edge. ML pipelines without the bill. -## The Problem +## AI-Native API -Databricks has become synonymous with "enterprise data platform": +```typescript +import { databricks } from 'databricks.do' // Full SDK +import { databricks } from 'databricks.do/tiny' // Minimal client +import { databricks } from 'databricks.do/sql' // SQL-only operations +``` -| The Databricks Tax | Reality | -|--------------------|---------| -| Unity Catalog | $6/DBU premium + compute costs | -| Serverless SQL | $0.22-$0.70/DBU depending on tier | -| MLflow "Enterprise" | Requires premium workspace | -| Jobs compute | $0.15-$0.40/DBU depending on tier | -| Real-time inference | Model serving at $0.07/1000 requests | -| Total platform | $500K-$1M+/year for serious data teams | - -**The dirty secret**: Most Databricks implementations: -- Explode costs when workloads scale -- Lock you into proprietary Unity Catalog -- Require Databricks-specific skills (not transferable) -- Make data sharing across clouds expensive -- Hide simple SQL behind Spark complexity - -Meanwhile, Databricks' core value proposition - **unified analytics on a lakehouse** - is fundamentally a storage and compute coordination problem. Edge compute with columnar storage solves this better. +Natural language for data workflows: -## The Solution +```typescript +import { databricks } from 'databricks.do' -**databricks.do** brings the lakehouse to the edge: +// Talk to it like a colleague +const revenue = await databricks`sales by region last quarter` +const top = await databricks`top 10 customers by revenue` +const trend = await databricks`monthly revenue trend this year` -```bash -npx create-dotdo databricks -``` +// Chain like sentences +await databricks`customers who churned last month` + .map(c => databricks`analyze churn reasons for ${c}`) -Your own Databricks alternative. Running on Cloudflare. AI-native from day one. +// Pipelines that build themselves +await databricks`run sales ETL daily at 6am` + .monitor() // AI watches for anomalies + .alert() // notify on failures +``` -| Databricks | databricks.do | -|------------|---------------| -| $500K-$1M+/year | **Free** (open source) | -| Unity Catalog lock-in | **Open Delta Lake** | -| Spark required | **SQL-first** | -| DBU-based billing | **Pay for actual compute** | -| Cloud-specific | **Edge-native** | -| AI as premium feature | **AI at the core** | +## The Problem ---- +Databricks has become synonymous with "enterprise data platform": -## Features +| The Databricks Tax | Reality | +|--------------------|---------| +| **Unity Catalog** | $6/DBU premium + compute costs | +| **Serverless SQL** | $0.22-$0.70/DBU depending on tier | +| **MLflow "Enterprise"** | Requires premium workspace | +| **Jobs compute** | $0.15-$0.40/DBU depending on tier | +| **Model serving** | $0.07/1000 requests | +| **Total platform** | $500K-$1M+/year for serious data teams | -### Unity Catalog +### The DBU Tax -Data governance without the premium tier. Discover, manage, and secure all your data assets. +The real cost of Databricks: -```typescript -import { databricks } from 'databricks.do' +- Costs explode when workloads scale +- Proprietary Unity Catalog lock-in +- Databricks-specific skills (not transferable) +- Cross-cloud data sharing is expensive +- Simple SQL hidden behind Spark complexity -// Create a catalog -await databricks.catalog.create({ - name: 'production', - comment: 'Production data assets', - owner: 'data-platform-team', -}) +Meanwhile, Databricks' core value - **unified analytics on a lakehouse** - is a storage and compute coordination problem. Edge compute with columnar storage solves this better. -// Create a schema -await databricks.schema.create({ - catalog: 'production', - name: 'sales', - comment: 'Sales domain data', - properties: { - domain: 'sales', - team: 'revenue-analytics', - }, -}) +## The Solution -// Grant permissions -await databricks.grants.update({ - securable_type: 'SCHEMA', - full_name: 'production.sales', - changes: [ - { - principal: 'data-analysts', - add: ['SELECT', 'READ_VOLUME'], - }, - { - principal: 'data-engineers', - add: ['SELECT', 'MODIFY', 'CREATE_TABLE'], - }, - ], -}) +**databricks.do** reimagines the lakehouse for data engineers: -// Data lineage tracking -const lineage = await databricks.lineage.get({ - table: 'production.sales.orders', - direction: 'both', // upstream and downstream -}) +``` +Databricks databricks.do +----------------------------------------------------------------- +$500K-$1M+/year Deploy in minutes +$6/DBU Unity Catalog $0 - open source +Spark required SQL-first +Cloud-specific Edge-native +AI as premium feature AI at the core +DBU-based billing Pay for actual compute ``` -### Delta Lake Tables +## One-Click Deploy -ACID transactions on cloud object storage. Time travel built-in. +```bash +npx create-dotdo databricks +``` -```typescript -// Create a Delta table -await databricks.tables.create({ - catalog: 'production', - schema: 'sales', - name: 'orders', - columns: [ - { name: 'order_id', type: 'BIGINT', nullable: false }, - { name: 'customer_id', type: 'BIGINT', nullable: false }, - { name: 'order_date', type: 'DATE', nullable: false }, - { name: 'amount', type: 'DECIMAL(18,2)', nullable: false }, - { name: 'status', type: 'STRING', nullable: false }, - ], - partitionedBy: ['order_date'], - clusteringBy: ['customer_id'], - properties: { - 'delta.autoOptimize.optimizeWrite': 'true', - 'delta.autoOptimize.autoCompact': 'true', - }, -}) +Your own Lakehouse. Running on infrastructure you control. AI-native from day one. -// Insert data with ACID guarantees -await databricks.tables.insert({ - table: 'production.sales.orders', - data: [ - { order_id: 1, customer_id: 100, order_date: '2025-01-15', amount: 99.99, status: 'completed' }, - { order_id: 2, customer_id: 101, order_date: '2025-01-15', amount: 249.99, status: 'pending' }, - ], -}) +```typescript +import { Databricks } from 'databricks.do' -// Time travel - query historical versions -const yesterdayData = await databricks.sql` - SELECT * FROM production.sales.orders - VERSION AS OF 42 -` - -// Or by timestamp -const historicalData = await databricks.sql` - SELECT * FROM production.sales.orders - TIMESTAMP AS OF '2025-01-14 00:00:00' -` - -// MERGE (upsert) operations -await databricks.tables.merge({ - target: 'production.sales.orders', - source: stagingOrders, - on: 'target.order_id = source.order_id', - whenMatched: { - update: { status: 'source.status', amount: 'source.amount' }, - }, - whenNotMatched: { - insert: '*', - }, +export default Databricks({ + name: 'company-lakehouse', + domain: 'data.company.com', + storage: 'r2', }) ``` -### Spark SQL +## Features -Full SQL analytics without the Spark cluster overhead. +### Data Catalog ```typescript -// SQL queries execute at the edge -const revenue = await databricks.sql` - SELECT - DATE_TRUNC('month', order_date) AS month, - SUM(amount) AS revenue, - COUNT(DISTINCT customer_id) AS unique_customers - FROM production.sales.orders - WHERE order_date >= '2024-01-01' - GROUP BY 1 - ORDER BY 1 -` - -// Window functions -const customerRanking = await databricks.sql` - SELECT - customer_id, - SUM(amount) AS total_spend, - RANK() OVER (ORDER BY SUM(amount) DESC) AS spend_rank, - PERCENT_RANK() OVER (ORDER BY SUM(amount) DESC) AS percentile - FROM production.sales.orders - GROUP BY customer_id -` - -// CTEs and complex queries -const cohortAnalysis = await databricks.sql` - WITH first_orders AS ( - SELECT - customer_id, - DATE_TRUNC('month', MIN(order_date)) AS cohort_month - FROM production.sales.orders - GROUP BY customer_id - ), - monthly_activity AS ( - SELECT - o.customer_id, - f.cohort_month, - DATE_TRUNC('month', o.order_date) AS activity_month, - SUM(o.amount) AS revenue - FROM production.sales.orders o - JOIN first_orders f ON o.customer_id = f.customer_id - GROUP BY 1, 2, 3 - ) - SELECT - cohort_month, - DATEDIFF(MONTH, cohort_month, activity_month) AS months_since_first, - COUNT(DISTINCT customer_id) AS active_customers, - SUM(revenue) AS cohort_revenue - FROM monthly_activity - GROUP BY 1, 2 - ORDER BY 1, 2 -` +// Just say what you need +await databricks`create production catalog` +await databricks`create sales schema in production` +await databricks`give analysts read access to sales` + +// AI infers what you need +await databricks`production.sales` // returns schema info +await databricks`lineage for orders` // returns data lineage +await databricks`who can access sales?` // returns permissions ``` -### MLflow Integration - -Model lifecycle management without the premium workspace. +### Delta Tables ```typescript -import { mlflow } from 'databricks.do/ml' - -// Create an experiment -const experiment = await mlflow.createExperiment({ - name: 'customer-churn-prediction', - artifact_location: 'r2://mlflow-artifacts/churn', - tags: { - team: 'data-science', - project: 'retention', - }, -}) - -// Log a run -const run = await mlflow.startRun({ - experiment_id: experiment.id, - run_name: 'xgboost-v1', -}) - -await mlflow.logParams(run.id, { - max_depth: 6, - learning_rate: 0.1, - n_estimators: 100, -}) - -await mlflow.logMetrics(run.id, { - accuracy: 0.89, - precision: 0.85, - recall: 0.92, - f1_score: 0.88, - auc_roc: 0.94, -}) - -// Log the model -await mlflow.logModel(run.id, { - artifact_path: 'model', - flavor: 'sklearn', - model: trainedModel, - signature: { - inputs: [ - { name: 'tenure', type: 'double' }, - { name: 'monthly_charges', type: 'double' }, - { name: 'total_charges', type: 'double' }, - ], - outputs: [ - { name: 'churn_probability', type: 'double' }, - ], - }, -}) - -await mlflow.endRun(run.id) - -// Register model to Model Registry -await mlflow.registerModel({ - name: 'customer-churn-model', - source: `runs:/${run.id}/model`, - description: 'XGBoost model for predicting customer churn', -}) - -// Promote to production -await mlflow.transitionModelVersion({ - name: 'customer-churn-model', - version: 1, - stage: 'Production', - archive_existing: true, -}) - -// Model serving (inference at the edge) -const prediction = await mlflow.predict({ - model: 'customer-churn-model', - stage: 'Production', - input: { - tenure: 24, - monthly_charges: 79.99, - total_charges: 1919.76, - }, -}) -// { churn_probability: 0.23 } +// Create tables naturally +await databricks`create orders table with order_id customer_id amount status` +await databricks`partition orders by date` +await databricks`add order_id 1 customer 100 amount 99.99 to orders` + +// Time travel is one word +await databricks`orders as of yesterday` +await databricks`orders version 42` +await databricks`orders changes since last week` + +// Merges read like English +await databricks`upsert staging_orders into orders on order_id` ``` -### Notebooks - -Interactive analytics without the cluster spin-up time. +### SQL Analytics ```typescript -import { notebooks } from 'databricks.do' - -// Create a notebook -const notebook = await notebooks.create({ - path: '/workspace/analytics/sales-analysis', - language: 'SQL', // SQL, Python, Scala, R - content: ` --- Cell 1: Load data -SELECT * FROM production.sales.orders LIMIT 10 - --- Cell 2: Aggregate -SELECT - status, - COUNT(*) as count, - SUM(amount) as total -FROM production.sales.orders -GROUP BY status - --- Cell 3: Visualization -%viz bar -SELECT - DATE_TRUNC('day', order_date) as date, - SUM(amount) as revenue -FROM production.sales.orders -WHERE order_date >= CURRENT_DATE - INTERVAL 30 DAYS -GROUP BY 1 -ORDER BY 1 - `, -}) - -// Run a notebook -const result = await notebooks.run({ - path: '/workspace/analytics/sales-analysis', - parameters: { - start_date: '2025-01-01', - end_date: '2025-01-31', - }, -}) - -// Schedule notebook execution -await notebooks.schedule({ - path: '/workspace/analytics/daily-report', - cron: '0 8 * * *', // 8 AM daily - timezone: 'America/Los_Angeles', - alerts: { - on_failure: ['data-team@company.com'], - }, -}) +// Just ask questions +await databricks`monthly revenue this year` +await databricks`customers ranked by spend` +await databricks`cohort analysis for Q1 signups` + +// Complex analysis, simple words +await databricks`customer lifetime value by acquisition channel` +await databricks`revenue attribution by marketing touchpoint` +await databricks`funnel conversion rates by segment` ``` -### SQL Warehouses - -Serverless SQL compute that scales to zero. +### MLflow ```typescript -import { warehouse } from 'databricks.do' - -// Create a SQL warehouse -const wh = await warehouse.create({ - name: 'analytics-warehouse', - size: 'Small', // Small, Medium, Large, X-Large - auto_stop_mins: 15, - enable_photon: true, // Vectorized query engine - max_num_clusters: 10, - spot_instance_policy: 'COST_OPTIMIZED', -}) - -// Execute queries against the warehouse -const result = await warehouse.query({ - warehouse_id: wh.id, - statement: ` - SELECT - product_category, - SUM(revenue) as total_revenue, - AVG(margin) as avg_margin - FROM production.sales.product_metrics - GROUP BY 1 - ORDER BY 2 DESC - LIMIT 10 - `, - wait_timeout: '30s', -}) - -// Query with parameters -const customerOrders = await warehouse.query({ - warehouse_id: wh.id, - statement: ` - SELECT * FROM production.sales.orders - WHERE customer_id = :customer_id - AND order_date >= :start_date - `, - parameters: [ - { name: 'customer_id', value: '12345', type: 'BIGINT' }, - { name: 'start_date', value: '2025-01-01', type: 'DATE' }, - ], -}) - -// Stream results for large queries -for await (const chunk of warehouse.streamQuery({ - warehouse_id: wh.id, - statement: 'SELECT * FROM production.logs.events', - chunk_size: 10000, -})) { - await processChunk(chunk) -} +// Experiments without the ceremony +await databricks`track churn prediction experiment` +await databricks`log accuracy 0.89 precision 0.85 recall 0.92` +await databricks`register churn model v1` + +// Deploy with one line +await databricks`deploy churn model to production` +await databricks`predict churn for customer 12345` + +// AI manages the lifecycle +await databricks`what's the best performing churn model?` +await databricks`compare model v1 vs v2` ``` -### DLT Pipelines (Delta Live Tables) - -Declarative ETL with automatic dependency management. +### Notebooks ```typescript -import { DLT } from 'databricks.do/pipelines' - -// Define a DLT pipeline -const pipeline = DLT.pipeline({ - name: 'sales-etl', - target: 'production.sales', - continuous: false, - development: false, -}) - -// Bronze layer - raw ingestion -const rawOrders = pipeline.table({ - name: 'raw_orders', - comment: 'Raw orders from source systems', - source: () => databricks.sql` - SELECT * FROM cloud_files( - 's3://raw-data/orders/', - 'json', - map('cloudFiles.inferColumnTypes', 'true') - ) - `, - expectations: { - 'valid_order_id': 'order_id IS NOT NULL', - 'valid_amount': 'amount > 0', - }, - expectation_action: 'ALLOW', // ALLOW, DROP, FAIL -}) - -// Silver layer - cleaned and conformed -const cleanedOrders = pipeline.table({ - name: 'cleaned_orders', - comment: 'Cleaned and validated orders', - source: () => databricks.sql` - SELECT - CAST(order_id AS BIGINT) AS order_id, - CAST(customer_id AS BIGINT) AS customer_id, - TO_DATE(order_date) AS order_date, - CAST(amount AS DECIMAL(18,2)) AS amount, - UPPER(TRIM(status)) AS status, - CURRENT_TIMESTAMP() AS processed_at - FROM LIVE.raw_orders - WHERE order_id IS NOT NULL - `, - expectations: { - 'unique_orders': 'COUNT(*) = COUNT(DISTINCT order_id)', - }, -}) - -// Gold layer - business aggregates -const dailySales = pipeline.table({ - name: 'daily_sales', - comment: 'Daily sales aggregations', - source: () => databricks.sql` - SELECT - order_date, - COUNT(*) AS order_count, - COUNT(DISTINCT customer_id) AS unique_customers, - SUM(amount) AS total_revenue, - AVG(amount) AS avg_order_value - FROM LIVE.cleaned_orders - WHERE status = 'COMPLETED' - GROUP BY order_date - `, -}) - -// Deploy the pipeline -await pipeline.deploy() +// Run analysis like you'd describe it +await databricks`run sales analysis notebook` +await databricks`schedule daily report at 8am Pacific` -// Run the pipeline -const update = await pipeline.start() -console.log(update.state) // STARTING -> RUNNING -> COMPLETED +// AI writes the code +await databricks`analyze sales trends and visualize as line chart` +await databricks`create dashboard for executive summary` ``` -### Lakehouse Architecture - -Unified platform for all your data workloads. +### SQL Warehouses ```typescript -import { lakehouse } from 'databricks.do' - -// Configure the lakehouse -const config = await lakehouse.configure({ - // Storage layer - storage: { - type: 'r2', // R2, S3, GCS, ADLS - bucket: 'company-lakehouse', - region: 'auto', - }, - - // Compute layer - compute: { - default_warehouse: 'analytics-warehouse', - spark_config: { - 'spark.sql.adaptive.enabled': 'true', - 'spark.sql.adaptive.coalescePartitions.enabled': 'true', - }, - }, - - // Governance layer - governance: { - default_catalog: 'production', - audit_logging: true, - data_lineage: true, - column_level_security: true, - }, - - // AI layer - ai: { - vector_search_enabled: true, - feature_store_enabled: true, - model_serving_enabled: true, - }, -}) +// Warehouses that manage themselves +await databricks`create analytics warehouse auto-stop 15 min` +await databricks`scale warehouse to handle Black Friday traffic` -// Lakehouse medallion architecture -const architecture = await lakehouse.createMedallion({ - bronze: { - catalog: 'raw', - retention_days: 90, - format: 'delta', - }, - silver: { - catalog: 'curated', - retention_days: 365, - format: 'delta', - z_ordering: true, - }, - gold: { - catalog: 'production', - retention_days: null, // Forever - format: 'delta', - materialized_views: true, - }, -}) +// Stream large results naturally +await databricks`stream all events from last month` + .each(batch => process(batch)) ``` ---- - -## AI-Native Analytics - -This is the revolution. Data engineering and analytics are fundamentally AI problems. - -### Natural Language to SQL - -Skip the SQL syntax. Just ask: +### ETL Pipelines ```typescript -import { databricks } from 'databricks.do' - -// Natural language queries -const result = await databricks`what were our top 10 customers by revenue last quarter?` -// Generates and executes: -// SELECT customer_id, SUM(amount) as revenue -// FROM production.sales.orders -// WHERE order_date >= DATE_TRUNC('quarter', CURRENT_DATE - INTERVAL 3 MONTHS) -// GROUP BY customer_id -// ORDER BY revenue DESC -// LIMIT 10 - -const analysis = await databricks`analyze the trend in order volume over the past year` -// Returns data + visualization + narrative - -const pipeline = await databricks`create an ETL pipeline to load customer data from S3` -// Generates DLT pipeline definition +// Pipelines in plain English +await databricks`create sales ETL bronze silver gold` +await databricks`run sales pipeline` +await databricks`backfill orders from January` + +// Quality expectations built-in +await databricks`fail if order_id is null` +await databricks`drop rows where amount <= 0` ``` -### Promise Pipelining for ML Workflows +## Promise Pipelining -Chain ML operations without callback hell: +Chain operations without callback hell: ```typescript -import { priya, ralph, tom } from 'agents.do' -import { databricks, mlflow } from 'databricks.do' - -// Build an ML pipeline with promise pipelining -const deployed = await databricks`load customer transaction data` - .map(data => databricks`clean and validate the data`) - .map(cleaned => databricks`engineer features for churn prediction`) - .map(features => mlflow`train an XGBoost model`) - .map(model => mlflow`evaluate model performance`) - .map(evaluated => mlflow`register model if metrics pass thresholds`) - .map(registered => mlflow`deploy to production endpoint`) +// Build an ML pipeline with one chain +const deployed = await databricks`load customer transactions` + .map(data => databricks`clean and validate ${data}`) + .map(clean => databricks`engineer churn features from ${clean}`) + .map(features => databricks`train XGBoost on ${features}`) + .map(model => databricks`evaluate ${model}`) + .map(evaluated => databricks`deploy if metrics pass`) // One network round trip. Record-replay pipelining. - -// AI agents orchestrate the workflow -const mlPipeline = await priya`design a customer segmentation model` - .map(spec => ralph`implement the feature engineering`) - .map(features => ralph`train the clustering model`) - .map(model => [priya, tom].map(r => r`review the model performance`)) - .map(reviewed => ralph`deploy to production`) ``` -### AI-Powered Data Quality - -Automatic anomaly detection and data quality monitoring: +### AI Agents as Data Engineers ```typescript -import { dataQuality } from 'databricks.do/ai' - -// Monitor data quality with AI -const monitor = await dataQuality.createMonitor({ - table: 'production.sales.orders', - baseline_window: '30 days', - metrics: [ - 'row_count', - 'null_rate', - 'distinct_count', - 'statistical_distribution', - ], - alert_on: { - row_count_change: 0.2, // 20% change - null_rate_increase: 0.05, // 5% increase - statistical_drift: 0.1, // Distribution shift - }, -}) +import { priya, ralph, tom, quinn } from 'agents.do' -// AI explains anomalies -const anomaly = await dataQuality.explain({ - table: 'production.sales.orders', - metric: 'row_count', - timestamp: '2025-01-15', -}) -// "Row count dropped 35% on 2025-01-15 compared to the 30-day average. -// Root cause analysis: -// - Source system API returned errors from 2-4 AM UTC -// - 12,453 orders failed to ingest -// - Recommendation: Re-run ingestion for affected time window" -``` +// The whole team, naturally +const spec = await priya`design customer 360 data model` +const pipeline = await ralph`implement ${spec} with DLT` +const review = await tom`review ${pipeline} architecture` +const tests = await quinn`create quality tests for ${pipeline}` -### AI Agents as Data Engineers +// Chain the team +await priya`plan Q1 data roadmap` + .map(plan => ralph`implement ${plan}`) + .map(code => [priya, tom, quinn].map(r => r`review ${code}`)) +``` -AI agents can build and maintain your data platform: +### AI-Powered Data Quality ```typescript -import { priya, ralph, tom, quinn } from 'agents.do' -import { databricks } from 'databricks.do' - -// Product manager defines requirements -const spec = await priya` - we need a customer 360 view that combines: - - transaction history - - support tickets - - product usage - - marketing engagement - create a data model spec -` - -// Developer implements the data pipeline -const pipeline = await ralph` - implement the customer 360 data model from ${spec} - use DLT for incremental processing - ensure GDPR compliance with column masking -` - -// Tech lead reviews the architecture -const review = await tom` - review the customer 360 pipeline architecture: - - data modeling best practices - - performance optimization - - cost efficiency - ${pipeline} -` - -// QA validates data quality -const validation = await quinn` - create data quality tests for customer 360: - - referential integrity - - business rule validation - - freshness SLAs -` +// Monitor automatically +await databricks`monitor orders for anomalies` +await databricks`alert if row count drops 20%` +await databricks`explain why orders dropped yesterday` +// "Row count dropped 35% - source API returned errors from 2-4 AM UTC. +// 12,453 orders failed to ingest. Re-run ingestion for affected window." ``` ---- - ## Architecture -databricks.do mirrors Databricks' architecture with Durable Objects: +### Durable Object per Workspace ``` - Cloudflare Edge - | - +---------------+---------------+ - | | | - +-----------+ +-----------+ +-----------+ - | Auth | | SQL | | MCP | - | Gateway | | Gateway | | Server | - +-----------+ +-----------+ +-----------+ - | | | - +-------+-------+-------+-------+ - | | - +------------+ +------------+ - | Workspace | | Workspace | - | DO | | DO | - +------------+ +------------+ - | - +--------------+--------------+ - | | | -+--------+ +-----------+ +---------+ -| Catalog| | Warehouse | | MLflow | -| DO | | DO | | DO | -+--------+ +-----------+ +---------+ - | | | -+---+---+ +------+------+ +----+----+ -| | | | | | | | -Delta Unity Query Vector Model Exp -Tables Catalog Engine Search Registry +LakehouseDO (config, users, catalogs) + | + +-- CatalogDO (schemas, tables, permissions) + | |-- SQLite: Metadata (encrypted) + | +-- R2: Delta tables (Parquet) + | + +-- WarehouseDO (queries, cache) + | |-- SQLite: Query state + | +-- Query engine + | + +-- MLflowDO (experiments, models) + | |-- SQLite: Tracking data + | +-- R2: Artifacts + | + +-- PipelineDO (ETL, schedules) + |-- SQLite: Pipeline state ``` -### Durable Object Structure - -| Durable Object | Databricks Equivalent | Purpose | -|----------------|----------------------|---------| -| `WorkspaceDO` | Workspace | Multi-tenant isolation | -| `CatalogDO` | Unity Catalog | Data governance | -| `SchemaDO` | Schema | Namespace management | -| `TableDO` | Delta Table | ACID table operations | -| `WarehouseDO` | SQL Warehouse | Query execution | -| `PipelineDO` | DLT Pipeline | ETL orchestration | -| `NotebookDO` | Notebook | Interactive analytics | -| `MLflowDO` | MLflow | Model lifecycle | -| `ExperimentDO` | Experiment | ML tracking | -| `ModelDO` | Model Registry | Model versioning | - ### Storage Tiers -``` -Hot (SQLite in DO) Warm (R2 Parquet) Cold (R2 Archive) ------------------ ----------------- ----------------- -Catalog metadata Delta table data Historical versions -Recent query cache ML artifacts Audit logs -Notebook state Feature store Compliance archive -MLflow tracking Large datasets Long-term retention -``` - -### Query Execution - -```typescript -// Query flow -SQL Query - | - v -Query Parser (Validate syntax) - | - v -Catalog Resolver (Resolve table references) - | - v -Access Control (Check permissions) - | - v -Query Optimizer (Generate execution plan) - | - v -Storage Layer (Fetch from R2/cache) - | - v -Execution Engine (Process at edge) - | - v -Results (Stream back to client) -``` - ---- +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Metadata, recent queries | <10ms | +| **Warm** | R2 Parquet | Delta tables, features | <100ms | +| **Cold** | R2 Archive | Historical versions, audit | <1s | ## vs Databricks | Feature | Databricks | databricks.do | |---------|------------|---------------| -| Pricing | $500K-$1M+/year | **Free** | -| Unity Catalog | $6/DBU premium | **Included** | -| SQL Warehouse | $0.22-$0.70/DBU | **Edge compute** | -| MLflow | Premium tier | **Included** | -| Infrastructure | Databricks managed | **Your Cloudflare** | -| Lock-in | Proprietary | **Open source** | -| Spark required | Yes | **SQL-first** | -| AI features | Premium add-ons | **Native** | - -### Cost Comparison - -**50-person data team with moderate workloads:** - -| | Databricks | databricks.do | -|-|------------|---------------| -| SQL Warehouse compute | $180,000/year | $0 | -| Unity Catalog | $36,000/year | $0 | -| Jobs compute | $120,000/year | $0 | -| MLflow/Model Serving | $48,000/year | $0 | -| Databricks Premium | $96,000/year | $0 | -| **Annual Total** | **$480,000** | **$50** (Workers) | -| **3-Year TCO** | **$1,440,000+** | **$150** | - ---- - -## Quick Start +| **Implementation** | DBU billing complexity | Deploy in minutes | +| **Annual Cost** | $500K-$1M+ | ~$50/month | +| **Architecture** | Spark clusters | Edge-native, global | +| **Data Catalog** | $6/DBU premium | Included | +| **AI** | Premium add-ons | AI-first design | +| **Data Location** | Databricks managed | Your Cloudflare account | +| **Lock-in** | Proprietary | MIT licensed | -### One-Click Deploy +## Use Cases -```bash -npx create-dotdo databricks - -# Follow prompts: -# - Workspace name -# - Storage configuration (R2 bucket) -# - Default catalog name -# - Authentication method -``` - -### Manual Setup - -```bash -git clone https://github.com/dotdo/databricks.do -cd databricks.do -npm install -npm run deploy -``` - -### First Query +### Data Engineering ```typescript -import { DatabricksClient } from 'databricks.do' +// Streaming pipelines in plain English +await databricks`stream events from Kafka to bronze` +await databricks`clean bronze events to silver` +await databricks`aggregate silver to gold metrics` -const databricks = new DatabricksClient({ - url: 'https://your-workspace.databricks.do', - token: process.env.DATABRICKS_TOKEN, -}) - -// 1. Create a catalog -await databricks.catalog.create({ name: 'demo' }) - -// 2. Create a schema -await databricks.schema.create({ - catalog: 'demo', - name: 'sales', -}) - -// 3. Create a table -await databricks.tables.create({ - catalog: 'demo', - schema: 'sales', - name: 'orders', - columns: [ - { name: 'id', type: 'BIGINT' }, - { name: 'customer', type: 'STRING' }, - { name: 'amount', type: 'DECIMAL(10,2)' }, - ], -}) - -// 4. Insert data -await databricks.tables.insert({ - table: 'demo.sales.orders', - data: [ - { id: 1, customer: 'Acme Corp', amount: 999.99 }, - { id: 2, customer: 'Globex Inc', amount: 1499.99 }, - ], -}) - -// 5. Query with SQL -const result = await databricks.sql` - SELECT customer, SUM(amount) as total - FROM demo.sales.orders - GROUP BY customer -` - -// 6. Or use natural language -const analysis = await databricks`what's our total revenue?` -// "Total revenue is $2,499.98 from 2 orders." +// Backfills are one line +await databricks`backfill orders from last month` ``` ---- - -## Migration from Databricks - -### Export from Databricks - -```bash -# Export Unity Catalog metadata -databricks unity-catalog export --catalog production --output ./export - -# Export notebooks -databricks workspace export_dir /workspace ./notebooks --format DBC - -# Export MLflow experiments -databricks experiments export --experiment-id 123 --output ./mlflow - -# Or use our migration tool -npx databricks.do migrate export \ - --workspace https://your-workspace.cloud.databricks.com \ - --token $DATABRICKS_TOKEN -``` - -### Import to databricks.do +### Data Science -```bash -npx databricks.do migrate import \ - --source ./export \ - --url https://your-workspace.databricks.do - -# Migrates: -# - Unity Catalog (catalogs, schemas, tables) -# - Table data (Delta format preserved) -# - Access controls and grants -# - MLflow experiments and models -# - Notebooks and dashboards -# - SQL queries and alerts +```typescript +// ML without the ceremony +await databricks`train churn model on customer features` +await databricks`tune hyperparameters for best accuracy` +await databricks`deploy if accuracy > 0.9` + +// Feature engineering +await databricks`create customer lifetime value features` +await databricks`store features in feature store` ``` -### Parallel Run +### Business Intelligence ```typescript -// Run both systems during transition -const bridge = databricks.migration.createBridge({ - source: { - type: 'databricks-cloud', - workspace: 'https://...', - token: process.env.DATABRICKS_TOKEN, - }, - target: { - type: 'databricks.do', - url: 'https://...', - }, - mode: 'dual-read', // Read from both, compare results -}) - -// Validation queries run against both -// Reconciliation reports generated automatically -// Cut over when confident +// Dashboards from questions +await databricks`create executive dashboard with revenue and customers` +await databricks`add filter for date range and region` +await databricks`schedule refresh daily at 6am` ``` ---- - -## Industry Use Cases - -### Data Engineering +### Migration from Databricks ```typescript -// Real-time data pipeline -const pipeline = DLT.pipeline({ - name: 'streaming-events', - continuous: true, -}) - -pipeline.table({ - name: 'events_bronze', - source: () => databricks.sql` - SELECT * FROM cloud_files( - 'r2://events/', - 'json', - map('cloudFiles.format', 'json') - ) - `, -}) +// One-liner migration +await databricks`import from cloud.databricks.com` -pipeline.table({ - name: 'events_silver', - source: () => databricks.sql` - SELECT - event_id, - user_id, - event_type, - TO_TIMESTAMP(event_time) as event_timestamp, - properties - FROM LIVE.events_bronze - `, -}) +// Or step by step +await databricks`import catalog production from databricks` +await databricks`import notebooks from /workspace` +await databricks`import mlflow experiments` ``` -### Data Science +## Deployment Options -```typescript -// Feature engineering + model training -const features = await databricks.sql` - SELECT - customer_id, - COUNT(*) as order_count, - SUM(amount) as total_spend, - AVG(amount) as avg_order_value, - MAX(order_date) as last_order, - DATEDIFF(CURRENT_DATE, MAX(order_date)) as days_since_last_order - FROM production.sales.orders - GROUP BY customer_id -` - -const model = await mlflow.autoML({ - task: 'classification', - target: 'churned', - features: features, - time_budget_minutes: 60, -}) +### Cloudflare Workers + +```bash +npx create-dotdo databricks +# Deploys to your Cloudflare account ``` -### Business Intelligence +### Private Cloud -```typescript -// Self-service analytics -const dashboard = await databricks.dashboard.create({ - name: 'Executive Summary', - queries: [ - { - name: 'Revenue Trend', - sql: `SELECT DATE_TRUNC('month', order_date) as month, SUM(amount) as revenue FROM production.sales.orders GROUP BY 1`, - visualization: 'line', - }, - { - name: 'Top Customers', - sql: `SELECT customer_id, SUM(amount) as spend FROM production.sales.orders GROUP BY 1 ORDER BY 2 DESC LIMIT 10`, - visualization: 'bar', - }, - ], - refresh_schedule: '0 8 * * *', -}) +```bash +docker run -p 8787:8787 dotdo/databricks ``` ---- - ## Roadmap -### Now +### Core Lakehouse - [x] Unity Catalog (catalogs, schemas, tables) - [x] Delta Lake tables with ACID - [x] SQL Warehouse queries - [x] MLflow tracking and registry -- [x] Notebooks (SQL) - [x] Natural language queries - -### Next -- [ ] DLT Pipelines (full implementation) -- [ ] Python notebooks -- [ ] Real-time streaming tables +- [ ] DLT Pipelines (full) +- [ ] Real-time streaming - [ ] Vector search -- [ ] Feature store -- [ ] Model serving endpoints -### Later -- [ ] Spark compatibility layer -- [ ] DBT integration -- [ ] Airflow integration -- [ ] Unity Catalog federation -- [ ] Cross-cloud Delta Sharing -- [ ] Photon-compatible query engine - ---- - -## Documentation - -| Guide | Description | -|-------|-------------| -| [Quick Start](./docs/quickstart.mdx) | Deploy in 5 minutes | -| [Unity Catalog](./docs/unity-catalog.mdx) | Data governance | -| [Delta Lake](./docs/delta-lake.mdx) | ACID tables | -| [SQL Warehouse](./docs/sql-warehouse.mdx) | Query execution | -| [MLflow](./docs/mlflow.mdx) | ML lifecycle | -| [DLT Pipelines](./docs/dlt.mdx) | ETL orchestration | -| [Migration](./docs/migration.mdx) | Moving from Databricks | - ---- +### AI +- [x] Natural language to SQL +- [x] AI data quality monitoring +- [ ] AutoML integration +- [ ] Feature store +- [ ] Model serving ## Contributing @@ -1087,29 +369,23 @@ databricks.do is open source under the MIT license. ```bash git clone https://github.com/dotdo/databricks.do cd databricks.do -npm install -npm test -npm run dev +pnpm install +pnpm test ``` -Key areas for contribution: -- Query engine optimization -- Delta Lake protocol compliance -- MLflow API compatibility -- Notebook execution runtime -- DLT pipeline orchestration - -See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - ---- - ## License -MIT +MIT License - For the data democratizers. ---

- The Lakehouse, simplified.
- Built on Cloudflare Workers. Powered by AI. No DBU pricing. + The $62B lakehouse ends here. +
+ SQL-first. AI-native. No DBU pricing. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/datadog/README.md b/rewrites/datadog/README.md index c2ce2cb9..606e06e1 100644 --- a/rewrites/datadog/README.md +++ b/rewrites/datadog/README.md @@ -6,46 +6,71 @@ Datadog dominates modern observability. But at $15-34/host/month for infrastruct **datadog.do** reimagines observability for the AI era. Full APM. Unlimited logs. Zero per-host pricing. -## The Problem +## AI-Native API -Datadog built an observability empire on: +```typescript +import { datadog } from 'datadog.do' // Full SDK +import { datadog } from 'datadog.do/tiny' // Minimal client +import { datadog } from 'datadog.do/metrics' // Metrics-only operations +``` -- **Per-host pricing** - $15/host/month (Infrastructure), $31/host/month (APM) -- **Per-GB log pricing** - $0.10/GB ingested, $1.27/GB indexed (15-day retention) -- **Custom metrics explosion** - $0.05/metric/month, scales to $100k+ easily -- **Data retention costs** - Historical data requires expensive plans -- **Feature fragmentation** - APM, Logs, Metrics, RUM, Security all priced separately -- **Unpredictable bills** - Usage spikes = surprise invoices +Natural language for observability: -A 500-host infrastructure with logs and APM? **$300k+/year**. With custom metrics and long retention? **$500k+**. +```typescript +import { datadog } from 'datadog.do' + +// Talk to it like an SRE +const health = await datadog`production health right now` +const errors = await datadog`errors in production last hour` +const slow = await datadog`slowest endpoints this week` + +// Chain like sentences +await datadog`services with high error rate` + .map(service => datadog`root cause for ${service}`) + .map(cause => datadog`recommended fix for ${cause}`) + +// Incidents that investigate themselves +await datadog`investigate the 3am outage` + .correlate() // cross-reference logs, traces, metrics + .timeline() // build incident timeline + .report() // generate post-mortem +``` -## The workers.do Way +## The Problem -It's 3am. Your pager goes off. Production is down. You need answers now - not after navigating five dashboards and writing three queries. Every minute of downtime costs money. Every metric you track costs money. You're paying to find problems and paying while they burn. +Datadog built an observability empire on: -What if incident response was a conversation? +| What Datadog Charges | The Reality | +|----------------------|-------------| +| **Per-host pricing** | $15/host (Infra), $31/host (APM) | +| **Per-GB logs** | $0.10/GB ingested, $1.27/GB indexed | +| **Custom metrics** | $0.05/metric/month, scales to $100k+ | +| **Retention** | Historical data requires expensive plans | +| **Fragmentation** | APM, Logs, Metrics, RUM all priced separately | +| **Unpredictable** | Usage spikes = surprise invoices | -```typescript -import { datadog, tom } from 'workers.do' +### The Observability Tax -// Natural language observability -const status = await datadog`what's the health of production right now?` -const cause = await datadog`why is the API slow?` -const alert = await datadog`alert when ${metric} exceeds ${threshold}` +A 500-host infrastructure with logs and APM? **$300k+/year**. With custom metrics and long retention? **$500k+**. -// Chain diagnostics into resolution -const resolution = await datadog`show me the error spike timeline` - .map(timeline => datadog`correlate with deployments and changes`) - .map(correlation => tom`identify root cause and suggest fix for ${correlation}`) -``` +### The Dashboard Maze -One import. Natural language. AI-powered incident response. +When production is down: +- Navigate five dashboards +- Write three queries +- Correlate manually +- Every minute costs money -That's observability that works for you. +### The Alert Fatigue + +- Too many alerts = ignored alerts +- Too few alerts = missed incidents +- Threshold tuning is a full-time job +- AI "assistants" add complexity, not clarity ## The Solution -**datadog.do** is Datadog reimagined: +**datadog.do** reimagines observability: ``` Traditional Datadog datadog.do @@ -66,449 +91,188 @@ npx create-dotdo datadog Your own Datadog. Running on Cloudflare. No per-host fees. -## Full-Stack Observability - -Everything you need to monitor your infrastructure: - ```typescript -import { datadog } from 'datadog.do' - -// Metrics -datadog.gauge('app.queue.size', 42, { queue: 'main' }) -datadog.count('app.requests.count', 1, { endpoint: '/api/users' }) -datadog.histogram('app.request.duration', 0.234, { endpoint: '/api/users' }) - -// Logs -datadog.log.info('User signed up', { - userId: 'user-123', - plan: 'pro', - source: 'auth-service', -}) - -// Traces -const span = datadog.trace.startSpan('http.request', { - service: 'api-gateway', - resource: '/api/users', -}) - -// Events -datadog.event({ - title: 'Deployment completed', - text: 'Version 2.3.1 deployed to production', - tags: ['environment:production', 'service:api'], +import { Datadog } from 'datadog.do' + +export default Datadog({ + name: 'acme-observability', + domain: 'monitor.acme.io', + retention: { + metrics: '90d', + logs: '365d', + traces: '30d', + }, }) ``` ## Features -### Infrastructure Monitoring - -Monitor hosts, containers, and services: +### Metrics ```typescript -import { Infrastructure } from 'datadog.do/infra' - -// Host metrics (auto-collected with agent) -// CPU, memory, disk, network, processes - -// Container metrics -const containers = await Infrastructure.containers({ - filter: 'kube_namespace:production', -}) - -// Kubernetes -const k8s = await Infrastructure.kubernetes({ - cluster: 'production', - metrics: ['pods', 'deployments', 'nodes'], -}) - -// Cloud integrations -await Infrastructure.integrate({ - provider: 'cloudflare', - metrics: ['workers', 'r2', 'd1'], -}) +// Query metrics naturally +await datadog`CPU usage for web servers last 24 hours` +await datadog`memory trend for api-gateway this week` +await datadog`disk usage above 80%` + +// AI infers what you need +await datadog`web servers` // returns host list +await datadog`web servers CPU` // returns metrics +await datadog`web servers CPU trending up` // returns analysis ``` -### Application Performance Monitoring (APM) - -End-to-end distributed tracing: +### Logs ```typescript -import { tracer } from 'datadog.do/apm' - -// Auto-instrument common libraries -tracer.use('http') -tracer.use('pg') -tracer.use('redis') -tracer.use('fetch') - -// Manual instrumentation -app.get('/api/users', async (c) => { - const span = tracer.startSpan('get_users') - - try { - const users = await span.trace('db.query', async () => { - return db.query('SELECT * FROM users') - }) - - span.setTag('user.count', users.length) - return c.json(users) - } catch (error) { - span.setError(error) - throw error - } finally { - span.finish() - } -}) - -// Service map auto-generated from traces -// Latency distributions, error rates, throughput +// Search logs naturally +await datadog`logs containing "timeout" in production` +await datadog`errors from api-gateway last hour` +await datadog`payment failures today` + +// Chain for investigation +await datadog`errors spiking in checkout` + .each(error => error.trace()) // get related traces ``` -### Log Management - -Unlimited log ingestion and analysis: +### Traces ```typescript -import { logs } from 'datadog.do/logs' - -// Structured logging -logs.info('Order processed', { - orderId: 'order-123', - amount: 99.99, - customer: 'user-456', -}) - -// Log parsing pipelines -const pipeline = logs.pipeline({ - name: 'Nginx Access Logs', - source: 'nginx', - processors: [ - { type: 'grok', pattern: '%{COMBINEDAPACHELOG}' }, - { type: 'date', source: 'timestamp', target: '@timestamp' }, - { type: 'geo', source: 'client_ip', target: 'geo' }, - { type: 'useragent', source: 'agent', target: 'browser' }, - ], -}) - -// Log queries -const results = await logs.query({ - query: 'service:api-gateway status:error', - from: '-15m', - to: 'now', - facets: ['@http.status_code', '@error.type'], -}) - -// Log archives (to R2) -logs.archive({ - query: '*', - destination: 'r2://logs-archive', - retention: '365d', -}) +// Distributed tracing in plain English +await datadog`slow requests to /api/users` +await datadog`traces with errors in payment-service` +await datadog`p99 latency for checkout flow` + +// Follow the request path +await datadog`trace the slow request from user-123` + .visualize() // flame graph ``` -### Metrics - -Custom metrics without the per-metric cost: +### Alerting ```typescript -import { metrics } from 'datadog.do/metrics' - -// Gauge (current value) -metrics.gauge('app.connections.active', 42, { - service: 'api', - region: 'us-east', -}) - -// Count (increments) -metrics.count('app.requests', 1, { - endpoint: '/api/users', - method: 'GET', - status: '200', -}) - -// Histogram (distributions) -metrics.histogram('app.latency', 0.234, { - endpoint: '/api/users', - percentiles: [0.5, 0.95, 0.99], -}) - -// Distribution (global percentiles) -metrics.distribution('app.request.duration', 0.234, { - service: 'api', -}) - -// Rate (per-second) -metrics.rate('app.throughput', eventCount, { - service: 'api', -}) - -// Set (unique values) -metrics.set('app.users.unique', userId, { - time_window: '1h', -}) +// Alerts as sentences +await datadog`alert when CPU > 90% for 5 minutes` +await datadog`alert when error rate > 5% for 5 minutes` +await datadog`alert when p99 latency > 2 seconds` +await datadog`alert when disk usage > 80%` + +// Smart alerts +await datadog`alert when traffic is anomalous` +await datadog`alert when checkout errors spike` + +// Composite alerts +await datadog`alert when high CPU and high error rate together` ``` ### Dashboards -Build real-time dashboards: - ```typescript -import { Dashboard, Widget } from 'datadog.do/dashboard' - -const infraDashboard = Dashboard({ - title: 'Infrastructure Overview', - layout: 'ordered', - widgets: [ - Widget.timeseries({ - title: 'CPU Usage', - query: 'avg:system.cpu.user{*} by {host}', - display: 'line', - }), - Widget.topList({ - title: 'Top Hosts by Memory', - query: 'top(avg:system.mem.used{*} by {host}, 10)', - }), - Widget.queryValue({ - title: 'Total Requests', - query: 'sum:app.requests{*}.as_count()', - precision: 0, - }), - Widget.heatmap({ - title: 'Request Latency', - query: 'avg:app.latency{*} by {endpoint}', - }), - Widget.hostmap({ - title: 'Host Map', - query: 'avg:system.cpu.user{*} by {host}', - color: 'cpu', - size: 'memory', - }), - Widget.logStream({ - title: 'Error Logs', - query: 'status:error', - columns: ['timestamp', 'service', 'message'], - }), - ], -}) -``` +// Create dashboards naturally +await datadog`dashboard for production infrastructure` +await datadog`dashboard for api-gateway performance` +await datadog`dashboard for checkout flow` -### Alerting +// Query dashboards +await datadog`show me the infrastructure dashboard` +``` -Proactive monitoring with alerts: +### Infrastructure ```typescript -import { Monitor } from 'datadog.do/monitors' - -// Metric alert -const cpuAlert = Monitor({ - name: 'High CPU Usage', - type: 'metric', - query: 'avg(last_5m):avg:system.cpu.user{*} by {host} > 90', - message: ` - CPU usage is above 90% on {{host.name}}. - - Current value: {{value}} - - @slack-ops-alerts - `, - thresholds: { - critical: 90, - warning: 80, - }, - renotify_interval: 300, -}) - -// Log alert -const errorAlert = Monitor({ - name: 'Error Rate Spike', - type: 'log', - query: 'logs("status:error").rollup("count").by("service").last("5m") > 100', - message: 'Error rate spike in {{service.name}}', -}) - -// APM alert -const latencyAlert = Monitor({ - name: 'High Latency', - type: 'apm', - query: 'avg(last_5m):avg:trace.http.request.duration{service:api} > 1', - message: 'API latency exceeds 1 second', -}) - -// Composite alert -const compositeAlert = Monitor({ - name: 'Service Degradation', - type: 'composite', - query: '${cpu_alert} && ${error_alert}', - message: 'Service experiencing both high CPU and error spikes', -}) - -// Anomaly detection -const anomalyAlert = Monitor({ - name: 'Traffic Anomaly', - type: 'metric', - query: 'avg(last_4h):anomalies(avg:app.requests{*}, "basic", 2) >= 1', - message: 'Unusual traffic pattern detected', -}) +// Host and container monitoring +await datadog`hosts with high CPU` +await datadog`containers in production namespace` +await datadog`pods restarting in kubernetes` + +// Cloud resources +await datadog`cloudflare workers by region` +await datadog`r2 buckets by size` +await datadog`d1 databases by query count` ``` -### Real User Monitoring (RUM) - -Monitor frontend performance: +### APM ```typescript -import { RUM } from 'datadog.do/rum' - -// Initialize RUM -RUM.init({ - applicationId: 'your-app-id', - clientToken: 'your-token', - site: 'your-org.datadog.do', - service: 'my-web-app', - trackInteractions: true, - trackResources: true, - trackLongTasks: true, -}) - -// Custom user actions -RUM.addAction('checkout_clicked', { - cartValue: 99.99, - itemCount: 3, -}) +// Application performance in plain English +await datadog`services with highest error rate` +await datadog`slowest database queries` +await datadog`api-gateway dependencies` + +// Service health +await datadog`is checkout service healthy?` +await datadog`what's blocking payment-service?` +``` -// Custom errors -RUM.addError(error, { - context: { userId: 'user-123' }, -}) +### RUM -// User identification -RUM.setUser({ - id: 'user-123', - email: 'user@example.com', - plan: 'pro', -}) +```typescript +// Real user monitoring +await datadog`page load times this week` +await datadog`javascript errors on checkout page` +await datadog`users affected by slow performance` + +// User journeys +await datadog`users dropping off at checkout` +await datadog`conversion rate by browser` ``` ### Synthetic Monitoring -Proactive testing: - ```typescript -import { Synthetics } from 'datadog.do/synthetics' - -// API test -const apiTest = Synthetics.api({ - name: 'Health Check', - request: { - method: 'GET', - url: 'https://api.example.com/health', - }, - assertions: [ - { type: 'statusCode', operator: 'is', target: 200 }, - { type: 'responseTime', operator: 'lessThan', target: 500 }, - { type: 'body', operator: 'contains', target: '"status":"healthy"' }, - ], - locations: ['us-east-1', 'eu-west-1', 'ap-southeast-1'], - frequency: 60, -}) - -// Browser test -const browserTest = Synthetics.browser({ - name: 'Login Flow', - startUrl: 'https://app.example.com/login', - steps: [ - { type: 'typeText', selector: '#email', value: 'test@example.com' }, - { type: 'typeText', selector: '#password', value: 'password' }, - { type: 'click', selector: 'button[type="submit"]' }, - { type: 'assertText', selector: '.welcome', value: 'Welcome' }, - ], - frequency: 300, -}) +// Proactive testing as sentences +await datadog`check api.acme.com/health every minute` +await datadog`test login flow from all regions` +await datadog`verify checkout completes under 3 seconds` ``` -## AI-Native Features - -### Natural Language Queries +## AI-Native Observability -Ask questions about your infrastructure: +### Incident Response ```typescript -import { ask } from 'datadog.do' - -// Simple questions -const q1 = await ask('what is the current CPU usage across all hosts?') -// { value: 45, unit: '%', trend: 'stable' } +// Ask questions about your infrastructure +await datadog`what's wrong with production right now?` +await datadog`why is the API slow?` +await datadog`what changed before the errors started?` // Diagnostic questions -const q2 = await ask('why is the API slow right now?') -// { -// diagnosis: 'Database connection pool saturated', -// evidence: [...], -// recommendations: ['Increase pool size', 'Add read replicas'] -// } - -// Comparative questions -const q3 = await ask('how does this week compare to last week?') -// { comparison: {...}, anomalies: [...] } - -// Root cause questions -const q4 = await ask('what caused the outage at 3pm?') -// { timeline: [...], rootCause: '...', impact: '...' } +await datadog`root cause of the 3am outage` +await datadog`how does this week compare to last week?` +await datadog`what's different about the slow requests?` ``` ### Watchdog (AI Detection) -AI-powered anomaly detection: - ```typescript -import { Watchdog } from 'datadog.do' - -// Enable AI monitoring -Watchdog.enable({ - services: ['api', 'web', 'worker'], - sensitivity: 'medium', -}) - -// Get AI-detected issues -const issues = await Watchdog.issues({ - from: '-24h', - severity: ['critical', 'high'], -}) - -for (const issue of issues) { - console.log(issue.title) - // "Latency spike in api-gateway" - - console.log(issue.impact) - // "Affecting 15% of requests" - - console.log(issue.rootCause) - // "Correlated with database connection spike" - - console.log(issue.relatedSpans) - // Links to affected traces -} +// AI finds anomalies automatically +await datadog`enable watchdog for production services` +await datadog`watchdog alerts last 24 hours` +await datadog`explain the latency anomaly in api-gateway` + +// AI correlates across signals +await datadog`correlate error spike with recent deployments` +await datadog`what else changed when checkout broke?` ``` ### AI Agents as SREs -AI agents for incident response: - ```typescript import { tom, quinn } from 'agents.do' import { datadog } from 'datadog.do' // Tech lead investigates incident -const investigation = await tom` - investigate the current elevated error rate in production - correlate logs, traces, and metrics to find the root cause -` +await tom`investigate the elevated error rate in production` + .using(datadog) + .report() // QA validates fix -const validation = await quinn` - verify that the deployment fixed the issue - compare error rates before and after -` +await quinn`verify the deployment fixed the issue` + .compare('error rate before and after') + +// Chain investigation into resolution +await datadog`services with errors right now` + .map(service => tom`diagnose ${service} and suggest fix`) + .map(fix => quinn`test that ${fix} resolves the issue`) ``` ## Architecture @@ -569,17 +333,15 @@ Metrics Logs Traces Events ### Query Engine ```typescript -// Metrics queries (Datadog Query Language compatible) -query('avg:system.cpu.user{*} by {host}') -query('sum:app.requests{env:prod}.as_count().rollup(sum, 60)') -query('top(avg:app.latency{*} by {endpoint}, 10, mean)') +// Natural language queries compile to DQL +await datadog`average CPU across web servers` +// -> avg:system.cpu.user{role:web} by {host} -// Log queries -query('service:api status:error @http.status_code:500') -query('service:api @duration:>1000') +await datadog`requests per second in production` +// -> sum:app.requests{env:prod}.as_count().rollup(sum, 60) -// Trace queries -query('service:api operation:http.request @duration:>1s') +await datadog`slowest 10 endpoints` +// -> top(avg:app.latency{*} by {endpoint}, 10, mean) ``` ## Agent Installation @@ -622,16 +384,14 @@ spec: ### Cloudflare Workers ```typescript -import { instrument } from 'datadog.do/worker' +import { datadog } from 'datadog.do' -export default instrument({ +// Auto-instrumentation - just wrap your worker +export default datadog.instrument({ async fetch(request, env, ctx) { // Your worker code - // Automatically captures traces, logs, metrics + // Traces, logs, metrics collected automatically }, -}, { - service: 'my-worker', - env: 'production', }) ``` @@ -639,46 +399,44 @@ export default instrument({ ### Agent Compatibility -The datadog.do agent is compatible with Datadog's agent protocol: +Point your existing agent to datadog.do: ```bash -# Switch site to your datadog.do instance -DD_SITE=your-org.datadog.do DD_API_KEY=your-key \ - datadog-agent run +DD_SITE=your-org.datadog.do DD_API_KEY=your-key datadog-agent run ``` -### API Compatibility - -Drop-in replacement for Datadog API: +### Dashboard Migration -``` -Endpoint Status ------------------------------------------------------------------ -POST /api/v1/series Supported -POST /api/v1/distribution_points Supported -POST /api/v1/check_run Supported -POST /api/v1/events Supported -POST /api/v1/logs Supported (v2 also) -POST /api/v1/intake Supported -GET /api/v1/query Supported +```typescript +// Migrate dashboards with one command +await datadog`import dashboards from datadog` +await datadog`import monitors from datadog` +await datadog`import synthetic tests from datadog` ``` -### Dashboard Migration +Or use the CLI: ```bash -# Export from Datadog -datadog-export dashboards --output ./dashboards - -# Import to datadog.do -npx datadog-migrate import ./dashboards +npx datadog-migrate import --from-datadog --all ``` ## Integrations +```typescript +// Enable integrations naturally +await datadog`connect to AWS` +await datadog`connect to Cloudflare` +await datadog`connect to PostgreSQL` + +// Or specify details +await datadog`monitor kubernetes cluster production` +await datadog`track GitHub Actions deployments` +``` + Pre-built integrations for common services: | Category | Integrations | -|----------|-------------| +|----------|-----------| | **Cloud** | AWS, GCP, Azure, Cloudflare | | **Containers** | Docker, Kubernetes, ECS | | **Databases** | PostgreSQL, MySQL, Redis, MongoDB | @@ -686,6 +444,20 @@ Pre-built integrations for common services: | **Languages** | Node.js, Python, Go, Java, Ruby | | **CI/CD** | GitHub Actions, GitLab CI, Jenkins | +## vs Datadog + +| Feature | Datadog | datadog.do | +|---------|---------|------------| +| **Infrastructure** | $15/host/month | $0 - run your own | +| **APM** | $31/host/month | $0 - run your own | +| **Logs** | $1.27/GB indexed | Unlimited (R2 storage) | +| **Custom metrics** | $0.05/metric/month | Unlimited | +| **Retention** | 15 days (logs) | Unlimited | +| **Data location** | Datadog's cloud | Your Cloudflare account | +| **Query language** | DQL | Natural language + DQL | +| **AI features** | Watchdog | AI-native from day one | +| **Lock-in** | Proprietary | MIT licensed | + ## Roadmap - [x] Metrics collection (agent) @@ -712,6 +484,23 @@ Observability shouldn't cost millions: Datadog showed the world what modern observability could be. **datadog.do** makes it accessible to everyone. +## Contributing + +datadog.do is open source under the MIT license. + +We especially welcome contributions from: +- SREs and DevOps engineers +- Observability experts +- AI/ML engineers +- Infrastructure engineers + +```bash +git clone https://github.com/dotdo/datadog.do +cd datadog.do +pnpm install +pnpm test +``` + ## License MIT License - Monitor everything. Alert on anything. Pay for storage, not seats. @@ -719,7 +508,12 @@ MIT License - Monitor everything. Alert on anything. Pay for storage, not seats. ---

- datadog.do is part of the dotdo platform. + The $18B valuation ends here.
- Website | Docs | Discord + AI-native. Open source. Your data. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/docusign/README.md b/rewrites/docusign/README.md index e2e53548..dcac609f 100644 --- a/rewrites/docusign/README.md +++ b/rewrites/docusign/README.md @@ -6,40 +6,33 @@ DocuSign turned "send a PDF for signature" into a $15B company charging $10-60 p **docusign.do** is the open-source alternative. Send documents for signature. Full contract lifecycle management. AI that actually reads and negotiates contracts. -## The workers.do Way - -You're a founder who just wants to close deals. But every contract becomes a negotiation nightmare - redlines flying back and forth, legal review backlogs, signature chasing. Meanwhile, DocuSign charges you per envelope to send PDFs. - -**workers.do** gives you AI that closes deals: +## AI-Native API ```typescript -import { docusign, mark } from 'workers.do' - -// Natural language for agreements -const contracts = await docusign`find contracts expiring in 90 days` -const risks = await docusign`analyze ${vendor} agreement for liability gaps` -const status = await docusign`which signatures are we waiting on for ${deal}` +import { docusign } from 'docusign.do' // Full SDK +import { docusign } from 'docusign.do/tiny' // Minimal client +import { docusign } from 'docusign.do/clm' // CLM-only operations ``` -Promise pipelining for contract workflows - one network round trip: +Natural language for agreements: ```typescript -// Proposal to signed deal -const signed = await docusign`generate ${service} agreement for ${customer}` - .map(contract => docusign`apply our standard negotiation playbook`) - .map(redlined => docusign`send for signature to ${customer.legal}`) - .map(executed => mark`notify sales team about ${executed}`) -``` +import { docusign } from 'docusign.do' -AI agents that negotiate for you: +// Talk to it like an assistant +const nda = await docusign`send NDA to alice@acme.com` +const pending = await docusign`envelopes pending signature` +const expiring = await docusign`contracts expiring in 90 days` -```typescript -import { priya, tom, sally } from 'agents.do' +// Chain like sentences +await docusign`envelopes pending over 7 days` + .map(env => docusign`remind ${env.recipient}`) -// Contract intelligence -await priya`compare ${vendor} terms against our standard positions` -await tom`identify non-standard clauses in ${agreement} requiring review` -await sally`draft response to ${vendor} redlines - protect our interests` +// Proposal to signed deal +await docusign`generate MSA for Acme Corp` + .map(contract => docusign`apply standard negotiation playbook`) + .map(redlined => docusign`send to legal@acme.com for signature`) + .notify(`Deal closed with Acme`) ``` ## The Problem @@ -85,162 +78,61 @@ Your own e-signature and CLM platform. Legally binding. AI-powered. Send, sign, done: ```typescript -import { sign } from 'docusign.do' - -// Create envelope -const envelope = await sign.envelopes.create({ - name: 'Service Agreement - Acme Corp', - documents: [ - { - name: 'Service Agreement', - file: agreementPdf, - fileType: 'pdf', - }, - ], - recipients: [ - { - role: 'signer', - name: 'John Smith', - email: 'john@acme.com', - order: 1, - fields: [ - { type: 'signature', page: 5, x: 100, y: 600, required: true }, - { type: 'date', page: 5, x: 300, y: 600, required: true }, - { type: 'initial', page: 2, x: 450, y: 700, required: true }, - { type: 'initial', page: 3, x: 450, y: 700, required: true }, - ], - }, - { - role: 'signer', - name: 'Jane Doe', - email: 'jane@vendor.com', - order: 2, - fields: [ - { type: 'signature', page: 5, x: 100, y: 650, required: true }, - { type: 'date', page: 5, x: 300, y: 650, required: true }, - ], - }, - { - role: 'cc', - name: 'Legal Team', - email: 'legal@acme.com', - }, - ], - message: { - subject: 'Please sign: Service Agreement', - body: 'Please review and sign the attached service agreement.', - }, - options: { - reminderDays: 3, - expireDays: 30, - }, -}) - -// Send for signature -await envelope.send() -``` +// Send documents for signature +await docusign`send service agreement to john@acme.com` +await docusign`send NDA to alice@startup.com and bob@startup.com` +await docusign`send offer letter to candidate@email.com cc hr@company.com` -### Signing Experience +// Check status naturally +await docusign`status of Acme agreement` +await docusign`who hasn't signed the NDA yet?` +await docusign`envelopes sent this week` +``` -Beautiful, mobile-friendly: +### Signing Workflows ```typescript -// Customize signing experience -await sign.branding.configure({ - logo: 'https://...', - primaryColor: '#0066CC', - buttonText: 'Sign Document', - signatureOptions: ['draw', 'type', 'upload'], - languageOptions: ['en', 'es', 'fr', 'de', 'zh'], -}) +// Sequential signing +await docusign`send contract to cfo@acme.com then ceo@acme.com` + +// Parallel signing +await docusign`send NDA to all founders at once` // In-person signing -await sign.envelopes.signInPerson({ - envelope: 'ENV-001', - hostEmail: 'host@company.com', - signerName: 'John Smith', - signerEmail: 'john@acme.com', -}) - -// Embedded signing (in your app) -const signingUrl = await sign.envelopes.embeddedSigningUrl({ - envelope: 'ENV-001', - signer: 'john@acme.com', - returnUrl: 'https://yourapp.com/signed', -}) +await docusign`start in-person signing for John Smith hosted by reception` + +// Embedded in your app +const url = await docusign`signing link for john@acme.com` ``` ### Templates -Reusable agreement templates: - ```typescript -// Create template -const template = await sign.templates.create({ - name: 'Standard NDA', - documents: [ - { - name: 'Non-Disclosure Agreement', - file: ndaTemplate, - }, - ], - roles: [ - { - name: 'Disclosing Party', - fields: [ - { type: 'text', label: 'Company Name', page: 1, x: 200, y: 150 }, - { type: 'signature', page: 3, x: 100, y: 500 }, - { type: 'date', page: 3, x: 300, y: 500 }, - ], - }, - { - name: 'Receiving Party', - fields: [ - { type: 'text', label: 'Company Name', page: 1, x: 200, y: 200 }, - { type: 'signature', page: 3, x: 100, y: 600 }, - { type: 'date', page: 3, x: 300, y: 600 }, - ], - }, - ], - defaultMessage: { - subject: 'NDA for your signature', - body: 'Please review and sign the attached NDA.', - }, -}) - -// Use template -await sign.envelopes.createFromTemplate({ - template: template.id, - recipients: { - 'Disclosing Party': { name: 'Acme Corp', email: 'legal@acme.com' }, - 'Receiving Party': { name: 'Vendor Inc', email: 'contracts@vendor.com' }, - }, - prefillData: { - 'Company Name': { 'Disclosing Party': 'Acme Corporation' }, - }, -}) +// Use templates naturally +await docusign`send standard NDA to alice@acme.com` +await docusign`send employee offer to candidate using senior-engineer template` + +// Create templates from examples +await docusign`save this as our standard MSA template` + +// List available templates +await docusign`our templates` +await docusign`NDA templates` ``` ### Bulk Send -Thousands of agreements, one click: +Thousands of agreements, one line: ```typescript -// Send same agreement to many recipients -await sign.bulk.send({ - template: 'standard-nda', - recipients: [ - { name: 'John Smith', email: 'john@company1.com', data: { company: 'Company 1' } }, - { name: 'Jane Doe', email: 'jane@company2.com', data: { company: 'Company 2' } }, - { name: 'Bob Wilson', email: 'bob@company3.com', data: { company: 'Company 3' } }, - // ...thousands more - ], - options: { - batchSize: 100, - delayBetweenBatches: '1 minute', - notifyOnComplete: 'admin@company.com', - }, -}) +// Bulk send to a list +await docusign`send annual NDA renewal to all vendors` +await docusign`send policy update to all employees` + +// With tracking +await docusign`send Q1 agreements to all customers` + .track() + .notify(`Q1 agreements complete`) ``` ## Contract Lifecycle Management @@ -252,45 +144,16 @@ Full CLM, not an afterthought: All your agreements, searchable: ```typescript -import { clm } from 'docusign.do' +// Search contracts naturally +await docusign`contracts with auto-renewal` +await docusign`active contracts expiring in 90 days` +await docusign`contracts worth over $100k` +await docusign`all Acme Corp agreements` // Import existing contracts -await clm.contracts.import({ - source: 'folder', - path: '/contracts', - extract: { - parties: true, - dates: true, - values: true, - terms: true, - }, -}) - -// Search contracts -const results = await clm.contracts.search({ - query: 'auto-renewal', - filters: { - status: 'active', - expiresWithin: '90 days', - value: { min: 100000 }, - }, -}) - -// Get contract details -const contract = await clm.contracts.get('CTR-001') -// { -// id: 'CTR-001', -// title: 'Master Service Agreement', -// parties: ['Acme Corp', 'Vendor Inc'], -// effectiveDate: '2024-01-15', -// expirationDate: '2027-01-14', -// value: 500000, -// renewalType: 'auto', -// noticePeriod: 90, -// keyTerms: [...], -// documents: [...], -// amendments: [...], -// } +await docusign`import contracts from /contracts folder` + +// AI extracts parties, dates, values, key terms automatically ``` ### Contract Requests @@ -298,48 +161,18 @@ const contract = await clm.contracts.get('CTR-001') Intake and approval workflows: ```typescript -// Create contract request -await clm.requests.create({ - type: 'new-vendor', - requestor: 'user-001', - details: { - vendorName: 'New Vendor Inc', - contractType: 'SaaS Subscription', - annualValue: 50000, - department: 'Engineering', - businessJustification: 'Required for CI/CD pipeline', - }, - attachments: ['vendor-proposal.pdf'], -}) - -// Approval workflow -await clm.workflows.define({ - name: 'Vendor Contract Approval', - trigger: { type: 'new-vendor' }, - steps: [ - { - name: 'Department Review', - approvers: ['department-head'], - condition: 'all', - }, - { - name: 'Legal Review', - approvers: ['legal-team'], - condition: 'any', - required: true, - }, - { - name: 'Finance Approval', - approvers: ['finance'], - condition: 'value > 25000', - }, - { - name: 'Executive Approval', - approvers: ['cfo'], - condition: 'value > 100000', - }, - ], -}) +// Request new contracts naturally +await docusign`request vendor contract for CloudTech $50k/year for Engineering` + +// Approvals flow automatically based on value +// - Department head reviews all +// - Legal reviews all +// - Finance approves > $25k +// - CFO approves > $100k + +// Check approval status +await docusign`status of CloudTech contract request` +await docusign`pending approvals for Engineering` ``` ### Clause Library @@ -347,42 +180,16 @@ await clm.workflows.define({ Standard clauses, consistent language: ```typescript -// Define clause library -await clm.clauses.create({ - name: 'Standard Limitation of Liability', - category: 'Liability', - text: ` - IN NO EVENT SHALL EITHER PARTY BE LIABLE TO THE OTHER FOR ANY - INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES, - REGARDLESS OF THE CAUSE OF ACTION OR THE FORM OF ACTION. - - THE TOTAL LIABILITY OF EITHER PARTY SHALL NOT EXCEED [AMOUNT]. - `, - variables: [ - { name: 'AMOUNT', type: 'currency', default: 'fees paid in prior 12 months' }, - ], - alternatives: [ - { - name: 'Mutual Cap', - text: '...total liability shall not exceed the greater of $[CAP] or...', - }, - { - name: 'Unlimited for IP', - text: '...except for claims arising from intellectual property infringement...', - }, - ], - fallbackPosition: 'mutual-cap', -}) - -// Use in contract authoring -await clm.contracts.author({ - template: 'msa-template', - clauses: [ - { id: 'standard-limitation-of-liability', variables: { AMOUNT: '$1,000,000' } }, - { id: 'standard-indemnification' }, - { id: 'standard-confidentiality' }, - ], -}) +// Query your clause library +await docusign`our standard liability clause` +await docusign`indemnification alternatives` +await docusign`confidentiality clauses` + +// Author contracts from clauses +await docusign`draft MSA for CloudTech using standard clauses` + +// Compare against standards +await docusign`how does this liability cap compare to our standard?` ``` ### Obligations Tracking @@ -390,29 +197,17 @@ await clm.contracts.author({ Never miss a deadline: ```typescript -// Extract obligations from contract -await clm.obligations.extract('CTR-001') - -// Track obligations -const obligations = await clm.obligations.list({ - contract: 'CTR-001', - upcoming: '30 days', -}) -// [ -// { type: 'renewal-notice', dueDate: '2025-10-15', action: 'Send 90-day notice' }, -// { type: 'audit-rights', dueDate: '2025-06-01', action: 'Annual audit window opens' }, -// { type: 'insurance', dueDate: '2025-12-31', action: 'Provide certificate renewal' }, -// ] - -// Set up alerts -await clm.obligations.alerts({ - contract: 'CTR-001', - notifications: [ - { daysBefore: 30, recipients: ['contract-owner'] }, - { daysBefore: 7, recipients: ['contract-owner', 'legal'] }, - { daysBefore: 1, recipients: ['contract-owner', 'legal', 'executive'] }, - ], -}) +// Track upcoming obligations +await docusign`obligations due in 30 days` +await docusign`renewal notices needed this quarter` +await docusign`insurance certificates expiring` + +// AI extracts obligations from contracts automatically +await docusign`extract obligations from Acme MSA` + +// Set up alerts naturally +await docusign`alert me 30 days before any renewal deadline` +await docusign`notify legal 7 days before compliance deadlines` ``` ### Amendment Management @@ -420,230 +215,114 @@ await clm.obligations.alerts({ Track changes over time: ```typescript -// Create amendment -await clm.amendments.create({ - contract: 'CTR-001', - title: 'First Amendment - Price Increase', - effectiveDate: '2025-07-01', - changes: [ - { - section: '4.1', - original: 'Annual fee: $50,000', - amended: 'Annual fee: $55,000', - reason: 'Annual price adjustment per Section 4.3', - }, - ], - signatories: [ - { party: 'Acme Corp', signer: 'legal@acme.com' }, - { party: 'Vendor Inc', signer: 'contracts@vendor.com' }, - ], -}) - -// View contract with all amendments -const consolidatedContract = await clm.contracts.consolidated('CTR-001') +// Create amendments naturally +await docusign`amend Acme contract: increase annual fee to $55k effective July 1` + +// View full history +await docusign`Acme contract with all amendments` +await docusign`what changed in the Acme contract?` + +// Send amendment for signature +await docusign`send price increase amendment to Acme for signature` ``` ## AI-Native Agreements -### AI Contract Analysis +### Contract Analysis ```typescript -import { tom } from 'agents.do' - -// Analyze contract -await tom` - Review this vendor contract and identify: - 1. Terms that deviate from our standard positions - 2. Missing protective clauses - 3. Unusual risk allocations - 4. Auto-renewal and termination provisions - 5. Hidden fees or escalation clauses - - Provide risk score and negotiation recommendations. -` - -// Tom analyzes and returns: -// "RISK SCORE: 7/10 (Elevated) -// -// Key Issues: -// 1. Liability cap is only $50K vs our standard of fees paid -// 2. Missing mutual confidentiality - only one-way -// 3. 60-day auto-renewal notice (our standard is 30) -// 4. Includes 5% annual escalator not discussed in proposal -// 5. Change of control provision is vendor-favorable -// -// Recommended Actions: -// 1. Request mutual liability cap at 12-month fees -// 2. Add mutual confidentiality provision -// 3. Negotiate 30-day notice period -// 4. Remove or cap annual escalator -// 5. Add mutual change of control rights" +// Analyze any contract +await docusign`analyze vendor contract for risks` +await docusign`review ${contract} against our standard positions` +await docusign`risk score for Acme MSA` + +// AI identifies automatically: +// - Terms deviating from standards +// - Missing protective clauses +// - Unusual risk allocations +// - Hidden fees or escalators +// - Auto-renewal traps ``` -### AI Contract Negotiation +### Contract Negotiation ```typescript -import { sally } from 'agents.do' - -// Generate redline -await sally` - Based on the contract analysis, generate a redline that: - 1. Proposes our standard liability language - 2. Adds mutual confidentiality - 3. Adjusts notice periods - 4. Removes the escalator clause - 5. Includes brief explanatory comments - - Tone: Professional, collaborative, focused on fairness -` - -// Auto-generate negotiation email -await sally` - Draft an email to the vendor's legal team: - - Acknowledge receipt of the contract - - Express interest in moving forward - - Summarize our proposed changes (high-level) - - Offer to discuss on a call - - Tone: Positive, business-focused, not adversarial -` -``` +// Generate redlines +await docusign`redline vendor contract using our playbook` +await docusign`propose our standard liability language` -### AI Contract Generation +// Draft negotiation responses +await docusign`draft response to vendor redlines - protect our interests` +await docusign`email to vendor legal: accept their indemnity, counter on liability` -```typescript -import { ralph } from 'agents.do' - -// Generate contract from term sheet -await ralph` - Create a Master Service Agreement based on these terms: - - Vendor: CloudTech Inc - - Service: Cloud infrastructure management - - Term: 3 years - - Annual fee: $120,000 - - SLA: 99.9% uptime - - Payment terms: Net 30 - - Both parties based in Delaware - - Use our standard MSA template. - Include appropriate SaaS-specific provisions. -` - -// Ralph generates complete contract -// Using clause library and standard positions +// AI maintains professional, collaborative tone ``` -### AI Clause Comparison +### Contract Generation ```typescript -import { priya } from 'agents.do' - -// Compare clauses across contracts -await priya` - Compare the indemnification clauses in: - - CTR-001 (Microsoft) - - CTR-002 (AWS) - - CTR-003 (Google Cloud) - - Which is most favorable to us? - What's the market standard? - Recommend our go-forward position. -` +// Generate from term sheets +await docusign`draft MSA for CloudTech: 3 years, $120k/year, 99.9% SLA` + +// AI uses clause library and standard positions automatically + +// Or from natural descriptions +await docusign`create NDA for discussing potential acquisition with Acme` +await docusign`draft consulting agreement for Jane Doe $200/hr` ``` -### AI Obligation Extraction +### Clause Comparison ```typescript -import { ada } from 'docusign.do/agents' +// Compare across contracts +await docusign`compare indemnification in Microsoft vs AWS vs Google contracts` +await docusign`which vendor has the best liability terms?` +await docusign`market standard for data processing agreements` +``` -// Extract all obligations from contract -await ada` - Read CTR-001 and extract ALL obligations for both parties: +### Obligation Extraction - For each obligation: - - Who is obligated (us or them) - - What the obligation is - - When it's due (one-time or recurring) - - Consequences of non-compliance - - Related contract section +```typescript +// Extract obligations automatically +await docusign`extract obligations from Acme MSA` - Format as structured data for our obligations tracker. -` +// Returns structured data: +// - Who is obligated (us or them) +// - What the obligation is +// - When it's due +// - Consequences of non-compliance ``` ## Legal Validity E-signatures are legally binding: -### ESIGN Act Compliance (US) +### Compliance Built In -```typescript -// US federal e-signature law since 2000 -await sign.compliance.esign({ - // Consumer consent - consent: { - required: true, - withdrawable: true, - recordRetention: true, - }, - - // Record retention - retention: { - duration: '7 years', - format: 'original', - accessible: true, - }, - - // Attribution - attribution: { - identity: 'email', - timestamp: 'tamper-evident', - auditTrail: 'comprehensive', - }, -}) -``` - -### UETA Compliance +ESIGN Act (US), UETA (all states), eIDAS (EU) - all supported automatically. Consumer consent, record retention, tamper-evident timestamps, comprehensive audit trails. ```typescript -// Uniform Electronic Transactions Act (all states except NY) -await sign.compliance.ueta({ - agreement: 'electronic-records', - attribution: true, - integrity: true, -}) -``` +// Check compliance status +await docusign`compliance status for Acme agreement` +await docusign`audit trail for ENV-001` -### eIDAS Compliance (EU) - -```typescript -// European e-signature regulation -await sign.compliance.eidas({ - signatureLevel: 'advanced', // or 'simple', 'qualified' - timestamping: 'qualified-service', - validation: 'long-term', - // For qualified signatures - qualifiedSignatureDevice: 'optional', -}) +// Get certificate of completion +await docusign`certificate for Acme NDA` +// Includes: All parties, timestamps, IP addresses, signature images, document hash ``` ### Audit Trail -Every action recorded: +Every action recorded automatically: ```typescript -const auditTrail = await sign.envelopes.auditTrail('ENV-001') -// [ -// { action: 'created', timestamp: '2025-01-15T10:00:00Z', actor: 'sender@company.com', ip: '...' }, -// { action: 'sent', timestamp: '2025-01-15T10:01:00Z', recipient: 'john@acme.com' }, -// { action: 'viewed', timestamp: '2025-01-15T14:30:00Z', actor: 'john@acme.com', ip: '...', device: '...' }, -// { action: 'signed', timestamp: '2025-01-15T14:35:00Z', actor: 'john@acme.com', ip: '...', signature: '...' }, -// { action: 'completed', timestamp: '2025-01-15T14:35:00Z' }, -// ] - -// Certificate of completion -const certificate = await sign.envelopes.certificate('ENV-001') -// Includes: All parties, timestamps, IP addresses, signature images, document hash +// View audit history +await docusign`audit trail for Acme agreement` + +// Returns complete history: +// - Created, sent, viewed, signed, completed +// - Timestamps, IP addresses, devices +// - Tamper-evident, legally admissible ``` ## Architecture @@ -674,49 +353,7 @@ OrgDO (config, users, branding) ### Document Integrity -```typescript -// Every document is hashed -const document = { - content: documentBuffer, - hash: await crypto.subtle.digest('SHA-256', documentBuffer), - hashAlgorithm: 'SHA-256', - timestamp: new Date().toISOString(), -} - -// Signature binds to document hash -const signature = { - documentHash: document.hash, - signerIdentity: 'john@acme.com', - signatureImage: signatureBlob, - timestamp: signedAt, - ip: signerIP, - userAgent: signerUA, -} - -// Any tampering detectable -const isValid = await verify(document.hash, signature.documentHash) -``` - -### Long-Term Validation - -```typescript -// Signatures remain valid even after certificate expiry -await sign.validation.configure({ - timestamping: { - provider: 'rfc3161-compliant', - embed: true, - }, - archival: { - format: 'PDF/A-3', - embedSignatures: true, - embedAuditTrail: true, - }, - retention: { - duration: '10 years', - format: 'unmodified', - }, -}) -``` +Every document is SHA-256 hashed. Signatures bind to document hash. Any tampering is detectable. RFC 3161 timestamping. PDF/A-3 archival format. 10-year retention. All automatic. ## Why Open Source for E-Signatures? @@ -753,6 +390,19 @@ DocuSign charges per envelope/user. At scale: - 10,000 envelopes/year = $30K-100K - Open source = marginal cost approaches zero +## vs DocuSign + +| Feature | DocuSign | docusign.do | +|---------|----------|-------------| +| **Pricing** | $10-60/user/month | $0 - run your own | +| **Per-Envelope Fees** | Yes, beyond limits | Unlimited | +| **CLM** | Expensive add-on | Included | +| **AI** | Premium tier | AI-native from day one | +| **Templates** | Proprietary | Open formats, portable | +| **Data Location** | Their servers | Your infrastructure | +| **Customization** | Configuration UI | Code it yourself | +| **Lock-in** | Years of migration | MIT licensed | + ## Deployment ### Cloudflare Workers @@ -775,13 +425,7 @@ kubectl apply -f docusign-do-deployment.yaml ### Hybrid -```typescript -// Edge for signing, origin for CLM -await sign.config.hybrid({ - edge: ['signing', 'templates', 'branding'], - origin: ['clm', 'analytics', 'bulk-operations'], -}) -``` +Edge for signing (fast worldwide), origin for CLM and bulk operations. Configure via environment or natural language. ## Roadmap @@ -846,7 +490,12 @@ MIT License - Sign freely. ---

- docusign.do is part of the dotdo platform. + The $15B toll booth ends here.
- Website | Docs | Discord + AI-native. Legally binding. Open source. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/dynamics/README.md b/rewrites/dynamics/README.md index 8640f38a..1455874d 100644 --- a/rewrites/dynamics/README.md +++ b/rewrites/dynamics/README.md @@ -11,496 +11,279 @@ Microsoft Dynamics 365 is the only enterprise platform that unifies CRM and ERP. **dynamics.do** is the open-source alternative. One Dataverse. Every module. AI-native from day one. Deploy in minutes. -## The workers.do Way +## AI-Native API -You need a unified platform for sales, service, finance, and operations. Microsoft wants $1.2M/year for Dynamics 365. Plus consultants. Plus Power Platform add-ons. Plus Copilot licenses. +```typescript +import { dynamics } from 'dynamics.do' // Full SDK +import { dynamics } from 'dynamics.do/tiny' // Minimal client +import { dynamics } from 'dynamics.do/sales' // Sales-only operations +``` -**workers.do** gives you an AI business team that just works: +Natural language for business workflows: ```typescript import { dynamics } from 'dynamics.do' -import { sally, ada, ralph } from 'agents.do' -// Natural language CRM + ERP -const forecast = await dynamics`show pipeline forecast for ${quarter}` -const cases = await dynamics`list all open support cases for ${customer}` -const inventory = await dynamics`check stock levels for ${product} across all warehouses` -``` +// Talk to it like a colleague +const pipeline = await dynamics`deals over 100k closing this quarter` +const atRisk = await dynamics`opportunities with no activity in 30 days` +const forecast = await dynamics`pipeline forecast for Q1` -### Promise Pipelining +// Chain like sentences +await dynamics`deals at risk this month` + .notify(`Schedule a check-in with the customer`) -Chain complex business workflows with a single network round trip: - -```typescript -const deal = await dynamics`close opportunity ${opportunityId}` - .map(opp => dynamics`create sales order from ${opp}`) +// Quote to cash in one breath +await dynamics`close Acme Corp deal` + .map(deal => dynamics`create order from ${deal}`) .map(order => dynamics`generate invoice for ${order}`) .map(invoice => [cfo, controller].map(r => r`approve ${invoice}`)) ``` -### AI Agents for Every Module - -```typescript -// Sales agent handles pipeline -await sally` - Review the pipeline for Q1. - Identify deals at risk of slipping. - Create follow-up tasks for the sales team. -` - -// Finance agent handles month-end -await ada` - Run month-end close for January. - Generate variance analysis vs budget. - Flag any unusual transactions. -` +## The Problem -// Operations agent handles supply chain -await ralph` - Check inventory levels against demand forecast. - Create purchase orders for items below safety stock. - Optimize warehouse replenishment. -` -``` +Microsoft Dynamics 365 dominates enterprise CRM + ERP: -One platform. Natural language. Your entire AI business team. +| What Microsoft Charges | The Reality | +|------------------------|-------------| +| **Sales** | $65-200/user/month | +| **Customer Service** | $50-95/user/month | +| **Finance** | $180/user/month | +| **Supply Chain** | $180/user/month | +| **Human Resources** | $120/user/month | +| **Full Suite** | $500+/user/month | -## The Problem +For a 200-user company: **$1.2M/year minimum**. -Enterprise CRM and ERP systems are broken. +### The Microsoft Tax -**Microsoft Dynamics 365 costs $65-200 per user per month.** A 500-person company pays $390,000 to $1.2M annually just for licensing. Add implementation partners, Microsoft consultants, and ongoing maintenance - enterprises hemorrhage millions. +Since the AI pivot: -| Module | D365 Pricing | What You Get | -|--------|--------------|--------------| -| Sales Professional | $65/user/mo | Basic CRM | -| Sales Enterprise | $95/user/mo | Advanced CRM | -| Customer Service | $50-95/user/mo | Support desk | -| Finance | $180/user/mo | GL, AP, AR | -| Supply Chain | $180/user/mo | Inventory, procurement | -| Human Resources | $120/user/mo | HCM | -| **Full Suite** | **$500+/user/mo** | Everything | +- Copilot licenses stacked on top of everything +- Power Platform add-ons for basic workflows +- Implementation partners charging $500/hour +- Months-long deployments +- Data locked in Microsoft's ecosystem -For a 200-user company needing full CRM + ERP: **$1.2M/year minimum**. +### The Interoperability Illusion -But the cost is only half the problem: +Dynamics talks OData. But: -- **Vendor lock-in** - Your business data trapped in Microsoft's ecosystem -- **Deployment complexity** - Months of implementation, armies of consultants -- **AI as afterthought** - Copilot bolted on, not built in -- **Licensing labyrinth** - Sales Professional vs. Enterprise vs. Premium vs. add-ons -- **Geographic restrictions** - Data residency nightmares, compliance theater +- Proprietary entity relationships underneath +- Custom APIs for critical workflows +- FetchXML for anything complex +- Power Platform for basic automation +- Your data, their rules -Meanwhile, the platform itself? It's Dataverse - CRUD operations over entities with business rules. We can build this. +## The Solution ---- +**dynamics.do** reimagines CRM + ERP for the AI era: -## The Solution +``` +Microsoft Dynamics 365 dynamics.do +----------------------------------------------------------------- +$1.2M/year (200 users) Deploy in minutes +$500/hour consultants Code it yourself +Months to customize Instant deploy +Copilot as add-on AI is the foundation +Microsoft data centers Your Cloudflare account +OData/FetchXML complexity Natural language +Vendor lock-in Open source, MIT licensed +``` -**dynamics.do** is an open-source, edge-native Dataverse alternative with AI at the core. +## One-Click Deploy ```bash npx create-dotdo dynamics ``` -One command. Your own Dynamics 365. Your data. Your rules. +A complete CRM + ERP. Running on infrastructure you control. Natural language from day one. -| Dynamics 365 | dynamics.do | -|--------------|-------------| -| $65-200+/user/month | **Free** (open source) | -| Months to deploy | **Minutes** | -| Microsoft data centers | **Your Cloudflare account** | -| AI is an add-on | **AI is the foundation** | -| Complex licensing | **MIT License** | -| Vendor lock-in | **OData v4 compatible** | +```typescript +import { Dynamics } from 'dynamics.do' ---- +export default Dynamics({ + name: 'acme-corp', + domain: 'erp.acme.com', + modules: ['sales', 'service', 'finance', 'supply-chain', 'hr'], +}) +``` ## Features ### Sales (CRM) -Full sales automation out of the box: - ```typescript -import { dynamics } from 'dynamics.do' - -// Create a lead -const lead = await dynamics.leads.create({ - firstname: 'John', - lastname: 'Smith', - companyname: 'Acme Corp', - emailaddress1: 'john@acme.com', - leadsourcecode: 'web' -}) - -// Qualify lead to opportunity -const { account, contact, opportunity } = await dynamics.leads.qualify(lead.id, { - createAccount: true, - createContact: true, - createOpportunity: true -}) - -// Manage opportunity through pipeline -await dynamics.opportunities.update(opportunity.id, { - estimatedvalue: 150000, - estimatedclosedate: '2025-03-31', - salesstage: 'propose' -}) +// Find anyone +const acme = await dynamics`Acme Corp` +const enterprise = await dynamics`accounts over 1M revenue` +const stale = await dynamics`opportunities no activity in 30 days` + +// AI infers what you need +await dynamics`Acme Corp` // returns account +await dynamics`contacts at Acme Corp` // returns contacts +await dynamics`Acme Corp pipeline` // returns opportunities ``` -**Included:** -- **Accounts & Contacts** - Company and person records with relationships -- **Leads** - Qualification workflows, lead scoring, conversion -- **Opportunities** - Pipeline management, probability forecasting -- **Quotes & Orders** - Product catalog, pricing rules, line items -- **Activities** - Tasks, emails, phone calls, appointments -- **Goals** - Sales quotas and KPI tracking - -### Customer Service - -Omnichannel support desk: +### Lead Management ```typescript -// Create a case -const incident = await dynamics.incidents.create({ - title: 'Product not working as expected', - customerid: 'CONTACT-001', - caseorigincode: 'email', - prioritycode: 'high', - entitlementid: 'ENT-001' // Links to service contract -}) +// Leads are one line +await dynamics`new lead John Smith from Acme Corp via web` -// Track SLA -const sla = await dynamics.incidents.getSlaStatus(incident.id) -// { firstResponse: { due: '2025-01-15T10:00:00Z', status: 'in-progress' } } +// Qualify naturally +await dynamics`qualify John Smith as opportunity` + .map(opp => dynamics`estimate ${opp} at 150k closing March`) -// Route to queue -await dynamics.incidents.route(incident.id, { queue: 'tier-2-support' }) - -// Resolve -await dynamics.incidents.resolve(incident.id, { - resolution: 'Provided configuration update', - billabletime: 30 -}) +// Bulk qualification just works +await dynamics`leads from last week with budget confirmed` + .qualify() ``` -**Included:** -- **Cases** - Incident management with SLA tracking -- **Queues** - Routing rules, assignment logic -- **Knowledge Base** - Article management with AI search -- **Entitlements** - Service contracts and support terms -- **Omnichannel** - Chat, email, voice unified routing - -### Finance - -Enterprise-grade financial management: +### Opportunities ```typescript -// General Ledger -await dynamics.finance.journalEntry({ - journalType: 'daily', - date: '2025-01-15', - lines: [ - { account: '6100-00', debit: 5000, dimension: { department: 'Engineering' } }, - { account: '2100-00', credit: 5000 } - ] -}) +// Pipeline at a glance +await dynamics`deals over 100k closing this quarter` +await dynamics`opportunities at risk` +await dynamics`stalled deals no activity 30 days` + +// Update naturally +await dynamics`move Acme deal to proposal stage` +await dynamics`Acme deal now 150k closing end of March` +``` -// Accounts Receivable -const invoice = await dynamics.finance.customerInvoice({ - customer: 'CUST-001', - lines: [ - { item: 'SERV-001', quantity: 10, unitPrice: 500 } - ], - paymentTerms: 'Net30' -}) +### Customer Service -// Apply payment -await dynamics.finance.customerPayment({ - customer: 'CUST-001', - amount: 5000, - invoices: [{ invoice: invoice.id, amount: 5000 }] -}) +```typescript +// Cases are one line +await dynamics`new case for Acme Corp product not working high priority` -// Accounts Payable -const vendorInvoice = await dynamics.finance.vendorInvoice({ - vendor: 'VEND-001', - invoiceNumber: 'INV-12345', - lines: [ - { account: '5100-00', amount: 10000, description: 'Raw materials' } - ] -}) +// Track naturally +await dynamics`open cases for Acme` +await dynamics`escalated cases this week` +await dynamics`cases breaching SLA` -// Payment proposal -const proposal = await dynamics.finance.paymentProposal({ - vendors: 'all', - dueBy: '2025-01-31', - method: 'ACH' -}) +// Route and resolve +await dynamics`route case 12345 to tier 2 support` +await dynamics`resolve case 12345 provided config update` ``` -**Included:** -- **General Ledger** - Chart of accounts, journal entries, dimensions -- **Accounts Receivable** - Customer invoicing, payments, aging -- **Accounts Payable** - Vendor bills, payment processing -- **Cash Management** - Bank accounts, reconciliation -- **Fixed Assets** - Asset tracking, depreciation -- **Budgeting** - Budget creation, variance analysis -- **Financial Reporting** - Statements, consolidation - -### Supply Chain Management - -End-to-end supply chain operations: +### Finance ```typescript -// Inventory Management -await dynamics.inventory.adjust({ - item: 'WIDGET-001', - warehouse: 'WH-01', - quantity: 100, - reason: 'physical-count' -}) - -// Check availability across locations -const availability = await dynamics.inventory.availability('WIDGET-001') -// { total: 500, byWarehouse: { 'WH-01': 300, 'WH-02': 200 }, reserved: 50, available: 450 } - -// Procurement -const po = await dynamics.procurement.purchaseOrder({ - vendor: 'VEND-001', - lines: [ - { item: 'RAW-001', quantity: 1000, unitPrice: 5.00 } - ], - requestedDate: '2025-02-01' -}) - -// Receive against PO -await dynamics.procurement.productReceipt({ - purchaseOrder: po.id, - lines: [{ item: 'RAW-001', quantity: 1000 }] -}) +// Just say it +await dynamics`journal entry 5000 from engineering to accounts payable` +await dynamics`invoice Acme Corp 10 consulting hours at 500` +await dynamics`record payment from Acme Corp 5000` + +// Payables just work +await dynamics`vendor invoice from Contoso 10000 for raw materials` +await dynamics`payment proposal for all vendors due this month via ACH` + +// Month-end in one breath +await dynamics`run month-end close for January` + .map(close => dynamics`generate variance analysis vs budget`) + .map(analysis => [cfo, controller].map(r => r`review ${analysis}`)) +``` -// Sales Order fulfillment -const salesOrder = await dynamics.sales.salesOrder({ - customer: 'CUST-001', - lines: [ - { item: 'WIDGET-001', quantity: 50 } - ] -}) +### Supply Chain -// Create shipment -await dynamics.warehouse.createShipment({ - salesOrder: salesOrder.id, - warehouse: 'WH-01' -}) +```typescript +// Inventory at a glance +await dynamics`stock levels for Widget-001 all warehouses` +await dynamics`items below reorder point` +await dynamics`inventory value by warehouse` + +// Procurement naturally +await dynamics`purchase order to Contoso for 1000 raw materials at 5 each` +await dynamics`receive 1000 raw materials against PO-001` + +// Order fulfillment +await dynamics`sales order for Acme 50 widgets` + .map(order => dynamics`ship ${order} from warehouse 1`) + .map(shipment => dynamics`invoice ${shipment}`) ``` -**Included:** -- **Inventory Management** - Stock levels, movements, valuation -- **Warehouse Management** - Locations, picking, packing -- **Procurement** - Purchase orders, vendor management -- **Sales Order Processing** - Order entry, fulfillment -- **Transportation** - Shipment management, carrier integration -- **Master Planning** - MRP, demand forecasting - ### Project Operations -Project-based businesses: - ```typescript -// Create project -const project = await dynamics.projects.create({ - name: 'Website Redesign', - customer: 'CUST-001', - type: 'time-and-material', - startDate: '2025-02-01', - endDate: '2025-04-30', - budget: 75000 -}) +// Projects are one line +await dynamics`new project Website Redesign for Acme 75k budget` -// Add work breakdown structure -await dynamics.projects.addTasks(project.id, [ - { name: 'Discovery', hours: 40, assignee: 'USR-001' }, - { name: 'Design', hours: 80, assignee: 'USR-002', predecessor: 'Discovery' }, - { name: 'Development', hours: 200, assignee: 'USR-003', predecessor: 'Design' } -]) - -// Record time -await dynamics.projects.timeEntry({ - project: project.id, - task: 'Discovery', - hours: 8, - date: '2025-02-03', - description: 'Requirements gathering' -}) +// Tasks naturally +await dynamics`add discovery phase 40 hours to Website Redesign` +await dynamics`log 8 hours on discovery requirements gathering today` -// Generate invoice -await dynamics.projects.invoice(project.id, { - throughDate: '2025-02-28', - includeExpenses: true -}) +// Billing just works +await dynamics`invoice Website Redesign through February including expenses` ``` -**Included:** -- **Project Planning** - WBS, scheduling, resource allocation -- **Time & Expense** - Entry, approval workflows -- **Project Accounting** - Budgets, actuals, recognition -- **Resource Management** - Skills, availability, utilization -- **Billing** - Time-and-material, fixed-price, milestones - ### Human Resources -Complete HCM solution: - ```typescript -// Employee lifecycle -const employee = await dynamics.hr.hire({ - name: 'Alice Johnson', - position: 'Software Engineer', - department: 'Engineering', - startDate: '2025-02-15', - compensation: { salary: 120000, currency: 'USD' } -}) +// Hiring is one line +await dynamics`hire Alice Johnson as Software Engineer in Engineering starting Feb 15 at 120k` -// Benefits enrollment -await dynamics.hr.enrollBenefits(employee.id, { - plans: ['medical-hmo', 'dental', '401k'], - coverage: { medical: 'employee+family' } -}) +// Benefits naturally +await dynamics`enroll Alice in medical HMO dental and 401k family coverage` -// Leave management -await dynamics.hr.requestLeave({ - employee: employee.id, - type: 'vacation', - startDate: '2025-03-10', - endDate: '2025-03-14' -}) +// Time off just works +await dynamics`Alice requesting vacation March 10 through 14` +await dynamics`pending leave requests for Engineering` -// Performance review -await dynamics.hr.performanceReview({ - employee: employee.id, - period: '2024', - rating: 4, - goals: [ - { objective: 'Ship authentication system', result: 'exceeded' } - ] -}) +// Reviews naturally +await dynamics`performance review for Alice 2024 rating 4 exceeded goals` ``` -**Included:** -- **Core HR** - Employee records, org structure -- **Recruiting** - Job postings, applications, hiring -- **Onboarding** - Task workflows, document collection -- **Benefits** - Enrollment, life events -- **Leave Management** - Accruals, requests, approvals -- **Performance** - Goals, reviews, feedback -- **Compensation** - Pay structures, adjustments - ---- - ## OData v4 Compatible -Standard OData - existing Power Platform integrations work. - -```bash -# Query accounts with related contacts -GET /api/data/v9.2/accounts?$select=name,revenue&$expand=contact_customer_accounts($select=fullname,emailaddress1)&$filter=revenue gt 1000000&$orderby=revenue desc&$top=10 - -# Create a sales order -POST /api/data/v9.2/salesorders -Content-Type: application/json -{ - "customerid_account@odata.bind": "/accounts(12345)", - "name": "SO-2025-001" -} - -# Execute action -POST /api/data/v9.2/leads(67890)/Microsoft.Dynamics.CRM.QualifyLead -Content-Type: application/json -{ - "CreateAccount": true, - "CreateContact": true, - "CreateOpportunity": true -} -``` - -Every OData operation you know: - -| Operation | Example | -|-----------|---------| -| `$select` | Return specific fields | -| `$filter` | Query conditions with operators | -| `$expand` | Include related entities | -| `$orderby` | Sort results | -| `$top` / `$skip` | Pagination | -| `$count` | Include total count | -| `$search` | Full-text search | -| `$apply` | Aggregations and grouping | - -### FetchXML Support - ```typescript -const response = await dynamics.query({ - fetchXml: ` - - - - - - - - - - - ` -}) -``` - ---- - -## AI-Native +// Same natural syntax, OData underneath +await dynamics`Acme Corp contacts` // returns Contact resources +await dynamics`invoices past due` // returns Invoice resources +await dynamics`everything for Acme 2024` // returns full data bundle -dynamics.do is built for AI agents, not retrofitted. - -### Natural Language Everything +// Bulk export for analytics +await dynamics`export all accounts since January` +``` -```typescript -import { dynamics } from 'dynamics.do' +### OData Resources Supported -// Sales queries -const deals = await dynamics.ask('Show me all opportunities closing this quarter over $50k') +| Category | Resources | +|----------|-----------| +| **Sales** | Account, Contact, Lead, Opportunity, Quote, SalesOrder, Product, PriceLevel | +| **Service** | Incident, Queue, KnowledgeArticle, Entitlement | +| **Finance** | JournalEntry, CustomerInvoice, VendorInvoice, Payment, FixedAsset | +| **Supply Chain** | Warehouse, InventoryAdjustment, PurchaseOrder, ProductReceipt | +| **HR** | Employee, Position, LeaveRequest, PerformanceReview | -// Finance queries -const aging = await dynamics.ask('What invoices are past due?') +### Power Platform Compatibility -// Supply chain -const stock = await dynamics.ask('Which items are below reorder point?') +Power Apps, Power Automate, Power BI - all work. Connect via OData feed and go. -// HR queries -const headcount = await dynamics.ask('How many engineers have we hired this year?') -``` +## AI-Native Business Operations ### AI Agents for Every Module ```typescript -import { sally, ada, ralph } from 'dynamics.do/agents' +import { sally, ada, ralph } from 'agents.do' -// Sales agent +// Sales agent handles pipeline await sally` Review the pipeline for Q1. Identify deals at risk of slipping. Create follow-up tasks for the sales team. ` -// Finance agent +// Finance agent handles month-end await ada` Run month-end close for January. Generate variance analysis vs budget. Flag any unusual transactions. ` -// Operations agent +// Operations agent handles supply chain await ralph` Check inventory levels against demand forecast. Create purchase orders for items below safety stock. @@ -508,25 +291,20 @@ await ralph` ` ``` -### AI-Powered Automation - -**Sales Intelligence:** -- Predictive lead scoring -- Opportunity risk analysis -- Next best action recommendations -- Email sentiment detection +### Intelligent Automation -**Finance Intelligence:** -- Anomaly detection in transactions -- Cash flow forecasting -- Automated bank reconciliation -- Invoice matching - -**Supply Chain Intelligence:** -- Demand forecasting -- Inventory optimization -- Supplier risk scoring -- Delivery prediction +```typescript +// AI suggests, you approve +await dynamics`what should we follow up on this week?` + .review() // shows recommendations + .approve() // you confirm + +// Proactive alerts just work +// - Deals going dark +// - Inventory running low +// - Invoices going overdue +// - SLAs about to breach +``` ### MCP Server @@ -537,462 +315,233 @@ Every entity, every action - exposed as AI tools: "mcpServers": { "dynamics": { "command": "npx", - "args": ["dynamics.do-mcp"], - "env": { - "DYNAMICS_URL": "https://your-dynamics.workers.dev", - "DYNAMICS_TOKEN": "your-token" - } + "args": ["dynamics.do-mcp"] } } } ``` -Available tools: - -| Tool | Description | -|------|-------------| -| `dynamics_query` | Query any entity with natural language or OData | -| `dynamics_create` | Create new records | -| `dynamics_update` | Update existing records | -| `dynamics_delete` | Delete records | -| `dynamics_execute` | Run actions and functions | -| `dynamics_metadata` | Explore entity definitions | -| `dynamics_associate` | Create relationships between records | -| `dynamics_workflow` | Trigger business process flows | -| `dynamics_finance` | Financial operations | -| `dynamics_inventory` | Inventory operations | - ---- - ## Architecture +### Durable Object per Business Unit + ``` - dynamics.do - | - +-----------+-------------+-------------+-----------+ - | | | | | - OData v4 FetchXML MCP Server WebSocket RPC - | | | | | - +-----------+-------------+-------------+-----------+ - | - Query Translator - | - +-------------------+-------------------+ - | | - Entity Router Security Layer - | | - +-------------------+-------------------+ - | - +---------------+---------------+ - | | | - +----------+ +----------+ +----------+ - | Sales | | Finance | | Supply | - | DO | | DO | | Chain DO | - +----------+ +----------+ +----------+ - | | | - +-------+-------+-------+-------+-------+ - | | | | | | - SQLite SQLite SQLite SQLite SQLite R2 - (CRM) (GL) (AP/AR) (Inv) (HR) (Docs) +BusinessUnitDO (config, users, roles) + | + +-- SalesDO (accounts, contacts, leads, opportunities) + | |-- SQLite: CRM records + | +-- R2: Documents, attachments + | + +-- ServiceDO (cases, queues, knowledge) + | |-- SQLite: Support data + | +-- R2: Case attachments + | + +-- FinanceDO (GL, AP, AR, fixed assets) + | |-- SQLite: Financial data + | +-- R2: Invoices, statements + | + +-- SupplyChainDO (inventory, procurement, shipping) + | |-- SQLite: Operations data + | + +-- HRDO (employees, positions, reviews) + |-- SQLite: HR data + +-- R2: Employee documents ``` -### Durable Objects Per Module - -Each business domain runs in its own Durable Object: +### Storage Tiers -| Durable Object | Module | Entities | -|----------------|--------|----------| -| `SalesDO` | Sales | Account, Contact, Lead, Opportunity, Quote, Order | -| `ServiceDO` | Customer Service | Case, Queue, Knowledge, Entitlement | -| `FinanceDO` | Finance | Journal, Customer Invoice, Vendor Invoice, Payment | -| `InventoryDO` | Supply Chain | Item, Warehouse, Stock, Movement | -| `ProcurementDO` | Procurement | Vendor, PO, Receipt | -| `ProjectDO` | Project Ops | Project, Task, Time, Expense | -| `HRDO` | Human Resources | Employee, Position, Leave, Performance | +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active records, recent transactions | <10ms | +| **Warm** | R2 + SQLite Index | Historical data (2-7 years) | <100ms | +| **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | ### Cross-Module Integration The magic of unified CRM+ERP - transactions flow automatically: ```typescript -// When a sales order is confirmed: -// 1. SalesDO creates the order -// 2. InventoryDO reserves stock -// 3. FinanceDO creates revenue recognition schedule -// 4. ProjectDO creates project (if project-based) - -const salesOrder = await dynamics.sales.confirm('SO-001') -// All of the above happens transactionally +// Quote to cash in one breath +await dynamics`close Acme Corp deal` + .map(deal => dynamics`create order from ${deal}`) // SalesDO + .map(order => dynamics`reserve inventory for ${order}`) // SupplyChainDO + .map(order => dynamics`generate invoice for ${order}`) // FinanceDO + .map(invoice => [cfo, controller].map(r => r`approve ${invoice}`)) ``` -### Tiered Storage +## vs Microsoft Dynamics 365 -| Tier | Storage | Use Case | -|------|---------|----------| -| Hot | SQLite in DO | Active records, recent transactions | -| Warm | R2 | Historical data, closed periods | -| Cold | R2 + Parquet | Analytics, compliance retention | +| Feature | Microsoft Dynamics 365 | dynamics.do | +|---------|----------------------|-----------| +| **Implementation** | $10M+ (enterprise) | Deploy in minutes | +| **Annual Cost** | $1.2M+ (200 users) | ~$100/month | +| **Architecture** | Azure monolith | Edge-native, global | +| **API** | OData + FetchXML | Natural language | +| **AI** | Copilot (add-on) | AI-first design | +| **Data Location** | Microsoft data centers | Your Cloudflare account | +| **Customization** | $500/hour consultants | Code it yourself | +| **Interoperability** | Power Platform lock-in | Open by default | +| **Updates** | Quarterly releases | Continuous deployment | +| **Lock-in** | Decades of migration | MIT licensed | ---- - -## Built-in Entities - -### Sales & Marketing - -| Entity | Description | -|--------|-------------| -| `account` | Companies and organizations | -| `contact` | Individual people | -| `lead` | Potential customers | -| `opportunity` | Sales opportunities | -| `quote` | Price proposals | -| `salesorder` | Confirmed orders | -| `product` | Product catalog | -| `pricelevel` | Price lists | -| `campaign` | Marketing campaigns | - -### Customer Service - -| Entity | Description | -|--------|-------------| -| `incident` | Service cases | -| `queue` | Work queues | -| `knowledgearticle` | Knowledge base | -| `entitlement` | Service contracts | - -### Finance - -| Entity | Description | -|--------|-------------| -| `msdyn_journalentry` | General ledger entries | -| `invoice` | Customer invoices | -| `msdyn_vendorinvoice` | Vendor bills | -| `msdyn_payment` | Payment records | -| `msdyn_fixedasset` | Fixed assets | - -### Supply Chain - -| Entity | Description | -|--------|-------------| -| `msdyn_warehouse` | Storage locations | -| `msdyn_inventoryadjustment` | Stock adjustments | -| `msdyn_purchaseorder` | Purchase orders | -| `msdyn_productreceipt` | Goods receipt | +## Use Cases -### Human Resources - -| Entity | Description | -|--------|-------------| -| `msdyn_employee` | Workers | -| `msdyn_position` | Job positions | -| `msdyn_leaverequest` | Time off requests | -| `msdyn_performancereview` | Reviews | - ---- - -## Business Process Flows - -Visual stage-gate workflows for any entity: +### Sales Operations ```typescript -await dynamics.workflows.createBusinessProcess({ - name: 'Lead to Opportunity', - entity: 'lead', - stages: [ - { - name: 'Qualify', - steps: [ - { attribute: 'companyname', required: true }, - { attribute: 'emailaddress1', required: true }, - { attribute: 'telephone1', required: false } - ] - }, - { - name: 'Develop', - steps: [ - { attribute: 'budgetamount', required: true }, - { attribute: 'decisionmaker', required: true } - ] - }, - { - name: 'Propose', - steps: [ - { attribute: 'qualifyingopportunityid', required: true } - ] - } - ] -}) -``` - -### Finance Workflows +// Weekly pipeline review +await dynamics`pipeline review this week` + .each(deal => deal.update().follow_up()) -```typescript -await dynamics.workflows.createApproval({ - name: 'Vendor Invoice Approval', - entity: 'msdyn_vendorinvoice', - conditions: [ - { when: 'amount > 10000', require: 'manager' }, - { when: 'amount > 50000', require: 'director' }, - { when: 'amount > 100000', require: 'cfo' } - ] -}) +// Quarterly forecast +await dynamics`forecast Q1 by region` ``` ---- - -## Security Model - -Row-level security matching Dynamics 365: +### Financial Close ```typescript -// Create a security role -await dynamics.security.createRole({ - name: 'Sales Representative', - privileges: [ - { entity: 'account', create: 'User', read: 'BusinessUnit', write: 'User', delete: 'None' }, - { entity: 'opportunity', create: 'User', read: 'BusinessUnit', write: 'User', delete: 'User' }, - { entity: 'lead', create: 'User', read: 'BusinessUnit', write: 'User', delete: 'User' } - ] -}) - -// Finance role with sensitive access -await dynamics.security.createRole({ - name: 'Accounts Payable Clerk', - privileges: [ - { entity: 'msdyn_vendorinvoice', create: 'BusinessUnit', read: 'BusinessUnit', write: 'BusinessUnit' }, - { entity: 'msdyn_payment', create: 'None', read: 'BusinessUnit', write: 'None' } // Can't create payments - ] -}) -``` - -Access levels: -- **None** - No access -- **User** - Own records only -- **BusinessUnit** - Records in user's business unit -- **ParentChild** - Parent and child business units -- **Organization** - All records - ---- - -## Power Platform Compatibility - -dynamics.do exposes the same APIs Power Platform expects: - -### Power Apps - -```javascript -// In Power Apps, your dynamics.do instance works like Dataverse -ClearCollect(Accounts, Filter(accounts, revenue > 1000000)) +// Month-end close +await dynamics`close January` + .map(period => dynamics`reconcile bank accounts`) + .map(period => dynamics`generate financial statements`) + .map(statements => [cfo, auditor].map(r => r`review ${statements}`)) ``` -### Power Automate +### Procurement Automation -```yaml -trigger: - type: dataverse - entity: opportunity - event: create - -actions: - - sendEmail: - to: sales-team@company.com - subject: "New Opportunity: {triggerOutputs()?['body/name']}" -``` - -### Power BI - -Connect via OData feed: -``` -https://your-dynamics.workers.dev/api/data/v9.2/ +```typescript +// Reorder workflow +await dynamics`items below reorder point` + .map(item => dynamics`create purchase order for ${item}`) + .map(po => dynamics`send ${po} to vendor`) ``` ---- +## Deployment Options -## Quick Start - -### One-Click Deploy +### Cloudflare Workers ```bash npx create-dotdo dynamics - -# Follow prompts: -# - Organization name -# - Modules to enable (Sales, Service, Finance, Supply Chain, HR) -# - Base currency -# - Fiscal year ``` -This creates a new Cloudflare Worker with: -- Full dynamics.do deployment -- SQLite database per module -- Sample data for selected modules -- MCP server endpoint enabled - -### Manual Setup +### Private Cloud ```bash -git clone https://github.com/dot-do/dynamics.git -cd dynamics -npm install -npm run deploy +docker run -p 8787:8787 dotdo/dynamics ``` -### Connect - -```typescript -import { DynamicsClient } from 'dynamics.do' - -const client = new DynamicsClient({ - url: 'https://your-dynamics.workers.dev', - token: 'your-token' -}) - -// Create an account -const account = await client.accounts.create({ - name: 'Contoso Ltd', - revenue: 5000000, - industrycode: 1 -}) - -// Create a customer invoice (Finance) -const invoice = await client.finance.customerInvoice({ - customer: account.id, - lines: [{ item: 'SERV-001', amount: 10000 }] -}) -``` - ---- - -## Migration from Dynamics 365 - -### Export from Dynamics +### On-Premises ```bash -# Use Dynamics 365 Data Export Service or XrmToolBox -dynamics365-export --environment contoso.crm.dynamics.com --entities account,contact,opportunity --output ./export +./dynamics-do-install.sh --on-premises --modules=all ``` -### Import to dynamics.do +## Why Open Source for CRM + ERP? -```bash -npx dynamics.do import ./export --url https://your-dynamics.workers.dev - -# Migrates: -# - All entity data -# - Relationships -# - Custom entities -# - Business process flows -# - Security roles -``` +### 1. True Interoperability -Supported formats: -- Dynamics 365 Data Export JSON -- CSV with relationship mapping -- XrmToolBox export format -- Solution package (.zip) +Microsoft talks interoperability while maintaining Power Platform lock-in. Open source means: +- OData native, not OData-bolted-on +- No vendor lock-in incentives +- Community-driven standards adoption +- Real data portability ---- +### 2. Innovation Velocity -## Comparison +Enterprise software moves slowly because vendors profit from the status quo. Open source enables: +- Business users to influence development directly +- Analysts to build on operational data +- Startups to integrate without vendor approval +- Enterprises to customize for their needs -| Feature | Dynamics 365 | Salesforce | SAP | dynamics.do | -|---------|--------------|------------|-----|-------------| -| Pricing | $65-200/user/mo | $25-330/user/mo | $95-500/user/mo | **Free** | -| CRM | Yes | Yes | Limited | **Yes** | -| ERP | Yes | No | Yes | **Yes** | -| Unified Platform | Yes | No | No | **Yes** | -| Deployment | Months | Weeks | Months | **Minutes** | -| AI | Copilot (add-on) | Einstein (add-on) | SAP AI (add-on) | **Built-in** | -| Open Source | No | No | No | **Yes (MIT)** | +### 3. Cost Liberation -### Cost Comparison +$1M+ implementations are business dollars diverted from growth. Open source means: +- Minutes to deploy, not months +- No implementation consultants +- No per-user licensing +- No vendor lock-in -**200-user company needing CRM + ERP:** +### 4. AI Enablement -| | Dynamics 365 | dynamics.do | -|-|--------------|-------------| -| Sales (100 users) | $79,200/yr | $0 | -| Service (50 users) | $47,500/yr | $0 | -| Finance (30 users) | $64,800/yr | $0 | -| Supply Chain (20 users) | $43,200/yr | $0 | -| **Annual Total** | **$234,700** | **$5** (Workers) | -| **5-Year TCO** | **$1,500,000+** | **$300** | - ---- +Closed CRM/ERP controls what AI you can use. Open source means: +- Integrate any LLM +- Build custom business intelligence +- Natural language everything +- Train models on your data (with governance) ## Roadmap -### Now -- [x] Core CRUD operations -- [x] OData v4 query support -- [x] Sales entities -- [x] Customer Service entities -- [x] MCP server for AI - -### Next -- [ ] Finance module (GL, AP, AR) -- [ ] Supply Chain (Inventory, Procurement) -- [ ] Business Process Flows -- [ ] Security roles and row-level access -- [ ] FetchXML parser - -### Later -- [ ] Project Operations -- [ ] Human Resources -- [ ] Plugins (pre/post operation) -- [ ] Real-time workflows -- [ ] Solution packaging -- [ ] Multi-currency -- [ ] Consolidation - ---- +### Core CRM +- [x] Accounts & Contacts +- [x] Leads & Opportunities +- [x] Quotes & Orders +- [x] Activities +- [ ] Campaigns +- [ ] Goals & Forecasting -## Documentation +### Customer Service +- [x] Cases +- [x] Queues +- [x] Knowledge Base +- [ ] Entitlements & SLAs +- [ ] Omnichannel Routing -| Guide | Description | -|-------|-------------| -| [Quick Start](./docs/quickstart.mdx) | Deploy in 5 minutes | -| [Sales Module](./docs/sales.mdx) | CRM functionality | -| [Finance Module](./docs/finance.mdx) | Financial management | -| [Supply Chain](./docs/supply-chain.mdx) | Inventory and procurement | -| [AI Features](./docs/ai.mdx) | Natural language, agents | -| [OData API](./docs/odata.mdx) | API reference | -| [Security](./docs/security.mdx) | Roles and permissions | -| [Migration](./docs/migration.mdx) | Moving from Dynamics 365 | +### Finance +- [x] General Ledger +- [x] Accounts Receivable +- [x] Accounts Payable +- [ ] Fixed Assets +- [ ] Cash Management +- [ ] Budgeting ---- +### Supply Chain +- [x] Inventory Management +- [x] Procurement +- [x] Sales Orders +- [ ] Warehouse Management +- [ ] Transportation +- [ ] Master Planning + +### HR +- [x] Employee Records +- [x] Leave Management +- [ ] Benefits +- [ ] Performance +- [ ] Recruiting +- [ ] Onboarding ## Contributing -Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md). +dynamics.do is open source under the MIT license. -### Development +We especially welcome contributions from: +- Business analysts +- Finance professionals +- Supply chain experts +- HR specialists +- CRM/ERP consultants ```bash -git clone https://github.com/dot-do/dynamics.git -cd dynamics -npm install -npm test -npm run dev +git clone https://github.com/dotdo/dynamics.do +cd dynamics.do +pnpm install +pnpm test ``` -### Key Areas - -- Finance module implementation -- Supply Chain entities -- OData query translation -- Business Process Flow engine -- Security model - ---- - ## License -MIT - see [LICENSE](LICENSE) +MIT License - For business operations everywhere. ---

- The complete business platform, liberated.
- CRM + ERP. Built on Cloudflare Workers. Designed for AI. Open to everyone. + The $1.2M/year CRM + ERP, liberated. +
+ Natural language. AI-first. Open source. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/fivetran/README.md b/rewrites/fivetran/README.md index fb7bb820..00b2c27b 100644 --- a/rewrites/fivetran/README.md +++ b/rewrites/fivetran/README.md @@ -1,171 +1,308 @@ # fivetran.do -Data pipelines that speak human. Stop configuring. Start commanding. +> Data Pipelines. Natural Language. Self-Healing. AI-First. -## The Hero +Fivetran charges $50k+/year for data pipelines. New connectors take 3-5 days. Something breaks at 2am and you're debugging a black box. You're locked into per-row pricing that scales against you. -Your data team is drowning. You're paying Fivetran $50k+/year for something that should be simple. Every new connector takes 3-5 days of back-and-forth with support. Something breaks at 2am and you're debugging a black box. You're locked into their ecosystem with no exit strategy. +**fivetran.do** is the open-source alternative. 500+ connectors. Natural language control. AI that fixes itself while you sleep. -You just want to say: *"sync Salesforce to our warehouse every 15 minutes"* and have it work. +## AI-Native API -## The Vision +```typescript +import { fivetran } from 'fivetran.do' // Full SDK +import { fivetran } from 'fivetran.do/tiny' // Minimal client +import { fivetran } from 'fivetran.do/sync' // Sync-only operations +``` -Data pipelines you can talk to. +Natural language for data pipelines: ```typescript -import { fivetran } from '@dotdo/fivetran' +import { fivetran } from 'fivetran.do' + +// Talk to it like a data engineer +const salesforce = await fivetran`sync salesforce to snowflake` +const payments = await fivetran`stripe to data lake hourly` +const broken = await fivetran`what's broken?` + +// Chain like sentences +await fivetran`stale connectors` + .refresh() + .verify() + +// Self-healing pipelines +await fivetran`sync salesforce` + .every('15 minutes') + .onError(e => fivetran`diagnose and fix ${e}`) +``` + +## The Problem + +Fivetran dominates data integration with per-row pricing that scales against you: + +| What Fivetran Charges | The Reality | +|-----------------------|-------------| +| **Monthly Active Rows** | $0.50-5.00 per 1M rows | +| **Connector Setup** | 3-5 business days | +| **Enterprise Tier** | $50,000+/year | +| **Schema Changes** | Manual migration dance | +| **Debugging** | Black box logs | +| **Vendor Lock-in** | Proprietary everything | + +### The Fivetran Tax + +Your data grows. Your bill grows faster: +- 10M rows: $500/month +- 100M rows: $5,000/month +- 1B rows: $50,000/month + +Every successful sync costs you more. Growth is punished. + +### The Real Problem + +Data engineers spend their time: +- Configuring dropdowns instead of building +- Debugging black boxes at 2am +- Waiting for connector support tickets +- Manually handling schema changes + +## The Solution + +**fivetran.do** reimagines data pipelines for the AI era: -fivetran`sync salesforce every 15 minutes to d1` -fivetran`connect stripe payments to data lake` -fivetran`what's broken?` // Returns status of failing connectors -fivetran`fix the broken connectors` // AI handles it +``` +Fivetran fivetran.do +----------------------------------------------------------------- +3-5 day connector setup Connect in one line +$50k+/year enterprise $500/year flat rate +Per-row pricing anxiety Sync freely +Black box debugging "why did salesforce fail?" +Manual schema migrations AI handles it +47 dropdown menus Natural language ``` -Not configuration files. Not 47 dropdown menus. Just say what you want. +## One-Click Deploy -## Promise Pipelining +```bash +npx create-dotdo fivetran +``` -Chain operations with zero round-trip latency. +A data pipeline platform. Running on infrastructure you control. AI-native from day one. ```typescript -const synced = await fivetran`list all connectors` - .map(connector => fivetran`sync ${connector}`) - .map(result => fivetran`verify ${result}`) -// One network round trip! +import { Fivetran } from 'fivetran.do' -// Parallel syncs with verification -const verified = await fivetran`find stale connectors` - .map(connector => fivetran`refresh ${connector}`) - .filter(result => result.status === 'healthy') - .map(result => fivetran`log ${result} to audit trail`) +export default Fivetran({ + name: 'my-data-platform', + domain: 'data.my-startup.com', +}) ``` -CapnWeb pipelining means your data flows as fast as your thoughts. - -## Agent Integration +## Features -Let your AI team handle data infrastructure. +### Connectors ```typescript -import { tom, ralph, priya } from 'agents.do' +// Connect anything in one line +const salesforce = await fivetran`connect salesforce` +const stripe = await fivetran`connect stripe` +const postgres = await fivetran`connect postgres ${env.DATABASE_URL}` + +// AI infers what you need +await fivetran`salesforce` // connects to salesforce +await fivetran`sync salesforce` // syncs existing connector +await fivetran`salesforce to snowflake` // full pipeline setup +``` -// Tom sets up the data infrastructure -tom`set up automated sync from Salesforce to our warehouse` +### Syncing -// Ralph builds pipelines for new features -ralph`create a pipeline to sync Stripe subscriptions for the billing dashboard` +```typescript +// Sync is one line +await fivetran`sync salesforce to d1` + .objects('Account', 'Contact', 'Opportunity') + .every('15 minutes') + .incremental() -// Priya monitors data quality -priya`check if our customer data is current and flag any sync issues` +// Full refresh when you need it +await fivetran`full sync stripe to warehouse` -// Chain of command -const pipeline = await priya`plan the data architecture for Q1` - .map(spec => tom`implement ${spec}`) - .map(impl => ralph`test ${impl}`) - .map(tested => priya`verify ${tested} meets requirements`) +// Batch syncs just work +await fivetran`sync all stale connectors` ``` -## The Stakes +### Destinations -| Pain Point | Fivetran | fivetran.do | -|------------|----------|-------------| -| **Monthly Cost** | $5,000-50,000+/mo | $50-500/mo (90% less) | -| **New Connector Lead Time** | 3-5 business days | Instant - just ask | -| **Schema Changes** | Manual migration dance | AI handles it automatically | -| **Debugging** | Black box logs | Natural language: "why did Salesforce sync fail?" | -| **Vendor Lock-in** | Proprietary everything | Open, portable, yours | -| **Scaling** | Per-row pricing anxiety | Flat rate, sync freely | +```typescript +// Natural as saying where data goes +await fivetran`salesforce to d1` +await fivetran`stripe to snowflake` +await fivetran`hubspot to bigquery` +await fivetran`all sources to data lake` + +// Archive to R2 +await fivetran`archive old data to r2 parquet` +``` -**The math**: A Series A startup paying Fivetran $4,000/month saves $43,200/year switching to fivetran.do. That's a full-time data engineer you don't need to hire. +### Scheduling -## Quick Start +```typescript +// Natural as talking to a scheduler +await fivetran`sync salesforce every 15 minutes` +await fivetran`stripe hourly` +await fivetran`hubspot daily at 6am` -```bash -npm install @dotdo/fivetran +// Bulk scheduling just works +await fivetran`all connectors every hour` ``` +### Transformations + ```typescript -import { fivetran } from '@dotdo/fivetran' +// Transform in plain English +await fivetran`sync users but lowercase emails and filter inactive` + +// Or be specific +await fivetran`sync accounts` + .transform(row => ({ + ...row, + email: row.email.toLowerCase(), + fullName: `${row.firstName} ${row.lastName}` + })) +``` + +## Promise Pipelining -// Natural language setup -await fivetran`connect to my salesforce org using ${env.SALESFORCE_CREDENTIALS}` -await fivetran`sync accounts, contacts, and opportunities to d1 every hour` -await fivetran`notify me on slack when sync fails` +```typescript +// Chain operations - one network round trip +await fivetran`all connectors` + .map(c => fivetran`sync ${c}`) + .map(r => fivetran`verify ${r}`) -// That's it. You're done. +// Parallel syncs with verification +await fivetran`stale connectors` + .refresh() + .filter(r => r.status === 'healthy') + .audit() ``` -## When You Need Control +## AI-Native Data Pipelines -The structured API is always there when you need precision. +### Self-Healing ```typescript -import { Fivetran } from '@dotdo/fivetran' - -const fivetran = new Fivetran({ id: 'my-data-platform' }) - -// Define a connector with explicit configuration -const salesforce = fivetran.createConnector({ - id: 'salesforce-prod', - source: { - type: 'salesforce', - credentials: env.SALESFORCE_CREDENTIALS, - objects: ['Account', 'Contact', 'Opportunity'] - }, - destination: { - type: 'd1', - database: env.ANALYTICS_DB - }, - sync: { - schedule: 'every 15 minutes', - mode: 'incremental' - } -}) +// Pipelines that fix themselves +await fivetran`sync salesforce` + .onError(async (error) => { + const diagnosis = await fivetran`why did ${error.connector} fail?` + const fix = await fivetran`fix ${diagnosis}` + if (!fix.resolved) { + await slack.notify('#data-team', diagnosis.summary) + } + }) + +// Or just ask +await fivetran`diagnose and fix all broken connectors` +``` -// Trigger manual sync -await salesforce.sync() +### Schema Evolution -// Check sync status -const status = await salesforce.status() -// { lastSync: '2024-01-15T10:30:00Z', status: 'healthy', rowsSynced: 15420 } +```typescript +// AI handles schema changes automatically +// - New columns detected +// - Type changes handled +// - Renames inferred +// - Breaking changes flagged for review + +// Or ask directly +await fivetran`salesforce schema changed?` +await fivetran`migrate accounts table to new schema` ``` -## Features +### Monitoring + +```typescript +// Query your pipelines like talking to your data team +await fivetran`what's the status of all connectors?` +await fivetran`which syncs failed in the last 24 hours?` +await fivetran`how many rows did we sync last week?` +await fivetran`why is salesforce slow?` + +// Close gaps at scale +await fivetran`stale connectors` + .refresh() + .verify() + .notify('#data-team') +``` + +### Agent Integration + +```typescript +import { tom, ralph, priya } from 'agents.do' + +// AI agents handle your data infrastructure +await tom`set up salesforce to warehouse pipeline` + .map(pipeline => ralph`test ${pipeline}`) + .map(tested => priya`verify ${tested} meets requirements`) -- **500+ Pre-built Connectors** - Salesforce, HubSpot, Stripe, databases, APIs -- **Natural Language Control** - Tagged templates for intuitive pipeline management -- **Promise Pipelining** - Chain operations with single round-trip via CapnWeb -- **Automated Schema Migrations** - AI handles source changes without breaking pipelines -- **Incremental Updates** - Sync only what changed, efficiently -- **AI-Native** - MCP tools for autonomous data operations -- **Self-Healing** - Automatic retry, backfill, and error resolution +// Chain of command - one network round trip +await priya`plan Q1 data architecture` + .map(spec => tom`implement ${spec}`) + .map(impl => ralph`test ${impl}`) +``` ## Architecture +### Durable Object per Connector + ``` - +----------------------+ - | fivetran.do | - | (Cloudflare Worker) | - +----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | ConnectorDO | | DestinationDO | | SyncDO | - | (source configs) | | (target schemas) | | (sync state) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | D1/R2 | | Cloudflare Queues| | MCP Tools | - | (data storage) | | (sync jobs) | | (AI operations) | - +------------------+ +------------------+ +------------------+ +PlatformDO (config, users, connectors, destinations) + | + +-- ConnectorDO (source configs, credentials) + | |-- SQLite: Connector state (encrypted) + | +-- R2: Credentials vault (encrypted) + | + +-- SyncDO (sync state, history, metrics) + | |-- SQLite: Sync records + | +-- Queues: Job scheduling + | + +-- DestinationDO (target configs, schemas) + | |-- SQLite: Schema cache + | +-- R2: Large payloads + | + +-- AIDO (diagnostics, healing, suggestions) + |-- LLM: Intent parsing, diagnostics + +-- MCP: Tool definitions ``` -**Key insight**: Durable Objects provide reliable connector state and sync coordination. Each connector gets its own DO for isolation. Queues handle job scheduling. AI interprets your intent and executes. +### Storage Tiers + +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active syncs, recent state | <10ms | +| **Warm** | R2 + SQLite Index | Historical sync data (30 days) | <100ms | +| **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | + +## vs Fivetran + +| Feature | Fivetran | fivetran.do | +|---------|----------|-------------| +| **Pricing** | Per-row (MAR) | Flat rate | +| **Setup** | 3-5 days | One line | +| **Schema Changes** | Manual migration | AI handles it | +| **Debugging** | Black box logs | Natural language | +| **Data Location** | Fivetran's cloud | Your Cloudflare account | +| **Customization** | Limited | Code it yourself | +| **AI** | None | AI-first design | +| **Lock-in** | Proprietary | MIT licensed | + +## Cost Comparison + +| Usage Level | Fivetran | fivetran.do | Savings | +|-------------|----------|-------------|---------| +| Startup (10M rows) | $500/mo | $50/mo | $5,400/yr | +| Growth (100M rows) | $5,000/mo | $250/mo | $57,000/yr | +| Scale (1B rows) | $50,000/mo | $1,000/mo | $588,000/yr | + +**Why the difference?** Fivetran charges per Monthly Active Row. fivetran.do runs on Cloudflare's efficient infrastructure with flat-rate pricing. Growth is rewarded, not punished. ## Connector Types @@ -173,185 +310,140 @@ const status = await salesforce.status() ```typescript // Databases -fivetran`sync postgres tables users, orders, products to warehouse` -fivetran`connect mysql with logical replication for real-time sync` -fivetran`stream mongodb collections to data lake` +await fivetran`postgres to warehouse` +await fivetran`mysql with replication` +await fivetran`mongodb collections to lake` // SaaS -fivetran`pull salesforce data every 15 minutes` -fivetran`sync hubspot contacts and deals hourly` -fivetran`connect stripe for payment analytics` +await fivetran`salesforce every 15 minutes` +await fivetran`hubspot contacts hourly` +await fivetran`stripe payments daily` // APIs -fivetran`fetch data from ${apiEndpoint} daily` -fivetran`sync graphql schema from ${endpoint}` +await fivetran`${apiEndpoint} daily` +await fivetran`graphql ${endpoint}` ``` ### Destinations ```typescript // Cloudflare-native -fivetran`send all data to d1 database analytics` -fivetran`archive to r2 in parquet format` +await fivetran`to d1` +await fivetran`archive to r2 parquet` // External warehouses -fivetran`sync to snowflake account ${account}` -fivetran`export to bigquery project ${project}` +await fivetran`to snowflake` +await fivetran`to bigquery` +await fivetran`to databricks` ``` -## Transformations +## Use Cases -```typescript -// SQL transforms (natural language) -fivetran`sync users but lowercase all emails and filter inactive` - -// Or explicit SQL -const connector = fivetran.createConnector({ - // ... - transform: { - type: 'sql', - query: ` - SELECT - id, - LOWER(email) as email, - created_at - FROM {{ source }} - WHERE status = 'active' - ` - } -}) +### Analytics Pipeline -// TypeScript transforms -const connector = fivetran.createConnector({ - // ... - transform: { - type: 'function', - fn: (row) => ({ - ...row, - email: row.email.toLowerCase(), - fullName: `${row.firstName} ${row.lastName}` - }) - } -}) +```typescript +// Full analytics pipeline in three lines +await fivetran`salesforce to snowflake hourly` +await fivetran`stripe to snowflake hourly` +await fivetran`hubspot to snowflake daily` ``` -## Monitoring +### Real-Time Sync ```typescript -// Natural language monitoring -fivetran`what's the status of all connectors?` -fivetran`which syncs failed in the last 24 hours?` -fivetran`how many rows did we sync last week?` - -// Programmatic -fivetran.onSyncComplete(async (event) => { - console.log(`Sync completed: ${event.connectorId}`) - console.log(`Rows synced: ${event.stats.rowsSynced}`) -}) - -fivetran.onSyncError(async (event) => { - // AI can automatically diagnose and fix - await fivetran`diagnose and fix ${event.error}` -}) +// Keep systems in sync +await fivetran`postgres to d1` + .every('1 minute') + .incremental() ``` -## AI Self-Healing - -When things break, fivetran.do fixes itself. +### Data Lake ```typescript -// Traditional approach: wake up at 2am, debug for hours -// fivetran.do approach: - -fivetran.onSyncError(async (error) => { - // AI diagnoses the issue - const diagnosis = await fivetran`why did ${error.connector} fail?` - - // AI fixes it if possible - const fix = await fivetran`fix ${diagnosis}` - - // If AI can't fix, escalate to human with full context - if (!fix.resolved) { - await slack.notify({ - channel: '#data-team', - message: `Sync issue needs human attention: ${diagnosis.summary}`, - context: diagnosis.fullContext - }) - } -}) +// Archive everything +await fivetran`all sources to r2` + .format('parquet') + .partition('date') ``` -## Cost Comparison +## Why Open Source for Data Pipelines? -| Usage Level | Fivetran Pricing | fivetran.do | Savings | -|-------------|------------------|-------------|---------| -| Startup (10 connectors) | $1,200/mo | $75/mo | $13,500/yr | -| Growth (50 connectors) | $5,000/mo | $250/mo | $57,000/yr | -| Scale (200 connectors) | $25,000/mo | $1,000/mo | $288,000/yr | -| Enterprise (500+) | $50,000+/mo | $2,500/mo | $570,000+/yr | +### 1. Cost Transparency -**Why the difference?** Fivetran charges per Monthly Active Row (MAR) with aggressive tier pricing. fivetran.do runs on Cloudflare's efficient infrastructure with usage-based pricing that scales linearly. +Fivetran pricing is opaque. Open source means: +- No per-row surprises +- Run it yourself for free +- Or pay for managed hosting -## The Rewrites Ecosystem +### 2. No Lock-in -fivetran.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare: +Your data, your infrastructure: +- Export anytime +- Switch destinations freely +- No proprietary formats -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| [airbyte.do](https://airbyte.do) | Airbyte | Data integration for AI | -| **fivetran.do** | Fivetran | Data pipelines for AI | -| [kafka.do](https://kafka.do) | Kafka | Event streaming for AI | +### 3. AI-First -Each rewrite follows the same pattern: -- Natural language first (tagged templates) -- Promise pipelining for efficient chains -- Durable Objects for state -- Compatible structured API when you need it +Closed platforms limit AI capabilities: +- Integrate any LLM +- Build custom diagnostics +- Train on your patterns -## The workers.do Platform +### 4. Community-Driven -fivetran.do is a core service of [workers.do](https://workers.do) - the platform for building Autonomous Startups. +500+ connectors maintained by: +- Data engineers +- Platform teams +- The community -```typescript -import { priya, ralph, tom, mark } from 'agents.do' -import { fivetran } from '@dotdo/fivetran' - -// AI agents that handle your entire data infrastructure -const dataTeam = { - architect: tom, // Designs data pipelines - builder: ralph, // Implements connectors - product: priya, // Defines data requirements - analytics: mark, // Creates reports -} - -// Natural language orchestration -await tom`design a data pipeline for our new analytics dashboard` - .map(design => ralph`implement ${design} with fivetran`) - .map(pipeline => priya`verify ${pipeline} meets product requirements`) - .map(verified => mark`create executive dashboard from ${verified}`) -``` +## Roadmap + +### Connectors +- [x] Databases (Postgres, MySQL, MongoDB) +- [x] SaaS (Salesforce, HubSpot, Stripe) +- [x] APIs (REST, GraphQL) +- [ ] CDC (Change Data Capture) +- [ ] Streaming (Kafka, Kinesis) -Both kinds of workers. Working for you. +### Destinations +- [x] D1 (Cloudflare SQL) +- [x] R2 (Object Storage) +- [x] Snowflake +- [x] BigQuery +- [ ] Databricks +- [ ] Redshift -## Why Cloudflare? +### AI +- [x] Natural Language Control +- [x] Self-Healing +- [x] Schema Evolution +- [ ] Anomaly Detection +- [ ] Cost Optimization -1. **Global Edge** - Connectors run close to data sources -2. **No Cold Starts** - Durable Objects stay warm -3. **Unlimited Duration** - Long-running syncs supported -4. **Built-in Queues** - Reliable job scheduling -5. **D1 + R2** - Native destinations for data lake -6. **AI-Native** - Natural language control with LLM integration +## Contributing -## Related Domains +fivetran.do is open source under the MIT license. -- **etl.do** - Extract, transform, load pipelines -- **pipelines.do** - Data pipeline orchestration -- **sync.do** - Real-time data synchronization -- **airbyte.do** - Open-source data integration +```bash +git clone https://github.com/dotdo/fivetran.do +cd fivetran.do +pnpm install +pnpm test +``` ## License -MIT +MIT License - Sync freely. + +--- + +

+ Stop paying per row. Start syncing freely. +
+ Natural language. Self-healing. Open source. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/fsx/README.md b/rewrites/fsx/README.md index b7624d79..5e6ae409 100644 --- a/rewrites/fsx/README.md +++ b/rewrites/fsx/README.md @@ -1,254 +1,384 @@ # fsx.do -Filesystem on Cloudflare Durable Objects - A virtual filesystem for the edge. +> Filesystem at the Edge. Natural Language. Tiered Storage. AI-Native. -## Features +Cloud storage vendors charge for egress, lock you into their APIs, and treat AI integration as an afterthought. Managing files across regions means complexity. Simple operations require SDKs, credentials, and configuration. -- **POSIX-like API** - Familiar fs operations (read, write, mkdir, readdir, stat, etc.) -- **Durable Object Storage** - SQLite-backed metadata with R2 blob storage -- **MCP Tools** - Model Context Protocol integration for AI-assisted file operations -- **Tiered Storage** - Hot/warm/cold storage tiers with automatic promotion - - Hot: Durable Object SQLite (low latency, small files) - - Warm: R2 object storage (large files, blobs) - - Cold: Archive storage (infrequent access) -- **Streaming** - ReadableStream/WritableStream support for large files -- **Permissions** - Unix-like permission model (rwx) -- **Symbolic Links** - Symlink and hardlink support -- **Watch** - File/directory change notifications +**fsx.do** is the edge-native alternative. POSIX-like semantics. Durable Object storage. Talk to your filesystem like a colleague. -## Installation +## AI-Native API -```bash -npm install fsx.do +```typescript +import { fsx } from 'fsx.do' // Full SDK +import { fsx } from 'fsx.do/tiny' // Minimal client +import { fsx } from 'fsx.do/stream' // Streaming-only operations ``` -## Quick Start - -### Basic Operations +Natural language for file operations: ```typescript -import { FSx } from 'fsx.do' +import { fsx } from 'fsx.do' -const fs = new FSx(env.FSX) +// Talk to it like a colleague +const files = await fsx`typescript files in src modified today` +const large = await fsx`files over 1MB in /uploads` +const logs = await fsx`error logs from last hour` -// Write a file -await fs.writeFile('/hello.txt', 'Hello, World!') +// Chain like sentences +await fsx`watch /config.json` + .on('change', file => fsx`reload ${file}`) -// Read a file -const content = await fs.readFile('/hello.txt', 'utf-8') +// Streaming that describes itself +await fsx`stream /var/log/app.log` + .pipe(fsx`compress to /backups/app.log.gz`) +``` -// Create directory -await fs.mkdir('/my-folder', { recursive: true }) +## The Problem -// List directory -const entries = await fs.readdir('/my-folder') +Cloud storage is fragmented and complex: -// Get file stats -const stats = await fs.stat('/hello.txt') -console.log(`Size: ${stats.size}, Modified: ${stats.mtime}`) +| What Cloud Vendors Charge | The Reality | +|---------------------------|-------------| +| **Egress Fees** | $0.09/GB (adds up fast) | +| **API Calls** | $0.004 per 10,000 requests | +| **Regional Complexity** | Multi-region = multi-config | +| **AI Integration** | DIY with external services | +| **Hot/Cold Management** | Manual lifecycle policies | -// Delete file -await fs.unlink('/hello.txt') -``` +### The Egress Tax -### Streaming Large Files +Every time data leaves the cloud: +- File downloads cost money +- API responses cost money +- Backups cost money +- Analytics cost money -```typescript -import { FSx } from 'fsx.do' +Edge-native means data stays close to users. No egress. -const fs = new FSx(env.FSX) +### The Complexity Tax -// Write stream -const writable = await fs.createWriteStream('/large-file.bin') -await someReadableStream.pipeTo(writable) +Simple file operations require: +- SDK installation and configuration +- Credential management +- Region selection +- Error handling boilerplate +- Retry logic -// Read stream -const readable = await fs.createReadStream('/large-file.bin') -for await (const chunk of readable) { - process.stdout.write(chunk) -} +A `readFile` shouldn't require 50 lines of setup. + +## The Solution + +**fsx.do** reimagines filesystems for the edge: -// Partial reads -const partial = await fs.createReadStream('/large-file.bin', { - start: 1000, - end: 2000 -}) +``` +Cloud Storage fsx.do +----------------------------------------------------------------- +Egress fees Edge-native (no egress) +Multi-region complexity Global by default +Manual lifecycle policies Automatic tiered storage +SDK boilerplate Natural language +External AI integration AI-native from day one +Vendor lock-in POSIX-like, portable ``` -### File Watching +## One-Click Deploy -```typescript -import { FSx } from 'fsx.do' +```bash +npx create-dotdo fsx +``` -const fs = new FSx(env.FSX) +A virtual filesystem. Running at the edge. Tiered storage automatic from day one. -// Watch a file -const watcher = fs.watch('/config.json', (eventType, filename) => { - console.log(`${eventType}: ${filename}`) -}) +```typescript +import { FSx } from 'fsx.do' -// Watch a directory recursively -const dirWatcher = fs.watch('/src', { recursive: true }, (eventType, filename) => { - console.log(`${eventType}: ${filename}`) +export default FSx({ + name: 'my-filesystem', + domain: 'files.myapp.com', + tiers: { + hot: 'sqlite', // Fast access + warm: 'r2', // Large files + cold: 'archive', // Long-term retention + }, }) - -// Stop watching -watcher.close() ``` -### MCP Tools +## Features + +### File Operations ```typescript -import { fsTools, invokeTool } from 'fsx.do/mcp' +// Just say what you need +const content = await fsx`read /hello.txt` +const readme = await fsx`show me the readme` +const config = await fsx`what's in config.json` + +// Write naturally +await fsx`write "Hello, World!" to /hello.txt` +await fsx`save ${data} as /output.json` + +// Copy, move, delete +await fsx`copy /src/file.ts to /backup/` +await fsx`move /old to /archive` +await fsx`delete /tmp/cache` +``` -// List available fs tools -console.log(fsTools.map(t => t.name)) -// ['fs_read', 'fs_write', 'fs_list', 'fs_mkdir', 'fs_delete', 'fs_move', 'fs_copy', ...] +### Directory Operations -// Invoke a tool -const result = await invokeTool('fs_list', { path: '/src', recursive: true }) +```typescript +// List and explore +const entries = await fsx`list /my-folder` +const tree = await fsx`show /src tree structure` +const nested = await fsx`everything in /project recursively` + +// Create and remove +await fsx`create directory /my-folder` +await fsx`make /a/b/c/d recursively` +await fsx`remove /old-folder and all contents` +``` -// Read file tool -const content = await invokeTool('fs_read', { path: '/README.md' }) +### Finding Files -// Search files -const matches = await invokeTool('fs_search', { - path: '/src', - pattern: '*.ts', - content: 'export function' -}) +```typescript +// Natural queries +const ts = await fsx`typescript files in /src` +const recent = await fsx`files modified in the last hour` +const large = await fsx`files larger than 10MB` +const logs = await fsx`log files containing "error"` + +// Combine naturally +const criticalLogs = await fsx`error logs from today over 1KB` ``` -### Durable Object +### Streaming Large Files ```typescript -import { FileSystemDO } from 'fsx.do/do' +// Stream naturally +await fsx`stream /large-file.bin` + .pipe(response) + +// Partial reads +const chunk = await fsx`bytes 1000-2000 of /large-file.bin` -// In your worker -export { FileSystemDO } +// Upload with streaming +await fsx`stream upload to /large-file.bin` + .from(request.body) +``` + +### File Watching -export default { - async fetch(request, env) { - const id = env.FSX.idFromName('user-123') - const stub = env.FSX.get(id) - return stub.fetch(request) - } -} +```typescript +// Watch like you'd ask someone to +await fsx`watch /config.json` + .on('change', () => fsx`reload config`) + +// Watch directories +await fsx`watch /src for changes` + .on('change', file => fsx`rebuild ${file}`) + +// Recursive watching +await fsx`watch /project recursively` + .on('add', file => console.log(`New: ${file}`)) + .on('remove', file => console.log(`Gone: ${file}`)) ``` ### Tiered Storage ```typescript -import { TieredFS } from 'fsx.do/storage' +// Storage tier is automatic based on: +// - File size (small = hot, large = warm) +// - Access frequency (frequent = promote, rare = demote) +// - Age (old = cold) -const fs = new TieredFS({ - hot: env.FSX, // Durable Object (fast, small files) - warm: env.R2_BUCKET, // R2 (large files) - cold: env.ARCHIVE, // Archive (infrequent) - thresholds: { - hotMaxSize: 1024 * 1024, // 1MB - warmMaxSize: 100 * 1024 * 1024 // 100MB - } -}) +await fsx`write ${smallConfig} to /config.json` // -> hot tier (SQLite) +await fsx`upload ${hugeVideo} to /videos/raw.mp4` // -> warm tier (R2) -// Automatic tier selection based on file size -await fs.writeFile('/small.txt', 'small content') // -> hot tier -await fs.writeFile('/large.bin', hugeBuffer) // -> warm tier +// Query by tier +const archived = await fsx`files in cold storage` +const active = await fsx`hot tier files accessed today` + +// Manual tier control when needed +await fsx`move /old-logs to cold storage` +await fsx`promote /important.db to hot tier` +``` + +### Permissions + +```typescript +// Unix-like permissions, natural language +await fsx`make /script.sh executable` +await fsx`set /private read-only` +await fsx`allow group write on /shared` + +// Check access +const canWrite = await fsx`can I write to /protected?` ``` -## API Overview +### Links -### Core Module (`fsx.do/core`) +```typescript +// Symlinks and hardlinks +await fsx`link /current to /releases/v2.0` +await fsx`symlink /latest to /versions/newest` -**File Operations** -- `readFile(path, encoding?)` - Read file contents -- `writeFile(path, data, options?)` - Write file contents -- `appendFile(path, data)` - Append to file -- `unlink(path)` - Delete file -- `rename(oldPath, newPath)` - Rename/move file -- `copyFile(src, dest)` - Copy file +// Resolve links +const target = await fsx`where does /current point?` +const real = await fsx`real path of /symlinked/file.txt` +``` -**Directory Operations** -- `mkdir(path, options?)` - Create directory -- `rmdir(path, options?)` - Remove directory -- `readdir(path, options?)` - List directory contents +## MCP Tools -**Metadata** -- `stat(path)` - Get file/directory stats -- `lstat(path)` - Get stats (don't follow symlinks) -- `access(path, mode?)` - Check file access -- `chmod(path, mode)` - Change permissions -- `chown(path, uid, gid)` - Change ownership +AI assistants can work with your filesystem: + +```typescript +import { fsx } from 'fsx.do/mcp' -**Links** -- `symlink(target, path)` - Create symbolic link -- `link(existingPath, newPath)` - Create hard link -- `readlink(path)` - Read symbolic link target -- `realpath(path)` - Resolve path (follow symlinks) +// Available as MCP tools +// fs_read, fs_write, fs_list, fs_search, fs_move, fs_copy, fs_delete... -**Streams** -- `createReadStream(path, options?)` - Get readable stream -- `createWriteStream(path, options?)` - Get writable stream +// AI can use these naturally +await fsx`find all TODO comments in typescript files` +await fsx`search for "deprecated" in /src` +await fsx`what files changed since yesterday?` +``` -**Watching** -- `watch(path, options?, listener?)` - Watch for changes -- `watchFile(path, options?, listener?)` - Watch specific file +## Architecture -### MCP Tools (`fsx.do/mcp`) +### Durable Object per Filesystem -- `fsTools` - Array of available filesystem tool definitions -- `invokeTool(name, params)` - Execute a tool by name -- `registerTool(tool)` - Add a custom tool +``` +FileSystemDO (config, metadata index) + | + +-- HotTier: SQLite + | |-- Small files (<1MB) + | +-- Frequently accessed + | + +-- WarmTier: R2 + | |-- Large files (1MB-100MB) + | +-- Standard access patterns + | + +-- ColdTier: R2 Archive + |-- Rarely accessed + +-- Long-term retention +``` -### Durable Object (`fsx.do/do`) +### Storage Tiers -- `FileSystemDO` - Main Durable Object class -- Handles all filesystem operations via fetch API +| Tier | Storage | Use Case | Latency | +|------|---------|----------|---------| +| **Hot** | SQLite | Small files, config, metadata | <10ms | +| **Warm** | R2 | Large files, media, logs | <100ms | +| **Cold** | R2 Archive | Backups, compliance, historical | <1s | -### Storage (`fsx.do/storage`) +### Automatic Tier Management -- `TieredFS` - Multi-tier filesystem with automatic placement -- `R2Storage` - R2-backed blob storage -- `SQLiteMetadata` - SQLite-backed metadata store +```typescript +// fsx.do automatically manages tiers: +// - New small files -> hot +// - Large files -> warm +// - Untouched for 30 days -> cold +// - Accessed from cold -> promote to warm + +// Override defaults per-filesystem +export default FSx({ + tiers: { + hotMaxSize: '2MB', // Larger hot tier + warmToClodAge: '90 days', // Slower cold transition + autoPromote: true, // Promote on access + }, +}) +``` ## File System Structure ``` / ├── .fsx/ # System directory -│ ├── metadata.db # SQLite metadata +│ ├── metadata.db # SQLite index │ └── config.json # FS configuration ├── home/ │ └── user/ │ ├── documents/ │ └── .config/ -└── tmp/ # Temporary files (auto-cleanup) +└── tmp/ # Temporary (auto-cleanup) +``` + +## vs Cloud Storage + +| Feature | S3/GCS/Azure | fsx.do | +|---------|--------------|--------| +| **Egress** | $0.09/GB | $0 (edge-native) | +| **Latency** | Region-bound | Global edge | +| **API** | SDK required | Natural language | +| **Tiering** | Manual policies | Automatic | +| **AI Integration** | External | MCP native | +| **File Operations** | Object-based | POSIX-like | +| **Streaming** | Multipart complexity | Native streams | + +## Use Cases + +### Config Management + +```typescript +// Watch and reload +await fsx`watch /config.json` + .on('change', () => fsx`reload config and restart workers`) ``` -## Configuration +### Log Aggregation ```typescript -const fs = new FSx(env.FSX, { - // Storage tiers - tiers: { - hotMaxSize: 1024 * 1024, // 1MB - warmEnabled: true, - coldEnabled: false, - }, +// Tail logs across services +await fsx`tail /var/log/*.log` + .pipe(fsx`stream to /aggregated/all.log`) - // Permissions - defaultMode: 0o644, // rw-r--r-- - defaultDirMode: 0o755, // rwxr-xr-x +// Archive old logs +await fsx`logs older than 7 days` + .each(log => fsx`compress ${log} to cold storage`) +``` - // Cleanup - tmpMaxAge: 24 * 60 * 60 * 1000, // 24 hours +### Asset Pipeline - // Limits - maxFileSize: 100 * 1024 * 1024, // 100MB - maxPathLength: 4096, -}) +```typescript +// Process uploads +await fsx`watch /uploads for new images` + .on('add', img => fsx`resize ${img} to /thumbnails`) +``` + +### Backup and Sync + +```typescript +// Sync directories +await fsx`sync /local to /backup` + +// Incremental backup +await fsx`files changed since last backup` + .each(file => fsx`copy ${file} to /backup`) +``` + +## SDK Entry Points + +```typescript +import { fsx } from 'fsx.do' // Full featured +import { fsx } from 'fsx.do/tiny' // Minimal, no deps +import { fsx } from 'fsx.do/stream' // Streaming only +import { fsx } from 'fsx.do/mcp' // MCP tools +import { fsx } from 'fsx.do/do' // Durable Object class ``` ## License MIT + +--- + +

+ Files at the edge. Natural as speech. +
+ POSIX-like. Tiered storage. AI-native. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/hubspot/README.md b/rewrites/hubspot/README.md index d114a9e6..e185d412 100644 --- a/rewrites/hubspot/README.md +++ b/rewrites/hubspot/README.md @@ -1,704 +1,483 @@ # hubspot.do -> You're a startup founder. You need CRM and marketing automation. HubSpot wants $3,600/month. Your runway is shrinking. Your competitors are closing deals while you're negotiating enterprise contracts. +> CRM and Marketing Automation. Edge-Native. AI-First. Open by Default. -> Your own HubSpot. One click. AI-native. Open source. +HubSpot charges $3,600/month for enterprise CRM. Your runway is shrinking. Your competitors are closing deals while you're negotiating contracts. -[![npm version](https://img.shields.io/npm/v/hubspot.do.svg)](https://www.npmjs.com/package/hubspot.do) -[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) +**hubspot.do** is the open-source alternative. Deploys in minutes. AI that writes emails, not just sends them. Your data, your infrastructure. ---- - -## The workers.do Way - -Talk to your CRM like a colleague: +## AI-Native API ```typescript -import { hubspot, sally, mark, priya } from 'workers.do' - -// Natural language queries -const leads = await hubspot`find leads from ${campaign}` -const engaged = await hubspot`show contacts who opened last email` -const pipeline = await hubspot`deals closing this month over $25k` - -// AI agents work your CRM -await sally`qualify inbound leads from today` -await mark`create nurture sequence for ${segment}` +import { hubspot } from 'hubspot.do' // Full SDK +import { hubspot } from 'hubspot.do/tiny' // Minimal client +import { hubspot } from 'hubspot.do/marketing' // Marketing-only operations ``` -### Promise Pipelining - -Chain operations without waiting. One network round trip: +Natural language for sales and marketing: ```typescript -// Find leads, send outreach, schedule follow-ups - all pipelined -const nurtured = await hubspot`find warm leads from webinar` - .map(lead => sally`send personalized outreach to ${lead}`) - .map(outreach => mark`schedule follow-up content for ${outreach}`) - -// Marketing to sales handoff -const mqls = await hubspot`find marketing qualified leads` - .map(lead => [priya, sally].map(r => r`review ${lead} for sales readiness`)) - .map(qualified => sally`create deal for ${qualified}`) +import { hubspot } from 'hubspot.do' + +// Talk to it like a colleague +const warm = await hubspot`leads from Google Ads campaign` +const engaged = await hubspot`contacts who opened email but didn't click` +const pipeline = await hubspot`deals closing this quarter over $50k` + +// Chain like sentences +await hubspot`leads from webinar` + .map(l => hubspot`send personalized follow-up to ${l}`) + +// Marketing that writes itself +await hubspot`create nurture sequence for trial users` + .draft() // AI writes the emails + .review() // you approve + .schedule() // sends automatically ``` ---- - ## The Problem -HubSpot is a $30 billion company. They charge $800 to $3,600 per month for enterprise features. You are paying for: +HubSpot dominates SMB marketing automation: + +| What HubSpot Charges | The Reality | +|---------------------|-------------| +| **Starter** | $20/mo (1,000 contacts, basic features) | +| **Professional** | $800/mo (automation, sequences) | +| **Enterprise** | $3,600/mo (advanced features) | +| **Additional Contacts** | $45/mo per 1,000 contacts | +| **Per-Seat Pricing** | Sales/Service hubs charge per user | -- **Their data centers** - You foot the bill for infrastructure you do not control +### The SaaS Tax + +You are paying for: + +- **Their infrastructure** - Data centers you don't control - **Their margins** - 80%+ gross margins extracted from your business -- **Their roadmap** - Features built for their largest customers, not you -- **Their lock-in** - Your customer data held hostage in their walled garden +- **Their roadmap** - Features built for enterprise, not startups +- **Their lock-in** - Customer data held hostage -And here is the worst part: **you are a tenant in someone else's system**. Your CRM data, your customer relationships, your sales pipeline - all living on servers you will never own, governed by terms you did not write, priced at whatever they decide next quarter. +### The AI Gap -This was the only option in 2006 when HubSpot launched. It is not the only option anymore. +HubSpot was built for humans clicking through web interfaces. Their "AI" features: +- Write subject lines (you still write the email) +- Suggest next actions (you still do them) +- Score leads (with rules you configure) ---- +Real AI writes the entire email. Real AI qualifies leads by reading conversations. Real AI closes care gaps without human intervention. ## The Solution -**hubspot.do** is a complete HubSpot alternative that you own. +**hubspot.do** reimagines CRM for startups and AI: + +``` +HubSpot hubspot.do +----------------------------------------------------------------- +$3,600/month enterprise Deploy in minutes +$45/1,000 contacts ~$5/month for 10,000 contacts +AI writes subject lines AI writes entire campaigns +Per-seat licensing Usage-based pricing +Data in their cloud Your Cloudflare account +Months to implement One command deploy +Vendor lock-in Open source, MIT licensed +``` + +## One-Click Deploy ```bash npx create-dotdo hubspot ``` -One command. Your own instance. Running on Cloudflare's global edge network. Pennies per month instead of hundreds or thousands of dollars. +A complete CRM. Running on infrastructure you control. AI-native from day one. ```typescript import { HubSpot } from 'hubspot.do' -const crm = new HubSpot(env.HUBSPOT) - -// Full CRM capabilities -await crm.contacts.create({ - email: 'alice@startup.com', - firstName: 'Alice', - lastName: 'Chen', - company: 'Acme Corp' -}) - -// Pipeline management -await crm.deals.create({ - name: 'Enterprise Contract', - amount: 50000, - stage: 'qualification', - contact: 'alice@startup.com' -}) - -// Marketing automation -await crm.workflows.trigger('welcome-sequence', { - contact: 'alice@startup.com' +export default HubSpot({ + name: 'my-startup', + domain: 'crm.my-startup.com', }) ``` -Not a "HubSpot alternative." Not a "lightweight CRM." This is **your own HubSpot** - the complete platform, reimagined for the AI era, running entirely under your control. - ---- - -## Why This Matters - -### For Startup Founders - -You are burning $10,000 to $40,000 per year on CRM software before you have product-market fit. That is runway. That is hiring budget. That is your survival. +**Note:** This is production CRM software. See compliance section below. -With hubspot.do: -- **Deploy in 60 seconds** - No sales calls, no procurement, no enterprise contracts -- **Pay for usage** - Pennies per thousand operations, not per seat -- **Own your data** - Export everything, migrate anywhere, no lock-in -- **Scale infinitely** - From zero to millions of contacts without upgrades +## Features -### For AI-First Companies +### Contacts -HubSpot was built for humans clicking through web interfaces. Your AI agents need something different: +```typescript +// Find anyone +const alice = await hubspot`Alice Chen at Acme` +const enterprise = await hubspot`contacts at companies over 500 employees` +const active = await hubspot`contacts who engaged this week` + +// AI infers what you need +await hubspot`Alice Chen` // returns contact +await hubspot`emails to Alice Chen` // returns email history +await hubspot`Alice Chen activity` // returns full timeline +``` -- **Every action is an API call** - No UI-only features, no hidden functionality -- **MCP tools built in** - Claude, GPT, and any LLM can operate your CRM directly -- **Real-time webhooks** - AI workflows triggered by every customer interaction -- **Structured data** - Schema-first design, not form fields bolted onto a legacy system +### Deals -### For Anyone Who Values Freedom +```typescript +// Pipeline is one line +const closing = await hubspot`deals closing this month` +const large = await hubspot`enterprise deals over $100k` +const stalled = await hubspot`deals stuck in negotiation over 30 days` + +// Move deals naturally +await hubspot`move Acme deal to proposal` +await hubspot`close won Acme deal $75k` +``` -Your customer relationships are your business. They should not be trapped in a vendor's database, subject to their pricing whims, their outages, their decisions about what features you need. +### Companies -hubspot.do gives you **sovereignty over your sales data**. +```typescript +// Company intelligence +const target = await hubspot`companies in healthcare over 1000 employees` +const engaged = await hubspot`companies with multiple contacts engaged` + +// Enrich automatically +await hubspot`enrich Acme Corp` + .firmographics() // company data + .technographics() // tech stack + .intent() // buying signals +``` ---- +### Marketing Automation -## One-Click Deploy +```typescript +// Campaigns that write themselves +await hubspot`create welcome sequence for new signups` + .draft() // AI writes emails + .review() // you approve + .activate() // starts sending + +// Segmentation is natural +await hubspot`contacts who downloaded whitepaper but not pricing` + .map(c => hubspot`send pricing follow-up to ${c}`) + +// A/B test without configuration +await hubspot`test subject lines for product launch email` +``` -### Cloudflare Workers +### Email Campaigns -```bash -npx create-dotdo hubspot +```typescript +// Send naturally +await hubspot`send product update to all customers` +await hubspot`email Alice about her expiring trial` + +// AI writes, you approve +await hubspot`draft re-engagement email for churned users` + .review() // see what AI wrote + .approve() // send it ``` -This scaffolds a complete hubspot.do instance configured for your Cloudflare account. - -### Manual Setup +### Lists and Segments ```typescript -// src/index.ts -import { HubSpotDO, HubSpotEntrypoint } from 'hubspot.do' +// Segmentation as sentences +const vips = await hubspot`customers spending over $10k annually` +const atrisk = await hubspot`customers with declining engagement` +const upsell = await hubspot`customers using feature X but not Y` -export { HubSpotDO } -export default HubSpotEntrypoint +// Smart lists update automatically +await hubspot`create list: enterprise leads from LinkedIn` ``` -```jsonc -// wrangler.jsonc -{ - "name": "my-hubspot", - "main": "src/index.ts", - "compatibility_date": "2025-01-01", - "compatibility_flags": ["nodejs_compat"], - "durable_objects": { - "bindings": [{ "name": "HUBSPOT", "class_name": "HubSpotDO" }] - }, - "migrations": [{ "tag": "v1", "new_sqlite_classes": ["HubSpotDO"] }] -} -``` +### Lead Scoring -```bash -npx wrangler deploy -``` +```typescript +// AI scores based on behavior, not rules +const hot = await hubspot`leads most likely to buy this week` +const mql = await hubspot`marketing qualified leads` -Your HubSpot is now live at the edge. Globally distributed. Zero cold starts. +// Custom scoring models +await hubspot`score leads based on pricing page visits and email opens` +``` ---- +### Workflows -## Features +```typescript +// Create workflows by describing them +await hubspot`when lead visits pricing, notify sales and send case study` -### CRM Core +// Complex sequences are still one line +await hubspot`nurture webinar attendees for 30 days then hand to sales` +``` -| Feature | Description | -|---------|-------------| -| **Contacts** | Full contact management with custom properties, lifecycle stages, and activity timeline | -| **Companies** | Organization records with hierarchies, domains, and relationship mapping | -| **Deals** | Sales pipeline with stages, amounts, close dates, and win probability | -| **Tickets** | Support ticketing with SLAs, priorities, and agent assignment | -| **Products** | Product catalog with SKUs, pricing, and inventory tracking | -| **Quotes** | Quote generation with line items, discounts, and approval workflows | - -### Marketing Hub - -| Feature | Description | -|---------|-------------| -| **Email Campaigns** | Drag-and-drop email builder with templates and A/B testing | -| **Forms** | Lead capture forms with conditional logic and progressive profiling | -| **Landing Pages** | Page builder with conversion tracking and personalization | -| **Lists** | Smart segmentation with AND/OR logic and real-time updates | -| **Workflows** | Visual automation builder for nurture sequences and lead scoring | -| **Analytics** | Campaign performance, attribution modeling, and ROI tracking | - -### Sales Hub - -| Feature | Description | -|---------|-------------| -| **Pipelines** | Multiple pipelines with customizable stages and probability | -| **Tasks** | Task queues with due dates, reminders, and sequences | -| **Meetings** | Calendar integration with booking links and round-robin | -| **Sequences** | Multi-step outreach automation with personalization | -| **Documents** | Document tracking with view notifications and analytics | -| **Forecasting** | Revenue forecasting with weighted pipeline and trends | - -### Service Hub - -| Feature | Description | -|---------|-------------| -| **Tickets** | Multi-channel ticket creation with routing and escalation | -| **Knowledge Base** | Self-service articles with search and feedback | -| **Customer Portal** | Branded portal for ticket submission and tracking | -| **Feedback Surveys** | NPS, CSAT, and CES surveys with automation triggers | -| **SLA Management** | Response and resolution time tracking with alerts | - -### Integrations - -| Feature | Description | -|---------|-------------| -| **Email Sync** | Two-way sync with Gmail, Outlook, and IMAP providers | -| **Calendar Sync** | Google Calendar and Microsoft 365 integration | -| **Calling** | VoIP integration with call logging and recording | -| **Slack** | Real-time notifications and deal room collaboration | -| **Zapier/n8n** | Webhook triggers for 5000+ app integrations | +### Meetings ---- +```typescript +// Scheduling without friction +await hubspot`find time for Alice Chen and our sales team` +await hubspot`book demo with Acme Corp next week` -## AI-Native +// Round-robin just works +await hubspot`assign inbound demos to sales team round robin` +``` -hubspot.do is built from the ground up for AI agents. Not retrofitted. Not an afterthought. +### Tickets and Support -### MCP Tools +```typescript +// Support is natural language +await hubspot`open tickets from enterprise customers` +await hubspot`tickets waiting on us over 24 hours` + +// AI handles routine responses +await hubspot`new ticket from Alice about billing` + .draft() // AI writes response + .review() // you check + .send() // resolved +``` -Every CRM operation is available as an MCP tool: +### Analytics ```typescript -import { hubspotTools, invokeTool } from 'hubspot.do/mcp' - -// List available tools -console.log(hubspotTools.map(t => t.name)) -// ['contacts_create', 'contacts_search', 'deals_create', 'deals_update', -// 'workflows_trigger', 'lists_add', 'emails_send', 'meetings_book', ...] - -// AI agents can invoke directly -await invokeTool('contacts_create', { - email: 'lead@prospect.com', - firstName: 'New', - lastName: 'Lead', - source: 'ai-outreach' -}) +// Ask questions, get answers +await hubspot`campaign performance this quarter` +await hubspot`which emails have best open rates` +await hubspot`attribution for closed deals` + +// Forecasting is a question +await hubspot`revenue forecast for Q2` +await hubspot`pipeline coverage for annual target` ``` -### Natural Language Queries +## AI-Native CRM -```typescript -import { HubSpot } from 'hubspot.do' +### Promise Pipelining -const crm = new HubSpot(env.HUBSPOT) +Chain operations without waiting. One network round trip: -// Natural language search -const results = await crm.search({ - natural: 'find all enterprise deals closing this quarter over $50k' -}) +```typescript +// Find leads, qualify, outreach - all pipelined +await hubspot`leads from product hunt launch` + .map(l => hubspot`enrich ${l}`) + .map(l => hubspot`score ${l}`) + .map(l => hubspot`send welcome sequence to ${l}`) -// AI-powered lead scoring -const score = await crm.contacts.score({ - contact: 'alice@startup.com', - model: 'engagement-propensity' -}) +// Marketing to sales handoff +await hubspot`marketing qualified leads this week` + .map(l => hubspot`assign ${l} to sales rep`) + .map(l => hubspot`create task: follow up with ${l}`) + +// Multi-reviewer approval +await hubspot`deals over $100k needing approval` + .map(d => [sales, finance].map(r => r`review ${d}`)) + .map(d => hubspot`move ${d} to contract`) ``` -### agents.do Integration - -hubspot.do is a core service of the [workers.do](https://workers.do) platform: +### AI Agents Work Your CRM ```typescript -import { sally, mark } from 'agents.do' -import { HubSpot } from 'hubspot.do' +import { hubspot, sally, mark, priya } from 'workers.do' + +// Sally handles sales +await sally`qualify inbound leads from today` +await sally`follow up with stalled deals` -// Sally handles sales outreach -const leads = await sally`find warm leads from yesterday's webinar` -for (const lead of leads) { - await sally`send personalized follow-up to ${lead}` -} +// Mark handles marketing +await mark`draft newsletter for this week` +await mark`create case study for Acme win` -// Mark writes marketing content -const campaign = await mark`create email sequence for product launch` -await crm.workflows.deploy(campaign) +// Priya handles product feedback +await priya`analyze support tickets for feature requests` // They work together -await sally.watch(crm.deals, { stage: 'closed-won' }, async (deal) => { - await mark`write case study about ${deal.company}` -}) +await hubspot`closed won deals this month` + .map(d => mark`write case study for ${d}`) + .map(d => sally`ask ${d.contact} for referral`) ``` -### Webhook Everything - -Every CRM event triggers webhooks your AI agents can consume: +### Real-Time Events ```typescript -import { HubSpot } from 'hubspot.do' - -const crm = new HubSpot(env.HUBSPOT) - -// React to any CRM event -crm.on('contact.created', async (contact) => { - await enrichContact(contact) - await scoreContact(contact) - await routeToRep(contact) -}) - -crm.on('deal.stage_changed', async (deal, prev, next) => { - if (next === 'negotiation') { - await notifyManager(deal) - await prepareContract(deal) - } -}) - -crm.on('ticket.created', async (ticket) => { - const response = await aiGenerateResponse(ticket) - await ticket.reply(response) -}) +// AI reacts to every CRM event +await hubspot`when contact created` + .map(c => hubspot`enrich and score ${c}`) + .map(c => hubspot`route ${c} to right rep`) + +await hubspot`when deal reaches negotiation` + .map(d => hubspot`notify manager about ${d}`) + .map(d => hubspot`prepare contract for ${d}`) + +await hubspot`when ticket created` + .map(t => hubspot`draft response for ${t}`) + .review() + .send() ``` ---- - ## API Compatible -hubspot.do implements the HubSpot API specification. Your existing integrations work unchanged. - -### Drop-In Replacement +hubspot.do implements the HubSpot API specification. Existing integrations work unchanged. ```typescript -// Before: HubSpot Cloud -import Hubspot from '@hubspot/api-client' -const client = new Hubspot.Client({ accessToken: process.env.HUBSPOT_TOKEN }) - -// After: hubspot.do (same code, just change the base URL) +// Change one line - everything else works import Hubspot from '@hubspot/api-client' const client = new Hubspot.Client({ - accessToken: process.env.HUBSPOT_TOKEN, - basePath: 'https://your-instance.hubspot.do' -}) - -// Everything works the same -const contact = await client.crm.contacts.basicApi.create({ - properties: { email: 'new@contact.com', firstname: 'New' } + basePath: 'https://your-instance.hubspot.do' // just change this }) ``` -### Supported API Endpoints - -| API | Compatibility | -|-----|---------------| -| **CRM Objects** | Full - contacts, companies, deals, tickets, products, quotes | -| **CRM Associations** | Full - all standard and custom association types | -| **CRM Properties** | Full - property groups, definitions, and validation | -| **CRM Pipelines** | Full - stages, probability, and automation | -| **CRM Search** | Full - filter groups, sorting, and pagination | -| **Marketing Emails** | Full - templates, campaigns, and analytics | -| **Forms** | Full - submissions, progressive profiling | -| **Workflows** | Full - actions, branches, and enrollment | -| **Files** | Full - upload, folders, and CDN delivery | -| **Webhooks** | Full - subscriptions and event delivery | - -### Migration Path +### Migration ```bash -# Export from HubSpot Cloud -npx hubspot.do migrate export --source=hubspot-cloud - -# Import to your instance -npx hubspot.do migrate import --target=https://your-instance.hubspot.do +# Export from HubSpot Cloud, import to your instance +npx hubspot.do migrate --from=hubspot-cloud --to=https://your-instance.hubspot.do ``` -Full data migration including contacts, companies, deals, activities, and custom properties. - ---- +Full data migration: contacts, companies, deals, activities, custom properties. ## Architecture +### Durable Object per Workspace + ``` -+------------------------------------------------------------------+ -| Your Application | -+------------------+-------------------+----------------------------+ -| HubSpot SDK | REST API | MCP Tools | -| (Compatible) | (HTTP/RPC) | (AI Agents) | -+------------------+---------+---------+----------------------------+ - | -+----------------------------v---------------------------------+ -| hubspot.do Worker | -+--------------------------------------------------------------+ -| | -| +----------+ +------------+ +-----------+ +-----------+ | -| | CRM | | Marketing | | Sales | | Service | | -| +----------+ +------------+ +-----------+ +-----------+ | -| | Contacts | | Emails | | Pipelines | | Tickets | | -| | Companies| | Forms | | Tasks | | KB | | -| | Deals | | Workflows | | Meetings | | Feedback | | -| | Tickets | | Analytics | | Sequences | | SLAs | | -| +----------+ +------------+ +-----------+ +-----------+ | -| | -+--------------------------------------------------------------+ -| Durable Object (HubSpotDO) | -+------------------------+-------------------------------------+ -| SQLite | R2 | -| (Hot Data Layer) | (Files, Attachments, Assets) | -+------------------------+-------------------------------------+ +HubSpotDO (config, users, pipelines) + | + +-- ContactsDO (demographics, activities) + | |-- SQLite: Contact records + | +-- R2: Email attachments + | + +-- DealsDO (pipeline, amounts, stages) + | |-- SQLite: Deal data + | +-- R2: Contracts, proposals + | + +-- MarketingDO (campaigns, workflows) + | |-- SQLite: Email tracking + | +-- R2: Templates, assets + | + +-- ServiceDO (tickets, knowledge base) + |-- SQLite: Ticket data + +-- R2: Attachments ``` -### Why Durable Objects? - -1. **Single-threaded consistency** - No race conditions on contact updates -2. **SQLite built-in** - Real relational database, not key-value hacks -3. **Automatic scaling** - Millions of CRM instances, zero configuration -4. **Global distribution** - Data lives at the edge, near your users -5. **Zero cold starts** - Always warm, always fast -6. **R2 integration** - Unlimited storage for files and attachments - ### Storage Tiers -| Tier | Storage | Use Case | -|------|---------|----------| -| **Hot** | SQLite | Active contacts, recent deals, live workflows | -| **Warm** | R2 | Historical activities, email content, attachments | -| **Cold** | R2 Archive | Compliance archives, audit trails | +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active contacts, recent deals | <10ms | +| **Warm** | R2 + Index | Historical activities (90+ days) | <100ms | +| **Cold** | R2 Archive | Compliance, audit trails | <1s | -```typescript -import { TieredHubSpot } from 'hubspot.do/storage' - -const crm = new TieredHubSpot({ - hot: env.HUBSPOT, - warm: env.R2_BUCKET, - cold: env.ARCHIVE, - thresholds: { - hotMaxContacts: 1_000_000, - activityRetentionDays: 90, - } -}) -``` +## vs HubSpot ---- +| Feature | HubSpot | hubspot.do | +|---------|---------|-----------| +| **Implementation** | Sales calls, procurement | Deploy in minutes | +| **Monthly Cost** | $800-3,600/mo | ~$5/mo | +| **AI** | Subject line suggestions | AI writes entire campaigns | +| **Data Location** | HubSpot's cloud | Your Cloudflare account | +| **Customization** | Limited to their UI | Code anything | +| **Lock-in** | Years of migration | MIT licensed | +| **API** | Rate limited | Unlimited | -## Getting Started +## Use Cases -### Installation +### Startup CRM -```bash -npm install hubspot.do +```typescript +// Your entire sales process +await hubspot`new lead Alice Chen from Product Hunt` +await hubspot`qualify Alice based on company size and role` +await hubspot`send personalized demo request to Alice` +await hubspot`schedule demo for Alice next Tuesday` +await hubspot`create deal Acme Corp $25k` ``` -### Basic Usage +### Marketing Automation ```typescript -import { HubSpot } from 'hubspot.do' - -const crm = new HubSpot(env.HUBSPOT) - -// Create a contact -const contact = await crm.contacts.create({ - email: 'alice@example.com', - firstName: 'Alice', - lastName: 'Smith', - company: 'Acme Corp', - phone: '+1-555-0123', - lifecycleStage: 'lead' -}) - -// Search contacts -const results = await crm.contacts.search({ - filters: [ - { property: 'lifecycleStage', operator: 'eq', value: 'lead' }, - { property: 'createdate', operator: 'gte', value: lastWeek } - ], - sorts: [{ property: 'createdate', direction: 'desc' }], - limit: 50 -}) - -// Create a deal -const deal = await crm.deals.create({ - name: 'Enterprise License', - amount: 25000, - stage: 'qualification', - closeDate: nextMonth, - associatedContacts: [contact.id] -}) - -// Update deal stage -await crm.deals.update(deal.id, { - stage: 'proposal', - amount: 30000 -}) - -// Trigger a workflow -await crm.workflows.enroll('new-lead-nurture', contact.id) +// Launch campaign in one conversation +await hubspot`create campaign for product launch` + .audience(`customers who used feature X`) + .sequence(`announce, demo invite, case study, pricing`) + .draft() + .review() + .schedule(`next Monday`) ``` -### Real-Time Updates +### Sales Pipeline ```typescript -import { HubSpot } from 'hubspot.do' - -const crm = new HubSpot(env.HUBSPOT) - -// Subscribe to contact changes -const unsubscribe = crm.contacts.watch({ - filters: [{ property: 'lifecycleStage', operator: 'eq', value: 'customer' }] -}, (event) => { - console.log(`New customer: ${event.contact.email}`) - sendWelcomePackage(event.contact) -}) +// Pipeline management as conversation +await hubspot`deals stuck in proposal over 2 weeks` + .map(d => hubspot`send nudge to ${d.contact}`) -// Subscribe to deal movements -crm.deals.watch({ - filters: [{ property: 'stage', operator: 'eq', value: 'closed-won' }] -}, async (event) => { - await celebrateWin(event.deal) - await provisionAccount(event.deal) -}) +await hubspot`deals closing this month` + .map(d => hubspot`forecast probability for ${d}`) ``` -### Marketing Automation +### Customer Success ```typescript -import { HubSpot } from 'hubspot.do' - -const crm = new HubSpot(env.HUBSPOT) - -// Create an email template -const template = await crm.emails.createTemplate({ - name: 'Welcome Email', - subject: 'Welcome to {{company.name}}!', - body: ` - Hi {{contact.firstName}}, +// Retention workflows +await hubspot`customers with declining engagement` + .map(c => hubspot`schedule check-in with ${c}`) - Thanks for signing up! Here's what to do next... - - Best, - The Team - ` -}) - -// Create a workflow -const workflow = await crm.workflows.create({ - name: 'New Signup Nurture', - trigger: { - type: 'form_submission', - formId: 'signup-form' - }, - actions: [ - { type: 'send_email', templateId: template.id, delay: 0 }, - { type: 'wait', duration: '3d' }, - { type: 'send_email', templateId: 'follow-up-1', delay: 0 }, - { type: 'branch', condition: 'has_opened_email', yes: 'qualify', no: 'wait' }, - { type: 'update_property', property: 'lifecycleStage', value: 'mql', id: 'qualify' } - ] -}) - -// Activate the workflow -await crm.workflows.activate(workflow.id) +await hubspot`customers approaching renewal` + .map(c => hubspot`send renewal reminder to ${c}`) ``` ---- +## Roadmap -## The Rewrites Ecosystem - -hubspot.do is part of the rewrites family - reimplementations of legacy SaaS platforms on Cloudflare Durable Objects: - -| Rewrite | Original | Monthly Cost Savings | -|---------|----------|---------------------| -| **hubspot.do** | HubSpot ($800-3600/mo) | 95-99% | -| [salesforce.do](https://salesforce.do) | Salesforce ($75-300/user/mo) | 95-99% | -| [zendesk.do](https://zendesk.do) | Zendesk ($55-115/agent/mo) | 90-95% | -| [intercom.do](https://intercom.do) | Intercom ($74-139/seat/mo) | 90-95% | -| [mailchimp.do](https://mailchimp.do) | Mailchimp ($13-350/mo) | 80-95% | -| [stripe.do](https://stripe.do) | Stripe Atlas ($500 + 2.9%) | 50-70% | - -Each rewrite follows the same pattern: -- **Your own instance** - Not multi-tenant, not shared, yours -- **Durable Object per workspace** - Isolated, consistent, scalable -- **SQLite for hot data** - Real database, real queries -- **R2 for files** - Unlimited storage, global CDN -- **MCP tools** - AI-native from day one -- **API compatible** - Existing integrations work - ---- - -## Pricing Comparison - -### HubSpot Cloud Pricing (2025) - -| Plan | Monthly Cost | What You Get | -|------|--------------|--------------| -| **Starter** | $20/mo | 1,000 contacts, basic features | -| **Professional** | $800/mo | 2,000 contacts, automation | -| **Enterprise** | $3,600/mo | 10,000 contacts, advanced features | - -Plus: $45/mo per additional 1,000 contacts. Per-seat pricing for Sales and Service Hubs. - -### hubspot.do Pricing - -| Resource | Cost | Notes | -|----------|------|-------| -| **Durable Object** | $0.15/million requests | Your CRM instance | -| **SQLite Storage** | $0.20/GB/month | Contact and deal data | -| **R2 Storage** | $0.015/GB/month | Files and attachments | -| **Workers** | Free tier: 100k/day | API and webhook handling | - -**Example: 10,000 contacts, 50,000 monthly operations** - -- HubSpot Enterprise: **$3,600/month** -- hubspot.do: **~$5/month** - -That is a **99.86% cost reduction**. - ---- - -## Frequently Asked Questions - -### Is this really a complete HubSpot replacement? - -hubspot.do implements the core CRM, Marketing, Sales, and Service Hub functionality that 90% of HubSpot users actually use. Enterprise features like predictive lead scoring and custom behavioral events are on the roadmap. - -### What about HubSpot's ecosystem of integrations? - -hubspot.do is API-compatible with HubSpot. Most integrations that work with HubSpot's API will work with hubspot.do by changing the base URL. We also provide native webhooks and MCP tools for AI-first integrations. - -### Can I migrate from HubSpot Cloud? - -Yes. We provide migration tools that export your contacts, companies, deals, activities, and custom properties from HubSpot Cloud and import them into your hubspot.do instance. - -### What about support? - -hubspot.do is open source (MIT license). Community support is available via GitHub Issues. Enterprise support contracts are available for organizations that need SLAs. - -### Is my data secure? - -Your data runs on Cloudflare's infrastructure with enterprise-grade security. You control the encryption keys. You control access. Your data never leaves infrastructure you control. - -### What if Cloudflare goes down? - -Cloudflare has a 99.99% uptime SLA. Your Durable Object data is automatically replicated across multiple regions. For additional resilience, enable continuous backup to your own R2 bucket or external storage. - ---- +### CRM Core +- [x] Contacts with custom properties +- [x] Companies with hierarchies +- [x] Deals with pipelines +- [x] Tickets with SLAs +- [x] Products and quotes +- [ ] Custom objects + +### Marketing +- [x] Email campaigns +- [x] Smart lists +- [x] Workflows +- [x] Forms +- [x] Landing pages +- [ ] Social media integration + +### Sales +- [x] Pipelines +- [x] Tasks and sequences +- [x] Meetings +- [x] Documents +- [x] Forecasting +- [ ] Call recording + +### AI +- [x] Natural language queries +- [x] Email drafting +- [x] Lead scoring +- [x] Campaign generation +- [ ] Predictive analytics +- [ ] Conversation intelligence ## Contributing -hubspot.do is open source and welcomes contributions. +hubspot.do is open source under the MIT license. ```bash -# Clone the repository -git clone https://github.com/drivly/hubspot.do.git +git clone https://github.com/dotdo/hubspot.do cd hubspot.do - -# Install dependencies -npm install - -# Run tests -npm test - -# Start local development -npm run dev +pnpm install +pnpm test ``` -See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. - ---- - ## License -MIT - see [LICENSE](LICENSE) - ---- - -## The Manifesto - -We are in the early days of a fundamental shift. - -For two decades, SaaS vendors built empires by hosting software you could have run yourself. They charged rent on your data. They dictated your upgrade path. They held your customer relationships hostage. - -The cloud was supposed to democratize computing. Instead, it created new landlords. - -**That era is ending.** - -Edge computing, Durable Objects, and AI agents are collapsing the complexity that justified SaaS premiums. The same CRM that costs $3,600/month can now run on infrastructure that costs $5/month. The same features that required a 500-person engineering team can be replicated by AI-assisted development in weeks. - -hubspot.do is not just a cheaper HubSpot. It is a statement: - -**Your customer data belongs to you.** - -**Your sales pipeline belongs to you.** - -**Your business relationships belong to you.** - -One click. Your own HubSpot. Forever. +MIT License - Your customers, your data, your CRM. ---

- Take back control. + The $3,600/month ends here. +
+ AI-native. Edge-first. Customer-owned. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/inngest/README.md b/rewrites/inngest/README.md index 31e6d936..ba68f55f 100644 --- a/rewrites/inngest/README.md +++ b/rewrites/inngest/README.md @@ -1,275 +1,207 @@ # inngest.do -Inngest on Cloudflare - Event-driven durable workflow execution without managing queues, infra, or state. +> Durable Workflows. Event-Driven. Natural Language. Edge-Native. -## The Problem +Inngest charges for function runs, limits concurrency, and forces you through their cloud. You define workflows in verbose configuration objects. Boilerplate everywhere. -Modern applications need background jobs and workflows: -- Process webhooks reliably -- Orchestrate multi-step operations -- Handle retries and failures gracefully -- Control concurrency and rate limits +**inngest.do** is the open-source alternative. Runs on your Cloudflare. Say what you want in plain English. Durable by default. -Traditional solutions require: -- Managing queue infrastructure -- Building retry logic -- Implementing state machines -- Operating distributed systems +## AI-Native API -## The Vision +```typescript +import { inngest } from 'inngest.do' // Full SDK +import { inngest } from 'inngest.do/tiny' // Minimal client +import { inngest } from 'inngest.do/workflows' // Workflow helpers +``` -Drop-in Inngest replacement running entirely on Cloudflare. +Natural language for durable workflows: ```typescript -import { Inngest } from '@dotdo/inngest' +import { inngest } from 'inngest.do' + +// Talk to it like a colleague +await inngest`when user signs up, send welcome email` +await inngest`process orders every 5 minutes` +await inngest`when payment fails, retry 3 times then notify support` + +// Chain like sentences +const workflow = await inngest`when user signs up` + .run('fetch user', () => fetchUser(event.data.id)) + .sleep('5 minutes') + .run('sync to CRM', () => syncToCRM(user)) + +// Concurrency as methods, not config +inngest`when task runs`.concurrency(5) +inngest`when api called`.throttle(10, 'per minute') +inngest`when search typed`.debounce('500ms') +``` + +No queues to manage. No state machines to build. Just say what you want. + +## The Problem -const inngest = new Inngest({ id: 'my-app' }) +Inngest (the company) dominates the durable workflow space: -// Define a function with durable steps -export const syncUser = inngest.createFunction( +| What Inngest Charges | The Reality | +|---------------------|-------------| +| **Function Runs** | $0.15-0.50 per 1,000 runs | +| **Concurrency Limits** | Pay more for higher limits | +| **Step Execution** | Charged per step | +| **Cloud Lock-in** | Must use Inngest Cloud | +| **Verbose Config** | `createFunction({ id, concurrency, throttle, retries... })` | + +### The Configuration Tax + +Every function requires a configuration object: + +```typescript +// The old way: configuration objects everywhere +inngest.createFunction( { id: 'sync-user', concurrency: { limit: 10 } }, { event: 'user/created' }, - async ({ event, step }) => { - // Each step is memoized - retries skip completed steps - const user = await step.run('fetch-user', () => - fetchUser(event.data.id) - ) - - // Sleep without blocking workers - await step.sleep('wait-for-propagation', '5m') - - // Continue from where we left off - const synced = await step.run('sync-to-crm', () => - syncToCRM(user) - ) - - return { synced } - } + async ({ event, step }) => { ... } ) - -// Send events to trigger functions -await inngest.send({ - name: 'user/created', - data: { id: '123' } -}) ``` -No queues to manage. No state machines to build. Just functions that work. - -## Features +You can't dictate that. Try saying "createFunction open brace id colon sync-user comma concurrency colon open brace limit colon 10..." -- **Durable Steps** - `step.run()`, `step.sleep()`, `step.waitForEvent()` with memoization -- **Event-Driven** - Trigger functions via events, cron, or webhooks -- **Concurrency Control** - Limits, throttling, debouncing per function or key -- **Automatic Retries** - Exponential backoff with configurable policies -- **TypeScript First** - Full type safety matching Inngest SDK -- **Edge Native** - Runs on Cloudflare's global network +## The Solution -## Architecture +**inngest.do** reimagines workflows for humans: ``` - +----------------------+ - | inngest.do | - | (Cloudflare Worker) | - +----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | StepDO | | EventDO | | ConcurrencyDO | - | (memoization) | | (subscriptions) | | (rate limits) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +-------------------+ - | Cloudflare Queues | - | (event delivery) | - +-------------------+ +Inngest (original) inngest.do +----------------------------------------------------------------- +$0.15-0.50 per 1,000 runs $0 - run your own +Concurrency limits tier-gated Unlimited, you control +Cloud lock-in Your Cloudflare account +createFunction({ config }) inngest\`when user signs up\` +Step IDs required Steps inferred from context +Event schemas boilerplate Natural language triggers ``` -**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each function execution gets its own StepDO for memoization. Event routing uses Queues for reliable delivery. - -## Installation +## One-Click Deploy ```bash -npm install @dotdo/inngest +npx create-dotdo inngest ``` -## Quick Start - -### Define Functions +A durable workflow engine. Running on your infrastructure. Say what you want. ```typescript -import { Inngest } from '@dotdo/inngest' +import { Inngest } from 'inngest.do' -const inngest = new Inngest({ id: 'my-app' }) +export default Inngest({ + name: 'my-app', + domain: 'workflows.my-app.com', +}) +``` -// Simple function -const helloWorld = inngest.createFunction( - { id: 'hello-world' }, - { event: 'test/hello' }, - async ({ event }) => { - return { message: `Hello ${event.data.name}!` } - } -) +## Features -// Multi-step function -const processOrder = inngest.createFunction( - { id: 'process-order' }, - { event: 'order/created' }, - async ({ event, step }) => { - const validated = await step.run('validate', () => - validateOrder(event.data.order) - ) - - const charged = await step.run('charge', () => - chargePayment(validated) - ) - - await step.run('notify', () => - sendConfirmation(charged) - ) - - return { orderId: charged.id, status: 'completed' } - } -) +### Durable Workflows + +```typescript +// Define workflows naturally +const syncUser = await inngest`when user signs up` + .run('fetch user', () => fetchUser(event.data.id)) + .sleep('5 minutes') + .run('sync to CRM', () => syncToCRM(user)) + .run('send welcome email', () => sendEmail(user)) + +// AI infers step boundaries +await inngest`when order placed, validate payment, reserve inventory, notify warehouse` + +// Long-running workflows just work +await inngest`when trial starts` + .sleep('14 days') + .run('send trial ending reminder', () => notifyTrialEnding()) ``` ### Step Methods ```typescript // Run and memoize a step -const result = await step.run('step-id', async () => { - return await someAsyncOperation() -}) +await inngest`sync user data` + .run('fetch from source', () => fetchData()) + .run('transform data', () => transform(data)) + .run('save to destination', () => save(transformed)) -// Sleep for a duration -await step.sleep('wait-period', '1h') // 1 hour -await step.sleep('brief-pause', '30s') // 30 seconds +// Sleep naturally +await inngest`wait 1 hour then send reminder` +await inngest`pause for 30 seconds` // Sleep until a specific time -await step.sleepUntil('scheduled-time', new Date('2024-12-25')) +await inngest`run on Christmas`.sleepUntil('2024-12-25') -// Wait for an external event -const approval = await step.waitForEvent('wait-for-approval', { - event: 'order/approved', - timeout: '24h', - match: 'data.orderId' -}) +// Wait for external events +await inngest`when order created, wait for approval up to 24 hours` -// Invoke another function -const result = await step.invoke('call-other', { - function: otherFunction, - data: { foo: 'bar' } -}) +// Invoke other workflows +await inngest`process order then notify shipping` ``` ### Concurrency Control ```typescript // Limit concurrent executions -inngest.createFunction( - { - id: 'limited-fn', - concurrency: { limit: 5 } // Max 5 parallel - }, - { event: 'task/run' }, - handler -) +inngest`when task runs`.concurrency(5) // Per-user concurrency -inngest.createFunction( - { - id: 'per-user-fn', - concurrency: { - limit: 1, - key: 'event.data.userId' // One per user - } - }, - { event: 'user/action' }, - handler -) +inngest`when user acts`.concurrency(1, 'per user') -// Rate limiting -inngest.createFunction( - { - id: 'throttled-fn', - throttle: { - limit: 10, - period: '1m' // 10 per minute - } - }, - { event: 'api/call' }, - handler -) +// Rate limiting - reads like English +inngest`when api called`.throttle(10, 'per minute') +inngest`when email sent`.throttle(100, 'per hour') // Debouncing -inngest.createFunction( - { - id: 'debounced-fn', - debounce: { - period: '5s', - key: 'event.data.userId' - } - }, - { event: 'search/query' }, - handler -) +inngest`when search typed`.debounce('500ms') +inngest`when form saved`.debounce('5 seconds', 'per user') ``` ### Event Triggers ```typescript -// Event trigger -{ event: 'user/created' } +// Event triggers - just say it +await inngest`when user signs up` +await inngest`when order placed` +await inngest`when payment fails` -// Cron trigger -{ cron: '0 9 * * *' } // Daily at 9am UTC +// Cron triggers - natural language +await inngest`every day at 9am` +await inngest`every 6 hours` +await inngest`every monday at midnight` // Multiple triggers -[ - { event: 'user/created' }, - { event: 'user/updated' }, - { cron: '0 */6 * * *' } // Every 6 hours -] +await inngest`when user signs up or updates profile` ``` ### Sending Events ```typescript -// Send a single event -await inngest.send({ - name: 'user/created', - data: { userId: '123', email: 'user@example.com' } -}) - -// Send multiple events -await inngest.send([ - { name: 'user/created', data: { userId: '123' } }, - { name: 'email/send', data: { to: 'user@example.com' } } -]) - -// Send with idempotency key -await inngest.send({ - id: 'unique-event-id', - name: 'order/created', - data: { orderId: 'abc' } -}) +// Send events naturally +await inngest`user signed up: ${userId}` +await inngest`order placed: ${orderId}` + +// Batch events +await inngest` + user ${userId} signed up + send welcome email to ${email} +` + +// With idempotency +await inngest`order ${orderId} created`.idempotent() ``` ### Serve with Hono ```typescript import { Hono } from 'hono' -import { serve } from '@dotdo/inngest/hono' +import { serve } from 'inngest.do/hono' const app = new Hono() -app.all('/api/inngest', serve({ - client: inngest, - functions: [syncUser, processOrder] -})) +app.all('/api/inngest', serve()) export default app ``` @@ -277,18 +209,12 @@ export default app ## Retry Policies ```typescript -inngest.createFunction( - { - id: 'retry-example', - retries: { - attempts: 5, - backoff: 'exponential', - maxDelay: '1h' - } - }, - { event: 'task/run' }, - handler -) +// Natural language retries +inngest`process payment`.retry(5, 'exponential') +inngest`send email`.retry(3, 'max 1 hour delay') + +// Or in the workflow definition +await inngest`when payment fails, retry 3 times then notify support` ``` Default retry policy: @@ -299,47 +225,160 @@ Default retry policy: ## Error Handling ```typescript -inngest.createFunction( - { id: 'error-handling' }, - { event: 'task/run' }, - async ({ event, step }) => { - try { - await step.run('risky-operation', () => riskyOp()) - } catch (error) { - // Handle or re-throw - await step.run('fallback', () => fallbackOp()) - } - } -) +// Try-catch in natural language +await inngest`when task runs` + .run('risky operation', () => riskyOp()) + .onError('run fallback', () => fallbackOp()) + +// Or with explicit handling +await inngest`process data or fallback to cached version` ``` -## The Rewrites Ecosystem +## Architecture -inngest.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare: +``` +Internet --> Cloudflare Worker --> Durable Objects --> SQLite + | | | + Event Router Step State Memoization + Concurrency Durability +``` -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| **inngest.do** | Inngest | Workflows/Jobs for AI | -| kafka.do | Kafka | Event streaming for AI | -| nats.do | NATS | Messaging for AI | +### Durable Object per Workflow -Each rewrite follows the same pattern: -- Durable Objects for state -- SQLite for persistence -- Cloudflare Queues for messaging -- Compatible API with the original +``` +WorkflowDO (execution state) + | + +-- StepDO (step memoization) + | |-- SQLite: Step results + | +-- Retry state + | + +-- EventDO (subscriptions) + | |-- SQLite: Event matching + | +-- Correlation IDs + | + +-- ConcurrencyDO (rate limits) + |-- SQLite: Counters + +-- Throttle/debounce state +``` + +### Storage + +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active workflows, step state | <10ms | +| **Warm** | R2 | Completed workflows | <100ms | +| **Cold** | R2 Archive | Historical runs | <1s | + +## vs Inngest + +| Feature | Inngest | inngest.do | +|---------|---------|-----------| +| **Pricing** | $0.15-0.50 per 1K runs | $0 - run your own | +| **Concurrency** | Tier-gated limits | Unlimited, you control | +| **Architecture** | Inngest Cloud | Edge-native, global | +| **API** | `createFunction({ config })` | `inngest\`when user signs up\`` | +| **Step IDs** | Required strings | Inferred from context | +| **Triggers** | Event schema objects | Natural language | +| **Data Location** | Inngest's servers | Your Cloudflare account | +| **Customization** | Limited | Code it yourself | +| **Lock-in** | Proprietary | MIT licensed | + +## Use Cases + +### Background Jobs + +```typescript +// Process uploads in the background +await inngest`when file uploaded, resize images, generate thumbnails, update database` + +// AI processes naturally +await inngest`when document uploaded, extract text, summarize, index for search` +``` + +### Scheduled Tasks + +```typescript +// Cron jobs in plain English +await inngest`every morning at 6am, send daily digest` +await inngest`every hour, sync inventory with warehouse` +await inngest`first monday of month, generate reports` +``` + +### Webhooks + +```typescript +// Process webhooks reliably +await inngest`when stripe payment succeeds, provision account, send receipt` +await inngest`when github push, run tests, deploy if passing` +``` + +### Multi-Step Workflows + +```typescript +// E-commerce order flow +await inngest`when order placed` + .run('validate payment', () => validatePayment()) + .run('reserve inventory', () => reserveInventory()) + .run('notify warehouse', () => notifyWarehouse()) + .sleep('1 hour') + .run('send confirmation', () => sendConfirmation()) + +// User onboarding +await inngest`when user signs up` + .run('create account', () => createAccount()) + .sleep('5 minutes') + .run('send welcome email', () => sendWelcome()) + .sleep('1 day') + .run('check if active', () => checkActivity()) + .run('send tips if inactive', () => sendTips()) +``` + +## Promise Pipelining + +Chain workflows without `Promise.all`: + +```typescript +// One network round trip +const result = await inngest`process order ${orderId}` + .run('validate', validateOrder) + .run('charge', chargePayment) + .run('fulfill', fulfillOrder) + .map(item => inngest`ship ${item.id}`) +``` ## Why Cloudflare? -1. **Global Edge** - Functions run close to users +1. **Global Edge** - Workflows run close to users 2. **No Cold Starts** - Durable Objects stay warm 3. **Unlimited Duration** - No execution timeouts 4. **Built-in Queues** - Reliable event delivery 5. **Single-Threaded DO** - No race conditions in step execution +## Why Natural Language? + +### The Dictation Test + +Every API should pass the dictation test: could you describe it to a voice assistant? + +```typescript +// OLD: Try dictating this +inngest.createFunction( + { id: 'sync-user', concurrency: { limit: 10 } }, + { event: 'user/created' }, + async ({ event, step }) => { ... } +) + +// NEW: Natural as speech +inngest`when user created, sync to CRM with concurrency 10` +``` + +### Benefits + +- **Onboarding** - New developers understand immediately +- **Maintenance** - Code reads like documentation +- **AI Integration** - LLMs can generate and modify workflows +- **Accessibility** - Voice coding becomes possible + ## Related Domains - **workflows.do** - Workflow orchestration @@ -347,6 +386,30 @@ Each rewrite follows the same pattern: - **jobs.do** - Background job queue - **cron.do** - Scheduled tasks +## Contributing + +inngest.do is open source under the MIT license. + +```bash +git clone https://github.com/dotdo/inngest.do +cd inngest.do +pnpm install +pnpm test +``` + ## License -MIT +MIT License + +--- + +

+ Workflows should read like sentences. +
+ Durable. Event-driven. Natural language. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/intercom/README.md b/rewrites/intercom/README.md index 1e93df83..4c94796c 100644 --- a/rewrites/intercom/README.md +++ b/rewrites/intercom/README.md @@ -1,232 +1,193 @@ # intercom.do -

- Customer Messaging. AI-First. Yours to Own. -

+> Customer Messaging. AI-First. Yours to Own. -

- npm version - npm downloads - license -

+Intercom charges $74-153/seat/month. AI features cost extra. Your conversations live on their servers. Your customer data belongs to them. For a 10-person support team: **$740-1,530/month** before AI features. ---- +**intercom.do** is the open-source alternative. AI built-in. Deploy in minutes. Own it forever. + +## AI-Native API + +```typescript +import { intercom } from 'intercom.do' // Full SDK +import { intercom } from 'intercom.do/tiny' // Minimal client +import { intercom } from 'intercom.do/inbox' // Inbox-only operations +``` + +Natural language for customer support: + +```typescript +import { intercom } from 'intercom.do' + +// Talk to it like a support manager +const waiting = await intercom`conversations waiting for reply` +const unassigned = await intercom`unassigned conversations older than 2 hours` +const vips = await intercom`messages from enterprise customers today` + +// Chain like sentences +await intercom`unassigned conversations` + .map(conv => intercom`assign ${conv} to available agent`) -Re-imagining customer messaging for the AI era. One-click deploy your own Intercom. +// AI responds, escalates when needed +await intercom`reply to user-123 about password reset` + .escalate('billing dispute') // routes to human if needed +``` ## The Problem -**Intercom charges $74-153/seat/month.** AI features cost extra. Your conversations live on their servers. Your customer data belongs to them. +Intercom dominates customer messaging with aggressive pricing: -| What Intercom Charges | What You Get | -|-----------------------|--------------| -| $74/seat/month | Basic messaging | -| $153/seat/month | Advanced automation | -| +$0.99/resolution | Fin AI Agent | -| Enterprise pricing | Custom bots | +| What Intercom Charges | The Reality | +|-----------------------|-------------| +| **Basic Seats** | $74/seat/month | +| **Advanced Seats** | $153/seat/month | +| **AI Resolutions** | $0.99 per resolution | +| **Custom Bots** | Enterprise pricing only | +| **Data Ownership** | Your data on their servers | +| **Vendor Lock-in** | Migration is painful | -For a 10-person support team: **$740-1,530/month** before AI features. +### The Intercom Tax -And you still don't own your data. +- Per-seat pricing punishes growth +- AI features cost extra on top +- Customer data is hostage +- Rate limits on your own conversations ## The Solution -**intercom.do** is open-source customer messaging with AI built-in. Deploy it once, own it forever. +**intercom.do** reimagines customer messaging: + +``` +Intercom intercom.do +---------------------------------------------------------- +$74-153/seat/month Deploy in minutes +$0.99/AI resolution AI included +Their servers Your Cloudflare account +Rate limited Unlimited +Vendor lock-in Open source, MIT licensed +``` + +## One-Click Deploy ```bash npx create-dotdo intercom ``` -That's it. Your own messenger. Your own inbox. Your data. +Your own messenger. Your own inbox. Your data. ```typescript -import { IntercomClient } from 'intercom.do' - -const intercom = new IntercomClient('https://support.yourcompany.com') +import { Intercom } from 'intercom.do' -// AI handles first response -await intercom.conversations.send({ - to: 'user@example.com', - message: 'How do I reset my password?', - // AI responds instantly, escalates if needed +export default Intercom({ + name: 'acme-support', + domain: 'support.acme.com', }) ``` ---- - ## Features -### Messenger Widget +### Conversations -Drop-in JavaScript widget that works anywhere: - -```html - - +```typescript +// Find conversations naturally +const open = await intercom`open conversations` +const waiting = await intercom`conversations waiting over 1 hour` +const urgent = await intercom`vip customers needing response` + +// AI infers what you need +await intercom`conversations from alice@example.com` // returns history +await intercom`unread messages today` // returns count and list +await intercom`escalated to billing team` // returns filtered ``` -- Customizable appearance (colors, position, launcher icon) -- Mobile-responsive design -- Pre-chat forms and qualification -- File attachments and rich media -- Typing indicators and read receipts - ### Team Inbox -Real-time collaborative inbox for your support team: +```typescript +// Assignments are one line +await intercom`assign conversation-123 to Sarah` +await intercom`route new conversations to available agents` -| Feature | Description | -|---------|-------------| -| **Unified Inbox** | All conversations from web, email, and API in one place | -| **Assignment** | Round-robin, load-balanced, or manual assignment | -| **Collision Detection** | See when teammates are viewing/typing | -| **Saved Replies** | Templated responses with variable substitution | -| **Internal Notes** | Private team communication within conversations | -| **Conversation Tags** | Organize and filter by custom categories | -| **SLA Timers** | Track response and resolution times | +// Bulk operations just work +await intercom`conversations waiting over 2 hours` + .map(conv => intercom`assign ${conv} to next available`) +``` ### Help Center -Self-service knowledge base with AI-powered search: - ```typescript -// Create articles programmatically -await intercom.articles.create({ - title: 'Getting Started Guide', - body: '

Welcome!

Here is how to get started...

', - state: 'published', - collection_id: 'getting-started' -}) +// Create articles naturally +await intercom`article: Getting Started with password reset steps` // AI-powered search -const results = await intercom.articles.search({ - query: 'how do I reset my password', - // Returns ranked results with relevance scores -}) -``` +await intercom`search articles about billing` +await intercom`suggest article for user asking about refunds` -- Markdown and rich text editor -- Collections and sections for organization -- SEO-optimized article pages -- FTS5 full-text search with ranking -- Suggested articles in messenger +// Bulk operations +await intercom`outdated articles` + .map(article => intercom`flag ${article} for review`) +``` ### Bots & Workflows -Visual workflow builder for automation: - ```typescript -// Define a qualification bot -const bot = intercom.bots.create({ - name: 'Lead Qualifier', - triggers: [{ event: 'conversation_started' }], - steps: [ - { type: 'message', content: 'Hi! What brings you here today?' }, - { type: 'buttons', options: ['Sales', 'Support', 'Partnership'] }, - { type: 'route', rules: [ - { match: 'Sales', assign_to: 'sales-team' }, - { match: 'Support', assign_to: 'ai-agent' }, - { match: 'Partnership', assign_to: 'partnerships@company.com' } - ]} - ] -}) -``` - -- No-code workflow builder UI -- Conditional branching logic -- User attribute collection -- Team and individual routing -- Scheduled messages -- Event-triggered automation +// Just describe what you want +await intercom`when new conversation, ask Sales or Support` +await intercom`route Sales to sales team, Support to AI agent` ---- +// AI builds the workflow +await intercom`create bot: qualify leads then route to sales` +``` ## AI Support Agents -AI agents handle L1 support out of the box. No per-resolution fees. +AI handles L1 support. No per-resolution fees. ### How It Works ``` -Customer Question - | - v - [AI Agent] - | - Can answer? - / \ - Yes No - | | - v v -Respond Escalate -instantly to human +Customer Question --> AI Agent --> Can answer? --> Yes --> Respond instantly + | + No --> Escalate to human ``` ### Knowledge Sources -AI learns from multiple sources: - ```typescript -// Train from your help center -await intercom.ai.train({ - sources: [ - { type: 'help_center' }, // Your articles - { type: 'conversations', days: 90 }, // Past conversations - { type: 'url', url: 'https://docs.yourcompany.com' }, - { type: 'pdf', path: './product-manual.pdf' } - ] -}) +// Train AI naturally +await intercom`learn from help center` +await intercom`learn from last 90 days of conversations` +await intercom`learn from docs.yourcompany.com` + +// Or batch it +await intercom`train on help center, recent conversations, and docs site` ``` ### Intelligent Escalation -AI knows when to hand off: - ```typescript -// Configure escalation rules -intercom.ai.configure({ - escalate_when: [ - 'customer_requests_human', - 'sentiment_negative', - 'confidence_below_0.7', - 'vip_customer', - 'billing_dispute' - ], - working_hours: { - timezone: 'America/Los_Angeles', - hours: { start: 9, end: 17 }, - days: ['mon', 'tue', 'wed', 'thu', 'fri'] - }, - outside_hours: 'collect_info_and_queue' -}) +// AI knows when to hand off +await intercom`escalate billing disputes to human` +await intercom`escalate when customer asks for manager` +await intercom`escalate negative sentiment to senior agents` + +// Working hours just work +await intercom`during business hours route to team, after hours collect info` ``` ### Conversation Handoff -Seamless transition from AI to human: - ```typescript -// AI provides context to human agent -{ - conversation_id: 'conv_123', - customer: { name: 'Alice', email: 'alice@example.com' }, - ai_summary: 'Customer asking about enterprise pricing. Mentioned 500 seats.', - ai_sentiment: 'positive', - suggested_response: 'Hi Alice! Happy to discuss enterprise pricing...', - relevant_articles: ['pricing-guide', 'enterprise-features'] -} -``` +// AI summarizes for human agents +await intercom`summarize conversation-123 for handoff` +// Returns: "Customer asking about enterprise pricing. Mentioned 500 seats. Positive sentiment." ---- +// Suggest responses +await intercom`suggest response for conversation-123` +``` -## Real-Time at the Edge +## Architecture -Built on Cloudflare Durable Objects with WebSocket support. Conversations happen at the edge, not in a distant data center. +Built on Cloudflare Durable Objects. Conversations happen at the edge. ### Global Performance @@ -237,283 +198,109 @@ Built on Cloudflare Durable Objects with WebSocket support. Conversations happen | WebSocket connection | Regional edge | Single region | | Offline resilience | Queue & sync | Lost messages | -### Architecture +### Durable Object per Workspace ``` -┌─────────────────────────────────────────────────────────────────────┐ -│ Your Customers │ -├─────────────────┬─────────────────┬─────────────────────────────────┤ -│ Web Widget │ Mobile SDK │ Email / API │ -├─────────────────┴─────────────────┴─────────────────────────────────┤ -│ Cloudflare Edge Network │ -├─────────────────────────────────────────────────────────────────────┤ -│ intercom.do Worker │ -│ ├── ConversationDO (WebSocket hub, message history) │ -│ ├── InboxDO (team view, assignments, routing) │ -│ ├── ArticleDO (help center, FTS5 search) │ -│ ├── UserDO (customer profiles, attributes) │ -│ └── AIDo (AI agent, knowledge base, embeddings) │ -├─────────────────────────────────────────────────────────────────────┤ -│ SQLite (hot) │ R2 (attachments) │ Vectorize (AI search) │ -└─────────────────────────────────────────────────────────────────────┘ +WorkspaceDO (config, users, roles, teams) + | + +-- ConversationDO (WebSocket hub, message history) + | |-- SQLite: Messages (hot) + | +-- R2: Attachments + | + +-- InboxDO (team view, assignments, routing) + | |-- SQLite: Assignment state + | + +-- ArticleDO (help center, FTS5 search) + | |-- SQLite: Articles with full-text search + | + +-- UserDO (customer profiles, attributes) + | |-- SQLite: Contact database + | + +-- AIAgentDO (AI agent, knowledge base, embeddings) + |-- Vectorize: Knowledge embeddings ``` -Each conversation runs in its own Durable Object: - -```typescript -// ConversationDO handles real-time messaging -export class ConversationDO extends DurableObject { - private connections = new Map() - - async webSocketMessage(ws: WebSocket, message: string) { - const { type, data } = JSON.parse(message) - - if (type === 'message') { - // Persist to SQLite - await this.sql.exec( - 'INSERT INTO messages (id, author, content, created_at) VALUES (?, ?, ?, ?)', - [data.id, data.author, data.content, Date.now()] - ) - - // Broadcast to all participants - for (const [id, conn] of this.connections) { - conn.send(JSON.stringify({ type: 'message', data })) - } - - // Trigger AI response if needed - if (data.author === 'customer' && this.aiEnabled) { - await this.env.AI_AGENT.respond(this.id, data) - } - } - } -} -``` - ---- - -## API Compatible +### Storage Tiers -Drop-in replacement for Intercom's REST API. Existing integrations just work. +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active conversations, recent messages | <10ms | +| **Warm** | R2 + SQLite Index | Archived conversations (30+ days) | <100ms | +| **Cold** | R2 Archive | Long-term retention (1+ year) | <1s | -### REST API +## vs Intercom -```bash -# List conversations -curl https://support.yourcompany.com/api/conversations \ - -H "Authorization: Bearer $API_KEY" - -# Send a message -curl -X POST https://support.yourcompany.com/api/conversations/123/reply \ - -H "Authorization: Bearer $API_KEY" \ - -H "Content-Type: application/json" \ - -d '{"type": "admin", "body": "Thanks for reaching out!"}' - -# Create a user -curl -X POST https://support.yourcompany.com/api/contacts \ - -H "Authorization: Bearer $API_KEY" \ - -d '{"email": "alice@example.com", "name": "Alice"}' -``` +| Feature | Intercom | intercom.do | +|---------|----------|-------------| +| **Base Price** | $74-153/seat/month | Deploy in minutes | +| **AI Resolutions** | $0.99 each | Included | +| **Architecture** | Centralized SaaS | Edge-native, global | +| **Data Location** | Their servers | Your Cloudflare account | +| **Custom Bots** | Enterprise only | Included | +| **Rate Limits** | Aggressive | Unlimited | +| **Lock-in** | Painful migration | MIT licensed | -### TypeScript SDK +## Customer Communication ```typescript -import { IntercomClient } from 'intercom.do' +// Messages read like you'd say them +await intercom`message alice@example.com about their subscription` +await intercom`send newsletter to pro customers` -const client = new IntercomClient({ - baseUrl: 'https://support.yourcompany.com', - apiKey: process.env.INTERCOM_API_KEY -}) +// Bulk outreach +await intercom`customers who signed up this week` + .map(user => intercom`welcome message to ${user}`) -// Conversations -const conversations = await client.conversations.list({ state: 'open' }) -await client.conversations.reply(convId, { body: 'Hello!' }) -await client.conversations.assign(convId, { assignee_id: 'admin_123' }) -await client.conversations.close(convId) - -// Contacts -const contact = await client.contacts.create({ email: 'user@example.com' }) -await client.contacts.update(contact.id, { custom_attributes: { plan: 'pro' } }) - -// Articles -const articles = await client.articles.search({ query: 'billing' }) -await client.articles.create({ title: 'FAQ', body: '...' }) - -// Events -await client.events.create({ - event_name: 'purchased', - user_id: 'user_123', - metadata: { plan: 'pro', amount: 99 } -}) +// Notify your team +await intercom`new conversation from enterprise customer` + .notify('slack', '#support-priority') ``` -### Webhooks +## Analytics ```typescript -// Receive real-time events -app.post('/webhooks/intercom', async (req, res) => { - const event = req.body - - switch (event.topic) { - case 'conversation.created': - await notifySlack(event.data) - break - case 'conversation.user.replied': - await updateCRM(event.data) - break - case 'conversation.admin.closed': - await triggerSurvey(event.data) - break - } - - res.sendStatus(200) -}) +// Query your metrics naturally +await intercom`response time this week` +await intercom`resolution rate by agent` +await intercom`busiest hours last month` + +// Trend analysis +await intercom`conversation volume trend` +await intercom`customer satisfaction over time` ``` ---- - -## One-Click Deploy +## Deployment Options ### Cloudflare Workers ```bash -# Create your intercom.do instance npx create-dotdo intercom - -# Follow the prompts: -# - Workspace name -# - Custom domain (optional) -# - AI model preference - -# Deploy -npm run deploy -``` - -### Configuration - -```jsonc -// wrangler.jsonc -{ - "name": "support", - "main": "src/index.ts", - "compatibility_date": "2025-01-01", - "compatibility_flags": ["nodejs_compat"], - "durable_objects": { - "bindings": [ - { "name": "CONVERSATION", "class_name": "ConversationDO" }, - { "name": "INBOX", "class_name": "InboxDO" }, - { "name": "ARTICLE", "class_name": "ArticleDO" }, - { "name": "USER", "class_name": "UserDO" }, - { "name": "AI_AGENT", "class_name": "AIAgentDO" } - ] - }, - "migrations": [ - { "tag": "v1", "new_sqlite_classes": ["ConversationDO", "InboxDO", "ArticleDO", "UserDO", "AIAgentDO"] } - ], - "vectorize": { - "bindings": [{ "binding": "VECTORIZE", "index_name": "knowledge-base" }] - }, - "ai": { "binding": "AI" }, - "r2_buckets": [ - { "binding": "ATTACHMENTS", "bucket_name": "intercom-attachments" } - ] -} ``` -### Custom Domain +### Private Cloud ```bash -# Add your support domain -wrangler domains add support.yourcompany.com +# Deploy to your infrastructure +docker run -p 8787:8787 dotdo/intercom ``` ---- - -## Migration from Intercom - -One-command migration from Intercom: +### On-Premises ```bash -npx intercom.do migrate --from-intercom - -# Migrates: -# - All conversations and messages -# - Contact database -# - Help center articles -# - Custom attributes -# - Team members and permissions -# - Saved replies -# - Tags and segments -``` - -### Data Export - -```typescript -import { IntercomMigration } from 'intercom.do/migrate' - -const migration = new IntercomMigration({ - source: { - apiKey: process.env.INTERCOM_API_KEY // Your Intercom API key - }, - destination: { - baseUrl: 'https://support.yourcompany.com', - apiKey: process.env.INTERCOM_DO_API_KEY - } -}) - -// Preview what will be migrated -const preview = await migration.preview() -console.log(`${preview.conversations} conversations`) -console.log(`${preview.contacts} contacts`) -console.log(`${preview.articles} articles`) - -// Run migration -await migration.run({ - conversations: true, - contacts: true, - articles: true, - onProgress: (progress) => console.log(`${progress.percent}% complete`) -}) +./intercom-do-install.sh --on-premises ``` ---- - -## Self-Hosting - -### Docker - -```bash -docker run -d \ - -p 8080:8080 \ - -e DATABASE_URL=sqlite:./data/intercom.db \ - -v intercom-data:/data \ - drivly/intercom.do -``` +## Migration from Intercom -### Node.js +One-command migration: ```bash -git clone https://github.com/drivly/intercom.do -cd intercom.do -npm install -npm run build -npm start +npx intercom.do migrate --from-intercom ``` ---- +Migrates everything: conversations, contacts, articles, custom attributes, team members, saved replies, tags, segments. -## Pricing Comparison - -| Feature | Intercom | intercom.do | -|---------|----------|-------------| -| Base price | $74-153/seat/month | $0 (self-host) | -| AI responses | $0.99/resolution | Included | -| Help center | Included | Included | -| Custom bots | Enterprise only | Included | -| Data ownership | Theirs | Yours | -| API limits | Rate limited | Unlimited | -| White label | Enterprise only | Always | - -### Cost Example +## Cost Example **10-person support team, 1,000 AI resolutions/month:** @@ -521,53 +308,82 @@ npm start |-|----------|-------------| | Seats | $740-1,530 | $0 | | AI resolutions | $990 | $0 | -| **Monthly total** | **$1,730-2,520** | **$5** (Workers) | +| **Monthly total** | **$1,730-2,520** | **~$5** (Workers) | | **Annual savings** | - | **$20,700-30,180** | ---- +## Use Cases -## Documentation +### Customer Support -| Guide | Description | -|-------|-------------| -| [Quick Start](./docs/quickstart.mdx) | Get running in 5 minutes | -| [Widget Customization](./docs/widget.mdx) | Styling and configuration | -| [AI Training](./docs/ai-training.mdx) | Teaching AI from your knowledge base | -| [Team Inbox](./docs/inbox.mdx) | Managing conversations | -| [Help Center](./docs/help-center.mdx) | Building your knowledge base | -| [Bots & Workflows](./docs/bots.mdx) | Automation and routing | -| [API Reference](./docs/api.mdx) | REST API documentation | -| [Migration Guide](./docs/migration.mdx) | Moving from Intercom | -| [Self-Hosting](./docs/self-hosting.mdx) | Running on your infrastructure | +Unified inbox, AI-first response, seamless escalation. Support teams focus on complex issues while AI handles the rest. ---- +### Sales Qualification + +```typescript +await intercom`new visitor on pricing page` + .map(visitor => intercom`qualify ${visitor} for sales`) + .map(lead => intercom`route ${lead} to sales team`) +``` + +### Proactive Outreach + +```typescript +// Engagement automation +await intercom`users who haven't logged in this week` + .map(user => intercom`send re-engagement message to ${user}`) + +await intercom`trial users expiring in 3 days` + .map(user => intercom`send trial extension offer to ${user}`) +``` + +## Roadmap + +### Core Messaging +- [x] Real-time conversations +- [x] Team inbox +- [x] AI responses +- [x] Help center +- [x] Bots & workflows +- [ ] Mobile SDK +- [ ] Email integration + +### AI +- [x] L1 support automation +- [x] Knowledge base training +- [x] Intelligent escalation +- [ ] Sentiment analysis +- [ ] Predictive support + +### Integrations +- [ ] Slack notifications +- [ ] CRM sync +- [ ] Zapier/n8n +- [ ] Custom webhooks ## Contributing -Contributions welcome! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. +intercom.do is open source under the MIT license. ```bash -# Development -git clone https://github.com/drivly/intercom.do +git clone https://github.com/dotdo/intercom.do cd intercom.do -npm install -npm run dev - -# Tests -npm test -npm run test:e2e +pnpm install +pnpm test ``` ---- - ## License -MIT - see [LICENSE](./LICENSE) +MIT License - Customer messaging for everyone. ---

- Customer messaging without the hostage pricing. + The per-seat pricing ends here.
- Built on Cloudflare Workers. Powered by AI. Owned by you. + AI-first. Edge-native. Customer-owned. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/jira/README.md b/rewrites/jira/README.md index acd35bac..7a98a59a 100644 --- a/rewrites/jira/README.md +++ b/rewrites/jira/README.md @@ -4,7 +4,36 @@ Jira became synonymous with issue tracking. It also became synonymous with slowness, complexity, and eye-watering enterprise pricing. Every developer knows the pain: waiting for pages to load, clicking through endless configuration screens, paying $8.15/user/month that somehow balloons to six figures annually. -**jira.do** reimagines issue tracking for the AI era. Deploy your own Jira. JQL-compatible queries. AI that triages bugs, writes acceptance criteria, and estimates story points while you sleep. +**jira.do** reimagines issue tracking for the AI era. Deploy your own Jira. Natural language queries. AI that triages bugs, writes acceptance criteria, and estimates story points while you sleep. + +## AI-Native API + +```typescript +import { jira } from 'jira.do' // Full SDK +import { jira } from 'jira.do/tiny' // Minimal client +import { jira } from 'jira.do/agile' // Scrum/Kanban operations +``` + +Natural language for project management: + +```typescript +import { jira } from 'jira.do' + +// Talk to it like a colleague +const bugs = await jira`critical bugs blocking release` +const myWork = await jira`my issues in current sprint` +const blocked = await jira`what's blocked and why?` + +// Chain like sentences +await jira`untriaged bugs this week` + .map(bug => jira`triage ${bug}`) + .map(bug => jira`assign ${bug} to on-call`) + +// Ship a release with one pipeline +await jira`completed since last release` + .map(issue => jira`add to release notes ${issue}`) + .notify(`#releases`) +``` ## The Problem @@ -38,9 +67,9 @@ $8-16/user/month $0 - run your own Slow page loads Instant (edge computing) Complex configuration Convention over configuration AI as premium add-on AI-native from day one -JQL queries JQL queries (compatible!) +JQL queries Natural language Atlassian servers Your Cloudflare account -Groovy automations TypeScript workflows +Groovy automations Say what you want Their ecosystem Open ecosystem ``` @@ -52,205 +81,89 @@ npx create-dotdo jira That's it. Your own Jira. Running on Cloudflare. Ready for issues. -Or deploy to an existing workers.do project: - -```bash -npx dotdo add jira -``` - -## The workers.do Way - -You're building a product. Your team needs issue tracking. Jira wants $100k/year and your developers spend more time waiting for pages to load than writing code. There's a better way. - -**Natural language. Tagged templates. AI agents that work.** - ```typescript -import { jira } from 'jira.do' -import { priya, ralph, quinn } from 'agents.do' +import { Jira } from 'jira.do' -// Talk to your issue tracker like a human -const bugs = await jira`find critical bugs from ${sprint}` -const blocked = await jira`which stories are blocked and why?` -const myWork = await jira`what am I working on?` -``` - -**Promise pipelining - chain work without Promise.all:** - -```typescript -// Ship a release with one pipeline -const shipped = await jira`find completed issues in ${cycle}` - .map(issue => mark`write release notes for ${issue}`) - .map(notes => priya`review ${notes}`) - -// AI triage pipeline -const triaged = await jira`show untriaged issues` - .map(issue => priya`analyze and prioritize ${issue}`) - .map(issue => ralph`estimate ${issue}`) - .map(issue => quinn`identify test cases for ${issue}`) +export default Jira({ + name: 'my-startup', + domain: 'issues.my-startup.com', +}) ``` -One network round trip. Record-replay pipelining. Workers working for you. - ## Features -### Issue Tracking - -Everything you expect from a modern issue tracker: - -| Feature | Description | -|---------|-------------| -| **Issues** | Bugs, stories, tasks, epics, subtasks | -| **Projects** | Organize work by team or product | -| **Boards** | Kanban and Scrum boards | -| **Backlogs** | Prioritized work queues | -| **Sprints** | Time-boxed iterations | -| **Roadmaps** | Visual timeline planning | -| **Components** | Categorize by system area | -| **Versions** | Track releases | -| **Labels** | Flexible tagging | -| **Custom Fields** | Your data, your schema | - -### JQL Compatible - -Write JQL exactly like Jira: +### Issues -```sql -project = BACKEND AND status = "In Progress" AND assignee = currentUser() -ORDER BY priority DESC, created ASC -``` - -Advanced queries work too: - -```sql -project in (FRONTEND, BACKEND, MOBILE) - AND resolution = Unresolved - AND priority in (Critical, High) - AND created >= -30d - AND labels in (security, performance) -ORDER BY updated DESC +```typescript +// Create issues naturally +await jira`create bug "Login broken on Safari" priority high` +await jira`new story "User can export dashboard as PDF"` +await jira`add task "Update dependencies" to current sprint` + +// Find issues by talking +await jira`my open bugs` +await jira`unassigned stories in backend` +await jira`blockers for release 2.0` +await jira`what did I close this week?` + +// AI infers what you need +await jira`BACKEND-123` // returns the issue +await jira`comments on BACKEND-123` // returns comments +await jira`history of BACKEND-123` // returns changelog ``` -The JQL parser compiles to SQLite - your queries run locally, instantly. - -### Agile Boards - -#### Scrum Board +### Sprints ```typescript -import { Board, Sprint } from 'jira.do' - -export const backendBoard = Board({ - name: 'Backend Scrum', - type: 'scrum', - project: 'BACKEND', - columns: ['To Do', 'In Progress', 'Code Review', 'Testing', 'Done'], - swimlanes: 'assignee', - quickFilters: [ - { name: 'Only My Issues', jql: 'assignee = currentUser()' }, - { name: 'Critical', jql: 'priority = Critical' }, - ], -}) - -// Sprint management -const currentSprint = await Sprint.active('BACKEND') -await Sprint.create({ - name: 'Sprint 42', - startDate: '2025-01-13', - endDate: '2025-01-24', - goal: 'Complete authentication refactor', -}) +// Sprint management as conversation +await jira`start sprint "Auth Refactor" two weeks` +await jira`current sprint progress` +await jira`what's left in this sprint?` +await jira`close sprint and move incomplete to backlog` + +// Sprint planning with AI +await jira`plan next sprint 40 points` // AI picks from backlog +await jira`can we fit BACKEND-456 in this sprint?` ``` -#### Kanban Board +### Boards ```typescript -export const supportBoard = Board({ - name: 'Support Kanban', - type: 'kanban', - project: 'SUPPORT', - columns: [ - { name: 'Inbox', limit: null }, - { name: 'Triage', limit: 5 }, - { name: 'Working', limit: 3 }, - { name: 'Review', limit: 2 }, - { name: 'Done', limit: null }, - ], -}) +// Scrum or Kanban, just ask +await jira`show backend board` +await jira`what's in code review?` +await jira`move BACKEND-123 to testing` + +// Board health +await jira`blocked items on frontend board` +await jira`what's been in progress too long?` ``` ### Workflows -Define issue workflows in TypeScript: - ```typescript -import { Workflow, Transition } from 'jira.do' - -export const bugWorkflow = Workflow({ - name: 'Bug Workflow', - statuses: ['Open', 'In Progress', 'In Review', 'Testing', 'Done'], - transitions: [ - Transition('Open', 'In Progress', { - validators: [{ field: 'assignee', isNotEmpty: true }], - }), - Transition('In Progress', 'In Review', { - postFunctions: [ - { action: 'addLabel', value: 'needs-review' }, - { action: 'notify', recipients: ['lead'] }, - ], - }), - Transition('In Review', 'Testing', { - conditions: [{ field: 'reviewer', approved: true }], - }), - Transition('Testing', 'Done', { - conditions: [{ field: 'qa', approved: true }], - postFunctions: [ - { action: 'setField', field: 'resolution', value: 'Fixed' }, - ], - }), - ], -}) +// Transition issues naturally +await jira`start work on BACKEND-123` +await jira`BACKEND-123 ready for review` +await jira`close BACKEND-123 as fixed` + +// Bulk transitions +await jira`close all done issues in sprint` +await jira`reopen BACKEND-100 through BACKEND-110` ``` ### Automations -Automate repetitive tasks with TypeScript: - ```typescript -import { Automation } from 'jira.do' - -// Auto-assign based on component -export const componentAssignment = Automation({ - name: 'Component Auto-Assignment', - trigger: { event: 'issue.created' }, - conditions: [ - { field: 'assignee', isEmpty: true }, - { field: 'components', isNotEmpty: true }, - ], - actions: async (issue) => { - const component = issue.components[0] - const lead = await component.getLead() - await issue.update({ assignee: lead }) - }, -}) - -// SLA breach warning -export const slaWarning = Automation({ - name: 'SLA Breach Warning', - trigger: { event: 'schedule', cron: '*/15 * * * *' }, - actions: async () => { - const issues = await Issue.query(` - priority = Critical - AND status != Done - AND created <= -4h - AND labels not in (sla-warned) - `) - for (const issue of issues) { - await issue.addLabel('sla-warned') - await issue.comment('SLA breach imminent. Escalating.') - await Slack.notify('#escalations', `Critical issue ${issue.key} approaching SLA breach`) - } - }, -}) +// Describe what you want, AI makes it happen +await jira`when bugs are created, assign to on-call` +await jira`notify #frontend when frontend issues are critical` +await jira`escalate if critical bugs sit untouched for 4 hours` + +// Or chain it yourself +await jira`new critical bugs` + .each(bug => jira`assign ${bug} to on-call`) + .notify(`#escalations`) ``` ## AI-Native Issue Tracking @@ -259,219 +172,102 @@ Here's where jira.do gets interesting. AI isn't an add-on. It's how modern issue ### AI Triage -When issues come in, AI handles the grunt work: - ```typescript -import { ai } from 'jira.do' - -// AI analyzes every new issue -export const aiTriage = Automation({ - name: 'AI Triage', - trigger: { event: 'issue.created' }, - actions: async (issue) => { - const analysis = await ai.triage(issue, { - estimatePoints: true, - suggestPriority: true, - suggestComponents: true, - suggestAssignee: true, - detectDuplicates: true, - writeAcceptanceCriteria: issue.type === 'Story', - }) - - // Apply AI suggestions - await issue.update({ - storyPoints: analysis.estimatedPoints, - priority: analysis.suggestedPriority, - components: analysis.suggestedComponents, - labels: analysis.suggestedLabels, - }) - - // Add AI-generated acceptance criteria - if (analysis.acceptanceCriteria) { - await issue.update({ - description: issue.description + '\n\n## Acceptance Criteria\n' + analysis.acceptanceCriteria - }) - } - - // Flag potential duplicates - if (analysis.duplicates.length > 0) { - await issue.addComment(`Potential duplicates: ${analysis.duplicates.map(d => d.key).join(', ')}`) - await issue.addLabel('possible-duplicate') - } - - // Auto-assign based on expertise - if (analysis.suggestedAssignee) { - await issue.update({ assignee: analysis.suggestedAssignee }) - } - }, -}) +// New issues triage themselves +await jira`triage new bugs` + .map(bug => jira`estimate ${bug}`) + .map(bug => jira`assign ${bug} by expertise`) + +// Or ask for analysis +await jira`analyze BACKEND-456` +// AI returns: priority suggestion, estimate, likely assignee, potential duplicates + +// Find duplicates before they multiply +await jira`is BACKEND-789 a duplicate?` +await jira`similar issues to "login timeout on mobile"` ``` -### AI Story Point Estimation - -Never play planning poker again: +### AI Estimation ```typescript -const estimate = await ai.estimatePoints(issue, { - historical: true, // Learn from past issues - codebase: true, // Analyze relevant code - teamVelocity: true, // Factor in team capacity -}) +// Never play planning poker again +await jira`estimate BACKEND-456` +// AI compares to similar historical issues, analyzes code complexity + +// Bulk estimation +await jira`estimate unpointed stories in backlog` -// Returns: -{ - points: 5, - confidence: 0.82, - reasoning: "Similar to BACKEND-234 which took 4 days. Code changes in auth module are well-isolated. Team velocity suggests 5 points is achievable in one sprint.", - comparables: ['BACKEND-234', 'BACKEND-198', 'BACKEND-312'] -} +// Sprint capacity planning +await jira`can backend team fit 45 points next sprint?` +await jira`what's realistic for next sprint?` ``` ### AI Acceptance Criteria -Product managers rejoice: - ```typescript -const story = Issue.create({ - type: 'Story', - summary: 'User can reset password via email', - description: 'Users need to reset their password when they forget it.', -}) +// AI writes acceptance criteria from story descriptions +await jira`write acceptance criteria for BACKEND-456` + +// Or generate from scratch +await jira`create story "password reset via email" with AC` +// AI generates full acceptance criteria, edge cases, security notes -// AI generates acceptance criteria -const ac = await ai.acceptanceCriteria(story) - -// Returns: -` -## Acceptance Criteria - -- [ ] User can click "Forgot Password" from login page -- [ ] User enters email address -- [ ] System validates email exists in database -- [ ] System sends password reset email within 30 seconds -- [ ] Email contains secure, time-limited reset link (expires in 1 hour) -- [ ] User can set new password meeting security requirements -- [ ] User receives confirmation email after password change -- [ ] User can log in with new password - -## Technical Notes -- Reset tokens should use cryptographically secure random generation -- Rate limit reset requests to prevent abuse (max 5 per hour per email) -- Log all password reset attempts for security audit -` +// Batch it +await jira`stories missing acceptance criteria` + .map(story => jira`write acceptance criteria for ${story}`) ``` ### AI Bug Analysis -AI reads stack traces so you don't have to: - ```typescript -const bugAnalysis = await ai.analyzeBug(issue, { - stackTrace: issue.description, - codebase: true, - recentCommits: true, -}) +// AI reads stack traces so you don't have to +await jira`analyze bug BACKEND-789` +// Returns: root cause, likely file, suggested fix, who to assign -// Returns: -{ - rootCause: "Null pointer exception in UserService.getProfile() when user.email is null", - likelyFile: "src/services/user-service.ts:142", - suggestedFix: "Add null check before accessing user.email property", - recentCommit: { - sha: "abc123", - message: "Refactor user profile loading", - author: "alice@company.com", - relevance: 0.91 - }, - assigneeSuggestion: "alice@company.com" // Author of related commit -} -``` - -### Natural Language Queries - -Skip JQL entirely: - -```typescript -import { jira } from 'jira.do' +// Find the culprit +await jira`who should fix BACKEND-789?` +// AI checks recent commits, code ownership, current workload -// Natural language to JQL -const bugs = await jira`show me all critical bugs from this week` -const myWork = await jira`what am I working on?` -const blocked = await jira`which stories are blocked and why?` -const velocity = await jira`how many points did the backend team complete last sprint?` +// Connect the dots +await jira`what commit caused BACKEND-789?` +await jira`related issues to BACKEND-789` ``` ### AI Sprint Planning -Let AI help plan sprints: - ```typescript -const sprintPlan = await ai.planSprint({ - team: 'backend', - capacity: 45, // story points - priorities: ['security', 'performance'], - mustInclude: ['BACKEND-500', 'BACKEND-501'], // Critical issues -}) +// Let AI plan your sprint +await jira`plan next sprint for backend team` +await jira`plan sprint prioritizing security issues` -// Returns: -{ - suggestedIssues: ['BACKEND-500', 'BACKEND-501', 'BACKEND-456', 'BACKEND-478'], - totalPoints: 43, - reasoning: "Prioritized security issues as requested. Included BACKEND-478 as it unblocks BACKEND-500. Left 2 points buffer for unexpected work.", - risks: [ - "BACKEND-456 has no acceptance criteria - recommend writing before sprint start", - "BACKEND-478 depends on external API that has been flaky" - ] -} +// What-if scenarios +await jira`what if we add BACKEND-456 to current sprint?` +await jira`sprint risks for current sprint` + +// Retrospective insights +await jira`why did last sprint slip?` +await jira`what patterns in our blocked issues?` ``` ## Real-Time Collaboration -Built on Durable Objects for true real-time: - ```typescript -// Subscribe to issue changes -const subscription = issue.subscribe((change) => { - console.log(`${change.field} changed from ${change.oldValue} to ${change.newValue} by ${change.user}`) -}) +// Watch issues live +await jira`watch BACKEND-123` + .on('change', delta => console.log(`${delta.field} changed`)) -// Subscribe to board changes -const boardSub = board.subscribe((event) => { - if (event.type === 'issue.moved') { - console.log(`${event.issue.key} moved to ${event.column}`) - } -}) +// Board presence +await jira`show backend board` + .on('move', issue => console.log(`${issue.key} moved`)) + .on('presence', users => console.log(`${users.length} viewing`)) -// Presence awareness -board.onPresence((users) => { - console.log(`${users.length} people viewing this board`) -}) +// Team activity stream +await jira`backend team activity today` ``` ## API Compatible -Drop-in compatibility with Jira's REST API: - -```typescript -// Standard Jira REST API endpoints -GET /rest/api/3/issue/{issueKey} -POST /rest/api/3/issue -PUT /rest/api/3/issue/{issueKey} -DELETE /rest/api/3/issue/{issueKey} - -GET /rest/api/3/project -GET /rest/api/3/project/{projectKey} - -POST /rest/api/3/search // JQL search -GET /rest/agile/1.0/board/{boardId}/sprint -POST /rest/agile/1.0/sprint - -GET /rest/api/3/user -GET /rest/api/3/field -GET /rest/api/3/workflow -``` - -Your existing Jira integrations work with minimal changes: +Drop-in compatibility with Jira's REST API. Your existing integrations work: ```typescript // Existing jira-client code @@ -485,6 +281,8 @@ const jira = new JiraClient({ const issue = await jira.findIssue('PROJ-123') ``` +Standard REST endpoints supported: issues, projects, boards, sprints, search, users, fields, workflows. + ## Architecture ### Durable Object per Project @@ -514,16 +312,11 @@ OrganizationDO (users, permissions, global config) ### Storage Tiers -```typescript -// Hot: SQLite in Durable Object -// Active issues, recent activity, current sprints - -// Warm: R2 object storage -// Closed issues from past 12 months, attachments - -// Cold: R2 archive -// Historical data, compliance retention -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active issues, current sprints | <10ms | +| **Warm** | R2 + SQLite Index | Closed issues (past 12 months) | <100ms | +| **Cold** | R2 Archive | Historical data, compliance | <1s | ### Multi-Tenancy @@ -537,6 +330,19 @@ opensource.jira.do <- Open source project Each org has isolated data, users, and configuration. +## vs Jira Cloud + +| Feature | Jira Cloud | jira.do | +|---------|-----------|---------| +| **Cost** | $8-16/user/month | ~$5/month total | +| **Page Load** | 3-5 seconds | <100ms | +| **Architecture** | Atlassian Cloud | Edge-native, global | +| **Queries** | JQL only | Natural language | +| **AI** | Enterprise add-on | Built-in from day one | +| **Data Location** | Atlassian data centers | Your Cloudflare account | +| **Customization** | Marketplace apps | Code it yourself | +| **Lock-in** | Years of migration | MIT licensed | + ## Migration from Jira One-time import of your existing Jira: @@ -585,20 +391,29 @@ npm run deploy # Production deployment ## Roadmap +### Core - [x] Issues, projects, components, versions -- [x] JQL parser and query engine +- [x] Natural language queries - [x] Scrum and Kanban boards - [x] Sprints and backlogs - [x] Workflows and transitions - [x] Custom fields - [x] REST API compatibility + +### AI - [x] AI triage and estimation +- [x] Acceptance criteria generation +- [x] Bug analysis and assignment +- [x] Sprint planning assistance +- [ ] Predictive velocity +- [ ] Burndown forecasting + +### Integrations - [ ] Advanced roadmaps - [ ] Confluence integration (confluence.do) - [ ] Tempo/time tracking -- [ ] Structure (hierarchies beyond epics) -- [ ] ScriptRunner compatibility -- [ ] Jira Service Management mode +- [ ] GitHub/GitLab sync +- [ ] Slack/Teams notifications ## Why Open Source? @@ -616,7 +431,7 @@ Jira showed the world what issue tracking could be. **jira.do** makes it fast, i We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. Key areas: -- JQL parser extensions +- Natural language query extensions - Board and workflow features - API compatibility - AI/MCP integrations @@ -629,7 +444,12 @@ MIT License - Use it however you want. Build your business on it. Fork it. Make ---

- jira.do is part of the dotdo platform. + The $96k/year issue tracker ends here.
- Website | Docs | Discord + Natural language. AI-native. Your infrastructure. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/launchdarkly/README.md b/rewrites/launchdarkly/README.md index 41f9fd3c..a5c5e40d 100644 --- a/rewrites/launchdarkly/README.md +++ b/rewrites/launchdarkly/README.md @@ -1,236 +1,334 @@ # launchdarkly.do -LaunchDarkly on Cloudflare Durable Objects - Feature flags and experimentation at the edge. +> Feature Flags and Experiments. Edge-Native. Natural Language. AI-First. -## The Problem +LaunchDarkly charges $1M+ annually for feature flags. Evaluations happen in their data centers. Experiments require a PhD to analyze. Configuration sprawls across dashboards. -Feature flags need to be fast. Really fast. Every millisecond of evaluation latency multiplies across millions of requests. +**launchdarkly.do** is the open-source alternative. Sub-millisecond evaluation at the edge. Natural language flag definitions. AI-powered experiment analysis. Deploy in minutes, not months. -Traditional feature flag services: -- Evaluate flags in a central region -- Add network latency to every request -- Struggle with real-time updates -- Charge per MAU, not usage +## AI-Native API -## The Vision +```typescript +import { flags } from 'launchdarkly.do' // Full SDK +import { flags } from 'launchdarkly.do/tiny' // Minimal client +import { experiments } from 'experiments.do' // Experiments only +``` -Feature flags evaluated at the edge. Sub-millisecond. Real-time updates. Pay for what you use. +Natural language for feature flags: ```typescript -import { init } from 'launchdarkly.do' -// or: import { init } from 'experiments.do' +import { flags, experiments } from 'launchdarkly.do' -const client = init({ - sdkKey: 'sdk-xxxx', - options: { kvNamespace: env.FLAGS_KV } -}) +// Talk to it like a colleague +const enabled = await flags`dark mode for ${user}` +const variant = await flags`checkout flow for pro users` +const config = await flags`pricing tier for enterprise` -await client.waitForInitialization() +// Experiments as questions +const results = await experiments`checkout redesign winning?` +const analysis = await experiments`which button color converts better?` -// Sub-ms evaluation at edge -const showNewUI = await client.variation('new-ui', context, false) -const buttonColor = await client.variation('button-color', context, 'blue') +// Chain like sentences +await flags`new onboarding for trial users` + .track(`completed signup`) + .analyze() ``` -Drop-in compatible with `@launchdarkly/node-server-sdk`. +## The Problem -## Features +LaunchDarkly dominates feature flag management: + +| What LaunchDarkly Charges | The Reality | +|---------------------------|-------------| +| **Enterprise Plan** | $1M+/year | +| **Per MAU** | $0.02-0.10 per user | +| **Experimentation** | Additional premium | +| **Evaluation Latency** | 50-200ms from their DC | +| **Configuration** | Complex JSON rules | +| **Analysis** | Stats degree required | + +### The Latency Tax + +Every feature flag evaluation: +- Request leaves the edge +- Travels to LaunchDarkly's data center +- Gets evaluated against rules +- Returns to your application + +50-200ms per flag. Multiply by flags per request. Multiply by requests per second. That's your latency tax. + +### The Configuration Sprawl + +```json +{ + "key": "new-checkout", + "rules": [{ + "clauses": [{ + "attribute": "plan", + "op": "in", + "values": ["pro", "enterprise"] + }], + "variation": 1 + }] +} +``` -- **Sub-ms Flag Evaluation** - FNV32a hashing, KV cache, edge evaluation -- **Real-time Updates** - SSE/WebSocket streaming, <1s propagation -- **Targeting Rules** - MongoDB-style conditions ($eq, $in, $gt, $regex...) -- **User Segments** - Group users by attributes for targeted rollouts -- **A/B Experiments** - Variation assignment, exposure tracking, metrics -- **Statistical Analysis** - Bayesian + Frequentist, CUPED variance reduction -- **SDK Compatibility** - Drop-in for LaunchDarkly Node SDK -- **Audit Log** - Full history of flag changes +This should be: `pro and enterprise users get new checkout` -## Architecture +### The Analysis Gap + +Experiment results come as p-values and confidence intervals. PMs don't know if 0.03 is good. Engineers don't know when to call it. + +## The Solution + +**launchdarkly.do** reimagines feature flags: ``` - +-----------------------+ - | launchdarkly.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | FlagEvaluatorDO | | ExperimentDO | | StreamingDO | - | SQLite + KV | | SQLite + Stats | | WebSocket/SSE | - +------------------+ +------------------+ +------------------+ +LaunchDarkly launchdarkly.do +----------------------------------------------------------------- +$1M+/year $0 - run your own +50-200ms latency <1ms at edge +Complex JSON rules Natural language +Stats PhD required AI explains results +Dashboard sprawl Code-first +Central evaluation Edge-native ``` -**Key Components**: -- `FlagEvaluatorDO` - Flag configuration and evaluation logic -- `ExperimentDO` - Experiment state, exposures, conversions, analysis -- `StreamingDO` - Real-time flag update distribution - -## Installation +## One-Click Deploy ```bash -npm install launchdarkly.do -# or -npm install experiments.do +npx create-dotdo launchdarkly ``` -## Quick Start - -### Basic Flag Evaluation +Feature flags at the edge. Running on infrastructure you control. ```typescript -import { init } from 'launchdarkly.do' +import { LaunchDarkly } from 'launchdarkly.do' -const client = init({ - sdkKey: env.LD_SDK_KEY, - options: { kvNamespace: env.FLAGS_KV } +export default LaunchDarkly({ + name: 'my-app', + domain: 'flags.my-app.com', }) +``` -await client.waitForInitialization() - -const context = { - kind: 'user', - key: 'user-123', - email: 'user@example.com', - custom: { - plan: 'pro', - country: 'US' - } -} +## Features -// Boolean flag -const enabled = await client.variation('feature-x', context, false) +### Boolean Flags -// String flag -const variant = await client.variation('checkout-flow', context, 'control') +```typescript +// Just ask +const enabled = await flags`dark mode for ${user}` +const beta = await flags`beta features enabled?` +const premium = await flags`is premium user?` + +// AI infers what you need +await flags`dark mode` // returns boolean +await flags`dark mode for ${user}` // evaluates for user +await flags`who has dark mode?` // returns user list +``` -// JSON flag -const config = await client.variation('pricing-config', context, { tier: 'basic' }) +### Multivariate Flags -// With evaluation reason -const detail = await client.variationDetail('feature-x', context, false) -// { value: true, variationIndex: 1, reason: { kind: 'RULE_MATCH', ruleIndex: 0 } } +```typescript +// Variations as natural language +const flow = await flags`checkout flow version for ${user}` +const tier = await flags`pricing tier for enterprise` +const layout = await flags`dashboard layout for mobile` + +// AI returns the right type +// String: 'single-page' | 'multi-step' | 'express' +// Config: { columns: 2, showSidebar: true } ``` -### Targeting Rules +### Targeting ```typescript -// Flag configuration with targeting rules -const flagConfig = { - key: 'new-checkout', - name: 'New Checkout Flow', - type: 'boolean', - defaultValue: false, - rules: [ - { - id: 'beta-users', - condition: { plan: { $in: ['pro', 'enterprise'] } }, - variations: [{ value: true, weight: 1.0 }] - }, - { - id: 'us-rollout', - condition: { country: 'US' }, - coverage: 0.5, // 50% of matching users - variations: [{ value: true, weight: 1.0 }] - } - ] -} +// Natural as describing who gets what +await flags`new checkout for pro and enterprise users` +await flags`dark mode for US and Canada` +await flags`beta features for @corp.com emails` + +// Percentage rollouts +await flags`new onboarding for 50% of trial users` +await flags`gradual rollout: 10% today, 50% next week, 100% in March` + +// Segments read like descriptions +await flags`power users get advanced dashboard` +await flags`churning users get retention offer` ``` ### Experiments ```typescript -import { Experiment } from 'launchdarkly.do' +// Run experiments as questions +const variant = await experiments`checkout redesign for ${user}` + .track(`completed purchase`) + +// Ask for results naturally +const results = await experiments`is checkout redesign winning?` +// "Treatment is winning with 92% confidence. 20% uplift in conversions." + +// AI explains what the stats mean +const analysis = await experiments`explain checkout redesign results` +// "The new checkout increased conversions from 3% to 3.6%. +// We're 92% confident this is real, not random chance. +// Expected revenue impact: $45,000/month." + +// Compare anything +await experiments`which button color converts better: red or blue?` +await experiments`single-page vs multi-step checkout?` +``` -const experiment = new Experiment(env.EXPERIMENTS) +### Metrics -// Get experiment assignment -const assignment = await experiment.assign('checkout-redesign', { - userId: 'user-123', - attributes: { plan: 'pro' } -}) -// { variation: 'treatment', payload: { layout: 'single-page' } } +```typescript +// Track events naturally +await flags`dark mode for ${user}` + .track(`clicked settings`) + .track(`toggled theme`) + +// Conversion tracking +await experiments`checkout redesign for ${user}` + .track(`viewed cart`) + .track(`started checkout`) + .track(`completed purchase`, { value: 99.99 }) + +// Funnel analysis +await experiments`checkout funnel for redesign experiment` +// Returns drop-off at each step with significance +``` -// Track conversion -await experiment.track('purchase', { - userId: 'user-123', - value: 99.99, - properties: { sku: 'WIDGET-001' } +### Real-time Updates + +```typescript +// Subscribe to flag changes +await flags`dark mode`.onChange(value => { + console.log('Dark mode changed:', value) }) -// Get results -const results = await experiment.getResults('checkout-redesign') -// { -// variations: [ -// { id: 'control', users: 5000, conversions: 150, rate: 0.03 }, -// { id: 'treatment', users: 5000, conversions: 180, rate: 0.036 } -// ], -// analysis: { -// method: 'bayesian', -// winner: 'treatment', -// chanceToWin: 0.92, -// uplift: 0.20 -// } -// } +// Or use streams +const stream = await flags`stream all changes` +for await (const change of stream) { + console.log(`${change.flag} is now ${change.value}`) +} ``` -### Real-time Streaming +## Architecture -```typescript -import { StreamingClient } from 'launchdarkly.do' +``` + +-----------------------+ + | launchdarkly.do | + | (Cloudflare Worker) | + +-----------------------+ + | + +---------------+---------------+ + | | | + +------------------+ +------------------+ +------------------+ + | FlagEvaluatorDO | | ExperimentDO | | StreamingDO | + | SQLite + KV | | SQLite + Stats | | WebSocket/SSE | + +------------------+ +------------------+ +------------------+ +``` -const stream = new StreamingClient({ - sdkKey: env.LD_SDK_KEY, - onUpdate: (flag) => { - console.log(`Flag ${flag.key} changed to`, flag.value) - } -}) +### Durable Object per Flag Namespace + +``` +FlagNamespaceDO (config, segments, audit) + | + +-- FlagsDO (flag definitions, targeting rules) + | |-- SQLite: Flag configs + | +-- KV: Hot cache for evaluation + | + +-- ExperimentsDO (experiments, exposures, conversions) + | |-- SQLite: Assignment log + | +-- Analytics: Metrics aggregation + | + +-- StreamingDO (real-time distribution) + |-- WebSocket: Live updates + +-- SSE: Fallback streaming +``` + +### Storage Tiers + +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | KV | Active flags, current experiments | <1ms | +| **Warm** | SQLite | Assignment logs, recent metrics | <10ms | +| **Cold** | R2 | Audit history, archived experiments | <100ms | + +## vs LaunchDarkly + +| Feature | LaunchDarkly | launchdarkly.do | +|---------|--------------|-----------------| +| **Annual Cost** | $1M+ Enterprise | ~$10/month | +| **Evaluation Latency** | 50-200ms | <1ms | +| **Configuration** | JSON rules, dashboards | Natural language | +| **Experiment Analysis** | Stats tables | AI explanations | +| **Data Location** | Their data centers | Your Cloudflare account | +| **Real-time Updates** | Seconds | <100ms | +| **Customization** | Limited SDK hooks | Code it yourself | +| **Lock-in** | Proprietary format | Open source, MIT licensed | -await stream.connect() -// Receives updates within <1s of flag changes +## AI-Native Experimentation + +### Natural Language Results + +Instead of: +``` +p-value: 0.023 +confidence interval: [0.012, 0.048] +relative uplift: 20.1% +sample ratio: 0.498 ``` -## API Overview +You get: +```typescript +await experiments`explain checkout redesign results` +// "The redesign is working. Conversions went from 3% to 3.6%. +// We're 97.7% confident this is real improvement. +// If we roll out to everyone, expect ~$45K more revenue monthly. +// Recommendation: Ship it." +``` -### LDClient +### Automatic Stopping -- `init(config)` - Initialize client -- `waitForInitialization()` - Wait for flags to load -- `variation(key, context, default)` - Evaluate flag -- `variationDetail(key, context, default)` - Evaluate with reason -- `allFlagsState(context)` - Get all flags for context -- `close()` - Shutdown client +```typescript +// AI knows when to call it +await experiments`checkout redesign` + .stopWhen(`95% confident`) + .notify(`results are in`) + +// Or ask +await experiments`should we stop the checkout experiment?` +// "Not yet. We're at 87% confidence. Need ~3 more days of data." +``` -### Targeting Operators +### Segment Discovery -| Operator | Description | Example | -|----------|-------------|---------| -| `$eq` | Equals | `{ plan: { $eq: 'pro' } }` | -| `$ne` | Not equals | `{ status: { $ne: 'banned' } }` | -| `$in` | In array | `{ country: { $in: ['US', 'CA'] } }` | -| `$nin` | Not in array | `{ country: { $nin: ['CN'] } }` | -| `$gt`, `$gte` | Greater than | `{ age: { $gte: 18 } }` | -| `$lt`, `$lte` | Less than | `{ usage: { $lt: 1000 } }` | -| `$exists` | Property exists | `{ premium: { $exists: true } }` | -| `$regex` | Regex match | `{ email: { $regex: '@corp\\.com$' } }` | -| `$and`, `$or` | Logical | `{ $and: [...] }` | +```typescript +// Find who benefits most +await experiments`who responds best to checkout redesign?` +// "Mobile users on iOS see 40% uplift vs 15% for desktop. +// Pro users see 25% uplift vs 10% for free users. +// Recommendation: Prioritize mobile rollout." +``` -### Statistical Analysis +## Deployment Options -**Bayesian** (default): -- Chance to win (probability treatment beats control) -- Expected loss (risk of choosing wrong) -- 95% credible intervals +### Cloudflare Workers -**Frequentist**: -- P-value with configurable significance (0.01, 0.05, 0.1) -- Confidence intervals -- Relative uplift +```bash +npx create-dotdo launchdarkly +``` -**Variance Reduction**: -- CUPED using pre-experiment data -- Winsorization for outliers -- SRM (Sample Ratio Mismatch) detection +### Self-Hosted + +```bash +# Docker +docker run -p 8787:8787 dotdo/launchdarkly + +# Kubernetes +kubectl apply -f launchdarkly-do.yaml +``` ## The Rewrites Ecosystem @@ -246,12 +344,52 @@ launchdarkly.do is part of the rewrites family: ## Why Durable Objects? -1. **Edge Evaluation** - Flags evaluated at nearest PoP +1. **Edge Evaluation** - Flags evaluated at nearest PoP, <1ms 2. **Consistent Bucketing** - Deterministic hashing, no race conditions 3. **Real-time Updates** - WebSocket hibernation for efficient streaming 4. **SQLite Storage** - Fast flag lookups, experiment metrics 5. **Global Scale** - Millions of concurrent evaluations +## Roadmap + +### Core Flags +- [x] Boolean flags +- [x] Multivariate flags +- [x] Percentage rollouts +- [x] User targeting +- [x] Segment targeting +- [ ] Scheduled flags +- [ ] Flag dependencies + +### Experiments +- [x] A/B testing +- [x] Multivariate testing +- [x] Bayesian analysis +- [x] Frequentist analysis +- [x] AI result explanations +- [ ] Multi-armed bandits +- [ ] Automatic stopping rules + +### Platform +- [x] Real-time streaming +- [x] Audit logging +- [x] KV caching +- [ ] OpenFeature provider +- [ ] LaunchDarkly migration tool + ## License -MIT +MIT License - Feature flags for everyone. + +--- + +

+ The $1M/year feature flag ends here. +
+ Edge-native. Natural language. AI-first. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/linear/README.md b/rewrites/linear/README.md index f2fac960..2904658f 100644 --- a/rewrites/linear/README.md +++ b/rewrites/linear/README.md @@ -6,6 +6,35 @@ Linear showed the world what issue tracking could feel like - instant, keyboard- **linear.do** is Linear reimagined. The same speed and elegance. AI that triages, writes, and ships. Deploy your own. Own your engineering workflow. +## AI-Native API + +```typescript +import { linear } from 'linear.do' // Full SDK +import { linear } from 'linear.do/tiny' // Minimal client +import { linear } from 'linear.do/graphql' // GraphQL-only operations +``` + +Natural language for engineering workflows: + +```typescript +import { linear } from 'linear.do' + +// Talk to it like a colleague +const blocked = await linear`what's blocking the release?` +const myWork = await linear`what should I work on today?` +const atRisk = await linear`issues at risk of missing deadline` + +// Chain like sentences +await linear`bugs reported this week` + .map(bug => linear`triage ${bug}`) + +// Issues that manage themselves +await linear`create issue "Add dark mode" in Frontend team` + .estimate() // AI estimates points + .assign() // AI suggests assignee + .schedule() // adds to current cycle +``` + ## The Problem Linear's pricing is simpler than Jira, but still adds up: @@ -53,36 +82,28 @@ Your own Linear. Running on Cloudflare. Faster than ever. npx dotdo add linear ``` -## The workers.do Way - -You're building a product. Your team needs issue tracking that doesn't suck. Linear is beautiful and fast, but it's still $17k/year with no self-hosting option. Your engineering workflow lives on someone else's servers. There's a better way. +## Promise Pipelining -**Natural language. Tagged templates. AI agents that work.** +Chain work without `Promise.all`: ```typescript import { linear } from 'linear.do' -import { priya, ralph, quinn } from 'agents.do' - -// Talk to your issue tracker like a human -const blocked = await linear`what's blocking the release?` -const myWork = await linear`what should I work on today?` -const velocity = await linear`how did we do last cycle?` -``` +import { priya, ralph, mark } from 'agents.do' -**Promise pipelining - chain work without Promise.all:** - -```typescript // Release pipeline -const shipped = await linear`find completed issues in ${cycle}` +await linear`completed issues this cycle` .map(issue => mark`write release notes for ${issue}`) .map(notes => priya`review ${notes}`) - .map(notes => linear`publish ${notes}`) // Triage pipeline -const triaged = await linear`get triage inbox` +await linear`triage inbox` .map(issue => priya`analyze and prioritize ${issue}`) .map(issue => ralph`estimate ${issue}`) - .map(issue => linear`route ${issue} to project`) + .map(issue => linear`route ${issue} to appropriate project`) + +// Review pipeline +await linear`issues ready for review` + .map(issue => [priya, ralph].map(r => r`review ${issue}`)) ``` One network round trip. Record-replay pipelining. Workers working for you. @@ -94,41 +115,27 @@ One network round trip. Record-replay pipelining. Workers working for you. The core of everything: ```typescript -import { Issue, Project, Cycle } from 'linear.do' - -const issue = await Issue.create({ - title: 'Implement WebSocket connection pooling', - description: ` - ## Problem - Current implementation creates new connection per request. - - ## Solution - Implement connection pooling with configurable limits. - - ## Acceptance Criteria - - [ ] Pool maintains up to 100 connections - - [ ] Automatic reconnection on failure - - [ ] Metrics exposed for monitoring - `, - team: 'backend', - priority: 'High', - labels: ['performance', 'infrastructure'], - project: 'q1-performance', - cycle: 'cycle-42', - estimate: 5, - assignee: '@alice', -}) +import { linear } from 'linear.do' -// Sub-issues -await issue.createSubIssue({ - title: 'Design connection pool architecture', - estimate: 2, -}) +// Create issues naturally +await linear`create issue "Implement WebSocket connection pooling" for backend` +await linear`add to q1-performance cycle 42 priority high` +await linear`assign to alice estimate 5 points` + +// Or all at once, like you'd say it +await linear` + create issue "Implement WebSocket connection pooling" in backend: + - priority high + - labels: performance, infrastructure + - project q1-performance + - cycle 42 + - estimate 5 points + - assign to alice +` -await issue.createSubIssue({ - title: 'Implement pool manager', - estimate: 3, -}) +// Sub-issues +await linear`add subtask "Design connection pool architecture" estimate 2` +await linear`add subtask "Implement pool manager" estimate 3` ``` ### The Speed @@ -136,24 +143,17 @@ await issue.createSubIssue({ Linear's signature instant feel, even faster on the edge: ```typescript -// Optimistic updates - UI updates before server confirms -await issue.update({ status: 'In Progress' }) -// UI already updated, server catches up - -// Keyboard-first -// ⌘K - Command menu -// C - Create issue -// X - Close issue -// A - Assign -// P - Set priority -// L - Add label -// G then I - Go to inbox -// G then M - Go to my issues - -// Real-time sync across all clients -issue.subscribe((change) => { - // Instant updates, no polling -}) +// Move issues naturally +await linear`move BACK-101 to in progress` +await linear`start working on BACK-101` // same thing +await linear`mark BACK-101 done` + +// Batch updates read like commands +await linear`close all completed issues in cycle 42` +await linear`assign my inbox to me` +await linear`move blocked issues to next cycle` + +// Real-time sync - all clients update instantly ``` ### Workflows @@ -161,27 +161,15 @@ issue.subscribe((change) => { Customizable issue states: ```typescript -import { Workflow } from 'linear.do' - -const engineeringWorkflow = Workflow.create({ - name: 'Engineering', - states: [ - // Backlog - { name: 'Backlog', type: 'backlog', color: '#bbb' }, - { name: 'Todo', type: 'unstarted', color: '#e2e2e2' }, - - // Started - { name: 'In Progress', type: 'started', color: '#f2c94c' }, - { name: 'In Review', type: 'started', color: '#bb87fc' }, - - // Completed - { name: 'Done', type: 'completed', color: '#5e6ad2' }, - - // Cancelled - { name: 'Canceled', type: 'canceled', color: '#95a2b3' }, - { name: 'Duplicate', type: 'canceled', color: '#95a2b3' }, - ], -}) +// Query workflow states +await linear`workflow states for backend team` +await linear`issues in review` +await linear`what's in progress?` + +// Move through workflows naturally +await linear`move BACK-101 to review` +await linear`BACK-101 needs review` // same thing +await linear`mark BACK-101 as duplicate of BACK-50` ``` ### Cycles (Sprints) @@ -189,29 +177,19 @@ const engineeringWorkflow = Workflow.create({ Time-boxed iterations: ```typescript -import { Cycle } from 'linear.do' - -const cycle = await Cycle.create({ - team: 'backend', - number: 42, - startsAt: '2025-01-13', - endsAt: '2025-01-24', - name: 'Connection Pooling Sprint', -}) - -// Auto-schedule issues into cycles -await cycle.addIssues(['BACK-101', 'BACK-102', 'BACK-103']) - -// Cycle metrics -const metrics = await cycle.getMetrics() -// { -// totalScope: 34, -// completed: 21, -// added: 5, -// removed: 2, -// scopeCreep: 0.15, -// burndownData: [...] -// } +// Create cycles naturally +await linear`create cycle 42 for backend starting Monday` +await linear`new sprint "Connection Pooling" Jan 13-24` + +// Add issues like you'd say it +await linear`add BACK-101 BACK-102 BACK-103 to cycle 42` +await linear`move my issues to current cycle` +await linear`schedule high priority bugs for this sprint` + +// Check progress +await linear`how's cycle 42 going?` +await linear`burndown for current cycle` +await linear`scope creep this sprint` ``` ### Projects @@ -219,24 +197,23 @@ const metrics = await cycle.getMetrics() Group issues into larger initiatives: ```typescript -import { Project, Milestone } from 'linear.do' - -const project = await Project.create({ - name: 'Q1 Performance', - description: 'Improve p99 latency by 50%', - team: 'backend', - lead: '@alice', - startDate: '2025-01-01', - targetDate: '2025-03-31', - milestones: [ - Milestone.create({ name: 'Baseline metrics', date: '2025-01-15' }), - Milestone.create({ name: 'Connection pooling', date: '2025-02-01' }), - Milestone.create({ name: 'Caching layer', date: '2025-02-15' }), - Milestone.create({ name: 'Target achieved', date: '2025-03-15' }), - ], -}) +// Create projects naturally +await linear`create project "Q1 Performance" for backend led by alice` +await linear`project goal: improve p99 latency by 50%` +await linear`project runs Jan through March` + +// Add milestones like a checklist +await linear` + milestones for Q1 Performance: + - Baseline metrics by Jan 15 + - Connection pooling by Feb 1 + - Caching layer by Feb 15 + - Target achieved by Mar 15 +` -// Project progress auto-calculated from issues +// Check progress +await linear`how's Q1 Performance doing?` +await linear`at risk projects this quarter` ``` ### Roadmaps @@ -244,18 +221,14 @@ const project = await Project.create({ Visualize the big picture: ```typescript -import { Roadmap } from 'linear.do' - -const roadmap = await Roadmap.create({ - name: 'Engineering 2025', - projects: ['q1-performance', 'q2-scaling', 'mobile-app', 'api-v2'], - view: 'timeline', // or 'table', 'board' - groupBy: 'team', - dateRange: { - start: '2025-01-01', - end: '2025-12-31', - }, -}) +// Query roadmaps naturally +await linear`engineering roadmap 2025` +await linear`what's planned for Q2?` +await linear`projects by team this year` + +// Create and update +await linear`create roadmap "Engineering 2025" with Q1 Performance, Q2 Scaling, Mobile App, API v2` +await linear`add Mobile App to Q3 roadmap` ``` ### Triage @@ -263,25 +236,19 @@ const roadmap = await Roadmap.create({ Handle incoming issues efficiently: ```typescript -import { Triage } from 'linear.do' - -// Triage inbox for team -const inbox = await Triage.getInbox('backend') -// Returns issues without: project, cycle, or estimate - -// Quick triage actions -await inbox[0].triage({ - project: 'q1-performance', - cycle: 'current', - priority: 'High', - estimate: 3, -}) - -// Or snooze -await inbox[1].snooze('next-week') - -// Or decline -await inbox[2].decline('Won\'t fix - out of scope') +// Check triage queue +await linear`triage inbox for backend` +await linear`untriaged issues this week` +await linear`issues needing attention` + +// Triage naturally +await linear`triage BACK-150 to Q1 Performance current cycle high priority 3 points` +await linear`snooze BACK-151 until next week` +await linear`close BACK-152 wont fix - out of scope` + +// Bulk triage with AI assist +await linear`triage inbox` + .map(issue => linear`auto-triage ${issue}`) ``` ### Views @@ -289,29 +256,18 @@ await inbox[2].decline('Won\'t fix - out of scope') Custom filtered views: ```typescript -import { View } from 'linear.do' - -const myWork = View.create({ - name: 'My Work', - filter: { - assignee: { eq: 'me' }, - status: { notIn: ['Done', 'Canceled'] }, - }, - sort: [ - { field: 'priority', direction: 'desc' }, - { field: 'updatedAt', direction: 'desc' }, - ], - groupBy: 'project', -}) - -const blockers = View.create({ - name: 'Blockers', - filter: { - labels: { contains: 'blocked' }, - status: { in: ['In Progress', 'In Review'] }, - }, - sort: [{ field: 'priority', direction: 'desc' }], -}) +// Query views naturally +await linear`my work` +await linear`my issues not done grouped by project` +await linear`blocked issues in progress or review` + +// Save views for reuse +await linear`save view "My Work": my issues not done sorted by priority` +await linear`save view "Blockers": issues labeled blocked in progress` + +// Use saved views +await linear`show my work` +await linear`blockers this sprint` ``` ## AI-Native Issue Tracking @@ -323,19 +279,15 @@ AI isn't a feature. It's how modern issue tracking works. AI handles the triage queue: ```typescript -import { ai } from 'linear.do' - -// Enable AI triage for team -await Team.enableAITriage('backend', { - autoLabel: true, // AI suggests labels - autoPriority: true, // AI suggests priority - autoAssign: true, // AI suggests assignee - autoEstimate: true, // AI estimates points - autoProject: true, // AI suggests project - duplicateDetection: true, -}) +// Enable AI triage naturally +await linear`enable AI triage for backend team` +await linear`auto-label auto-assign auto-estimate new issues` + +// AI triage in action +await linear`triage inbox` + .map(issue => linear`analyze and triage ${issue}`) -// When issue comes in, AI does: +// When issue comes in, AI: // 1. Analyzes title and description // 2. Searches for duplicates // 3. Suggests labels from content @@ -353,43 +305,17 @@ AI helps write better issues: ```typescript // Draft issue from quick thought -const issue = await ai.draftIssue(` - login page is slow for some users -`) - -// AI creates: -{ - title: 'Slow login page performance for subset of users', - description: ` -## Problem -Some users are experiencing slow load times on the login page. - -## Investigation Needed -- [ ] Identify which users are affected -- [ ] Check for geographic patterns -- [ ] Review login page performance metrics -- [ ] Check third-party auth provider latency - -## Potential Causes -- CDN cache misses -- OAuth provider latency -- Database query performance -- Asset loading - -## Metrics to Track -- Login page load time (p50, p95, p99) -- OAuth callback duration -- Time to interactive - `, - suggestedLabels: ['performance', 'investigation', 'login'], - suggestedPriority: 'Medium', -} - -// From bug report -const bugIssue = await ai.draftIssue({ - type: 'bug', - input: 'users seeing 500 error when uploading files larger than 10mb', -}) +await linear`draft issue: login page is slow for some users` + +// AI creates full issue with: +// - Clear title +// - Problem statement +// - Investigation checklist +// - Potential causes +// - Suggested labels, priority + +// Bug reports expand automatically +await linear`bug: users seeing 500 error uploading files over 10mb` // AI adds: // - Steps to reproduce @@ -404,19 +330,17 @@ AI understands your codebase: ```typescript // AI analyzes related code when creating issues -const issue = await Issue.create({ - title: 'Add rate limiting to upload endpoint', - aiContext: { - codebase: true, // AI reads relevant code - history: true, // AI checks similar past issues - }, -}) +await linear`create issue "Add rate limiting to upload endpoint" with code context` // AI automatically adds: // - Links to relevant files // - Notes about existing patterns // - Suggested implementation approach // - Potential impact areas + +// Ask about code impact +await linear`what would changing the auth middleware affect?` +await linear`issues related to upload endpoint` ``` ### AI Estimation @@ -424,27 +348,20 @@ const issue = await Issue.create({ AI estimates story points: ```typescript -const estimate = await ai.estimate(issue, { - factors: { - codebase: true, // Analyze code complexity - history: true, // Compare to past issues - team: 'backend', // Team velocity context - }, -}) - -// Returns: -{ - estimate: 5, - confidence: 0.84, - reasoning: 'Similar to BACK-234 (5 points, 4 days). Requires changes in 3 files. No external dependencies.', - comparables: [ - { id: 'BACK-234', points: 5, duration: '4 days', similarity: 0.89 }, - { id: 'BACK-198', points: 3, duration: '2 days', similarity: 0.72 }, - ], - risks: [ - 'Touches auth middleware - needs careful testing', - ], -} +// Estimate issues naturally +await linear`estimate BACK-100` +await linear`how long would this take?` + +// AI returns: +// - Estimate: 5 points +// - Confidence: 84% +// - Reasoning: Similar to BACK-234 (5 points, 4 days) +// - Risks: Touches auth middleware - needs careful testing + +// Bulk estimation +await linear`estimate all unestimated issues in triage` +await linear`untriaged issues` + .map(issue => linear`estimate ${issue}`) ``` ### AI Cycle Planning @@ -452,36 +369,19 @@ const estimate = await ai.estimate(issue, { AI helps plan sprints: ```typescript -const plan = await ai.planCycle({ - team: 'backend', - capacity: 40, // Story points - constraints: { - mustInclude: ['BACK-100'], // Critical issues - priorities: ['security', 'performance'], - }, -}) - -// Returns: -{ - suggestedIssues: [ - { id: 'BACK-100', points: 8, reason: 'Critical - requested' }, - { id: 'BACK-105', points: 5, reason: 'Security priority' }, - { id: 'BACK-108', points: 5, reason: 'Unblocks BACK-105' }, - { id: 'BACK-112', points: 8, reason: 'Performance priority' }, - { id: 'BACK-115', points: 5, reason: 'Quick win, low risk' }, - { id: 'BACK-120', points: 8, reason: 'Performance priority' }, - ], - totalPoints: 39, - bufferRemaining: 1, - risks: [ - 'BACK-108 has external dependency on API provider', - 'BACK-112 needs design review before start', - ], - recommendations: [ - 'Start BACK-108 early due to external dependency', - 'Schedule BACK-112 design review in first 2 days', - ], -} +// Plan cycles naturally +await linear`plan next cycle for backend with 40 points capacity` +await linear`plan sprint including BACK-100 prioritizing security and performance` + +// AI suggests: +// - Issues to include with reasoning +// - Total points (39/40) +// - Risks and dependencies +// - Recommendations for sequencing + +// Accept or adjust +await linear`accept cycle plan` +await linear`remove BACK-120 from plan, add BACK-125 instead` ``` ### AI Release Notes @@ -489,31 +389,20 @@ const plan = await ai.planCycle({ AI writes release notes from completed issues: ```typescript -const releaseNotes = await ai.generateReleaseNotes({ - cycle: 'cycle-42', - format: 'changelog', // or 'blog', 'email', 'slack' -}) - -// Returns: -` -## v2.5.0 (2025-01-24) - -### New Features -- **Connection pooling for WebSocket** - Improved performance for high-traffic connections. Connections are now pooled and reused, reducing p99 latency by 40%. -- **Rate limiting for uploads** - Added configurable rate limits to prevent abuse. - -### Bug Fixes -- Fixed file upload failures for files larger than 10MB -- Resolved memory leak in long-running connections - -### Performance -- Database query optimization for user lookups (2x faster) -- Reduced cold start time by 30% - -### Internal -- Upgraded to TypeScript 5.4 -- Added comprehensive connection pooling tests -` +// Generate release notes naturally +await linear`release notes for cycle 42` +await linear`changelog for this sprint` +await linear`write release announcement for v2.5` + +// Different formats +await linear`release notes for cycle 42 as blog post` +await linear`release notes for cycle 42 for slack` + +// AI generates from completed issues: +// - New features with impact +// - Bug fixes +// - Performance improvements +// - Breaking changes ``` ### Natural Language Queries @@ -521,10 +410,13 @@ const releaseNotes = await ai.generateReleaseNotes({ Query issues conversationally: ```typescript -const blocked = await ai.query`what's blocking the release?` -const myWork = await ai.query`what should I work on today?` -const velocity = await ai.query`how did we do last cycle compared to average?` -const risk = await ai.query`which issues are at risk of missing the deadline?` +// Ask anything +await linear`what's blocking the release?` +await linear`what should I work on today?` +await linear`how did we do last cycle compared to average?` +await linear`which issues are at risk of missing the deadline?` +await linear`who's overloaded this sprint?` +await linear`show me bugs reported this week` ``` ## GitHub Integration @@ -532,57 +424,23 @@ const risk = await ai.query`which issues are at risk of missing the deadline?` Deep GitHub integration built-in: ```typescript -import { GitHub } from 'linear.do/integrations' - -// Link issues to PRs -// PRs auto-link when branch includes issue ID (e.g., back-101-connection-pool) - -// Auto-transitions -await GitHub.configure({ - onPROpened: Issue.moveTo('In Review'), - onPRMerged: Issue.moveTo('Done'), - onPRClosed: Issue.moveTo('In Progress'), -}) - -// Sync GitHub issues -await GitHub.sync({ - repo: 'company/backend', - labels: ['linear-sync'], - targetProject: 'q1-performance', -}) +// Connect naturally +await linear`connect to github company/backend` +await linear`sync github issues labeled linear-sync to Q1 Performance` + +// Auto-transitions (enabled by default) +// - PR opened -> issue moves to In Review +// - PR merged -> issue moves to Done +// - Branch with issue ID auto-links + +// Query GitHub context +await linear`PRs linked to BACK-101` +await linear`issues waiting on code review` ``` -## API & Integrations +## API Compatibility -Full API compatibility: - -```typescript -// GraphQL API (like Linear) -POST /graphql - -query { - issues(filter: { team: { key: { eq: "BACK" } } }) { - nodes { - id - title - state { name } - assignee { name } - } - } -} - -mutation { - issueCreate(input: { - teamId: "..." - title: "New issue" - description: "..." - }) { - issue { id identifier } - } -} -``` - -Existing Linear SDK works: +Full Linear API compatibility - existing SDKs just work: ```typescript import { LinearClient } from '@linear/sdk' @@ -592,11 +450,14 @@ const client = new LinearClient({ apiUrl: 'https://your-org.linear.do/graphql', // Just change URL }) +// Existing code works unchanged const issues = await client.issues({ filter: { team: { key: { eq: 'BACK' } } }, }) ``` +GraphQL API at `/graphql`. Same schema. Same queries. Same mutations. + ## Keyboard-First Design ``` diff --git a/rewrites/looker/README.md b/rewrites/looker/README.md index c473dcc3..e2f0c1a6 100644 --- a/rewrites/looker/README.md +++ b/rewrites/looker/README.md @@ -6,48 +6,47 @@ Looker revolutionized BI with its semantic modeling layer. But at $5,000/month m **looker.do** reimagines modern BI for the AI era. Conversational analytics. LookML-compatible. One-click deploy. -## The Problem - -Google acquired Looker for $2.6B and built an enterprise moat on: - -- **Enterprise pricing** - $5,000/month minimum, often $50-300k/year for real deployments -- **LookML complexity** - Requires dedicated engineers to maintain the semantic layer -- **Google Cloud lock-in** - Deep BigQuery integration, harder to use with other clouds -- **Developer-focused** - Business users can't create their own models -- **Slow iteration** - LookML changes require dev cycles, PR reviews, deployments -- **Embedded costs** - Charging per-user for embedded analytics in your product - -A mid-size company with 100 analysts? **$100k+/year** just for BI access. - -## The workers.do Way +## AI-Native API -Your board wants metrics. Your team needs self-service analytics. Looker wants $5k/month minimum and a dedicated LookML engineer. +```typescript +import { looker } from 'looker.do' // Full SDK +import { looker } from 'looker.do/tiny' // Minimal client +import { looker } from 'looker.do/embed' // Embedding only +``` -**What if analytics just understood what you meant?** +Natural language for business intelligence: ```typescript import { looker } from 'looker.do' -import { priya, tom } from 'agents.do' -// Conversational analytics -const answer = await looker`how many users signed up this week?` -const trends = await looker`show revenue by product category over time` +// Talk to it like a colleague +const revenue = await looker`revenue by region this quarter` +const trends = await looker`show signups week over week` const deep = await looker`why did churn spike in Q3?` -// AI agents build your semantic layer -const model = await priya`create LookML model for our SaaS metrics` -const review = await tom`review the model for performance issues` +// Chain like sentences +await looker`top customers by revenue` + .notify(`Your quarterly report is ready`) + +// Dashboards that build themselves +await looker`customer analytics for ${customer.id}` + .embed({ sessionLength: 3600 }) ``` -**Promise pipelining** - chain analytics without `Promise.all`: +## The Problem -```typescript -const insights = await looker`query ${customerData}` - .map(data => looker`build explore from ${data}`) - .map(explore => [priya, tom].map(r => r`review ${explore} for accuracy`)) -``` +Google acquired Looker for $2.6B and built an enterprise moat on: + +| What Google Charges | The Reality | +|---------------------|-------------| +| **Minimum License** | $5,000/month to start | +| **Enterprise Deploys** | $50-300k/year for real usage | +| **LookML Engineers** | Dedicated staff to maintain semantic layer | +| **Embedded Analytics** | Per-user fees kill product analytics | +| **Cloud Lock-in** | Deep BigQuery integration | +| **Iteration Speed** | LookML changes require dev cycles | -One network round trip. Natural language. Your semantic layer, AI-generated. +A mid-size company with 100 analysts? **$100k+/year** just for BI access. ## The Solution @@ -72,118 +71,96 @@ npx create-dotdo looker Your own Looker instance. Running on Cloudflare. Zero licensing fees. -## Conversational Analytics +```typescript +import { Looker } from 'looker.do' + +export default Looker({ + name: 'my-analytics', + domain: 'analytics.mycompany.com', + connections: ['postgresql://...'], +}) +``` + +## Features + +### Conversational Analytics Skip the Explores. Just ask: ```typescript -import { looker } from 'looker.do' - // Ask questions in plain English const answer = await looker`how many users signed up last week?` -const trends = await looker`show me revenue trends by product category` +const trends = await looker`revenue trends by product category` const deep = await looker`why did churn increase in Q3?` // AI generates LookML, runs query, returns results -console.log(answer.data) // Query results -console.log(answer.lookml) // Generated LookML +console.log(answer.data) // Query results +console.log(answer.lookml) // Generated LookML console.log(answer.visualization) // Auto-generated chart -console.log(answer.narrative) // Natural language explanation +console.log(answer.narrative) // Natural language explanation ``` -## Features - -### LookML Semantic Layer - -Define your data model once, query it everywhere: - -```lookml -# models/ecommerce.model.lkml -connection: "production" - -include: "/views/*.view.lkml" +### Explores -explore: orders { - label: "Order Analysis" - description: "Analyze customer orders and revenue" +```typescript +// Natural language IS the explore +const orders = await looker`orders by month last 90 days` +const geo = await looker`revenue by country completed only` +const top = await looker`top 500 customers by revenue` - join: customers { - type: left_outer - sql_on: ${orders.customer_id} = ${customers.id} ;; - relationship: many_to_one - } +// Comparative queries +await looker`this quarter vs last quarter by product` +await looker`revenue YoY by region` - join: products { - type: left_outer - sql_on: ${orders.product_id} = ${products.id} ;; - relationship: many_to_one - } -} +// Get the SQL (for debugging/auditing) +const sql = await looker`orders by status`.toSQL() ``` -```lookml -# views/orders.view.lkml -view: orders { - sql_table_name: public.orders ;; - - dimension: id { - primary_key: yes - type: number - sql: ${TABLE}.id ;; - } +### Dashboards - dimension_group: created { - type: time - timeframes: [raw, date, week, month, quarter, year] - sql: ${TABLE}.created_at ;; - } +```typescript +// Create dashboards naturally +const dashboard = await looker`executive dashboard for revenue` + +// Or compose from queries +await looker`create dashboard "Revenue Analytics"` + .add(looker`total revenue this month vs last`) + .add(looker`order count trend by week`) + .add(looker`revenue by category bar chart`) + .add(looker`top 10 customers table`) + +// Filter across all tiles +await looker`revenue dashboard for APAC region` +``` - dimension: status { - type: string - sql: ${TABLE}.status ;; - } +### Saved Queries (Looks) - measure: total_revenue { - type: sum - sql: ${TABLE}.amount ;; - value_format_name: usd - } +```typescript +// Save any query for reuse +await looker`top 10 products by revenue this quarter` + .save(`Top Products Q1`) - measure: order_count { - type: count - drill_fields: [id, created_date, customers.name, total_revenue] - } +// Reference saved queries +const products = await looker`load Top Products Q1` - measure: average_order_value { - type: average - sql: ${TABLE}.amount ;; - value_format_name: usd - } -} +// Schedule delivery +await looker`Top Products Q1` + .schedule({ daily: '8am', to: 'team@company.com' }) ``` -### AI-Generated LookML +### LookML Semantic Layer -Let AI create your semantic layer: +AI builds your semantic layer from natural descriptions: ```typescript -import { generateLookML } from 'looker.do' - // Point at your database, AI generates LookML -const model = await generateLookML({ - connection: 'postgresql://...', - schema: 'public', - tables: ['orders', 'customers', 'products'], -}) +const model = await looker`generate LookML for orders, customers, products` -// Returns complete LookML files -console.log(model.files) -// { -// 'models/generated.model.lkml': '...', -// 'views/orders.view.lkml': '...', -// 'views/customers.view.lkml': '...', -// 'views/products.view.lkml': '...', -// } +// Describe what you want to analyze +const saas = await looker` + create model for our SaaS business + track MRR, ARR, churn rate, LTV, CAC +` // AI infers: // - Primary keys and relationships @@ -192,249 +169,80 @@ console.log(model.files) // - Drill fields and formatting ``` -### Explores - -Interactive data exploration: - -```typescript -import { Explore } from 'looker.do' - -// Create an explore programmatically -const orderExplore = Explore({ - model: 'ecommerce', - explore: 'orders', - fields: [ - 'orders.created_month', - 'orders.total_revenue', - 'orders.order_count', - 'customers.country', - ], - filters: { - 'orders.created_date': 'last 90 days', - 'orders.status': 'completed', - }, - sorts: ['orders.total_revenue desc'], - limit: 500, -}) - -// Run the query -const results = await orderExplore.run() - -// Get SQL (for debugging/auditing) -const sql = orderExplore.toSQL() -``` - -### Dashboards - -Compose explores into dashboards: - -```typescript -import { Dashboard, Tile } from 'looker.do' - -export const RevenueDashboard = Dashboard({ - title: 'Revenue Analytics', - layout: 'grid', - filters: [ - { name: 'Date Range', field: 'orders.created_date', default: 'last 30 days' }, - { name: 'Region', field: 'customers.region', type: 'multiselect' }, - ], - tiles: [ - Tile.singleValue({ - title: 'Total Revenue', - explore: 'orders', - measure: 'orders.total_revenue', - comparison: { type: 'previous_period' }, - }), - Tile.singleValue({ - title: 'Order Count', - explore: 'orders', - measure: 'orders.order_count', - }), - Tile.lineChart({ - title: 'Revenue Trend', - explore: 'orders', - dimensions: ['orders.created_week'], - measures: ['orders.total_revenue'], - span: 2, - }), - Tile.barChart({ - title: 'Revenue by Category', - explore: 'orders', - dimensions: ['products.category'], - measures: ['orders.total_revenue'], - sorts: ['orders.total_revenue desc'], - }), - Tile.table({ - title: 'Top Customers', - explore: 'orders', - dimensions: ['customers.name', 'customers.email'], - measures: ['orders.total_revenue', 'orders.order_count'], - limit: 10, - }), - ], -}) -``` - -### Looks (Saved Queries) +Or define LookML directly when you need precision: -Save and share queries: +```lookml +# models/ecommerce.model.lkml +connection: "production" +include: "/views/*.view.lkml" -```typescript -import { Look } from 'looker.do' - -export const TopProducts = Look({ - title: 'Top 10 Products by Revenue', - model: 'ecommerce', - explore: 'orders', - fields: [ - 'products.name', - 'products.category', - 'orders.total_revenue', - 'orders.order_count', - ], - filters: { - 'orders.created_date': 'this quarter', - }, - sorts: ['orders.total_revenue desc'], - limit: 10, - visualization: { - type: 'table', - showRowNumbers: true, - columnWidths: { 'products.name': 200 }, - }, -}) +explore: orders { + label: "Order Analysis" + join: customers { + sql_on: ${orders.customer_id} = ${customers.id} ;; + } +} ``` ## AI-Native Features -### Conversational Queries - -Ask questions, get answers: - -```typescript -import { ask } from 'looker.do' - -// Simple questions -const q1 = await ask('What was revenue last month?') -// AI: Queries orders.total_revenue with date filter - -// Comparative questions -const q2 = await ask('How does this month compare to last month?') -// AI: Adds period-over-period comparison - -// Drill-down questions -const q3 = await ask('Break that down by region') -// AI: Adds customers.region dimension - -// Follow-up questions (context-aware) -const q4 = await ask('Which products drove the increase?') -// AI: Pivots to product analysis, focuses on growth -``` - -### Auto-Modeling +### Follow-Up Questions -AI creates LookML from your questions: +Context-aware conversation: ```typescript -import { autoModel } from 'looker.do' - -// Describe what you want to analyze -const model = await autoModel({ - description: ` - I want to analyze our SaaS business. Key metrics are: - - MRR and ARR - - Customer acquisition cost - - Lifetime value - - Churn rate - - Revenue by plan type - `, - database: 'postgresql://...', -}) - -// AI discovers relevant tables, creates model -console.log(model.explores) // ['subscriptions', 'customers', 'payments'] -console.log(model.measures) // ['mrr', 'arr', 'cac', 'ltv', 'churn_rate'] +const q1 = await looker`revenue last month` +const q2 = await looker`break that down by region` // remembers context +const q3 = await looker`which products drove the increase?` // follows the thread +const q4 = await looker`export that to CSV` // exports q3 results ``` ### Insight Discovery -Automatic anomaly detection and insights: - ```typescript -import { discoverInsights } from 'looker.do' - -const insights = await discoverInsights({ - explore: 'orders', - timeframe: 'last 30 days', - focus: ['revenue', 'conversion'], -}) - -// Returns ranked insights -for (const insight of insights) { - console.log(insight.headline) - // "Mobile conversion dropped 23% on March 15" - - console.log(insight.explanation) - // "Coincides with app update v2.3.1. Checkout flow changed." +// AI finds what's interesting +const insights = await looker`what's unusual in revenue this month?` - console.log(insight.recommendation) - // "Investigate checkout funnel on mobile devices" - - console.log(insight.look) - // Link to saved look with relevant query -} +// Returns ranked discoveries +// - "Mobile conversion dropped 23% on March 15" +// - "Enterprise segment up 45% week over week" +// - "Product X has 3x average return rate" ``` ### AI Agents as Analysts -AI agents can build complete analytics: - ```typescript import { priya, tom } from 'agents.do' -import { looker } from 'looker.do' // Product manager builds analytics -const productAnalytics = await priya` - create a complete analytics model for our product usage data - I need to track feature adoption, user engagement, and retention +const analytics = await priya` + create complete analytics for product usage + track feature adoption, engagement, retention ` -// Tech lead reviews the LookML -const review = await tom` - review this LookML model for performance issues and best practices - ${productAnalytics.lookml} -` +// Tech lead reviews the model +const review = await tom`review ${analytics.lookml} for performance` + +// Promise pipelining - one network round trip +const verified = await looker`analyze ${customerData}` + .map(data => looker`build explore from ${data}`) + .map(explore => [priya, tom].map(r => r`review ${explore}`)) ``` ## Embedded Analytics -Embed Looker in your product without per-user fees: - -### SSO Embed +Embed in your product without per-user fees: ```typescript -import { createEmbedUrl } from 'looker.do/embed' - -// Generate signed embed URL -const embedUrl = await createEmbedUrl({ - dashboard: 'customer_analytics', - user: { - external_user_id: customer.id, - first_name: customer.name, - permissions: ['access_data', 'see_looks'], - models: ['customer_facing'], - user_attributes: { - customer_id: customer.id, // Row-level security - }, - }, - session_length: 3600, -}) +// Natural language embedding +const embed = await looker`customer analytics for ${customer.id}` + .embed({ sessionLength: 3600 }) -// Embed in your app -return - -// Or use the API -await pd.webForms.submit(form.id, { - name: 'Bob Smith', - email: 'bob@startup.com', - company: 'StartupXYZ', - budget: '$10k-50k', -}) +// Create lead capture +await pipedrive`create contact form for enterprise leads` + .embed() // returns iframe code + .notify('#sales') + .createDeal('inbound') ``` -### Calling (Built In) - -No $2.50+/user add-on: +### Calling ```typescript -// Make a call (via browser or integration) -const call = await pd.calls.start({ - to: person.phone[0], - deal_id: deal.id, - person_id: person.id, -}) - -// Call is automatically logged when ended -call.on('ended', async (result) => { - await pd.activities.create({ - type: 'call', - deal_id: deal.id, - person_id: person.id, - done: true, - duration: result.duration, - note: result.transcription, // AI transcription included - outcome: result.outcome, - }) -}) - -// VoIP integrations -await pd.calls.configure({ - provider: 'twilio', // or 'vonage', 'aircall', 'ringcentral' - credentials: { ... }, -}) +// Click to call +await pipedrive`call Alice Chen` + .transcribe() // AI listens + .summarize() // creates activity note + .suggest() // next steps + +// Review calls +await pipedrive`my calls today` +await pipedrive`calls with no follow-up` ``` ---- - ## AI-Native Sales -### AI Sales Assistant - -AI that actually helps close deals: +### Daily Briefing ```typescript -import { pd } from 'pipedrive.do' - -// AI analyzes your pipeline daily -const insights = await pd.ai.dailyInsights() - -console.log(insights) -// { -// focus: [ -// { deal: 'Acme Corp', action: 'Send proposal', reason: 'Demo went well, momentum building' }, -// { deal: 'BigTech Inc', action: 'Call champion', reason: 'No activity in 5 days' }, -// ], -// risks: [ -// { deal: 'StartupXYZ', risk: 'Stalled', days: 14, suggestion: 'Offer limited-time discount' }, -// ], -// wins: [ -// { deal: 'MegaCorp', probability: 95, tip: 'Get signature this week' }, -// ], -// forecast: { -// thisMonth: { committed: 125000, likely: 85000, possible: 200000 }, -// nextMonth: { committed: 50000, likely: 120000, possible: 350000 }, -// }, -// } +// Start your day +await pipedrive`what should I focus on today?` +// AI analyzes your pipeline: +// - Hot deals to close +// - Stalled deals needing attention +// - Activities due +// - Follow-ups needed ``` -### AI Email Writer +### Deal Intelligence ```typescript -// AI writes personalized emails -const email = await pd.ai.composeEmail({ - deal_id: deal.id, - type: 'follow_up', - context: 'Had a great demo yesterday, they seemed interested in the analytics features', -}) - -console.log(email) +// AI analyzes any deal +await pipedrive`analyze Acme deal` // { -// subject: 'Re: Analytics Deep Dive for Acme', -// body: 'Hi Alice,\n\nThank you for your time yesterday...', -// tone: 'professional', -// callToAction: 'Schedule technical review call', +// health: 'at risk', +// reason: 'No activity in 7 days, champion on vacation', +// action: 'Schedule with technical buyer instead', // } -// AI suggests email improvements -const improved = await pd.ai.improveEmail({ - subject: 'Checking in', - body: 'Hi, wanted to see if you had any questions about the proposal.', -}) -// { -// subject: 'Quick question about the Acme proposal', -// body: 'Hi Alice,\n\nI wanted to make sure the proposal addresses...', -// improvements: ['More specific subject line', 'Added value proposition', 'Clear CTA'], -// } +// Predict outcomes +await pipedrive`will Acme close this quarter?` +await pipedrive`what's blocking BigTech?` ``` -### AI Lead Scoring +### Lead Scoring ```typescript -// Automatic lead scoring -const score = await pd.ai.scoreLead(person.id) - -console.log(score) -// { -// score: 87, -// grade: 'A', -// factors: [ -// { factor: 'Company size', impact: +15, reason: '201-500 employees matches ICP' }, -// { factor: 'Engagement', impact: +20, reason: 'Opened 5 emails, clicked 3' }, -// { factor: 'Title', impact: +10, reason: 'VP-level decision maker' }, -// { factor: 'Industry', impact: +12, reason: 'Technology sector' }, -// ], -// recommendation: 'Hot lead - prioritize immediate outreach', -// } - -// Bulk scoring -const scoredLeads = await pd.ai.scoreLeads({ - pipeline_id: pipeline.id, - stage: 'lead_in', -}) -// Returns all leads sorted by score +// AI scores automatically +await pipedrive`score my leads` +await pipedrive`hot leads this week` +await pipedrive`leads matching our ICP` ``` -### AI Forecasting +### Forecasting ```typescript // Revenue forecasting -const forecast = await pd.ai.forecast({ - period: 'Q1', - team: 'all', -}) +await pipedrive`Q1 forecast` +await pipedrive`forecast by rep` +await pipedrive`commit vs pipeline this month` -console.log(forecast) -// { -// period: 'Q1 2025', -// predicted: 850000, -// confidence: 0.78, -// breakdown: { -// committed: 320000, // 95%+ probability -// likely: 280000, // 70-94% probability -// possible: 250000, // 30-69% probability -// }, -// trends: { -// vsLastQuarter: +15, -// vsSamePeriodLastYear: +42, -// }, -// risks: [ -// { deal: 'BigDeal Corp', value: 150000, risk: 'Champion leaving company' }, -// ], -// } - -// Deal-level predictions -const prediction = await pd.ai.predictDeal(deal.id) -// { -// willClose: 0.72, -// predictedCloseDate: '2025-02-15', -// predictedValue: 72000, // vs current 75000 -// nextBestAction: 'Schedule executive sponsor meeting', -// } +// AI explains the numbers +await pipedrive`why is forecast down vs last quarter?` ``` -### AI Activity Suggestions +### Sally: Your AI SDR ```typescript import { sally } from 'agents.do' -import { pd } from 'pipedrive.do' - -// Sally is your AI SDR -await sally` - Review my pipeline and tell me what I should focus on today. - Prioritize based on deal value and likelihood to close. - Create activities for my top 5 priorities. -` - -// Sally analyzes and creates activities: -// - Call Acme Corp (discovery follow-up) -// - Email BigTech (send case study) -// - Meeting prep for MegaCorp demo -// - Contract review for StartupXYZ -// - LinkedIn connect with new champion at Enterprise Inc - -// Automatic activity creation based on deal stage -pd.deals.on('stage_changed', async (deal, fromStage, toStage) => { - const activities = await pd.ai.suggestActivities({ - deal_id: deal.id, - stage: toStage, - }) - - for (const activity of activities) { - await pd.activities.create(activity) - } -}) + +// Sally works your pipeline +await sally`review my pipeline and flag stalled deals` +await sally`prioritize my activities by deal value` +await sally`draft follow-ups for all proposals sent this week` + +// Sally takes action (you approve) +await sally`schedule next steps for all hot deals` + .review() // you check her work + .execute() // she does it ``` ---- +## Promise Pipelining + +Chain operations without waiting. One network round trip: + +```typescript +import { pipedrive, sally, priya } from 'pipedrive.do' + +// Find stalled deals, analyze, create action plans - all pipelined +const revived = await pipedrive`stalled deals over 7 days` + .map(deal => priya`analyze why ${deal} is stuck`) + .map(analysis => sally`create re-engagement plan for ${analysis}`) + +// Activity-based selling flow +const priorities = await pipedrive`hot deals by close date` + .map(deal => sally`schedule next activity for ${deal}`) + .map(activity => [priya, sally].map(r => r`review ${activity}`)) + +// Pipeline sweep +await pipedrive`deals closing this week` + .map(deal => pipedrive`what's needed to close ${deal}?`) + .map(gaps => sally`address ${gaps}`) +``` ## API Compatible @@ -632,16 +314,12 @@ Drop-in replacement for Pipedrive's REST API: ```typescript // Before: Pipedrive Cloud import Pipedrive from 'pipedrive' -const client = new Pipedrive.ApiClient() client.basePath = 'https://api.pipedrive.com/v1' // After: pipedrive.do (just change the base path) -import Pipedrive from 'pipedrive' -const client = new Pipedrive.ApiClient() client.basePath = 'https://your-instance.pipedrive.do/api/v1' // All APIs work the same -const dealsApi = new Pipedrive.DealsApi(client) const deals = await dealsApi.getDeals() ``` @@ -659,132 +337,65 @@ const deals = await dealsApi.getDeals() | **Webhooks** | Full - all event types | | **Users** | Full - teams, permissions, goals | ---- - ## Architecture -``` -┌─────────────────────────────────────────────────────────────────────┐ -│ pipedrive.do Worker │ -├─────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌─────────────────────────────────────────────────────────────────┐│ -│ │ WorkspaceDO (per team) ││ -│ ├─────────────────────────────────────────────────────────────────┤│ -│ │ ││ -│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ ││ -│ │ │ Pipelines │ │ Deals │ │ Persons & Orgs │ ││ -│ │ │ & Stages │ │ Engine │ │ Relationship Graph │ ││ -│ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ ││ -│ │ ││ -│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ ││ -│ │ │ Activities │ │ Email │ │ AI Sales Assistant │ ││ -│ │ │ Scheduler │ │ Sync │ │ (llm.do integration) │ ││ -│ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ ││ -│ │ ││ -│ │ ┌─────────────────────────────────────────────────────────────┐││ -│ │ │ WebSocket Hub │││ -│ │ │ Real-time pipeline updates & collaboration │││ -│ │ └─────────────────────────────────────────────────────────────┘││ -│ │ ││ -│ └─────────────────────────────────────────────────────────────────┘│ -│ │ -├─────────────────────────────────────────────────────────────────────┤ -│ SQLite (deals, contacts) │ R2 (files) │ Vectorize (AI search) │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -### Real-Time Collaboration - -Every pipeline action broadcasts instantly: - -```typescript -// Connect to workspace -const ws = pd.realtime.connect() - -ws.on('deal:created', (deal) => { - // Someone created a deal -}) - -ws.on('deal:moved', (deal, fromStage, toStage) => { - // Someone moved a deal (drag-drop) - playWonSound() // if moved to Closed Won -}) - -ws.on('activity:completed', (activity) => { - // Someone completed an activity -}) - -ws.on('user:typing', (user, context) => { - // Show typing indicator in shared views -}) -``` - ### Durable Object per Workspace -Each sales team runs in isolated infrastructure: - ``` -acme.pipedrive.do <- Acme's workspace -startup.pipedrive.do <- Startup's workspace -agency.pipedrive.do <- Agency's workspace (manages multiple clients) +WorkspaceDO (config, users, pipelines) + | + +-- DealsDO (deals, stages, products) + | |-- SQLite: Deal records + | +-- R2: Attachments, proposals + | + +-- ContactsDO (persons, organizations) + | |-- SQLite: Contact data + | +-- Relationship graph + | + +-- ActivitiesDO (calls, meetings, tasks) + | |-- SQLite: Activity log + | +-- Calendar sync + | + +-- MailDO (emails, templates) + | |-- SQLite: Thread data + | +-- R2: Attachments + | + +-- AIDO (insights, scoring, forecasting) + |-- llm.do integration + +-- Vectorize for semantic search ``` ---- - -## Pricing Comparison - -### Pipedrive Pricing (2025) - -| Plan | Per User/Month | 20 Users/Year | -|------|---------------|---------------| -| Essential | $14 | $3,360 | -| Advanced | $34 | $8,160 | -| Professional | $49 | $11,760 | -| Power | $64 | $15,360 | -| Enterprise | $99 | $23,760 | +### Storage Tiers -Plus add-ons: -- LeadBooster: +$32.50/company/month -- Web Visitors: +$41/company/month -- Campaigns: +$13.33/company/month -- Smart Docs: +$32.50/company/month -- Projects: +$6.70/company/month -- Calling: +$2.50/user/month +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active deals, recent activities | <10ms | +| **Warm** | R2 + Index | Historical deals (1-3 years) | <100ms | +| **Cold** | R2 Archive | Compliance retention | <1s | -**20-person team on Professional with all add-ons: ~$18,000/year** - -### pipedrive.do Pricing - -| Resource | Cost | Notes | -|----------|------|-------| -| Durable Object | $0.15/million requests | Your workspace | -| SQLite Storage | $0.20/GB/month | All your data | -| R2 Storage | $0.015/GB/month | Attachments | -| Workers | Free tier: 100k/day | API calls | -| AI | ~$0.01/query | Via llm.do | - -**Example: 20 users, 10,000 deals/year, 50k activities** +### Real-Time Collaboration -| | Pipedrive Professional | pipedrive.do | -|-|----------------------|--------------| -| Software | $11,760/year | ~$15/month | -| Add-ons | $1,500+/year | $0 (included) | -| AI features | Limited | Unlimited | -| **Total** | **$13,260+/year** | **~$180/year** | +Every pipeline action broadcasts instantly. Drag a deal, everyone sees it. Bell rings when deals close. -**Savings: 98.6%** +## vs Pipedrive ---- +| Feature | Pipedrive | pipedrive.do | +|---------|-----------|--------------| +| **Pricing** | $14-99/user/month | ~$15/month total | +| **AI** | Premium add-on | Native, unlimited | +| **Data** | Their servers | Your Cloudflare | +| **Add-ons** | $100+/month extra | Everything included | +| **API** | Rate limited | Unlimited | +| **Lock-in** | Data export fees | MIT licensed | ## Migration -One-command migration from Pipedrive: +One command from Pipedrive: ```bash npx pipedrive.do migrate --from-pipedrive -# Migrates: +# Migrates everything: # - Pipelines and stages # - All deals with history # - Persons and organizations @@ -792,11 +403,8 @@ npx pipedrive.do migrate --from-pipedrive # - Products and line items # - Email history # - Custom fields -# - Automations (converted) ``` ---- - ## Contributing pipedrive.do is open source under the MIT license. @@ -806,11 +414,8 @@ git clone https://github.com/dotdo/pipedrive.do cd pipedrive.do pnpm install pnpm test -pnpm dev ``` ---- - ## License MIT License @@ -818,9 +423,9 @@ MIT License ---

- Sell more. Pay less. Own everything. + The $1.5B CRM ends here.
- Visual pipeline CRM without the visual tax. + Visual pipeline. AI-native. Activity-based.

Website | Docs | diff --git a/rewrites/posthog/README.md b/rewrites/posthog/README.md index 38de3f9b..1e54cd82 100644 --- a/rewrites/posthog/README.md +++ b/rewrites/posthog/README.md @@ -1,282 +1,285 @@ # posthog.do -PostHog on Cloudflare Durable Objects - Product analytics for every AI agent. +> Product Analytics. Edge-Native. AI-First. Open by Default. -## The Problem +PostHog charges $450/month for 2M events. Requires self-hosting for privacy. Treats AI agents as afterthoughts. Feature flags need configuration UIs. Funnels require point-and-click builders. -AI agents need analytics. Millions of them. Running in parallel. Each isolated. Each with their own telemetry. +**posthog.do** is the open-source alternative. Privacy-first. Edge-native. Deploys in seconds. AI agents are first-class citizens. -Traditional analytics were built for humans: -- One shared analytics instance for many users -- Centralized data warehouses -- Manual event schema management -- Expensive per-project +## AI-Native API -AI agents need the opposite: -- One analytics instance per agent -- Distributed by default -- Automatic schema evolution -- Free at the instance level, pay for usage - -## The Vision +```typescript +import { posthog } from 'posthog.do' // Full SDK +import { posthog } from 'posthog.do/tiny' // Minimal client +import { posthog } from 'posthog.do/flags' // Feature flags only +``` -Every AI agent gets their own PostHog. +Natural language for product analytics: ```typescript -import { tom, ralph, priya } from 'agents.do' -import { PostHog } from 'posthog.do' +import { posthog } from 'posthog.do' -// Each agent has their own isolated analytics -const tomAnalytics = PostHog.for(tom) -const ralphAnalytics = PostHog.for(ralph) -const priyaAnalytics = PostHog.for(priya) +// Talk to it like a PM +const funnel = await posthog`users who clicked buy but didn't checkout` +const cohort = await posthog`power users who logged in 10+ times last week` +const trend = await posthog`signups by country this month` -// Full PostHog API -await tomAnalytics.capture('code_reviewed', { pr: 123, approved: true }) -await ralphAnalytics.capture('build_completed', { success: true, duration: 42 }) -await priyaAnalytics.capture('spec_written', { feature: 'auth' }) +// Chain like sentences +await posthog`pro users who haven't used feature X` + .notify(`Check out our new feature X!`) + +// Feature flags that read like questions +const enabled = await posthog`is new-checkout-flow on for user-123?` +const variant = await posthog`which pricing-experiment variant for user-456?` ``` -Not a shared analytics platform with project keys. Not a multi-tenant nightmare. Each agent has their own complete PostHog instance. +## The Problem -## Features +PostHog dominates open-source analytics but: -- **Event Capture** - `posthog.capture()` with full API compatibility -- **Feature Flags** - `posthog.isFeatureEnabled()` evaluated at the edge -- **Experiments** - A/B testing with deterministic bucketing -- **Analytics Queries** - Funnels, retention, cohorts -- **MCP Tools** - Model Context Protocol for AI-native analytics -- **Session Recording** - DOM snapshots (future) -- **Surveys** - In-app feedback (future) +| What PostHog Charges | The Reality | +|----------------------|-------------| +| **Cloud Pricing** | $450/month for 2M events | +| **Self-Hosting** | Complex Kubernetes setup | +| **Feature Flags** | UI configuration required | +| **AI Agents** | No first-class support | +| **Edge Performance** | Centralized servers | -## Architecture +### AI Agents Need Analytics Too + +Traditional analytics track humans clicking buttons. But AI agents need: +- One analytics instance per agent +- Distributed by default +- Automatic schema evolution +- Events captured at the edge + +## The Solution + +**posthog.do** reimagines analytics for AI-native applications: ``` - +-----------------------+ - | posthog.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | EventsDO (Tom) | | EventsDO (Ralph) | | EventsDO (...) | - | SQLite + AE | | SQLite + AE | | SQLite + AE | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +--------------------+--------------------+ - | | | - +----------+ +-----------+ +------------+ - | KV | | D1 | | Analytics | - | (flags) | | (meta) | | Engine | - +----------+ +-----------+ +------------+ +PostHog Cloud posthog.do +----------------------------------------------------------------- +$450/month for 2M events Usage-based, starts free +Complex self-hosting Deploy in seconds +Centralized servers Edge-native, global +UI configuration Code-first, natural language +Human-only analytics AI agents are first-class ``` -**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each agent's analytics is a Durable Object. SQLite handles recent events. Analytics Engine handles aggregations at scale. - -## Installation +## One-Click Deploy ```bash -npm install posthog.do +npx create-dotdo posthog ``` -## Quick Start - -### Event Capture +Product analytics on infrastructure you control. Edge-native from day one. ```typescript import { PostHog } from 'posthog.do' -const posthog = new PostHog(env.POSTHOG) - -// Capture events -await posthog.capture('$pageview', { - distinct_id: 'user-123', - properties: { - $current_url: 'https://example.com/dashboard', - plan: 'pro' - } +export default PostHog({ + name: 'my-app', + domain: 'analytics.my-app.com', }) +``` -// Capture with timestamp -await posthog.capture('purchase_completed', { - distinct_id: 'user-123', - timestamp: new Date('2024-01-15T10:30:00Z'), - properties: { - amount: 99.99, - currency: 'USD' - } -}) +## Features + +### Event Capture -// Batch capture -await posthog.captureBatch([ - { event: 'item_viewed', distinct_id: 'user-123', properties: { item_id: 'abc' } }, - { event: 'item_added_to_cart', distinct_id: 'user-123', properties: { item_id: 'abc' } } -]) +```typescript +// Just say what happened +await posthog`user-123 viewed dashboard on pro plan` +await posthog`user-123 purchased $99.99 USD` +await posthog`user-123 clicked buy then added to cart` + +// AI infers the schema +await posthog`user-123 viewed dashboard` // creates $pageview +await posthog`user-123 signed up` // creates signup_completed +await posthog`user-123 purchased item abc` // creates purchase with item_id ``` ### Feature Flags ```typescript -import { PostHog } from 'posthog.do' - -const posthog = new PostHog(env.POSTHOG) - -// Simple boolean flag -const enabled = await posthog.isFeatureEnabled('new-checkout-flow', 'user-123') - -// With context for targeting -const variant = await posthog.getFeatureFlag('pricing-experiment', 'user-123', { - personProperties: { - plan: 'pro', - company_size: 50 - }, - groups: { - company: 'acme-corp' - } -}) +// Flags are questions +const enabled = await posthog`is new-checkout-flow on for user-123?` +const variant = await posthog`which pricing-experiment for user-456?` +const flags = await posthog`all flags for user-123` -// Get all flags for a user -const flags = await posthog.getAllFlags('user-123') -// { 'new-checkout-flow': true, 'pricing-experiment': 'variant-b', ... } +// With context - just describe it +const result = await posthog`is premium-feature on for user-123 on pro plan at acme-corp?` ``` ### Experiments ```typescript -import { PostHog } from 'posthog.do' - -const posthog = new PostHog(env.POSTHOG) - -// Get experiment variant (deterministic) -const variant = await posthog.getExperimentVariant('signup-flow', 'user-123') +// Get experiment variant +const variant = await posthog`signup-flow variant for user-123` // 'control' | 'variant-a' | 'variant-b' -// Track experiment exposure -await posthog.capture('$experiment_started', { - distinct_id: 'user-123', - properties: { - $experiment_name: 'signup-flow', - $experiment_variant: variant - } -}) +// Track the whole experiment naturally +await posthog`user-123 started signup-flow experiment as ${variant}` +await posthog`user-123 completed signup in signup-flow experiment` +``` -// Track conversion -await posthog.capture('signup_completed', { - distinct_id: 'user-123', - properties: { - experiment: 'signup-flow', - variant: variant - } -}) +### Funnels + +```typescript +// Funnels read like sentences +await posthog`funnel: pageview -> signup -> purchase in Jan 2024` +await posthog`funnel: landing -> trial -> paid last 30 days`.by('source') +await posthog`funnel: add to cart -> checkout -> payment for pro users` + +// Get conversion rates +const conversion = await posthog`how many users go from trial to paid?` +const dropoff = await posthog`where do users drop off in onboarding?` ``` -### Analytics Queries +### Retention ```typescript -import { PostHog } from 'posthog.do' +// Retention in plain English +await posthog`retention: signup -> return visit last month` +await posthog`retention: first purchase -> second purchase by week` +await posthog`do pro users come back more than free users?` +``` -const posthog = new PostHog(env.POSTHOG) +### Trends -// Funnel analysis -const funnel = await posthog.query.funnel({ - steps: ['$pageview', 'signup_started', 'signup_completed'], - dateRange: { from: '2024-01-01', to: '2024-01-31' } -}) -// { steps: [{ event: '$pageview', count: 1000, dropoff: 0.3 }, ...] } +```typescript +// Trends are just questions +await posthog`signups per day this month` +await posthog`purchases by country last 30 days` +await posthog`pageviews vs signups this week`.by('plan') -// Retention analysis -const retention = await posthog.query.retention({ - startEvent: 'signup_completed', - returnEvent: '$pageview', - dateRange: { from: '2024-01-01', to: '2024-01-31' } -}) -// { cohorts: [{ date: '2024-01-01', size: 100, retention: [1.0, 0.4, 0.3, ...] }] } - -// Event trends -const trends = await posthog.query.trends({ - events: ['$pageview', 'signup_completed'], - breakdown: 'plan', - dateRange: { from: '2024-01-01', to: '2024-01-31' }, - interval: 'day' -}) +// Breakdowns are natural +await posthog`revenue by plan by month in 2024` ``` -### MCP Tools +### Cohorts ```typescript -import { posthogTools, invokeTool } from 'posthog.do/mcp' +// Define cohorts naturally +await posthog`users who signed up in January` +await posthog`power users: 10+ logins last week` +await posthog`churned: no activity in 30 days` + +// Use cohorts in queries +await posthog`retention for power users` +await posthog`funnel for users who came from Google` +``` -// List available analytics tools -console.log(posthogTools.map(t => t.name)) -// ['capture_event', 'get_feature_flag', 'query_funnel', 'query_retention', ...] +### User Identification -// Invoke a tool -const result = await invokeTool('capture_event', { - event: 'task_completed', - distinct_id: 'agent-tom', - properties: { task_type: 'code_review' } -}) +```typescript +// Identify and alias +await posthog`user-123 is john@example.com on pro plan` +await posthog`anonymous-456 is actually user-123` -// AI-native analytics access -await invokeTool('query_funnel', { - natural: 'show me the signup funnel for pro users last month' -}) +// Set properties +await posthog`user-123 upgraded to enterprise` +await posthog`user-123 works at Acme Corp with 50 employees` ``` -## API Overview +## Agent Analytics -### Capture (`posthog.do`) +Every AI agent gets their own analytics instance: -- `capture(event, options)` - Capture single event -- `captureBatch(events)` - Capture multiple events -- `identify(distinctId, properties)` - Set person properties -- `alias(alias, distinctId)` - Link identities +```typescript +import { tom, ralph, priya } from 'agents.do' +import { posthog } from 'posthog.do' -### Feature Flags (`posthog.do/flags`) +// Each agent has isolated analytics +await posthog.for(tom)`reviewed PR 123 and approved` +await posthog.for(ralph)`build completed in 42s` +await posthog.for(priya)`wrote spec for auth feature` -- `isFeatureEnabled(flag, distinctId, context?)` - Boolean check -- `getFeatureFlag(flag, distinctId, context?)` - Get variant -- `getAllFlags(distinctId, context?)` - Get all flags +// Query agent performance +await posthog.for(tom)`average review time this week` +await posthog`which agent completes tasks fastest?` +``` -### Experiments (`posthog.do/experiments`) +## Architecture -- `getExperimentVariant(experiment, distinctId)` - Get bucket -- `getExperimentResults(experiment)` - Get statistics +``` + +-----------------------+ + | posthog.do | + | (Cloudflare Worker) | + +-----------------------+ + | + +---------------+---------------+ + | | | + +------------------+ +------------------+ +------------------+ + | EventsDO (User) | | EventsDO (Agent) | | EventsDO (...) | + | SQLite + AE | | SQLite + AE | | SQLite + AE | + +------------------+ +------------------+ +------------------+ + | | | + +---------------+---------------+ + | + +--------------------+--------------------+ + | | | + +----------+ +-----------+ +------------+ + | KV | | D1 | | Analytics | + | (flags) | | (meta) | | Engine | + +----------+ +-----------+ +------------+ +``` + +**Key insight**: Durable Objects provide single-threaded, strongly consistent state. Each project's analytics is a Durable Object. SQLite handles recent events. Analytics Engine handles aggregations at scale. -### Analytics (`posthog.do/analytics`) +## vs PostHog -- `query.funnel(options)` - Funnel analysis -- `query.retention(options)` - Retention analysis -- `query.trends(options)` - Event trends -- `query.paths(options)` - User paths +| Feature | PostHog | posthog.do | +|---------|---------|------------| +| **Pricing** | $450/month for 2M events | Usage-based, starts free | +| **Self-Hosting** | Complex Kubernetes | Deploy in seconds | +| **Configuration** | UI point-and-click | Natural language | +| **Feature Flags** | Dashboard required | Code-first | +| **AI Agents** | Not supported | First-class citizens | +| **Edge Performance** | Centralized | Global edge | +| **Data Location** | Their servers | Your Cloudflare account | +| **Open Source** | Yes | Yes, MIT licensed | -## The Rewrites Ecosystem +## Promise Pipelining -posthog.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare Durable Objects: +Chain operations without `Promise.all`: + +```typescript +// Find users, analyze, and act - one network round trip +await posthog`users who abandoned checkout` + .map(user => posthog`${user} purchase history`) + .filter(history => history.total > 100) + .notify(`Come back for 10% off!`) + +// Parallel cohort analysis +await posthog`compare pro vs free users` + .retention() + .funnel(`signup -> activation -> payment`) + .export() +``` -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| **posthog.do** | PostHog | Product analytics for AI | -| mongo.do | MongoDB | Document database for AI | -| kafka.do | Kafka | Event streaming for AI | +## MCP Tools -Each rewrite follows the same pattern: -- Durable Object per instance (per agent) -- SQLite for hot tier storage -- Cloudflare primitives for scale (KV, Analytics Engine, D1) -- MCP tools for AI-native access -- Compatible API with the original +AI-native analytics access via Model Context Protocol: + +```typescript +// AI agents query analytics naturally +await agent`analyze our conversion funnel` +// Agent uses posthog MCP tools automatically + +// Available tools +// - capture_event: Track user actions +// - query_funnel: Analyze conversion paths +// - query_retention: Measure user return rates +// - get_feature_flag: Check flag status +// - define_cohort: Create user segments +``` ## Why Durable Objects? 1. **Single-threaded consistency** - No race conditions in event ordering -2. **Per-instance isolation** - Each agent's analytics is completely separate +2. **Per-instance isolation** - Each project's analytics is completely separate 3. **Automatic scaling** - Millions of instances, zero configuration 4. **Global distribution** - Events captured at the edge 5. **SQLite inside** - Real SQL for analytics queries @@ -288,25 +291,18 @@ posthog.do is a core service of [workers.do](https://workers.do) - the platform ```typescript import { priya, ralph, tom, mark } from 'agents.do' -import { PostHog } from 'posthog.do' - -// AI agents with full-stack analytics -const startup = { - product: priya, - engineering: ralph, - tech: tom, - marketing: mark, -} - -// Track agent activity -for (const [role, agent] of Object.entries(startup)) { - const analytics = PostHog.for(agent) - await analytics.capture('agent_started', { - role, - started: new Date(), - status: 'active' - }) -} +import { posthog } from 'posthog.do' + +// Track your AI startup naturally +await posthog`priya wrote spec for auth feature` +await posthog`ralph implemented auth in 2 hours` +await posthog`tom reviewed and approved auth PR` +await posthog`mark wrote launch blog post` + +// Query startup metrics +await posthog`average feature completion time this week` +await posthog`which agent ships fastest?` +await posthog`funnel: spec -> implement -> review -> ship` ``` Both kinds of workers. Working for you. @@ -314,3 +310,16 @@ Both kinds of workers. Working for you. ## License MIT + +--- + +

+ Analytics that speak your language. +
+ Edge-native. AI-first. Open source. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/powerbi/README.md b/rewrites/powerbi/README.md index 6c843f33..6a49ea3e 100644 --- a/rewrites/powerbi/README.md +++ b/rewrites/powerbi/README.md @@ -20,7 +20,13 @@ Microsoft built a BI empire on: A 500-person enterprise? **$60k+/year** for Pro. Premium capacity? **$60k+ additional**. -## The workers.do Way +## AI-Native API + +```typescript +import { powerbi } from 'powerbi.do' // Full SDK +import { powerbi } from 'powerbi.do/tiny' // Minimal client +import { powerbi } from 'powerbi.do/dax' // DAX-only operations +``` Your CFO lives in Excel. Your team needs dashboards. Microsoft wants $120k/year and a Power BI Desktop license for every analyst. @@ -30,7 +36,7 @@ Your CFO lives in Excel. Your team needs dashboards. Microsoft wants $120k/year import { powerbi } from 'powerbi.do' import { priya, mark } from 'agents.do' -// Excel-native analytics +// Talk to it like a colleague const report = await powerbi`analyze ${excelFile} with revenue trends` const dashboard = await powerbi`build KPI dashboard from ${spreadsheet}` const insight = await powerbi`why did margins drop in February?` @@ -82,20 +88,12 @@ The killer feature: true Excel integration without Microsoft: ```typescript import { powerbi } from 'powerbi.do' -// Connect directly to Excel files -const report = await powerbi.fromExcel({ - file: 'sales_data.xlsx', - tables: ['Sales', 'Products', 'Regions'], -}) - -// Or connect to Google Sheets -const report2 = await powerbi.fromSheet({ - spreadsheetId: '1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms', - ranges: ['Sales!A1:Z1000', 'Products!A1:D100'], -}) +// Just point at your data +const report = await powerbi`load sales_data.xlsx with tables Sales Products Regions` +const sheets = await powerbi`connect to Google Sheet ${spreadsheetId}` // Excel formulas work as DAX measures -const report3 = await powerbi` +await powerbi` analyze ${excelFile} with measures: - Total Sales = SUM(Sales[Amount]) - YoY Growth = CALCULATE([Total Sales], SAMEPERIODLASTYEAR('Date'[Date])) @@ -106,7 +104,16 @@ const report3 = await powerbi` ### DAX Compatible -Write DAX formulas exactly like Power BI: +Write DAX formulas exactly like Power BI - or describe them naturally: + +```typescript +// Natural language generates DAX +await powerbi`profit margin as [Profit] / [Revenue] in percent` +await powerbi`YTD sales vs same period last year` +await powerbi`running total of sales by date` +``` + +Or write DAX directly: ```dax // Measures @@ -149,170 +156,114 @@ CALCULATE( ### Data Model -Define relationships and hierarchies: +Define relationships and hierarchies naturally: ```typescript -import { DataModel, Table, Relationship } from 'powerbi.do' - -export const SalesModel = DataModel({ - tables: { - Sales: Table({ - source: 'sales.csv', - columns: { - OrderID: { type: 'integer', key: true }, - Date: { type: 'date' }, - ProductID: { type: 'integer' }, - CustomerID: { type: 'integer' }, - Amount: { type: 'decimal' }, - Quantity: { type: 'integer' }, - }, - }), - - Products: Table({ - source: 'products.csv', - columns: { - ProductID: { type: 'integer', key: true }, - Name: { type: 'string' }, - Category: { type: 'string' }, - Subcategory: { type: 'string' }, - Price: { type: 'decimal' }, - }, - hierarchies: { - 'Product Hierarchy': ['Category', 'Subcategory', 'Name'], - }, - }), - - Date: Table({ - type: 'date', - start: '2020-01-01', - end: '2025-12-31', - // Auto-generates Year, Quarter, Month, Week, Day columns - }), - }, - - relationships: [ - Relationship({ - from: { table: 'Sales', column: 'ProductID' }, - to: { table: 'Products', column: 'ProductID' }, - type: 'many-to-one', - }), - Relationship({ - from: { table: 'Sales', column: 'Date' }, - to: { table: 'Date', column: 'Date' }, - type: 'many-to-one', - }), - ], -}) +import { powerbi } from 'powerbi.do' + +// Load tables from files +await powerbi`load sales.csv as Sales` +await powerbi`load products.csv as Products` +await powerbi`create date table 2020 to 2025` + +// Relationships in plain English +await powerbi`relate Sales.ProductID to Products.ProductID` +await powerbi`relate Sales.Date to Date.Date` + +// Hierarchies +await powerbi`Products hierarchy: Category > Subcategory > Name` + +// Or describe the whole model at once +await powerbi` + model from: + - sales.csv as Sales + - products.csv as Products + - date table 2020 to 2025 + + relationships: + - Sales.ProductID -> Products.ProductID + - Sales.Date -> Date.Date +` ``` ### Reports -Build interactive reports: +Build interactive reports with natural language: ```typescript -import { Report, Page, Visual } from 'powerbi.do' - -export const SalesReport = Report({ - name: 'Sales Analysis', - theme: 'corporate', - pages: [ - Page({ - name: 'Overview', - visuals: [ - Visual.card({ - title: 'Total Revenue', - measure: '[Total Sales]', - format: 'currency', - position: { x: 0, y: 0, width: 200, height: 100 }, - }), - Visual.card({ - title: 'YoY Growth', - measure: '[YoY Growth %]', - format: 'percent', - conditionalFormatting: { - positive: 'green', - negative: 'red', - }, - position: { x: 220, y: 0, width: 200, height: 100 }, - }), - Visual.lineChart({ - title: 'Revenue Trend', - axis: 'Date[Month]', - values: ['[Total Sales]', '[Same Period Last Year]'], - position: { x: 0, y: 120, width: 600, height: 300 }, - }), - Visual.barChart({ - title: 'Sales by Category', - axis: 'Products[Category]', - values: ['[Total Sales]'], - sort: 'descending', - position: { x: 620, y: 120, width: 400, height: 300 }, - }), - ], - slicers: [ - { field: 'Date[Year]', type: 'dropdown' }, - { field: 'Products[Category]', type: 'list' }, - { field: 'Customers[Region]', type: 'tile' }, - ], - }), - Page({ - name: 'Product Details', - visuals: [ - Visual.matrix({ - rows: ['Products[Category]', 'Products[Subcategory]'], - columns: ['Date[Quarter]'], - values: ['[Total Sales]', '[Profit Margin]'], - subtotals: true, - }), - ], - }), - ], -}) +import { powerbi } from 'powerbi.do' + +// Create a report with one sentence +const report = await powerbi`executive dashboard for sales data` + +// Or describe what you want to see +await powerbi` + Sales Analysis report: + - card: Total Revenue as currency + - card: YoY Growth as percent, green if positive red if negative + - line chart: revenue by month vs last year + - bar chart: sales by category descending + + slicers: Year, Category, Region +` + +// Add pages naturally +await powerbi`add Product Details page to ${report}` +await powerbi`matrix: Category > Subcategory by Quarter showing Sales and Margin` + +// AI builds the layout - you refine if needed +await powerbi`make the revenue chart bigger` +await powerbi`move slicers to sidebar` ``` ### Power Query (M) -Transform data with Power Query: +Transform data naturally or with M code: ```typescript -import { PowerQuery } from 'powerbi.do' +import { powerbi } from 'powerbi.do' + +// Natural language transformations +await powerbi`transform sales.csv: filter Amount > 0, add YearMonth column` +await powerbi`unpivot months into Month and Value columns` +await powerbi`split Name column by comma into First and Last` -const transformedSales = PowerQuery(` +// Chain transformations +await powerbi`load sales.csv` + .map(data => powerbi`filter ${data} where Amount > 0`) + .map(data => powerbi`add column YearMonth = Year * 100 + Month`) + .map(data => powerbi`remove duplicates by OrderID`) + +// Or use M code directly when you need precision +await powerbi` + M code: let Source = Csv.Document(File.Contents("sales.csv")), - #"Promoted Headers" = Table.PromoteHeaders(Source), - #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers", { - {"Date", type date}, - {"Amount", type number} - }), - #"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Amount] > 0), - #"Added YearMonth" = Table.AddColumn(#"Filtered Rows", "YearMonth", - each Date.Year([Date]) * 100 + Date.Month([Date]) - ) + Filtered = Table.SelectRows(Source, each [Amount] > 0) in - #"Added YearMonth" -`) + Filtered +` ``` ### Quick Measures -AI-generated DAX: +Describe what you want, get DAX: ```typescript -import { quickMeasure } from 'powerbi.do' +import { powerbi } from 'powerbi.do' -// Describe what you want, get DAX -const measure = await quickMeasure({ - model: SalesModel, - description: 'rolling 3-month average of sales', -}) +// Just say it +const rolling = await powerbi`measure: rolling 3-month average of sales` // Returns: AVERAGEX(DATESINPERIOD('Date'[Date], MAX('Date'[Date]), -3, MONTH), [Total Sales]) -const measure2 = await quickMeasure({ - model: SalesModel, - description: 'year-to-date sales compared to same period last year', -}) +const ytdCompare = await powerbi`measure: YTD sales vs same period last year` // Returns complex DAX with time intelligence + +// Common patterns in plain English +await powerbi`measure: profit margin as Profit / Revenue in percent` +await powerbi`measure: running total of sales by date` +await powerbi`measure: percent of parent category` +await powerbi`measure: rank products by sales descending` ``` ## AI-Native Features @@ -322,23 +273,19 @@ const measure2 = await quickMeasure({ Ask questions about your data: ```typescript -import { qna } from 'powerbi.do' - -// Simple questions -const answer1 = await qna('what were total sales last month?') -// Returns: { value: 1234567, visualization: } +import { powerbi } from 'powerbi.do' -// Comparative questions -const answer2 = await qna('how does Europe compare to North America?') -// Returns: { data: [...], visualization: } +// Just ask +const answer = await powerbi`what were total sales last month?` +const compare = await powerbi`how does Europe compare to North America?` +const trend = await powerbi`show me the sales trend over time` -// Trend questions -const answer3 = await qna('show me the sales trend over time') -// Returns: { data: [...], visualization: } +// Drill down conversationally +await powerbi`break that down by product category` +await powerbi`now just show electronics` +await powerbi`why did margins drop in Q3?` -// Drill-down questions -const answer4 = await qna('break that down by product category') -// Returns: { data: [...], visualization: } +// Answers come with visualizations automatically ``` ### AI Insights @@ -346,21 +293,19 @@ const answer4 = await qna('break that down by product category') Automatic insight discovery: ```typescript -import { insights } from 'powerbi.do' - -const findings = await insights({ - model: SalesModel, - focus: 'Sales[Amount]', - timeframe: 'last quarter', -}) - -// Returns ranked insights -for (const finding of findings) { - console.log(finding.type) // 'anomaly' | 'trend' | 'driver' - console.log(finding.description) // "Sales spiked 45% on March 15" - console.log(finding.explanation) // "Driven by Product X promotion" - console.log(finding.visual) // Auto-generated visualization -} +import { powerbi } from 'powerbi.do' + +// Ask for insights +const findings = await powerbi`insights on sales last quarter` +const anomalies = await powerbi`what's unusual in the data?` +const drivers = await powerbi`what's driving revenue growth?` + +// AI finds and explains patterns automatically: +// - "Sales spiked 45% on March 15, driven by Product X promotion" +// - "West region outperforming by 23% due to new store openings" +// - "Customer churn increased in segment B after price change" + +// Each insight comes with a visualization ``` ### Smart Narratives @@ -368,37 +313,38 @@ for (const finding of findings) { Auto-generated text summaries: ```typescript -import { smartNarrative } from 'powerbi.do' +import { powerbi } from 'powerbi.do' -const narrative = await smartNarrative({ - visual: salesTrendChart, - style: 'executive-summary', -}) +// Generate executive summaries +const summary = await powerbi`summarize sales performance for the board` +// "Sales increased 12% quarter-over-quarter, driven primarily by strong +// performance in Electronics (+23%). West region led at 18%, Northeast declined 5%..." -// Returns: "Sales increased 12% quarter-over-quarter, driven primarily -// by strong performance in the Electronics category (+23%). The West -// region led growth at 18%, while the Northeast declined 5%..." +// Different tones for different audiences +await powerbi`explain this chart for the CEO` +await powerbi`technical summary of data quality issues` +await powerbi`weekly metrics email for the sales team` ``` ### AI Agents as Analysts -AI agents can create complete reports: +AI agents create complete reports with promise pipelining: ```typescript -import { priya, mark } from 'agents.do' +import { priya, mark, tom } from 'agents.do' import { powerbi } from 'powerbi.do' -// Product manager builds analytics -const productReport = await priya` - create a Power BI report analyzing our product usage data - include adoption metrics, engagement scores, and churn indicators -` +// Agents work like analysts +const productReport = await priya`analyze our product usage data` + .visualize() // auto-generates Power BI report + .share() // sends to stakeholders -// Marketing builds campaign report -const campaignReport = await mark` - build a report showing our marketing campaign performance - with spend, conversions, and ROI by channel -` +// Chain agents for review workflows +const report = await mark`build campaign performance report` + .map(draft => [priya, tom].map(r => r`review ${draft}`)) + .map(approved => powerbi`publish ${approved} to workspace`) + +// One round trip - natural language throughout ``` ## Real-Time Streaming @@ -406,57 +352,25 @@ const campaignReport = await mark` No refresh limitations: ```typescript -import { StreamingDataset } from 'powerbi.do' - -const realtimeSales = StreamingDataset({ - name: 'Real-time Sales', - columns: { - timestamp: { type: 'datetime' }, - product: { type: 'string' }, - amount: { type: 'number' }, - region: { type: 'string' }, - }, -}) - -// Push data in real-time -for await (const sale of salesStream) { - await realtimeSales.push({ - timestamp: new Date(), - product: sale.product, - amount: sale.amount, - region: sale.region, - }) -} +import { powerbi } from 'powerbi.do' -// Report updates automatically +// Create a streaming dataset +const stream = await powerbi`stream: Real-time Sales with timestamp product amount region` + +// Push data naturally +await powerbi`push to Real-time Sales: ${sale}` + +// Or pipe from any source +await powerbi`stream from ${kafkaTopic} to Real-time Sales` +await powerbi`stream from ${webhookUrl} to Real-time Sales` + +// Reports update automatically - no 8/day limit ``` ## Embedding Embed reports in your applications: -### JavaScript SDK - -```typescript -import { PowerBIEmbed } from 'powerbi.do/embed' - -const embed = new PowerBIEmbed({ - container: document.getElementById('report'), - report: 'sales-analysis', - accessToken: await getEmbedToken(), - settings: { - filterPaneEnabled: false, - navContentPaneEnabled: true, - }, -}) - -// Interact programmatically -await embed.setFilters([ - { table: 'Date', column: 'Year', values: [2024] }, -]) -const data = await embed.exportData() -``` - ### React Component ```tsx @@ -465,10 +379,9 @@ import { PowerBIReport } from 'powerbi.do/react' function Dashboard() { return ( console.log(data)} - height={600} /> ) } @@ -477,16 +390,25 @@ function Dashboard() { ### Row-Level Security ```typescript -import { createEmbedToken } from 'powerbi.do/embed' - -const token = await createEmbedToken({ - report: 'sales-analysis', - roles: ['RegionalManager'], - filters: { - 'Customers[Region]': user.region, // Row-level security - }, - expiry: Date.now() + 3600000, -}) +import { powerbi } from 'powerbi.do' + +// Embed with security in one line +const token = await powerbi`embed sales-analysis for ${user.email} filtered to ${user.region}` + +// RLS just works - users see only their data +``` + +### JavaScript SDK + +```typescript +import { powerbi } from 'powerbi.do' + +// Embed and control naturally +const embed = await powerbi`embed sales-analysis in ${container}` + +// Interact naturally +await powerbi`filter ${embed} to Year 2024` +await powerbi`export ${embed} data to Excel` ``` ## Architecture @@ -537,29 +459,32 @@ DAX Expression Parse Tree Execution Plan ### Column Store -Power BI uses columnar storage for performance: +Columnar storage for fast aggregations - automatic optimization: ```typescript -// Data stored in column format for fast aggregations -ColumnStore({ - table: 'Sales', - columns: { - Date: Int32Array, // Dates as integers - ProductID: Int32Array, // Dictionary encoded - Amount: Float64Array, // Native numbers - }, - dictionaries: { - ProductID: ['Widget A', 'Widget B', ...], - }, - indexes: { - Date: BitIndex, // Bitmap index for filtering - }, -}) +import { powerbi } from 'powerbi.do' + +// You don't configure storage - it optimizes automatically +// Dates as integers, dictionary encoding, bitmap indexes +// All handled by the engine + +// Just query naturally +await powerbi`sum of Amount by Date for last 5 years` // <10ms ``` ## Data Connections -Connect to any source: +Connect to any source naturally: + +```typescript +import { powerbi } from 'powerbi.do' + +// Just say where your data is +await powerbi`connect to ${postgresUrl}` +await powerbi`import from Snowflake sales_db.transactions` +await powerbi`load data from ${restApiEndpoint}` +await powerbi`sync with Google Sheet ${spreadsheetId}` +``` | Source | Type | Notes | |--------|------|-------| @@ -577,16 +502,21 @@ Connect to any source: ## Migration from Power BI -### Export Your Reports +### Import Your Reports ```bash -# Export PBIX file -# (Manual export from Power BI Desktop) - -# Import to powerbi.do +# One command migration npx powerbi-migrate import ./report.pbix ``` +```typescript +import { powerbi } from 'powerbi.do' + +// Or import programmatically +await powerbi`import ${pbixFile}` +await powerbi`migrate workspace from Power BI service` +``` + ### DAX Compatibility Full DAX function support: diff --git a/rewrites/rippling/README.md b/rewrites/rippling/README.md index 02390bee..88378fcf 100644 --- a/rewrites/rippling/README.md +++ b/rewrites/rippling/README.md @@ -4,6 +4,37 @@ You're a startup founder. You just hired employee #25. You need to create their email, add them to Slack, provision GitHub, ship a laptop, set up payroll, enroll benefits... Two systems. Duplicate data. Manual processes. And both want $35/employee/month. Your growth shouldn't come with a per-seat tax. +## AI-Native API + +```typescript +import { rippling } from 'rippling.do' // Full SDK +import { rippling } from 'rippling.do/tiny' // Minimal client +import { rippling } from 'rippling.do/hr' // HR-only operations +``` + +Natural language for people operations: + +```typescript +import { rippling, rio } from 'rippling.do' + +// Talk to it like a colleague +const sarah = await rippling`Sarah Chen` +const engineers = await rippling`engineers in Austin` +const onboarding = await rippling`new hires this month` + +// Chain like sentences +await rippling`hire Sarah Chen as Engineer` + .provision(['github', 'linear', 'aws']) + .shipLaptop() + .scheduleOnboarding() + +// Offboarding that handles itself +await rippling`Sarah's last day is Friday` + .revokeAccess() + .recoverDevices() + .transferFiles() +``` + ## The workers.do Way ```typescript @@ -71,18 +102,10 @@ npx create-dotdo rippling Your unified people + IT platform is live. ```typescript -import { rippling } from 'rippling.do' +import { rippling, rio } from 'rippling.do' -// Add an employee - everything else happens automatically -await rippling.employees.hire({ - name: 'Sarah Chen', - email: 'sarah@startup.com', - role: 'Software Engineer', - department: 'Engineering', - startDate: '2025-01-15', - manager: 'alex-kim', - location: 'San Francisco' -}) +// Hire in natural language +await rippling`hire Sarah Chen as Software Engineer in Engineering` // This triggers: // - Email account creation (Google Workspace) @@ -99,42 +122,22 @@ await rippling.employees.hire({ The magic is the **employee record as the source of truth**. ```typescript -// The employee record -const sarah = { - id: 'sarah-chen', - name: 'Sarah Chen', - email: 'sarah@startup.com', - department: 'Engineering', - role: 'Software Engineer', - manager: 'alex-kim', - startDate: '2025-01-15', - status: 'active' -} - -// This drives EVERYTHING: - -// Apps (what SaaS they can access) -sarah.apps = ['slack', 'github', 'notion', 'figma', 'linear'] - -// Devices (what hardware they have) -sarah.devices = ['macbook-pro-m3-sarah'] - -// Access (what permissions they have) -sarah.access = { - github: { teams: ['engineering'], role: 'member' }, - slack: { channels: ['#engineering', '#all-hands'] }, - aws: { role: 'developer', accounts: ['staging'] } -} +// Query anyone naturally +const sarah = await rippling`Sarah Chen` +const engineers = await rippling`engineers in San Francisco` +const onPTO = await rippling`employees on PTO this week` + +// AI infers what you need +await rippling`Sarah Chen` // returns employee +await rippling`Sarah's devices` // returns device list +await rippling`Sarah's app access` // returns provisioned apps ``` When Sarah's record changes, everything updates: ```typescript -// Sarah gets promoted to Tech Lead -await rippling.employees.update('sarah-chen', { - role: 'Tech Lead', - level: 'L5' -}) +// Promote in natural language +await rippling`promote Sarah Chen to Tech Lead` // Automatically: // - GitHub: Added to 'tech-leads' team @@ -147,11 +150,8 @@ await rippling.employees.update('sarah-chen', { When Sarah leaves: ```typescript -// Sarah leaves the company -await rippling.employees.terminate('sarah-chen', { - lastDay: '2025-06-30', - reason: 'resignation' -}) +// Offboard naturally +await rippling`Sarah's last day is June 30` // Automatically: // - All app access revoked within minutes @@ -170,25 +170,26 @@ await rippling.employees.terminate('sarah-chen', { Everything about a person in one place. ```typescript -const employee = await rippling.employees.get('sarah-chen') +// Get employee with natural language +const sarah = await rippling`Sarah Chen` // HR data -employee.name // Sarah Chen -employee.department // Engineering -employee.manager // Alex Kim -employee.salary // $150,000 -employee.startDate // 2025-01-15 +sarah.name // Sarah Chen +sarah.department // Engineering +sarah.manager // Alex Kim +sarah.salary // $150,000 +sarah.startDate // 2025-01-15 // IT data -employee.email // sarah@startup.com -employee.apps // ['slack', 'github', 'notion', ...] -employee.devices // ['macbook-pro-m3-sarah'] -employee.accessLevel // L3 (department-specific access) +sarah.email // sarah@startup.com +sarah.apps // ['slack', 'github', 'notion', ...] +sarah.devices // ['macbook-pro-m3-sarah'] +sarah.accessLevel // L3 (department-specific access) // Computed -employee.tenure // 2 months -employee.isFullyProvisioned // true -employee.complianceStatus // 'green' +sarah.tenure // 2 months +sarah.isFullyProvisioned // true +sarah.complianceStatus // 'green' ``` ### App Provisioning @@ -196,45 +197,19 @@ employee.complianceStatus // 'green' Connect your SaaS apps. Provisioning becomes automatic. ```typescript -// Connect apps -await rippling.apps.connect('google-workspace', { - serviceAccount: process.env.GOOGLE_SERVICE_ACCOUNT, - domain: 'startup.com' -}) - -await rippling.apps.connect('slack', { - token: process.env.SLACK_ADMIN_TOKEN -}) - -await rippling.apps.connect('github', { - appId: process.env.GITHUB_APP_ID, - installationId: process.env.GITHUB_INSTALLATION_ID, - privateKey: process.env.GITHUB_PRIVATE_KEY -}) - -// Define provisioning rules -await rippling.provisioning.rule({ - name: 'Engineering apps', - when: { department: 'Engineering' }, - provision: [ - { app: 'github', team: 'engineering', role: 'member' }, - { app: 'linear', team: 'engineering' }, - { app: 'slack', channels: ['#engineering', '#dev'] }, - { app: 'notion', workspace: 'engineering' }, - { app: 'aws', role: 'developer', accounts: ['staging'] } - ] -}) - -await rippling.provisioning.rule({ - name: 'Tech Lead apps', - when: { role: 'Tech Lead' }, - provision: [ - { app: 'github', team: 'tech-leads', role: 'maintainer' }, - { app: 'slack', channels: ['#tech-leads', '#architecture'] }, - { app: 'aws', role: 'tech-lead', accounts: ['staging', 'production'] }, - { app: 'pagerduty', role: 'responder' } - ] -}) +// Connect apps in natural language +await rippling`connect Google Workspace for startup.com` +await rippling`connect Slack` +await rippling`connect GitHub` + +// Define provisioning rules naturally +await rippling`when engineers join`.provision(['github', 'linear', 'slack', 'notion', 'aws']) +await rippling`when tech leads join`.provision(['pagerduty', 'aws production']) +await rippling`when managers join`.provision(['lattice', 'expensify']) + +// Or just describe what you want +await rippling`engineers get GitHub, Linear, Slack #engineering, and AWS staging` +await rippling`tech leads also get production AWS and PagerDuty` ``` ### Device Management @@ -242,42 +217,22 @@ await rippling.provisioning.rule({ Laptops, phones, monitors - tracked and managed. ```typescript -// Inventory management -await rippling.devices.add({ - type: 'laptop', - model: 'MacBook Pro 14" M3', - serialNumber: 'C02XX1234567', - purchaseDate: '2025-01-01', - cost: 2499, - status: 'in-stock' -}) +// Add devices naturally +await rippling`add MacBook Pro M3 serial C02XX1234567` +await rippling`add 27" LG monitor to inventory` -// Assign to employee -await rippling.devices.assign('macbook-C02XX1234567', { - employee: 'sarah-chen', - assignedDate: '2025-01-15' -}) +// Assign to employees +await rippling`give Sarah the new MacBook` +await rippling`ship laptop to Jamie in Austin` -// MDM integration -await rippling.devices.enroll('macbook-C02XX1234567', { - mdm: 'jamf', - profile: 'engineering-standard' -}) - -// Track all devices -const devices = await rippling.devices.list() -// [ -// { serial: 'C02XX1234567', type: 'laptop', assignee: 'sarah-chen', status: 'active' }, -// { serial: 'C02YY7654321', type: 'laptop', assignee: null, status: 'in-stock' }, -// { serial: 'MON123456', type: 'monitor', assignee: 'alex-kim', status: 'active' } -// ] +// Track devices +await rippling`Sarah's devices` +await rippling`all laptops in stock` +await rippling`devices needing refresh` // Off-boarding device recovery -await rippling.devices.recover('macbook-C02XX1234567', { - action: 'return', - returnAddress: 'Office HQ', - wipeAfterReturn: true -}) +await rippling`recover Sarah's laptop`.returnTo('Office HQ').wipeOnReturn() +await rippling`wipe Jamie's MacBook remotely` ``` ### Access Control @@ -285,41 +240,19 @@ await rippling.devices.recover('macbook-C02XX1234567', { Manage who can access what. ```typescript -// Define access levels -await rippling.access.defineLevel('L1', { - name: 'Basic Access', - description: 'New employees, contractors', - apps: { - slack: { channels: ['#all-hands'] }, - notion: { pages: ['public-wiki'] }, - google: { drive: 'view-only' } - } -}) - -await rippling.access.defineLevel('L2', { - name: 'Team Access', - inherits: 'L1', - description: 'Full-time employees', - apps: { - slack: { channels: ['#team-${department}'] }, - notion: { pages: ['${department}-workspace'] }, - google: { drive: 'team-drive' } - } -}) - -await rippling.access.defineLevel('L3', { - name: 'Sensitive Access', - inherits: 'L2', - description: 'Senior employees', - requires: { tenure: '6 months', review: 'manager' }, - apps: { - aws: { accounts: ['staging'] }, - github: { repos: ['private-*'] } - } -}) - -// Employees automatically get access based on role + tenure -// No manual provisioning needed +// Define access levels naturally +await rippling`new employees get Slack #all-hands, Notion public wiki, view-only Drive` +await rippling`full-time employees also get team channels and team drive` +await rippling`after 6 months with manager approval, employees get AWS staging and private repos` + +// Grant access on demand +await rippling`Sarah needs Datadog access`.approvedUntil('90 days') +await rippling`give Jamie production AWS access` +await rippling`add the interns to #summer-2025` + +// Revoke just as easily +await rippling`remove Jamie's production access` +await rippling`revoke all access for terminated employees` ``` ### IT Service Desk @@ -327,31 +260,18 @@ await rippling.access.defineLevel('L3', { Employees request, IT approves, system provisions. ```typescript -// Employee requests software -await rippling.requests.create({ - employee: 'sarah-chen', - type: 'app-access', - app: 'datadog', - reason: 'Need to debug production issues', - urgency: 'medium' -}) - -// IT approves -await rippling.requests.approve('request-123', { - approver: 'it-admin', - notes: 'Approved for 90 days' -}) - -// System automatically provisions -// - Datadog account created -// - SSO configured -// - Employee notified -// - Access logged - -// Track all requests -const requests = await rippling.requests.list({ - status: 'pending' -}) +// Employees request naturally +await rippling`Sarah requests Datadog for debugging production` +await rippling`Jamie needs a new monitor for home office` + +// IT approves with one line +await rippling`approve Sarah's Datadog request for 90 days` +await rippling`approve Jamie's monitor, ship to home address` + +// Track requests +await rippling`pending IT requests` +await rippling`requests this week` +await rippling`Jamie's open requests` ``` ### Compliance @@ -359,33 +279,20 @@ const requests = await rippling.requests.list({ Track software licenses, access reviews, security. ```typescript -// Software license tracking -const licenses = await rippling.compliance.licenses() -// { -// slack: { type: 'per-seat', count: 45, used: 42, cost: '$12/seat/mo' }, -// figma: { type: 'per-seat', count: 20, used: 18, cost: '$15/seat/mo' }, -// github: { type: 'per-seat', count: 50, used: 35, cost: '$4/seat/mo' } -// } +// License tracking +await rippling`how many Slack seats are we using?` +await rippling`unused Figma licenses` +await rippling`software spend this quarter` // Access reviews -await rippling.compliance.accessReview({ - scope: 'all-employees', - apps: ['aws', 'github', 'notion'], - reviewers: ['department-managers'], - due: '2025-03-01' -}) +await rippling`run AWS access review by March 1` +await rippling`who has production access?` +await rippling`access not reviewed in 90 days` // Security posture -const security = await rippling.compliance.security() -// { -// mfaEnrollment: '98%', -// ssoAdoption: '100%', -// deviceCompliance: '95%', -// issues: [ -// { type: 'mfa-disabled', employees: ['jamie-wong'], risk: 'medium' }, -// { type: 'device-outdated', devices: ['macbook-C02ZZ123'], risk: 'low' } -// ] -// } +await rippling`security issues` +await rippling`employees without MFA` +await rippling`devices with outdated OS` ``` ## AI Assistant @@ -606,25 +513,14 @@ export const MyAppConnector = Connector({ name: 'my-app', auth: 'oauth2', - provision: async (employee, config) => { - // Create account in my-app - await myAppApi.createUser({ - email: employee.email, - name: employee.name, - role: config.role - }) - }, - - deprovision: async (employee) => { - // Remove account - await myAppApi.deleteUser(employee.email) - }, - - sync: async () => { - // Reconcile state - const users = await myAppApi.listUsers() - return users - } + // What happens when someone joins + provision: (employee) => myAppApi.createUser(employee), + + // What happens when someone leaves + deprovision: (employee) => myAppApi.deleteUser(employee.email), + + // Keep state in sync + sync: () => myAppApi.listUsers() }) ``` @@ -633,60 +529,20 @@ export const MyAppConnector = Connector({ The rules engine that makes it all work. ```typescript -// Rules are evaluated in order -// First match wins (unless 'additive: true') - -await rippling.provisioning.rules([ - // Everyone gets these - { - name: 'All employees', - when: { status: 'active' }, - provision: [ - { app: 'google-workspace' }, - { app: 'slack', channels: ['#all-hands', '#random'] }, - { app: '1password', vault: 'company' } - ] - }, - - // Department-specific - { - name: 'Engineering', - when: { department: 'Engineering' }, - additive: true, - provision: [ - { app: 'github', org: 'startup', team: 'engineering' }, - { app: 'linear', team: 'engineering' }, - { app: 'aws', role: 'developer', accounts: ['staging'] } - ] - }, - - // Role-specific - { - name: 'Managers', - when: { hasDirectReports: true }, - additive: true, - provision: [ - { app: 'slack', channels: ['#managers'] }, - { app: 'lattice', role: 'manager' }, - { app: 'expensify', role: 'approver' } - ] - }, - - // Special access - { - name: 'Production access', - when: { - department: 'Engineering', - level: { gte: 'L4' }, - tenure: { gte: '6 months' } - }, - additive: true, - provision: [ - { app: 'aws', accounts: ['production'], role: 'senior-developer' }, - { app: 'pagerduty', role: 'responder' } - ] - } -]) +// Rules read like sentences +await rippling`everyone gets Google Workspace, Slack, and 1Password` +await rippling`engineers also get GitHub, Linear, and AWS staging` +await rippling`managers also get Slack #managers, Lattice, and Expensify` + +// Conditional access reads naturally +await rippling`senior engineers with 6+ months get production AWS and PagerDuty` +await rippling`after SOC 2 training, employees can access customer data` + +// Chain provisioning with promise pipelining +await rippling`new engineers this quarter` + .map(emp => rippling`provision ${emp} with standard engineering stack`) + .map(emp => rippling`ship laptop to ${emp}`) + .map(emp => rippling`schedule onboarding call for ${emp}`) ``` ## Self-Service IT @@ -694,23 +550,14 @@ await rippling.provisioning.rules([ Employees handle common IT tasks themselves. ```typescript -// Employee self-service portal -await rippling.self.requestApp('sarah-chen', 'datadog', { - reason: 'Debugging production issues' -}) - -await rippling.self.reportIssue('sarah-chen', { - type: 'device', - device: 'macbook-C02XX1234567', - issue: 'Battery draining quickly', - priority: 'low' -}) - -await rippling.self.requestDevice('sarah-chen', { - type: 'monitor', - model: 'LG 27" 4K', - reason: 'Remote work setup' -}) +// Employees request what they need naturally +await rippling`I need Datadog for debugging production` +await rippling`my laptop battery is draining fast` +await rippling`I need a monitor for my home office` + +// Managers can act on behalf of reports +await rippling`Sarah needs Figma for the new project` +await rippling`order a standing desk for Jamie` ``` ## Pricing @@ -731,11 +578,10 @@ await rippling.self.requestDevice('sarah-chen', { Import from Rippling: ```typescript -import { migrate } from 'rippling.do/migrate' +import { rippling } from 'rippling.do' -await migrate.fromRippling({ - apiKey: process.env.RIPPLING_API_KEY -}) +// One line migration +await rippling`import everything from Rippling` // Imports: // - All employee records diff --git a/rewrites/salesforce/README.md b/rewrites/salesforce/README.md index da9db300..c7dca50b 100644 --- a/rewrites/salesforce/README.md +++ b/rewrites/salesforce/README.md @@ -1,97 +1,70 @@ # salesforce.do -> You're a startup founder. You need CRM. Salesforce wants $500k/year and a 12-month implementation. Your sales team needs to close deals, not wait for consultants. +> CRM for sales teams. Edge-Native. Open by Default. AI-First. -

- The $300B CRM. Reimagined. Open Source. AI-Native. -

- -

- npm version - npm downloads - license -

+Salesforce charges $25-330 per user per month. Einstein AI costs extra. Implementation takes months. Apex is proprietary. SOQL is locked in. Your customer data lives on their servers. A 100-person sales team on Enterprise with Einstein: **$315,000/year**. ---- +**salesforce.do** is the open-source alternative. Deploy in minutes, not months. AI-native from day one. Your data, your infrastructure, your rules. -## The workers.do Way - -Talk to your CRM like a colleague: +## AI-Native API ```typescript -import { salesforce, sally, priya, tom } from 'workers.do' - -// Natural language queries -const pipeline = await salesforce`show pipeline for Q1` -const leads = await salesforce`find enterprise leads from ${campaign}` -const forecast = await salesforce`forecast revenue for next quarter` - -// AI agents work your CRM -await sally`qualify all new leads from yesterday` -await sally`send follow-up to deals with no activity in 7 days` +import { salesforce } from 'salesforce.do' // Full SDK +import { salesforce } from 'salesforce.do/tiny' // Minimal client +import { salesforce } from 'salesforce.do/soql' // SOQL-only operations ``` -### Promise Pipelining - -Chain operations without waiting. One network round trip: +Natural language for sales workflows: ```typescript -// Find leads, send proposals, get reviews - all pipelined -const closed = await salesforce`find qualified leads over $50k` - .map(lead => sally`send proposal to ${lead}`) - .map(proposal => [priya, tom].map(r => r`review ${proposal}`)) +import { salesforce } from 'salesforce.do' -// AI-driven pipeline management -const atRisk = await salesforce`find stalled opportunities` - .map(opp => priya`analyze risk factors for ${opp}`) - .map(analysis => sally`create re-engagement plan for ${analysis}`) -``` +// Talk to it like a colleague +const pipeline = await salesforce`deals closing this quarter` +const hot = await salesforce`hot leads from website this week` +const stalled = await salesforce`opportunities with no activity in 2 weeks` ---- +// Chain like sentences +await salesforce`qualified leads over $50k` + .notify(`Ready to send proposal?`) -Salesforce is the world's largest enterprise software company. They charge $25-330 per user per month. Einstein AI costs extra. Implementation takes months. Apex is proprietary. SOQL is locked in. Your customer data lives on their servers. - -**salesforce.do** is the open-source alternative. Deploy your own Salesforce org in one click. SOQL-compatible. Apex-compatible. AI agents are first-class citizens. Your data, your infrastructure, your rules. +// Deals that update themselves +await salesforce`new deal Acme Corp $150k enterprise` + .track() // activity logging + .forecast() // AI predicts close + .alert() // notify on changes +``` ## The Problem -Salesforce built a $300B empire on a simple model: charge per seat, forever. +Salesforce charges $25-330 per user per month: -| What Salesforce Charges | The Real Cost | +| What Salesforce Charges | The Reality | |------------------------|---------------| -| **Essentials** | $25/user/month (stripped down) | -| **Professional** | $80/user/month (no customization) | -| **Enterprise** | $165/user/month (most popular) | -| **Unlimited** | $330/user/month (everything) | +| **Implementation** | $50k-500k+ (3-18 months) | +| **Annual Maintenance** | $165-330/user/month | | **Einstein AI** | +$50-150/user/month on top | -| **Implementation** | $50k-500k+ (consultants) | -| **Annual commitment** | Required (no monthly) | - -**A 100-person sales team on Enterprise with Einstein: $315,000/year.** - -And you still don't own your data. You can't run it yourself. You're locked into Apex, SOQL, and their ecosystem. Leaving means rewriting everything. +| **Admin Salary** | $80k-150k/year (required) | +| **Vendor Lock-in** | Apex/SOQL proprietary | ### The Einstein Tax -Salesforce's AI strategy is pure extraction: - -- Einstein Activity Capture: +$50/user/month -- Einstein Opportunity Scoring: +$75/user/month -- Einstein Lead Scoring: +$75/user/month -- Einstein Forecasting: +$75/user/month -- Einstein GPT: +$50+/user/month +Since Einstein launched: +- Lead Scoring: +$75/user/month +- Opportunity Scoring: +$75/user/month +- Forecasting: +$75/user/month +- GPT: +$50+/user/month Want AI features? Double your bill. -### The Implementation Nightmare +### The Integration Nightmare -Average Salesforce implementation: -- **Timeline**: 3-18 months -- **Consulting fees**: $50k-500k+ -- **Admin salary**: $80k-150k/year (you need a full-time admin) -- **Developer costs**: $120k-200k/year (for Apex customization) - -Total first-year cost for a mid-market company: **$500k-1M+** +Salesforce talks API-first. But: +- SOQL is proprietary +- Apex can't run anywhere else +- Data exports are painful +- Custom objects trap your schema +- Implementation takes months ## The Solution @@ -104,24 +77,17 @@ $25-330/user/month $0 - run your own Einstein AI premium AI-native from day one Vendor lock-in Open source, MIT licensed Their servers Your Cloudflare account -Apex proprietary TypeScript (Apex compatible) -SOQL queries SOQL queries (100% compatible) -Custom Objects Custom Objects (100% compatible) -Triggers Triggers (TypeScript) -Flows Workflows (code-first) -18-month implementation 60-second deployment +18-month implementation Deploy in minutes $150k/year admin Self-managing with AI ``` ---- - ## One-Click Deploy ```bash npx create-dotdo salesforce ``` -That's it. Your own Salesforce org. Running on Cloudflare's global edge network. +A full CRM. Running on infrastructure you control. SOQL-compatible from day one. ```typescript import { Salesforce } from 'salesforce.do' @@ -129,886 +95,262 @@ import { Salesforce } from 'salesforce.do' export default Salesforce({ name: 'my-org', domain: 'crm.my-company.com', - edition: 'unlimited', // All features, always }) ``` -Or deploy manually: - -```bash -git clone https://github.com/dotdo/salesforce.do -cd salesforce.do -npm install -npm run deploy -``` - -Your own Salesforce. In 60 seconds. Forever. - ---- - ## Features -### Sales Cloud - -Everything you need to manage your sales pipeline: - -#### Accounts & Contacts +### Accounts & Contacts ```typescript -import { sf } from 'salesforce.do' - -// Create an account -const account = await sf.Account.create({ - Name: 'Acme Corporation', - Industry: 'Technology', - AnnualRevenue: 5000000, - Website: 'https://acme.com', - BillingCity: 'San Francisco', - BillingState: 'CA', -}) +// Create anyone +const acme = await salesforce`create account Acme Corp in Technology, $5M revenue` +const alice = await salesforce`Alice Chen VP Engineering at Acme, alice@acme.com` -// Create contacts at that account -await sf.Contact.create({ - FirstName: 'Alice', - LastName: 'Chen', - Email: 'alice@acme.com', - Title: 'VP of Engineering', - AccountId: account.Id, -}) - -// Relationship queries -const accountWithContacts = await sf.query(` - SELECT Id, Name, - (SELECT Id, Name, Email, Title FROM Contacts) - FROM Account - WHERE Id = '${account.Id}' -`) +// Find anyone +await salesforce`Alice Chen` // returns contact +await salesforce`contacts at Acme` // returns list +await salesforce`VPs at accounts over $10M` // AI infers what you need ``` -#### Leads & Lead Conversion +### Leads ```typescript -// Create a lead -const lead = await sf.Lead.create({ - FirstName: 'Bob', - LastName: 'Smith', - Company: 'StartupXYZ', - Email: 'bob@startupxyz.com', - LeadSource: 'Web', - Status: 'Open', -}) +// New leads, one line +await salesforce`new lead Bob Smith from StartupXYZ, website inquiry` -// AI-powered lead scoring (built in, no extra charge) -const score = await sf.einstein.scoreLead(lead.Id) +// AI scores automatically (no Einstein tax) +await salesforce`score Bob Smith` // { score: 87, factors: ['Title match', 'Company size', 'Engagement'] } -// Convert lead to account/contact/opportunity -const conversion = await sf.Lead.convert(lead.Id, { - createOpportunity: true, - opportunityName: 'StartupXYZ - Enterprise Deal', -}) +// Convert when ready +await salesforce`convert Bob Smith to opportunity` ``` -#### Opportunities & Pipeline +### Opportunities ```typescript -// Create an opportunity -const opp = await sf.Opportunity.create({ - Name: 'Acme Corp - Enterprise License', - AccountId: account.Id, - Amount: 150000, - CloseDate: '2025-03-31', - StageName: 'Qualification', - Probability: 20, -}) +// Deals are one line +await salesforce`new deal Acme Corp $150k enterprise, close March` +await salesforce`move Acme deal to proposal stage` +await salesforce`close Acme deal won` -// Move through stages -await sf.Opportunity.update(opp.Id, { - StageName: 'Proposal/Price Quote', - Probability: 60, -}) - -// AI-powered forecasting (built in) -const forecast = await sf.einstein.forecastOpportunity(opp.Id) -// { predictedClose: '2025-03-15', confidence: 0.78, risk: 'medium' } +// Pipeline queries read like questions +await salesforce`deals closing this month` +await salesforce`pipeline by stage` +await salesforce`deals over $100k at risk` -// Pipeline analytics -const pipeline = await sf.query(` - SELECT StageName, SUM(Amount), COUNT(Id) - FROM Opportunity - WHERE IsClosed = false - GROUP BY StageName -`) +// AI forecasting (built in) +await salesforce`forecast Q1` ``` -#### Activities & Tasks +### Activities ```typescript -// Log a call -await sf.Task.create({ - Subject: 'Discovery Call with Alice', - WhoId: contact.Id, - WhatId: opp.Id, - Type: 'Call', - Status: 'Completed', - Description: 'Discussed technical requirements...', - ActivityDate: new Date(), -}) +// Log activities naturally +await salesforce`called Alice about technical requirements` +await salesforce`emailed Acme proposal` +await salesforce`meeting with Bob Tuesday 2pm` -// Schedule follow-up -await sf.Task.create({ - Subject: 'Send proposal', - WhoId: contact.Id, - WhatId: opp.Id, - OwnerId: currentUser.Id, - ActivityDate: nextWeek, - Priority: 'High', -}) - -// AI generates activity summary -const summary = await sf.einstein.summarizeActivities(opp.Id) +// Query activity +await salesforce`Acme activity this week` +await salesforce`deals with no activity in 7 days` ``` -### Service Cloud - -Full customer support platform: - -#### Cases +### Cases ```typescript -// Create a case -const supportCase = await sf.Case.create({ - Subject: 'Cannot login to dashboard', - Description: 'Getting 403 error when trying to access...', - ContactId: contact.Id, - AccountId: account.Id, - Priority: 'High', - Origin: 'Email', - Status: 'New', -}) +// Support in one line +await salesforce`new case from Acme: login issues, high priority` +await salesforce`escalate Acme case to tier 2` +await salesforce`close Acme case resolved` -// AI auto-classifies and routes -await sf.einstein.classifyCase(supportCase.Id) -// Automatically sets: Type, Category, Assignment, Priority - -// Escalation rules -await sf.Case.escalate(supportCase.Id, { - reason: 'SLA breach imminent', - escalateTo: 'Tier 2 Support', -}) +// AI classifies and routes automatically +// AI suggests knowledge articles ``` -#### Knowledge Base +### Campaigns ```typescript -// Create knowledge article -await sf.Knowledge__kav.create({ - Title: 'How to reset your password', - UrlName: 'reset-password', - Summary: 'Step-by-step guide to resetting...', - ArticleBody__c: '1. Click "Forgot Password"...', - IsPublished: true, -}) - -// AI-powered article suggestions for cases -const suggestions = await sf.einstein.suggestArticles(supportCase.Id) - -// Attach article to case -await sf.CaseArticle.create({ - CaseId: supportCase.Id, - KnowledgeArticleId: suggestions[0].Id, -}) -``` - -#### Entitlements & SLAs - -```typescript -// Define SLA -const sla = await sf.Entitlement.create({ - Name: 'Premium Support', - AccountId: account.Id, - StartDate: '2025-01-01', - EndDate: '2025-12-31', - SlaProcessId: premiumSlaProcess.Id, -}) - -// SLA milestones tracked automatically -const milestones = await sf.query(` - SELECT Id, Name, TargetDate, CompletionDate, IsViolated - FROM CaseMilestone - WHERE CaseId = '${supportCase.Id}' -`) -``` - -### Marketing Cloud (Built In) - -No separate product, no separate licensing: - -#### Campaigns - -```typescript -// Create campaign -const campaign = await sf.Campaign.create({ - Name: 'Q1 Product Launch', - Type: 'Email', - Status: 'Planned', - StartDate: '2025-01-15', - EndDate: '2025-02-15', - BudgetedCost: 50000, -}) - -// Add members -await sf.CampaignMember.create({ - CampaignId: campaign.Id, - LeadId: lead.Id, - Status: 'Sent', -}) - -// Track responses -await sf.CampaignMember.update(member.Id, { - Status: 'Responded', - FirstRespondedDate: new Date(), -}) - -// ROI tracking -const roi = await sf.query(` - SELECT Id, Name, NumberOfLeads, NumberOfConvertedLeads, - AmountWonOpportunities, ActualCost - FROM Campaign - WHERE Id = '${campaign.Id}' -`) +// Campaigns read like briefs +await salesforce`create Q1 launch campaign, email, $50k budget` +await salesforce`add website leads to Q1 campaign` +await salesforce`Q1 campaign ROI` ``` -#### Email Automation +### Custom Objects ```typescript -// Define email template -const template = await sf.EmailTemplate.create({ - Name: 'Welcome Email', - Subject: 'Welcome to {!Account.Name}', - Body: 'Hi {!Contact.FirstName}...', - FolderId: marketingFolder.Id, -}) +// Define custom objects naturally +await salesforce`create Invoice object with amount, status, account lookup` -// Send via workflow -await sf.workflows.trigger('send-welcome-email', { - contactId: contact.Id, - templateId: template.Id, -}) +// Use them immediately +await salesforce`new invoice for Acme $5000 due next Friday` +await salesforce`Acme invoices this quarter` +await salesforce`overdue invoices` ``` -### Custom Objects +## Promise Pipelining -Define your own objects with full metadata support: +Chain operations without waiting. One network round trip: ```typescript -import { SObject } from 'salesforce.do' - -// Define custom object -export const Invoice__c = SObject({ - name: 'Invoice__c', - label: 'Invoice', - pluralLabel: 'Invoices', - description: 'Customer invoices', - - fields: { - InvoiceNumber__c: { - type: 'AutoNumber', - format: 'INV-{0000}', - }, - Amount__c: { - type: 'Currency', - required: true, - precision: 16, - scale: 2, - }, - Status__c: { - type: 'Picklist', - values: ['Draft', 'Sent', 'Paid', 'Overdue', 'Cancelled'], - default: 'Draft', - }, - Account__c: { - type: 'Lookup', - referenceTo: 'Account', - relationshipName: 'Invoices', - }, - DueDate__c: { - type: 'Date', - }, - LineItems__c: { - type: 'MasterDetail', - referenceTo: 'InvoiceLineItem__c', - }, - }, - - validationRules: [ - { - name: 'Amount_Must_Be_Positive', - condition: 'Amount__c <= 0', - errorMessage: 'Invoice amount must be greater than zero', - }, - ], - - triggers: { - afterInsert: async (records, context) => { - // Auto-notify accounting - for (const invoice of records) { - await context.notify('accounting@company.com', { - subject: `New Invoice: ${invoice.InvoiceNumber__c}`, - body: `Amount: $${invoice.Amount__c}`, - }) - } - }, - beforeUpdate: async (newRecords, oldRecords, context) => { - for (const [invoice, oldInvoice] of zip(newRecords, oldRecords)) { - if (invoice.Status__c === 'Sent' && oldInvoice.Status__c === 'Draft') { - invoice.SentDate__c = new Date() - } - } - }, - }, -}) -``` - -### Apex-Compatible Triggers - -Write triggers in TypeScript with Salesforce semantics: +import { salesforce, sally, priya, tom } from 'workers.do' -```typescript -import { Trigger } from 'salesforce.do' - -export const OpportunityTrigger = Trigger('Opportunity', { - // Before triggers - modify records before save - beforeInsert: async (newRecords, context) => { - for (const opp of newRecords) { - // Auto-set probability based on stage - opp.Probability = context.getStageDefaultProbability(opp.StageName) - } - }, - - beforeUpdate: async (newRecords, oldRecords, context) => { - for (const [opp, oldOpp] of zip(newRecords, oldRecords)) { - // Validate stage transitions - if (!isValidStageTransition(oldOpp.StageName, opp.StageName)) { - throw new TriggerError(`Invalid stage transition: ${oldOpp.StageName} -> ${opp.StageName}`) - } - } - }, - - // After triggers - perform related operations - afterUpdate: async (newRecords, oldRecords, context) => { - const closedWon = [] - - for (const [opp, oldOpp] of zip(newRecords, oldRecords)) { - // Opportunity just closed won - if (opp.StageName === 'Closed Won' && oldOpp.StageName !== 'Closed Won') { - closedWon.push(opp) - } - } - - if (closedWon.length > 0) { - // Bulk insert tasks - await context.insert('Task', closedWon.map(opp => ({ - Subject: `Onboard ${opp.Name}`, - WhatId: opp.Id, - OwnerId: opp.OwnerId, - ActivityDate: addDays(new Date(), 7), - Priority: 'High', - }))) - - // Notify sales manager - await context.notify('sales-manager', { - type: 'deal_closed', - opportunities: closedWon, - }) - } - }, - - afterDelete: async (oldRecords, context) => { - // Log deletions for compliance - await context.insert('AuditLog__c', oldRecords.map(opp => ({ - ObjectType__c: 'Opportunity', - RecordId__c: opp.Id, - Action__c: 'Delete', - DeletedBy__c: context.userId, - DeletedAt__c: new Date(), - }))) - }, -}) -``` +// Find leads, send proposals, get reviews - all pipelined +await salesforce`qualified leads over $50k` + .map(lead => sally`send proposal to ${lead}`) + .map(proposal => [priya, tom].map(r => r`review ${proposal}`)) -### Flows (Process Automation) +// AI-driven pipeline management +await salesforce`stalled opportunities` + .map(opp => priya`analyze risk factors for ${opp}`) + .map(analysis => sally`create re-engagement plan for ${analysis}`) -Visual workflow builder with code-first option: +// Bulk outreach +await salesforce`enterprise leads from Q1 campaign` + .map(lead => sally`personalized follow-up for ${lead}`) + .schedule() -```typescript -import { Flow } from 'salesforce.do' - -export const LeadNurture = Flow({ - name: 'Lead Nurture Sequence', - description: 'Automated lead nurturing workflow', - - trigger: { - type: 'record', - object: 'Lead', - when: 'Created', - condition: 'LeadSource = "Web"', - }, - - variables: { - emailsSent: { type: 'Number', default: 0 }, - lastEmailDate: { type: 'Date' }, - }, - - steps: [ - // Immediate welcome email - { - id: 'welcome', - type: 'sendEmail', - template: 'Welcome_Lead', - to: '{!$Record.Email}', - then: 'wait1', - }, - - // Wait 3 days - { - id: 'wait1', - type: 'wait', - duration: '3d', - then: 'checkEngagement', - }, - - // Check if they've engaged - { - id: 'checkEngagement', - type: 'decision', - conditions: [ - { - name: 'Engaged', - condition: '{!$Record.HasOpened__c} = true', - then: 'sendCaseStudy', - }, - { - name: 'Not Engaged', - then: 'sendFollowUp', - }, - ], - }, - - // Branch: engaged leads get case study - { - id: 'sendCaseStudy', - type: 'sendEmail', - template: 'Case_Study', - then: 'assignToSdr', - }, - - // Branch: not engaged get follow-up - { - id: 'sendFollowUp', - type: 'sendEmail', - template: 'Follow_Up_1', - then: 'wait2', - }, - - // Another wait - { - id: 'wait2', - type: 'wait', - duration: '5d', - then: 'checkAgain', - }, - - // Final check - { - id: 'checkAgain', - type: 'decision', - conditions: [ - { - name: 'Still No Engagement', - condition: '{!$Record.HasOpened__c} = false AND {!$Record.HasClicked__c} = false', - then: 'markCold', - }, - { name: 'Default', then: 'assignToSdr' }, - ], - }, - - // Mark as cold - { - id: 'markCold', - type: 'updateRecord', - record: '{!$Record}', - values: { Status: 'Cold', Rating: 'Cold' }, - }, - - // Assign to SDR - { - id: 'assignToSdr', - type: 'assignRecord', - record: '{!$Record}', - assignmentRules: 'SDR_Round_Robin', - then: 'createTask', - }, - - // Create follow-up task - { - id: 'createTask', - type: 'createRecord', - object: 'Task', - values: { - Subject: 'Follow up with {!$Record.Name}', - WhoId: '{!$Record.Id}', - OwnerId: '{!$Record.OwnerId}', - ActivityDate: '{!$Flow.CurrentDate + 1}', - Priority: 'High', - }, - }, - ], -}) +// Close care gaps at scale +await salesforce`deals with no activity in 2 weeks` + .map(deal => sally`check in on ${deal}`) + .track() ``` ---- - ## SOQL Compatible -Write SOQL queries exactly like Salesforce: - -```sql --- Basic query -SELECT Id, Name, Amount, StageName, CloseDate -FROM Opportunity -WHERE StageName != 'Closed Lost' - AND Amount > 50000 - AND CloseDate = THIS_QUARTER -ORDER BY Amount DESC -LIMIT 10 - --- Relationship query (parent) -SELECT Id, Name, Account.Name, Account.Industry -FROM Contact -WHERE Account.AnnualRevenue > 1000000 - --- Relationship query (child) -SELECT Id, Name, - (SELECT Id, LastName, Email FROM Contacts), - (SELECT Id, Amount, StageName FROM Opportunities WHERE IsClosed = false) -FROM Account -WHERE Industry = 'Technology' - --- Aggregate query -SELECT StageName, SUM(Amount) totalAmount, COUNT(Id) dealCount -FROM Opportunity -WHERE CloseDate = THIS_YEAR -GROUP BY StageName -HAVING SUM(Amount) > 100000 -ORDER BY SUM(Amount) DESC - --- Date functions -SELECT Id, Name -FROM Lead -WHERE CreatedDate = LAST_N_DAYS:30 - AND ConvertedDate = null - --- TYPEOF for polymorphic relationships -SELECT Id, Subject, - TYPEOF What - WHEN Account THEN Name, Industry - WHEN Opportunity THEN Amount, StageName - END -FROM Task -WHERE Status = 'Completed' -``` - -The SOQL parser compiles to SQLite with full optimization: +Natural language or SOQL - your choice: ```typescript -import { sf } from 'salesforce.do' +// Natural language +await salesforce`deals over $100k closing this month` -// Direct SOQL -const opps = await sf.query(` - SELECT Id, Name, Amount +// Or SOQL when you need precision +await salesforce.query(` + SELECT Id, Name, Amount, CloseDate FROM Opportunity WHERE Amount > 100000 + AND CloseDate = THIS_MONTH + AND IsClosed = false `) - -// Query builder (type-safe) -const opps = await sf.Opportunity - .select('Id', 'Name', 'Amount') - .where('Amount', '>', 100000) - .orderBy('Amount', 'desc') - .limit(10) - .execute() ``` ---- - ## AI-Native CRM -### AI SDR (Sales Development Representative) +### AI SDR -Your AI SDR qualifies leads 24/7: +Sally qualifies leads 24/7: ```typescript import { sally } from 'agents.do' -import { sf } from 'salesforce.do' // Sally is your AI SDR -await sally` - Review all new leads from yesterday. - Score them based on our ICP criteria. - Send personalized outreach to hot leads. - Schedule discovery calls for engaged prospects. -` +await sally`qualify all new leads from yesterday` +await sally`send personalized outreach to hot leads` +await sally`schedule discovery calls for engaged prospects` // She updates Salesforce directly -// - Lead.Status updated to 'Working' -// - Lead.Rating set based on analysis -// - Tasks created for follow-ups +// - Lead status updated // - Emails sent via connected inbox // - Meetings scheduled via calendar ``` ### AI Pipeline Manager -AI keeps your pipeline healthy: +Priya keeps your pipeline healthy: ```typescript import { priya } from 'agents.do' -import { sf } from 'salesforce.do' -// Priya manages pipeline hygiene -await priya` - Find all opportunities with no activity in 14+ days. - Analyze each one for risk factors. - Create tasks for the opportunity owners. - Flag deals that should be moved to "At Risk". -` +// Pipeline hygiene +await priya`find deals with no activity in 2 weeks` +await priya`flag at-risk opportunities` +await priya`create tasks for stalled deals` // AI forecasting (no Einstein tax) -const forecast = await sf.einstein.forecast({ - period: 'Q1', - team: 'North America', -}) - -console.log(forecast) +await salesforce`forecast Q1 North America` // { // committed: 2_450_000, // bestCase: 3_200_000, -// pipeline: 5_800_000, -// atRisk: [{ oppId: '...', reason: 'Champion left company' }], -// recommendations: ['Accelerate Acme deal', 'Add multi-thread at BigCorp'] +// atRisk: ['Acme - champion left company'] // } ``` ### AI Case Resolution -AI handles tier-1 support automatically: +Quinn handles tier-1 support: ```typescript import { quinn } from 'agents.do' -import { sf } from 'salesforce.do' - -// Quinn handles incoming cases -sf.Case.on('created', async (caseRecord) => { - // AI analyzes the case - const analysis = await quinn` - Analyze this support case: - Subject: ${caseRecord.Subject} - Description: ${caseRecord.Description} - - 1. Classify the issue type - 2. Search knowledge base for solutions - 3. Determine if AI can resolve or needs human - ` - - if (analysis.canResolve) { - // AI sends response - await sf.CaseComment.create({ - ParentId: caseRecord.Id, - Body: analysis.response, - IsPublished: true, - }) - - // Close if customer confirms - await sf.Case.update(caseRecord.Id, { - Status: 'Awaiting Customer Response', - AI_Resolved__c: true, - }) - } else { - // Route to human with context - await sf.Case.update(caseRecord.Id, { - OwnerId: analysis.suggestedQueue, - AI_Summary__c: analysis.summary, - AI_Suggested_Solution__c: analysis.suggestedSolution, - }) - } -}) + +// AI handles incoming cases automatically +// - Classifies the issue +// - Searches knowledge base +// - Responds or routes to human +await quinn`handle incoming support cases` ``` -### Natural Language Queries +### Natural Language Everything Skip SOQL entirely: ```typescript -import { sf } from 'salesforce.do' - -// Natural language to results -const deals = await sf`show me all deals over $100k closing this month` -const contacts = await sf`who are the VPs at our top 10 accounts by revenue?` -const atRisk = await sf`which opportunities are at risk of slipping?` +// Just ask +await salesforce`deals over $100k closing this month` +await salesforce`VPs at our top 10 accounts` +await salesforce`opportunities at risk of slipping` // AI-powered insights -const insights = await sf`what should I focus on today?` -// "You have 3 opportunities with meetings today totaling $450k. -// The Acme deal ($150k) has gone quiet - consider a check-in. -// Two new leads match your ideal customer profile and should be prioritized." -``` - -### MCP Tools for Every Object - -Every SObject exposes MCP tools for AI agents: - -```typescript -import { sfTools } from 'salesforce.do/mcp' - -// All standard and custom objects have CRUD tools -console.log(sfTools.map(t => t.name)) -// [ -// 'Account_create', 'Account_read', 'Account_update', 'Account_delete', 'Account_query', -// 'Contact_create', 'Contact_read', 'Contact_update', 'Contact_delete', 'Contact_query', -// 'Opportunity_create', 'Opportunity_read', 'Opportunity_update', 'Opportunity_delete', 'Opportunity_query', -// 'Lead_create', 'Lead_read', 'Lead_update', 'Lead_delete', 'Lead_query', 'Lead_convert', -// 'Case_create', 'Case_read', 'Case_update', 'Case_delete', 'Case_query', 'Case_escalate', -// 'Task_create', 'Task_read', 'Task_update', 'Task_delete', 'Task_query', -// 'Invoice__c_create', 'Invoice__c_read', ... // Custom objects too! -// ] - -// AI agents invoke directly -await invokeTool('Opportunity_create', { - Name: 'New Deal from AI SDR', - AccountId: '001...', - Amount: 75000, - StageName: 'Qualification', - CloseDate: '2025-06-30', -}) +await salesforce`what should I focus on today?` +// "You have 3 meetings totaling $450k. +// Acme deal has gone quiet - consider a check-in. +// Two new leads match your ICP." ``` ---- - ## jsforce Compatible -Your existing jsforce integrations work unchanged: +Existing jsforce integrations work unchanged: ```typescript import jsforce from 'jsforce' -// Point at your salesforce.do instance const conn = new jsforce.Connection({ loginUrl: 'https://your-org.salesforce.do', }) -await conn.login('user@example.com', 'password') - // All jsforce APIs work -const accounts = await conn.query('SELECT Id, Name FROM Account LIMIT 10') - -const result = await conn.sobject('Lead').create({ - FirstName: 'New', - LastName: 'Lead', - Company: 'Test Corp', -}) - -await conn.sobject('Opportunity').update({ - Id: '006...', - StageName: 'Proposal/Price Quote', -}) - -// Bulk operations -const bulkResult = await conn.sobject('Contact') - .insertBulk(contactsToInsert) - -// Streaming API -conn.streaming.topic('/topic/NewLeads').subscribe((message) => { - console.log('New lead:', message.sobject) -}) +const accounts = await conn.query('SELECT Id, Name FROM Account') +await conn.sobject('Lead').create({ FirstName: 'New', LastName: 'Lead', Company: 'Test' }) ``` ### Migration from Salesforce -One-command migration: - ```bash npx salesforce.do migrate --from=production - -# Migrates: -# - All standard objects -# - Custom objects and fields -# - Record data -# - Relationships -# - Users and permissions -# - Reports and dashboards -# - Workflows and flows -# - Apex triggers (converted to TypeScript) +# Migrates everything: objects, fields, data, users, workflows ``` ---- - ## Architecture ### Durable Object per Org -Each Salesforce org runs in a dedicated Durable Object: - ``` -┌─────────────────────────────────────────────────────────────────────┐ -│ salesforce.do Worker │ -├─────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌─────────────────────────────────────────────────────────────────┐│ -│ │ OrgDO (per customer) ││ -│ ├─────────────────────────────────────────────────────────────────┤│ -│ │ ││ -│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││ -│ │ │ Metadata │ │ Schema │ │ Users & │ ││ -│ │ │ Engine │ │ Cache │ │ Perms │ ││ -│ │ └─────────────┘ └─────────────┘ └─────────────┘ ││ -│ │ ││ -│ │ ┌─────────────────────────────────────────────────────────────┐││ -│ │ │ SOQL Query Engine │││ -│ │ │ ┌─────────┐ ┌─────────────┐ ┌────────────────┐ │││ -│ │ │ │ Parser │─▶│ Optimizer │─▶│ SQLite Compiler│ │││ -│ │ │ └─────────┘ └─────────────┘ └────────────────┘ │││ -│ │ └─────────────────────────────────────────────────────────────┘││ -│ │ ││ -│ │ ┌─────────────────────────────────────────────────────────────┐││ -│ │ │ Trigger Framework │││ -│ │ │ before/after × insert/update/delete/undelete │││ -│ │ └─────────────────────────────────────────────────────────────┘││ -│ │ ││ -│ └─────────────────────────────────────────────────────────────────┘│ -│ │ -├─────────────────────────────────────────────────────────────────────┤ -│ SQLite (hot data) │ R2 (attachments) │ Vectorize (AI search) │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -### Dynamic Metadata Engine - -Custom objects and fields are managed at runtime: - -```typescript -// Create custom object at runtime -await sf.describe.createSObject({ - fullName: 'Project__c', - label: 'Project', - fields: [ - { fullName: 'Status__c', type: 'Picklist', values: ['New', 'Active', 'Complete'] }, - { fullName: 'Budget__c', type: 'Currency' }, - ], -}) - -// Schema is immediately queryable -const projects = await sf.query('SELECT Id, Name, Status__c FROM Project__c') + OrgDO (per customer) + | + +-- AccountsDO (accounts, contacts) + | |-- SQLite: Account/Contact records + | +-- R2: Documents, attachments + | + +-- SalesDO (leads, opportunities, activities) + | |-- SQLite: Sales data + | +-- Search indexes + | + +-- ServiceDO (cases, knowledge) + | |-- SQLite: Support data + | + +-- MarketingDO (campaigns, members) + |-- SQLite: Campaign data ``` ### Storage Tiers @@ -1016,7 +358,7 @@ const projects = await sf.query('SELECT Id, Name, Status__c FROM Project__c') | Tier | Storage | Use Case | Query Speed | |------|---------|----------|-------------| | **Hot** | SQLite | Active records (< 2 years) | <10ms | -| **Warm** | R2 + SQLite | Historical data (2-7 years) | <100ms | +| **Warm** | R2 + SQLite Index | Historical data (2-7 years) | <100ms | | **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | ### Multi-Tenancy @@ -1027,109 +369,57 @@ bigcorp.salesforce.do <- BigCorp's org startup.salesforce.do <- Startup's org ``` -Each org is completely isolated: -- Separate Durable Object -- Separate SQLite database -- Separate R2 bucket -- No data mixing, ever - ---- - -## Pricing Comparison - -### Salesforce Pricing (2025) - -| Edition | Per User/Month | 100 Users/Year | -|---------|---------------|----------------| -| Essentials | $25 | $30,000 | -| Professional | $80 | $96,000 | -| Enterprise | $165 | $198,000 | -| Unlimited | $330 | $396,000 | -| + Einstein AI | +$50-150 | +$60,000-180,000 | - -**Plus**: Implementation ($50k-500k), Admin salary ($100k+), Developer costs ($150k+) - -### salesforce.do Pricing - -| Resource | Cost | Notes | -|----------|------|-------| -| Durable Object | $0.15/million requests | Your org | -| SQLite Storage | $0.20/GB/month | All your data | -| R2 Storage | $0.015/GB/month | Attachments | -| Workers | Free tier: 100k/day | API requests | -| AI (optional) | ~$0.01/query | Via llm.do | - -**Example: 100 users, 50,000 records, 100k API calls/month** +Each org is completely isolated. Separate database, separate storage, no data mixing. -| | Salesforce Enterprise | salesforce.do | -|-|----------------------|---------------| -| Software | $198,000/year | ~$20/month | -| AI features | +$120,000/year | ~$10/month | -| Implementation | $100,000 | $0 | -| Admin | $120,000/year | AI-managed | -| **Total Year 1** | **$538,000** | **~$360** | +## vs Salesforce -**Savings: 99.93%** - ---- +| Feature | Salesforce | salesforce.do | +|---------|------------|---------------| +| **Implementation** | $50k-500k+ | Deploy in minutes | +| **Annual Cost** | $165-330/user/month | ~$20/month | +| **AI Features** | +$50-150/user/month | Built in | +| **Data Location** | Salesforce servers | Your Cloudflare account | +| **Customization** | Apex/SOQL proprietary | TypeScript | +| **Lock-in** | Years of migration | MIT licensed | ## Roadmap -### Completed - -- [x] Standard Objects (Account, Contact, Lead, Opportunity, Case, Task, Event) -- [x] Custom Objects with full metadata -- [x] SOQL parser and query engine -- [x] Relationship queries (parent and child) -- [x] Aggregate functions -- [x] Triggers (all 8 contexts) -- [x] jsforce compatibility layer -- [x] AI-native querying -- [x] MCP tools for all objects -- [x] Lead conversion -- [x] Case management - -### In Progress - -- [ ] Process Builder / Flows (visual + code) -- [ ] Reports and Dashboards +### Core CRM +- [x] Accounts & Contacts +- [x] Leads & Conversion +- [x] Opportunities & Pipeline +- [x] Activities & Tasks +- [x] Cases +- [x] Campaigns +- [x] Custom Objects +- [ ] Reports & Dashboards - [ ] Approval Processes -- [ ] Permission Sets and Profiles -- [ ] Field-Level Security -- [ ] Sharing Rules -### Planned +### Compatibility +- [x] SOQL Parser +- [x] jsforce API +- [x] MCP Tools +- [ ] Apex Runtime +- [ ] Lightning Components -- [ ] Lightning Web Components runtime -- [ ] Apex runtime (TypeScript-compatible subset) -- [ ] Data Loader compatibility -- [ ] Salesforce DX compatibility -- [ ] CPQ (Configure, Price, Quote) -- [ ] Field Service - ---- +### AI +- [x] Natural Language Queries +- [x] Lead Scoring +- [x] Forecasting +- [x] Case Classification +- [ ] Predictive Analytics ## Contributing -salesforce.do is open source under the MIT license. Contributions welcome. +salesforce.do is open source under the MIT license. ```bash git clone https://github.com/dotdo/salesforce.do cd salesforce.do pnpm install pnpm test -pnpm dev ``` -Key contribution areas: -- SOQL parser extensions -- Standard object implementations -- jsforce compatibility -- AI/MCP integrations -- Documentation - ---- - ## License MIT License - Use it however you want. @@ -1139,7 +429,7 @@ MIT License - Use it however you want.

The $300B CRM monopoly ends here.
- Your data. Your infrastructure. Your AI. Your CRM. + Your data. Your infrastructure. Your AI.

Website | Docs | diff --git a/rewrites/sap/README.md b/rewrites/sap/README.md index 0523d162..d040f600 100644 --- a/rewrites/sap/README.md +++ b/rewrites/sap/README.md @@ -1,82 +1,56 @@ # sap.do -> Enterprise ERP for Manufacturing. AI-Native. Zero Complexity. +> Enterprise ERP for Manufacturing. Edge-Native. Open by Default. AI-First. -[![npm version](https://img.shields.io/npm/v/sap.do.svg)](https://www.npmjs.com/package/sap.do) -[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) - ---- - -SAP is the $200B gorilla of enterprise software. S/4HANA implementations cost $10M-$100M+. Even "lightweight" SAP Business One requires $50,000-200,000 in licensing plus consultants charging $200-400/hour. +SAP is the $200B gorilla of enterprise software. S/4HANA implementations cost $10M-$100M+. Even "lightweight" SAP Business One requires $50,000-200,000 in licensing plus consultants charging $200-400/hour. ABAP developers are increasingly rare. Customization means vendor lock-in. **sap.do** is the open-source alternative. Manufacturing-grade ERP on Cloudflare Workers. AI runs your MRP. Deploy in minutes, not years. OData compatible for existing integrations. -## The workers.do Way - -You're running a manufacturing business. You need MRP, BOMs, production orders, quality control. SAP wants $10M and 3 years. Your competitors are shipping while you're still in "discovery." - -**workers.do** gives you an AI manufacturing team that understands your shop floor: +## AI-Native API ```typescript -import { sap } from 'sap.do' -import { ops, planner, quality } from 'agents.do' - -// Natural language manufacturing operations -const mrp = await sap`plan production for ${salesOrder}` -const bom = await sap`explode BOM for ${finishedGood} with all components` -const capacity = await sap`check capacity for ${workCenter} next ${weeks} weeks` +import { sap } from 'sap.do' // Full SDK +import { sap } from 'sap.do/tiny' // Minimal client +import { sap } from 'sap.do/mrp' // MRP-only operations ``` -### Promise Pipelining - -Chain complex manufacturing workflows with a single network round trip: +Natural language for manufacturing workflows: ```typescript -const produced = await sap`create production order for ${material}` - .map(order => sap`release ${order} to shop floor`) - .map(order => sap`confirm operations for ${order}`) - .map(order => [quality, ops].map(r => r`inspect ${order}`)) -``` - -### AI Agents for Manufacturing +import { sap } from 'sap.do' -```typescript -// Planner agent handles MRP -await planner` - Run MRP for next quarter. - Identify materials with lead time risk. - Suggest purchase orders to place this week. -` +// Talk to it like a planner +const gaps = await sap`materials below safety stock` +const overdue = await sap`open purchase orders over $10k` +const critical = await sap`production orders late this week` -// Shop floor agent handles execution -await shopfloor`What should I work on next?` -// "Production order PRD-001 for FINISHED-001 is highest priority. -// Materials staged at bin STAGING-01. -// Estimated completion: 2.5 hours." +// Chain like sentences +await sap`raw materials expiring in 30 days` + .notify(`Expedite usage or dispose`) -// Quality agent handles inspections -await quality` - Predict quality outcomes for incoming batch from ${vendor}. - Flag any high-risk materials for 100% inspection. -` +// Production that runs itself +await sap`create production order for 100 FINISHED-001` + .release() // to shop floor + .confirm() // operations complete + .receipt() // goods to inventory ``` -One import. Natural language. Your AI manufacturing team. - ## The Problem SAP has become synonymous with "enterprise complexity": -| The SAP Tax | Reality | -|-------------|---------| -| SAP Business One | $95-150/user/month + implementation | -| SAP S/4HANA Cloud | $350-500+/user/month | -| Implementation | 6-24 months, often fails | -| Consultants | $200-400/hour (Deloitte, Accenture, etc.) | -| Technical debt | ABAP developers increasingly rare | -| Total cost | 3-5x software cost in services | +| What SAP Charges | The Reality | +|------------------|-------------| +| **SAP Business One** | $95-150/user/month + implementation | +| **SAP S/4HANA Cloud** | $350-500+/user/month | +| **Implementation** | 6-24 months, often fails | +| **Consultants** | $200-400/hour (Deloitte, Accenture, etc.) | +| **Technical debt** | ABAP developers increasingly rare | +| **Total cost** | 3-5x software cost in services | + +### The SAP Tax -**The dirty secret**: Most SAP implementations: +Most SAP implementations: - Take 2x longer than planned - Cost 3x the original budget - Deliver 50% of promised functionality @@ -86,708 +60,352 @@ Meanwhile, SAP's core value proposition - **materials planning for manufacturing ## The Solution -**sap.do** brings manufacturing ERP to the edge: +**sap.do** reimagines manufacturing ERP for the edge: + +``` +SAP S/4HANA sap.do +----------------------------------------------------------------- +$10M-100M implementation Deploy in minutes +$1-5M/year maintenance $0 - run your own +ABAP (1983 technology) TypeScript +Consultant dependency Self-service +AI as "innovation" AI at the core +Batch MRP (nightly) Real-time MRP +Vendor lock-in Open source, MIT licensed +``` + +## One-Click Deploy ```bash npx create-dotdo sap ``` -Your own SAP alternative. Running on Cloudflare. AI-native from day one. +A manufacturing ERP. Running on infrastructure you control. AI-native from day one. -| SAP | sap.do | -|-----|--------| -| $95-500+/user/month | **Free** (open source) | -| 6-24 month implementation | **Minutes** | -| ABAP (1983 technology) | **TypeScript** | -| Consultant dependency | **Self-service** | -| AI as "innovation" | **AI at the core** | -| Vendor lock-in | **OData v4 compatible** | +```typescript +import { SAP } from 'sap.do' ---- +export default SAP({ + name: 'acme-manufacturing', + domain: 'erp.acme-mfg.com', + modules: ['mm', 'pp', 'sd', 'qm'], +}) +``` ## Features ### Material Master -The foundation of manufacturing ERP. Every item, every attribute, every view. - ```typescript -import { sap } from 'sap.do' +// Create materials naturally +await sap`create material MAT-001: Aluminum Extrusion 6061-T6, raw, KG, $4.50` +await sap`create material FINISHED-001: Widget Assembly, finished, EA, $45.00` -// Create material master -await sap.materials.create({ - number: 'MAT-001', - description: 'Aluminum Extrusion 6061-T6', - type: 'raw', // raw, semifinished, finished, service - baseUnit: 'KG', - - // Purchasing view - purchasing: { - purchasingGroup: 'RAW-METALS', - orderUnit: 'KG', - minOrderQty: 100, - standardVendor: 'VENDOR-001', - plannedDeliveryDays: 14 - }, - - // Inventory view - inventory: { - valuationClass: 'RAW-MATERIALS', - priceControl: 'moving-average', - standardPrice: 4.50 - }, - - // MRP view - mrp: { - mrpType: 'PD', // MRP - mrpController: 'MRP-001', - lotSize: 'EQ', // Exact lot - safetyStock: 500, - plannedDeliveryDays: 14 - }, - - // Quality view - quality: { - inspectionType: 'incoming', - controlKey: 'QM-01' - } -}) +// AI infers what you need +await sap`MAT-001` // returns material +await sap`MAT-001 inventory` // returns stock levels +await sap`MAT-001 where used` // returns BOM usage +await sap`materials below safety stock` // returns shortage list ``` -### Bill of Materials (BOM) - -Define what goes into what. Multi-level, with variants and alternatives. +### Bill of Materials ```typescript -// Create a product BOM -await sap.bom.create({ - material: 'FINISHED-001', - plant: 'PLANT-01', - usage: 'production', // production, engineering, costing - baseQuantity: 1, - validFrom: '2025-01-01', - - components: [ - { - material: 'MAT-001', - quantity: 2.5, - unit: 'KG', - operation: '0010' // Links to routing - }, - { - material: 'MAT-002', - quantity: 4, - unit: 'EA', - operation: '0020' - }, - { - material: 'SEMI-001', - quantity: 1, - unit: 'EA', - operation: '0030', - // Phantom - explodes into its components - phantom: true - } - ] -}) +// Define BOMs naturally +await sap` + FINISHED-001 BOM: + - 2.5 KG MAT-001 at operation 10 + - 4 EA MAT-002 at operation 20 + - 1 EA SEMI-001 at operation 30 phantom +` -// Explode BOM to see full component tree -const explosion = await sap.bom.explode('FINISHED-001', { - plant: 'PLANT-01', - levels: 'all', // or specific depth - includePhantoms: false -}) -``` +// Explode to see full component tree +await sap`explode BOM for FINISHED-001 at PLANT-01 all levels` -### Routing / Work Centers +// What-if analysis +await sap`cost impact if MAT-001 price increases 10%` +``` -Define how things get made. +### Work Centers and Routing ```typescript -// Define work center -await sap.workCenters.create({ - id: 'WC-ASSEMBLY-01', - plant: 'PLANT-01', - description: 'Assembly Line 1', - capacity: { - available: 8, // hours per day - utilization: 0.85, - efficiency: 0.95 - }, - costCenter: 'CC-PROD-01', - activityTypes: ['labor', 'machine'] -}) - -// Create routing for finished product -await sap.routings.create({ - material: 'FINISHED-001', - plant: 'PLANT-01', - usage: 'production', - - operations: [ - { - number: '0010', - description: 'Cut raw material', - workCenter: 'WC-CUT-01', - setupTime: 15, // minutes - machineTime: 5, // per unit - laborTime: 3 - }, - { - number: '0020', - description: 'Machine components', - workCenter: 'WC-CNC-01', - setupTime: 30, - machineTime: 12, - laborTime: 2 - }, - { - number: '0030', - description: 'Final assembly', - workCenter: 'WC-ASSEMBLY-01', - setupTime: 10, - machineTime: 0, - laborTime: 20 - }, - { - number: '0040', - description: 'Quality inspection', - workCenter: 'WC-QC-01', - setupTime: 5, - laborTime: 10, - controlKey: 'QM-01' - } - ] -}) +// Define work centers +await sap`create work center WC-CNC-01 at PLANT-01, 8 hours daily, 85% utilization` +await sap`create work center WC-ASSEMBLY-01 at PLANT-01, 8 hours daily` + +// Define routing +await sap` + FINISHED-001 routing: + - Op 10: Cut raw material at WC-CUT-01, 15 min setup, 5 min/unit + - Op 20: Machine components at WC-CNC-01, 30 min setup, 12 min/unit + - Op 30: Final assembly at WC-ASSEMBLY-01, 10 min setup, 20 min/unit + - Op 40: Quality inspection at WC-QC-01, 5 min setup, 10 min/unit +` ``` -### Material Requirements Planning (MRP) - -The heart of manufacturing ERP. Now AI-powered. +### Material Requirements Planning ```typescript -// Run MRP for a plant -const mrpRun = await sap.mrp.run({ - plant: 'PLANT-01', - planningHorizon: '90 days', - scope: 'all' // or specific materials -}) +// Run MRP naturally +await sap`run MRP for PLANT-01 next 90 days` +await sap`what materials do we need to order this week?` +await sap`which work centers are overloaded next month?` -// Results -// { -// plannedOrders: [ -// { material: 'MAT-001', quantity: 500, date: '2025-02-01', type: 'purchase' }, -// { material: 'SEMI-001', quantity: 100, date: '2025-02-10', type: 'production' } -// ], -// exceptionMessages: [ -// { material: 'MAT-003', message: 'Safety stock below threshold', priority: 'high' } -// ], -// capacityLoad: { -// 'WC-CNC-01': { load: 0.92, status: 'overloaded' }, -// 'WC-ASSEMBLY-01': { load: 0.78, status: 'ok' } -// } -// } +// AI explains decisions +await sap`why is MRP suggesting 750 units of MAT-001?` +// "Vendor has 23% lead time variability. Buffer covers 2 standard deviations. +// Total cost increase: $340, stockout risk reduction: 89%" -// Convert planned order to production order -await sap.production.createOrder({ - material: 'FINISHED-001', - quantity: 50, - startDate: '2025-02-15', - plant: 'PLANT-01' -}) +// Convert planned to production +await sap`convert planned orders to production orders for FINISHED-001` ``` ### Production Orders -Shop floor execution and tracking. - ```typescript -// Create production order -const order = await sap.production.createOrder({ - material: 'FINISHED-001', - quantity: 100, - plant: 'PLANT-01', - scheduledStart: '2025-02-01', - priority: 'normal' -}) +// Production in natural language +await sap`create production order for 100 FINISHED-001, start Feb 1` +await sap`release PRD-001 to shop floor` -// Release for production -await sap.production.release(order.id) - -// Confirm operations (shop floor feedback) -await sap.production.confirm({ - order: order.id, - operation: '0010', - quantity: 100, - actualSetupTime: 18, - actualMachineTime: 520, - actualLaborTime: 310, - scrap: 2 -}) +// Confirm operations +await sap`confirm PRD-001 op 10: 100 good, 2 scrap, 18 min setup, 520 min machine` -// Goods receipt from production -await sap.production.goodsReceipt({ - order: order.id, - quantity: 98, // 100 - 2 scrap - storageLocation: 'FG-WAREHOUSE' -}) +// Complete production +await sap`goods receipt 98 FINISHED-001 from PRD-001 to FG-WAREHOUSE` + +// Chain the full cycle +await sap`create production order for 100 FINISHED-001` + .release() + .confirm() + .receipt() ``` ### Warehouse Management -Bin-level inventory control for complex warehouses. - ```typescript -// Define storage structure -await sap.warehouse.createStorageType({ - warehouse: 'WH-01', - storageType: 'RACK', - description: 'Rack Storage', - mixedStorage: false, - capacity: 100, // per bin - pickStrategy: 'FIFO' -}) - -// Putaway -await sap.warehouse.putaway({ - material: 'MAT-001', - quantity: 500, - batch: 'BATCH-2025-001', - suggestBin: true // AI suggests optimal bin -}) +// Put materials away +await sap`putaway 500 KG MAT-001 batch BATCH-2025-001 suggest bin` // AI considers: picking frequency, weight, size, product affinity // Pick for production -await sap.warehouse.pick({ - productionOrder: 'PRD-001', - material: 'MAT-001', - quantity: 50, - strategy: 'FIFO' -}) +await sap`pick materials for PRD-001 FIFO` -// Inventory count -await sap.warehouse.physicalInventory({ - warehouse: 'WH-01', - storageType: 'RACK', - countDate: '2025-01-31' -}) +// Stock queries +await sap`where is MAT-001?` +await sap`what's in bin RACK-A-01?` +await sap`inventory value by storage location` + +// Cycle count +await sap`cycle count warehouse WH-01 rack storage today` ``` ### Quality Management -Inspection, defect tracking, certificates. - ```typescript -// Create inspection lot -const inspection = await sap.quality.createInspectionLot({ - material: 'MAT-001', - batch: 'BATCH-2025-001', - origin: 'goods-receipt', - vendor: 'VENDOR-001' -}) +// Inspection on receipt +await sap`inspect batch BATCH-2025-001 from VENDOR-001` + .record(`tensile: 42000 PSI pass, hardness: 95 Rockwell B pass`) + .accept() -// Record results -await sap.quality.recordResults({ - inspectionLot: inspection.id, - characteristics: [ - { name: 'Tensile Strength', value: 42000, unit: 'PSI', status: 'pass' }, - { name: 'Hardness', value: 95, unit: 'Rockwell B', status: 'pass' }, - { name: 'Dimensional Tolerance', value: 0.002, unit: 'IN', status: 'pass' } - ] -}) - -// Usage decision -await sap.quality.usageDecision({ - inspectionLot: inspection.id, - decision: 'accept', - postToInventory: true -}) +// Quality queries +await sap`batches pending inspection` +await sap`vendor quality scores last 12 months` +await sap`defect trend for MAT-001` -// Quality certificates -const cert = await sap.quality.generateCertificate({ - inspectionLot: inspection.id, - format: 'CoC' // Certificate of Conformance -}) +// Certificates +await sap`generate CoC for batch BATCH-2025-001` ``` ### Sales & Distribution -Customer orders through delivery and billing. - ```typescript -// Create sales order -const order = await sap.sales.createOrder({ - customer: 'CUST-001', - orderType: 'standard', - items: [ - { - material: 'FINISHED-001', - quantity: 50, - plant: 'PLANT-01', - requestedDeliveryDate: '2025-02-15' - } - ], - shippingCondition: 'standard', - paymentTerms: 'NET30' -}) +// Sales orders naturally +await sap`sales order CUST-001: 50 FINISHED-001, deliver Feb 15, NET30` // Check availability -const atp = await sap.sales.checkAvailability(order.id) -// { item: 'FINISHED-001', requested: 50, confirmed: 50, date: '2025-02-15' } - -// Create delivery -const delivery = await sap.sales.createDelivery({ - salesOrder: order.id, - pickDate: '2025-02-14', - shipDate: '2025-02-15' -}) +await sap`can we deliver 50 FINISHED-001 by Feb 15?` +// "Yes. 30 in stock, 20 from PRD-001 completing Feb 10." -// Pick, pack, ship -await sap.warehouse.pick({ delivery: delivery.id }) -await sap.shipping.createShipment({ - delivery: delivery.id, - carrier: 'FEDEX', - service: 'ground' -}) +// Ship the order +await sap`pick pack ship SO-001 via FedEx Ground` + .invoice() -// Billing -await sap.billing.createInvoice({ delivery: delivery.id }) +// Order to cash in one chain +await sap`sales order CUST-001: 50 FINISHED-001` + .atp() // check availability + .pick() // pick from warehouse + .ship() // create shipment + .invoice() // bill customer ``` ### Financial Accounting -General ledger, cost accounting, and profitability analysis. - ```typescript -// Goods movement posts to FI automatically +// Goods movements post automatically // Purchase receipt -> Inventory Dr, GR/IR Cr // Invoice receipt -> GR/IR Dr, AP Cr -// Goods issue to production -> WIP Dr, Inventory Cr +// Production issue -> WIP Dr, Inventory Cr // Production receipt -> FG Inventory Dr, WIP Cr -// Cost center reporting -const costs = await sap.controlling.costCenterReport({ - costCenter: 'CC-PROD-01', - period: '2025-01', - includePlanActualVariance: true -}) +// Cost queries +await sap`actual cost FINISHED-001 this month` +await sap`variance analysis PLANT-01 January` +await sap`profitability by product line Q1` // Product costing -const cost = await sap.controlling.calculateCost({ - material: 'FINISHED-001', - costingVariant: 'standard', - lotSize: 100 -}) +await sap`roll up cost for FINISHED-001 lot size 100` // { // materialCost: 125.00, // laborCost: 45.00, // overheadCost: 22.50, -// totalCost: 192.50, -// components: [...] +// totalCost: 192.50 // } - -// Profitability analysis -const profitability = await sap.controlling.profitabilityAnalysis({ - dimension: 'product', - period: '2025-01' -}) ``` ---- - ## AI-Native Manufacturing -This is the revolution. MRP has always been a computational problem. AI solves it better. +MRP has always been a computational problem. AI solves it better. ### AI Material Requirements Planning -Traditional MRP is deterministic: fixed lead times, fixed lot sizes, fixed safety stock. AI MRP is adaptive. - ```typescript -import { mrp } from 'sap.do/ai' - -// AI-powered MRP considers: -// - Historical lead time variability -// - Supplier reliability scores -// - Demand forecast uncertainty -// - Current capacity constraints -// - Material availability trends -// - Economic order quantities - -const aiMrp = await mrp.plan({ - plant: 'PLANT-01', - horizon: '90 days', - optimization: 'cost', // or 'service-level', 'balanced' - constraints: { - maxInventory: 1000000, // $1M inventory cap - serviceLevel: 0.98 // 98% fill rate - } -}) +// AI MRP considers everything +await sap`plan materials for next quarter optimizing cost` +// AI weighs: lead time variability, supplier reliability, demand uncertainty, +// capacity constraints, inventory costs, stockout costs + +// Ask why +await sap`why order MAT-001 on Feb 3 instead of Feb 15?` +// "Vendor has 23% lead time variability. February shows 12% late deliveries +// historically. Earlier order covers 2 sigma of demand uncertainty. +// Cost increase: $340. Stockout risk reduction: 89%." -// AI explains its decisions -// "Recommending order of 750 units MAT-001 on Feb 3 instead of Feb 15: -// - Vendor VENDOR-001 has 23% lead time variability -// - Historical data shows 12% late deliveries in February -// - Buffer covers 2 standard deviations of demand uncertainty -// - Total cost increase: $340, stockout risk reduction: 89%" +// What-if scenarios +await sap`what if MAT-001 lead time increases to 21 days?` +await sap`impact of losing VENDOR-001 as supplier?` ``` ### AI Production Scheduling -Finite capacity scheduling that actually works. - ```typescript -import { scheduler } from 'sap.do/ai' - -const schedule = await scheduler.optimize({ - plant: 'PLANT-01', - horizon: '2 weeks', - objectives: ['on-time-delivery', 'minimize-changeovers', 'maximize-utilization'], - weights: [0.5, 0.3, 0.2] -}) - -// AI sequence optimization: -// "Resequenced work center WC-CNC-01: -// - Grouped similar materials to reduce setup time -// - Total setup time reduced from 14.5 hours to 8.2 hours -// - All delivery dates still met -// - Capacity utilization improved from 78% to 89%" - -// Real-time rescheduling when disruptions occur -await scheduler.handleDisruption({ - type: 'machine-breakdown', - workCenter: 'WC-CNC-01', - duration: '4 hours' -}) +// Optimize the schedule +await sap`optimize schedule for PLANT-01 next 2 weeks` +// "Resequenced WC-CNC-01: grouped similar materials. +// Setup time reduced from 14.5 hours to 8.2 hours. +// All delivery dates still met. Utilization up from 78% to 89%." + +// Handle disruptions +await sap`WC-CNC-01 down for 4 hours` // AI automatically reschedules affected orders + +// Capacity queries +await sap`can we add 200 FINISHED-001 to next week's schedule?` ``` ### AI Quality Prediction -Predict defects before they happen. - ```typescript -import { quality } from 'sap.do/ai' - -// Train on historical inspection data -await quality.train({ - inspectionLots: 'last-2-years', - features: ['vendor', 'batch', 'material', 'season', 'process-parameters'] -}) - // Predict incoming quality -const prediction = await quality.predict({ - material: 'MAT-001', - vendor: 'VENDOR-001', - batch: 'BATCH-2025-010' -}) -// { -// prediction: 'accept', -// confidence: 0.94, -// riskFactors: [ -// { factor: 'vendor_score', value: 0.92, impact: 'low' }, -// { factor: 'seasonal_pattern', value: 'q1_normal', impact: 'none' } -// ], -// recommendation: 'Skip inspection, vendor track record excellent' -// } +await sap`predict quality for incoming batch from VENDOR-001` +// "Accept. Confidence: 94%. Vendor score: 0.92. Recommendation: skip inspection." -// Anomaly detection during production -const anomaly = await quality.monitor({ - productionOrder: 'PRD-001', - operation: '0020', - processData: realtimeSensorData -}) +// Process monitoring +await sap`monitor PRD-001 operation 20 for anomalies` // "Warning: CNC spindle vibration trending 15% above normal. // Recommendation: Inspect tooling before next batch." + +// Supplier quality +await sap`which vendors need quality improvement focus?` ``` ### AI Demand Forecasting -Better than any statistical method. - ```typescript -import { forecast } from 'sap.do/ai' - -const demand = await forecast.predict({ - material: 'FINISHED-001', - horizon: '12 months', - granularity: 'weekly', - includeFactors: ['seasonality', 'trends', 'promotions', 'economic'] -}) - -// AI explains the forecast +// Forecast demand +await sap`forecast FINISHED-001 next 12 months weekly` // "Predicted Q2 demand: 4,200 units (+15% vs Q1) -// Drivers: -// - Seasonal uptick: +8% (consistent with last 3 years) -// - Market trend: +5% (industry growth) -// - Planned promotion week 18: +400 units -// Uncertainty range: 3,800 - 4,600 (90% confidence)" - -// Integrate with MRP -await sap.mrp.run({ - plant: 'PLANT-01', - demandSource: demand // Uses AI forecast -}) -``` +// Drivers: Seasonal uptick +8%, market trend +5%, promotion week 18 +400 +// Range: 3,800 - 4,600 (90% confidence)" -### AI Shop Floor Assistant +// Feed into MRP +await sap`run MRP using AI forecast` +``` -Natural language interface for operators. +### Shop Floor Assistant ```typescript -import { shopfloor } from 'sap.do/agents' - -// Operator asks questions -await shopfloor`What should I work on next?` -// "Production order PRD-001 for FINISHED-001 is highest priority. -// Next operation: 0020 (Machine components) at WC-CNC-01. -// Materials staged at bin STAGING-01. +// Operators talk naturally +await sap`what should I work on next?` +// "PRD-001 for FINISHED-001 is highest priority. +// Operation 20 at WC-CNC-01. Materials at STAGING-01. // Estimated completion: 2.5 hours." -await shopfloor`WC-CNC-01 is down for maintenance` -// "Acknowledged. I've: -// - Updated work center status to 'down' -// - Rescheduled affected orders to WC-CNC-02 -// - Notified production planning -// - Created maintenance notification MN-2025-042" - -await shopfloor`Record 95 pieces good, 5 scrap for operation 0030` -// "Confirmed operation 0030: -// - Good: 95 pieces -// - Scrap: 5 pieces (5% scrap rate) -// - Note: Scrap rate above standard (2%). Create quality notification?" -``` +await sap`WC-CNC-01 is down for maintenance` +// "Updated status. Rescheduled affected orders to WC-CNC-02. +// Notified planning. Created maintenance notification MN-2025-042." ---- +await sap`record 95 good 5 scrap for PRD-001 op 30` +// "Confirmed. Scrap rate 5% vs standard 2%. Create quality notification?" +``` ## OData v4 Compatible -Existing SAP integrations work with standard OData. - -### Entity Sets +Existing SAP integrations work unchanged. ```bash # Query materials -GET /sap/opu/odata4/sap/API_MATERIAL/A_Material?$filter=MaterialType eq 'FERT'&$select=Material,MaterialDescription +GET /odata/v4/MaterialService/Materials?$filter=MaterialType eq 'FERT' -# Get BOM -GET /sap/opu/odata4/sap/API_BOM/A_BillOfMaterial?$filter=Material eq 'FINISHED-001'&$expand=to_BillOfMaterialItem +# Get BOM with components +GET /odata/v4/BOMService/BOMs?$filter=Material eq 'FINISHED-001'&$expand=Components # Production orders -GET /sap/opu/odata4/sap/API_PRODUCTION_ORDER/A_ProductionOrder?$filter=OrderStatus eq 'Released' -``` - -### Deep Insert - -```typescript -// Create sales order with items in one call -await fetch('/sap/opu/odata4/sap/API_SALES_ORDER/A_SalesOrder', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - SalesOrderType: 'OR', - SoldToParty: 'CUST-001', - to_Item: [ - { Material: 'FINISHED-001', RequestedQuantity: 100 } - ] - }) -}) +GET /odata/v4/ProductionService/Orders?$filter=Status eq 'Released' ``` -### Actions and Functions - -```typescript -// Release production order -await fetch('/sap/opu/odata4/sap/API_PRODUCTION_ORDER/A_ProductionOrder(\'PRD-001\')/Release', { - method: 'POST' -}) - -// Check ATP -const atp = await fetch('/sap/opu/odata4/sap/API_ATP/CheckAvailability', { - method: 'POST', - body: JSON.stringify({ - Material: 'FINISHED-001', - Plant: 'PLANT-01', - RequestedQuantity: 100, - RequestedDate: '2025-02-15' - }) -}) -``` - ---- - ## Architecture -sap.do mirrors SAP's organizational structure with Durable Objects: - -``` - Cloudflare Edge - | - +---------------+---------------+ - | | | - +-----------+ +-----------+ +-----------+ - | Auth | | OData | | MCP | - | Gateway | | Gateway | | Server | - +-----------+ +-----------+ +-----------+ - | | | - +-------+-------+-------+-------+ - | | - +------------+ +------------+ - | Client DO | | Client DO | - | (Tenant) | | (Tenant) | - +------------+ +------------+ - | - +----------------+----------------+ - | | | -+----------+ +----------+ +----------+ -| Company | | Company | | Company | -| Code DO | | Code DO | | Code DO | -+----------+ +----------+ +----------+ - | -+----+----+----+----+----+----+ -| | | | | | | -MM PP SD WM QM FI CO +### Durable Object per Company + +``` +CompanyDO (config, plants, company codes) + | + +-- MaterialsDO (material masters, BOMs) + | |-- SQLite: Material data + | +-- R2: Drawings, specs + | + +-- ProductionDO (orders, routings, work centers) + | |-- SQLite: Production data + | +-- R2: Confirmations, history + | + +-- InventoryDO (stock, movements, warehouses) + | |-- SQLite: Current stock + | +-- R2: Movement history + | + +-- SalesDO (orders, deliveries, billing) + | |-- SQLite: Open orders + | +-- R2: Completed documents + | + +-- QualityDO (inspections, certificates) + |-- SQLite: Active lots + +-- R2: Certificates, results ``` -### Durable Object Structure - -| Durable Object | SAP Equivalent | Purpose | -|----------------|----------------|---------| -| `ClientDO` | Client (Mandant) | Tenant isolation | -| `CompanyCodeDO` | Company Code | Legal entity | -| `PlantDO` | Plant | Manufacturing site | -| `StorageLocationDO` | Storage Location | Inventory location | -| `MaterialMasterDO` | Material Master | Product data | -| `ProductionOrderDO` | Production Order | Manufacturing execution | -| `SalesDocumentDO` | Sales Document | Customer orders | -| `PurchaseDocumentDO` | Purchase Document | Vendor orders | -| `InspectionLotDO` | Inspection Lot | Quality management | -| `CostCenterDO` | Cost Center | Cost accounting | - ### Storage Tiers -``` -Hot (SQLite in DO) Warm (R2 Parquet) Cold (R2 Archive) ------------------ ----------------- ----------------- -Current inventory Historical moves 7+ year archive -Open orders Closed orders Audit retention -Active materials Inactive materials Deleted masters -Last 2 years FI 3-7 years FI Compliance data -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active materials, open orders | <10ms | +| **Warm** | R2 + SQLite Index | Historical transactions (2-7 years) | <100ms | +| **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | ### Real-Time Material Ledger -Unlike batch-processed SAP, sap.do provides real-time actual costing: - ```typescript // Every goods movement updates actual cost in real-time // No month-end "material ledger close" -const actualCost = await sap.materials.getActualCost({ - material: 'MAT-001', - plant: 'PLANT-01', - asOf: 'now' // Real-time, not period-end -}) +await sap`actual cost MAT-001 now` // { // movingAverage: 4.52, // lastPurchasePrice: 4.65, @@ -797,264 +415,154 @@ const actualCost = await sap.materials.getActualCost({ // } ``` ---- - ## vs SAP | Feature | SAP S/4HANA | SAP Business One | sap.do | |---------|-------------|------------------|--------| -| Pricing | $350+/user/mo | $95-150/user/mo | **Free** | -| Implementation | 12-36 months | 3-9 months | **Minutes** | -| Infrastructure | SAP/Hyperscaler | On-prem/Cloud | **Your Cloudflare** | -| Programming | ABAP | SDK | **TypeScript** | -| AI/ML | SAP AI Core ($$$) | Limited | **Native** | -| MRP | Batch (nightly) | Batch | **Real-time** | -| Customization | Consultants required | Consultants required | **Self-service** | -| OData | Yes | Limited | **Full v4** | - -### Cost Comparison - -**100-user manufacturing company:** - -| | SAP Business One | sap.do | -|-|------------------|--------| -| Year 1 licenses | $150,000 | $0 | -| Implementation | $200,000 | $0 | -| Annual maintenance | $30,000 | $0 | -| Customization | $50,000 | $0 | -| **Year 1 Total** | **$430,000** | **$5** (Workers) | -| **5-Year TCO** | **$1,100,000+** | **$300** | +| **Implementation** | $10M-100M+ | $200K-500K | Deploy in minutes | +| **Annual Cost** | $1-5M+ | $50K-150K | ~$100/month | +| **Architecture** | Monolithic, HANA | On-prem/Cloud | Edge-native, global | +| **Programming** | ABAP | SDK | TypeScript | +| **AI** | SAP AI Core ($$$) | Limited | AI-first design | +| **MRP** | Batch (nightly) | Batch | Real-time | +| **Data Location** | SAP's data centers | On-prem/Cloud | Your Cloudflare account | +| **Customization** | $500/hour consultants | $200/hour | Code it yourself | +| **Lock-in** | Decades of migration | Years | MIT licensed | ---- - -## Quick Start - -### One-Click Deploy +## Use Cases -```bash -npx create-dotdo sap +### Job Shops -# Follow prompts: -# - Company name -# - Industry template (discrete, process, make-to-order) -# - Currency and fiscal year -# - Initial plant setup +```typescript +// Quote to cash in natural language +await sap`quote CUST-001: 500 custom brackets, aluminum, 2 week lead` + .approve() + .convert() // to sales order + .plan() // MRP generates requirements + .produce() // create production orders + .ship() + .invoice() ``` -### Manual Setup +### Discrete Manufacturing -```bash -git clone https://github.com/dotdo/sap.do -cd sap.do -npm install -npm run deploy +```typescript +// High-volume production +await sap`schedule FINISHED-001 for 10,000 units next month` +await sap`balance line for ASSEMBLY-LINE-1 takt time 30 seconds` +await sap`kanban replenishment for MAT-001 min 500 max 1500` ``` -### First Production Order +### Process Manufacturing ```typescript -import { SAPClient } from 'sap.do' - -const sap = new SAPClient({ - url: 'https://your-company.sap.do', - token: process.env.SAP_TOKEN -}) - -// 1. Create material -await sap.materials.create({ - number: 'WIDGET-001', - description: 'Demo Widget', - type: 'finished', - baseUnit: 'EA' -}) - -// 2. Create BOM -await sap.bom.create({ - material: 'WIDGET-001', - components: [ - { material: 'RAW-001', quantity: 2 }, - { material: 'RAW-002', quantity: 1 } - ] -}) - -// 3. Run MRP -await sap.mrp.run({ material: 'WIDGET-001' }) - -// 4. Create production order -await sap.production.createOrder({ - material: 'WIDGET-001', - quantity: 100 -}) +// Formula management +await sap` + FORMULA-001 batch 1000 KG: + - 400 KG ingredient A + - 350 KG ingredient B + - 250 KG ingredient C + yield 95% +` -// Manufacturing! +await sap`create batch BATCH-2025-001 from FORMULA-001` + .record(`potency 98%, moisture 2.1%`) + .release() ``` ---- - -## Migration from SAP - -### Export from SAP +### Multi-Plant Operations -```bash -# Use SAP standard extraction programs -# SE38 -> MM60 (material master) -# SE38 -> CS12 (BOM explosion) -# LSMW for mass data -# or... - -# Our migration tool -npx sap.do export \ - --system ECC \ - --client 100 \ - --connection rfc.json +```typescript +// Cross-plant planning +await sap`transfer 500 MAT-001 from PLANT-01 to PLANT-02` +await sap`where should we produce FINISHED-001 for minimum cost?` +await sap`consolidate requirements across all plants` ``` -### Import to sap.do +## Migration from SAP -```bash -npx sap.do migrate \ - --source ./sap-export \ - --url https://your-company.sap.do - -# Migrates: -# - Material masters (all views) -# - Bills of material -# - Routings and work centers -# - Vendors and customers -# - Open orders -# - Inventory balances -# - Cost masters +```typescript +// Parallel run both systems +await sap`migrate from SAP ECC client 100` + .materials() // material masters + .boms() // bills of material + .routings() // work centers and routings + .inventory() // current stock + .openOrders() // sales and production + .verify() // reconciliation ``` -### Parallel Run +Natural language migration: ```typescript -// Run both systems during transition -const bridge = sap.migration.createBridge({ - source: { type: 'sap-ecc', connection: rfcConfig }, - target: { type: 'sap.do', url: 'https://...' }, - mode: 'dual-write' -}) - -// All transactions written to both -// Reconciliation reports available -// Cut over when confident +await sap`import materials from SAP export file` +await sap`reconcile inventory with SAP as of Jan 31` +await sap`which transactions differ between systems?` ``` ---- - ## Industry Templates -Pre-configured setups for common manufacturing scenarios: - -### Discrete Manufacturing - ```bash -npx create-dotdo sap --template discrete +npx create-dotdo sap --template discrete # Make-to-stock, routing-based +npx create-dotdo sap --template process # Formula/batch, co-products +npx create-dotdo sap --template mto # Make-to-order, configurable +npx create-dotdo sap --template jobshop # Quote-to-cash, high-mix low-volume ``` -- Make-to-stock production -- Routing-based operations -- Standard costing -- Finite capacity scheduling - -### Process Manufacturing - -```bash -npx create-dotdo sap --template process -``` - -- Formula/recipe management -- Batch tracking -- Co-products and by-products -- Potency and yield variance - -### Make-to-Order / Engineer-to-Order - -```bash -npx create-dotdo sap --template mto -``` - -- Project-based manufacturing -- Configurable products -- Customer-specific BOMs -- Cost-plus pricing - ---- - ## Roadmap -### Now -- [x] Material Master (all views) +### Core ERP +- [x] Material Master - [x] Bill of Materials -- [x] Basic MRP +- [x] MRP - [x] Production Orders - [x] Inventory Management +- [x] Sales & Distribution +- [x] Quality Management - [x] OData v4 compatibility - -### Next -- [ ] Routings and Work Centers -- [ ] Finite Capacity Scheduling -- [ ] Quality Management -- [ ] AI MRP Optimization -- [ ] Sales & Distribution -- [ ] Purchasing - -### Later - [ ] Warehouse Management (WM/EWM) - [ ] Product Cost Controlling -- [ ] Batch Management - [ ] Variant Configuration -- [ ] EDI/IDoc Integration -- [ ] Multi-Plant Planning - ---- -## Documentation - -| Guide | Description | -|-------|-------------| -| [Quick Start](./docs/quickstart.mdx) | Deploy in 5 minutes | -| [Material Master](./docs/material-master.mdx) | Product data setup | -| [BOM & Routing](./docs/bom-routing.mdx) | Manufacturing engineering | -| [MRP](./docs/mrp.mdx) | Material requirements planning | -| [Production](./docs/production.mdx) | Shop floor execution | -| [AI Features](./docs/ai.mdx) | AI-powered manufacturing | -| [OData API](./docs/odata.mdx) | API reference | -| [Migration](./docs/migration.mdx) | Moving from SAP | - ---- +### AI +- [x] Natural Language Interface +- [x] AI MRP Optimization +- [x] Shop Floor Assistant +- [x] Quality Prediction +- [ ] Demand Forecasting +- [ ] Predictive Maintenance +- [ ] Autonomous Scheduling ## Contributing sap.do is open source under the MIT license. +We especially welcome contributions from: +- Manufacturing engineers +- Supply chain professionals +- SAP consultants ready to escape ABAP +- AI/ML researchers + ```bash git clone https://github.com/dotdo/sap.do cd sap.do -npm install -npm test -npm run dev +pnpm install +pnpm test ``` -Key areas for contribution: -- PP (Production Planning) modules -- CO (Controlling) cost rollup -- OData entity coverage -- Industry-specific features -- AI/ML model improvements - -See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - ---- - ## License -MIT +MIT License - For manufacturing everywhere. ---

- Manufacturing ERP, simplified.
- Built on Cloudflare Workers. Powered by AI. No consultants required. + The $200B gorilla meets its match. +
+ AI-first. Real-time MRP. No consultants required. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/segment/README.md b/rewrites/segment/README.md index 95b6740f..7b22f31c 100644 --- a/rewrites/segment/README.md +++ b/rewrites/segment/README.md @@ -1,311 +1,383 @@ # segment.do -Segment on Cloudflare Durable Objects - A Customer Data Platform for every AI agent. +> Customer Data Platform. Edge-Native. Open by Default. AI-First. -## The Problem - -AI agents need to track events. Millions of them. In real-time. Each routed to dozens of destinations. +Twilio paid $3.2 billion for Segment. Now they charge per Monthly Tracked User, gate identity resolution behind enterprise tiers, and treat your customer data as their asset. Simple event tracking costs more than your database. -Traditional CDPs were built for humans: -- Centralized infrastructure -- Complex pricing per MTU -- Slow identity resolution -- Data leaves your control +**segment.do** is the open-source alternative. Edge-native. GDPR-compliant by design. Deploys in minutes, not weeks. AI agents and humans share the same analytics pipeline. -AI agents need the opposite: -- Edge-native, sub-millisecond ingestion -- Simple usage-based pricing -- Real-time identity resolution -- Data stays in your infrastructure +## AI-Native API -## The Vision +```typescript +import { segment } from 'segment.do' // Full SDK +import { segment } from 'segment.do/tiny' // Minimal client +import { segment } from 'segment.do/server' // Server-side only +``` -Every AI agent gets their own analytics pipeline. +Natural language for analytics: ```typescript -import { tom, ralph, priya } from 'agents.do' -import { Analytics } from 'segment.do' - -// Each agent tracks their own events -const tomAnalytics = Analytics.for(tom) -const ralphAnalytics = Analytics.for(ralph) -const priyaAnalytics = Analytics.for(priya) - -// Full Segment-compatible API -await tomAnalytics.track('Code Review Completed', { - pr: 123, - approved: true, - linesReviewed: 450 -}) +import { segment } from 'segment.do' -await ralphAnalytics.identify('ralph@agents.do', { - role: 'developer', - expertise: ['typescript', 'rust'] -}) +// Talk to it like a colleague +await segment`user-123 signed up from Google Ads` +await segment`user-123 started checkout with $99.99 in cart` +await segment`user-123 completed order #456 revenue $99.99` -await priyaAnalytics.page('Roadmap', { - section: 'Q1 Planning' -}) +// Chain like sentences +await segment`users who abandoned checkout` + .notify(`Complete your purchase for 10% off`) + +// Identity resolution just works +await segment`link anonymous-xyz to user-123` +await segment`merge user-123 with user-456` ``` -Not a shared analytics account. Not MTU-based billing. Each agent has their own complete analytics pipeline. +## The Problem -## Features +Twilio Segment dominates customer data: -- **Segment-Compatible API** - Drop-in replacement for analytics.js -- **Identity Resolution** - Durable Object-based identity graph -- **300+ Destinations** - Route events to GA4, Mixpanel, Amplitude, etc. -- **Warehouse Sync** - Real-time export to R2/Parquet/Iceberg -- **First-Party Tracking** - Bypass ad blockers with same-domain collection -- **Edge-Native** - Sub-millisecond ingestion at the edge -- **GDPR Compliant** - Process data in-region, no cross-border transfers +| What Segment Charges | The Reality | +|----------------------|-------------| +| **10K MTU** | $120/month | +| **100K MTU** | $1,200/month | +| **1M MTU** | $25,000+/month custom | +| **Identity Resolution** | Enterprise add-on | +| **Warehouse Sync** | Enterprise add-on | +| **First-Party Tracking** | DIY proxy setup | -## Architecture +### The MTU Tax + +Every user you track costs money. But: + +- Segment counts everyone, even anonymous visitors +- Price tiers punish growth +- "Business" tier gates essential features +- Enterprise sales calls for anything serious + +### The Data Hostage Problem + +Your customer data lives in Segment's infrastructure: + +- They process your events in their cloud +- Cross-border data transfers (GDPR nightmare) +- Vendor lock-in via proprietary schemas +- Data portability means paying for exports + +## The Solution + +**segment.do** reimagines CDP for the edge: ``` - +-----------------------+ - | segment.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | IdentityDO (Tom) | |IdentityDO (Ralph)| | IdentityDO (...) | - | SQLite Graph | | SQLite Graph | | SQLite Graph | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +-------------------+ - | Destinations | - | (Queues + Workers)| - +-------------------+ - | - +---------------+---------------+ - | | | - +-----------+ +-----------+ +-----------+ - | GA4 | | Mixpanel | | Webhooks | - +-----------+ +-----------+ +-----------+ +Twilio Segment segment.do +----------------------------------------------------------------- +$120/mo for 10K MTU ~$5/mo for 10K MTU +Enterprise for identity Identity resolution included +Enterprise for warehouse Warehouse sync included +Proxy DIY for first-party First-party native +Their infrastructure Your Cloudflare account +Cross-border transfers Process in-region +Proprietary schemas Open source, MIT licensed ``` -**Key insight**: Durable Objects provide single-threaded identity resolution. Each user's identity graph is a Durable Object. Workers handle ingestion. Queues handle destination delivery. - -## Installation +## One-Click Deploy ```bash -npm install segment.do +npx create-dotdo segment ``` -## Quick Start - -### Browser SDK +A complete CDP. Running on infrastructure you control. GDPR-compliant from day one. ```typescript -import { Analytics } from 'segment.do' +import { Segment } from 'segment.do' -const analytics = Analytics({ - writeKey: 'your-write-key', - // First-party tracking (bypass ad blockers) - apiHost: 'analytics.yoursite.com' +export default Segment({ + name: 'my-analytics', + domain: 'analytics.mysite.com', }) +``` -// Track events -analytics.track('Button Clicked', { - buttonId: 'cta-signup', - page: '/pricing' -}) +## The Vision -// Identify users -analytics.identify('user_123', { - email: 'user@example.com', - plan: 'pro' -}) +Every AI agent gets their own analytics pipeline. -// Track page views -analytics.page('Pricing', { - section: 'enterprise' -}) +```typescript +import { tom, ralph, priya } from 'agents.do' +import { segment } from 'segment.do' + +// Each agent tracks naturally +await segment`tom reviewed PR 123 - 450 lines approved` +await segment`ralph deployed v2.1.0 to production` +await segment`priya updated Q1 roadmap` + +// Query your agents like a database +await segment`tom activity this week` +await segment`most active agents today` ``` -### Server SDK +Not a shared analytics account. Not MTU-based billing. Each agent has their own complete analytics pipeline. + +## Features -```typescript -import { Analytics } from 'segment.do/server' - -const analytics = Analytics({ writeKey: 'your-write-key' }) - -// Track server-side events -await analytics.track({ - userId: 'user_123', - event: 'Order Completed', - properties: { - orderId: 'order_456', - revenue: 99.99, - products: [ - { id: 'prod_1', name: 'Widget', price: 49.99 } - ] - } -}) +### Event Tracking -// Batch multiple events -await analytics.batch([ - { type: 'track', userId: 'user_123', event: 'Checkout Started' }, - { type: 'identify', userId: 'user_123', traits: { cartValue: 99.99 } } -]) +```typescript +// Just say what happened +await segment`user-123 clicked signup button` +await segment`user-123 viewed pricing page` +await segment`user-123 searched for widgets` + +// Revenue tracking reads like a receipt +await segment`user-123 purchased widget for $49.99` +await segment`user-123 subscribed to pro plan $99/month` +await segment`user-123 upgraded from starter to business` + +// Batch events read like a story +await segment` + user-123: + - viewed homepage + - clicked pricing + - started trial + - invited teammate +` ``` ### Identity Resolution ```typescript -import { Analytics } from 'segment.do' +// Link anonymous to known +await segment`link anon-xyz to user-123` -const analytics = Analytics({ writeKey: 'your-write-key' }) +// Merge identities across devices +await segment`merge mobile-user with desktop-user` -// Anonymous tracking -analytics.track('Page Viewed') // Uses anonymousId +// B2B account association +await segment`user-123 joined Acme Corp enterprise plan` -// Later, identify the user -analytics.identify('user_123', { - email: 'user@example.com' -}) -// Anonymous events are now merged with user_123 +// The identity graph updates automatically +await segment`who is user-123?` +``` -// Alias for cross-device tracking -analytics.alias('user_123', 'previous_anonymous_id') +### Destinations + +```typescript +// Route events naturally +await segment`send to GA4 and Mixpanel` +await segment`send purchases to Facebook Pixel` +await segment`webhook all signups to Slack` + +// Query destinations +await segment`where does user signup go?` +await segment`active destinations` ``` -### Group (B2B) +### Warehouse Sync ```typescript -import { Analytics } from 'segment.do' +// Query your events like a database +await segment`signups this week` +await segment`revenue by campaign last month` +await segment`users who viewed but didnt purchase` + +// Export for analysis +await segment`export signups to parquet` +await segment`sync to BigQuery hourly` +``` -const analytics = Analytics({ writeKey: 'your-write-key' }) +## Architecture -// Associate user with organization -analytics.group('org_456', { - name: 'Acme Corp', - plan: 'enterprise', - employees: 500 -}) +### Durable Object per Identity -// Events now include group context -analytics.track('Feature Used', { - feature: 'advanced-reporting' -}) ``` +Event Ingestion Flow: + +Browser/Server --> Cloudflare Worker --> IdentityDO --> Destinations + | | | + Edge Auth SQLite Graph Queues + Workers + (same domain) (per-user) (fan-out) +``` + +### Storage Tiers + +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Recent events (30 days) | <10ms | +| **Warm** | R2 + Index | Historical (30-365 days) | <100ms | +| **Cold** | R2 Archive | Compliance (1+ years) | <1s | + +### Identity Graph + +Each user is a Durable Object. Single-threaded consistency for: +- Anonymous to known identity merging +- Cross-device identity resolution +- B2B account association +- GDPR right-to-erasure + +## Browser Tracking -## API Overview +```typescript +import { segment } from 'segment.do' + +// Just drop it in - no config needed +await segment`page view pricing` +await segment`button click signup-cta` +await segment`form submit contact-us` -### Track API +// Automatic context capture +// - UTM parameters +// - Referrer +// - Device info +// - Geo location -| Endpoint | Description | -|----------|-------------| -| `POST /v1/track` | Track a single event | -| `POST /v1/page` | Track a page view | -| `POST /v1/screen` | Track a mobile screen view | -| `POST /v1/identify` | Identify a user with traits | -| `POST /v1/group` | Associate user with group | -| `POST /v1/alias` | Merge user identities | -| `POST /v1/batch` | Send multiple events | +// First-party tracking (bypass ad blockers) +// Deploy to analytics.yoursite.com +``` -### Event Schema +## Server Tracking ```typescript -interface SegmentEvent { - type: 'track' | 'identify' | 'page' | 'screen' | 'group' | 'alias' - anonymousId?: string - userId?: string - timestamp: string - context: { - ip?: string - userAgent?: string - locale?: string - campaign?: { source, medium, term, content, name } - device?: { type, manufacturer, model } - os?: { name, version } - } - // Type-specific fields - properties?: Record // track, page, screen - traits?: Record // identify, group - event?: string // track only - groupId?: string // group only -} +import { segment } from 'segment.do/server' + +// Server-side events +await segment`user-123 completed order $99.99` +await segment`user-123 subscription renewed` +await segment`user-123 api call to /users endpoint` + +// Backend events with full context +await segment` + user-123 checkout completed: + - order: order-456 + - revenue: $99.99 + - items: 3 + - coupon: SAVE10 +` ``` -## Destinations +## Population Analytics + +```typescript +// Query your users like a database +await segment`users who signed up this week` +await segment`users from Google Ads who converted` +await segment`churned users last 30 days` + +// Cohort analysis +await segment`users who did X but not Y` +await segment`users active in January inactive in February` + +// Funnel analysis +await segment`signup to purchase funnel this month` +``` -### Supported Destinations +## Supported Destinations | Category | Destinations | |----------|--------------| -| Analytics | GA4, Mixpanel, Amplitude, Heap, Posthog | -| Marketing | HubSpot, Mailchimp, Intercom, Customer.io | -| Advertising | Google Ads, Facebook Pixel, LinkedIn | -| Data Warehouse | BigQuery, Snowflake, ClickHouse, R2 | -| Custom | Webhooks, HTTP API | +| **Analytics** | GA4, Mixpanel, Amplitude, Heap, Posthog | +| **Marketing** | HubSpot, Mailchimp, Intercom, Customer.io | +| **Advertising** | Google Ads, Facebook Pixel, LinkedIn | +| **Data Warehouse** | BigQuery, Snowflake, ClickHouse, R2 | +| **Custom** | Webhooks, HTTP API | -### Destination Configuration +### Destination Routing ```typescript -// Configure destinations in dashboard or via API -const config = { - destinations: { - ga4: { - enabled: true, - measurementId: 'G-XXXXX', - apiSecret: 'xxxxx' - }, - mixpanel: { - enabled: true, - projectToken: 'xxxxx' - }, - webhook: { - enabled: true, - url: 'https://your-server.com/webhook', - headers: { 'X-API-Key': 'xxxxx' } - } - } -} +// Route by event type +await segment`send signups to HubSpot` +await segment`send purchases to GA4 and Facebook` +await segment`send all events to BigQuery` + +// Conditional routing +await segment`send enterprise signups to Salesforce` +await segment`send errors to PagerDuty` ``` -## Warehouse Sync +## vs Twilio Segment + +| Feature | Twilio Segment | segment.do | +|---------|----------------|------------| +| **10K MTU** | $120/mo | ~$5/mo | +| **100K MTU** | $1,200/mo | ~$20/mo | +| **1M MTU** | $25,000+/mo | ~$150/mo | +| **Identity Resolution** | Enterprise add-on | Included | +| **Warehouse Sync** | Enterprise add-on | Included | +| **First-Party Tracking** | DIY proxy | Native | +| **Data Location** | Segment's cloud | Your account | +| **GDPR Compliance** | Their responsibility | Your control | +| **Lock-in** | Proprietary schemas | MIT licensed | -Real-time export to R2 in Parquet/Iceberg format: +## Use Cases + +### E-commerce ```typescript -import { query } from 'segment.do/warehouse' - -// Query events directly -const results = await query.sql(` - SELECT - event, - count(*) as count, - avg(revenue) as avgRevenue - FROM events - WHERE timestamp > now() - INTERVAL 7 DAY - GROUP BY event - ORDER BY count DESC -`) - -// Export to external warehouse -await query.export({ - format: 'parquet', - destination: 's3://your-bucket/events/', - partitionBy: ['date', 'event'] -}) +// Track the full customer journey +await segment`user-123 viewed product widget-500` +await segment`user-123 added widget to cart` +await segment`user-123 started checkout $49.99` +await segment`user-123 completed purchase order-789` + +// Abandonment recovery +await segment`users with abandoned carts` + .notify(`Complete your purchase for free shipping`) +``` + +### SaaS + +```typescript +// Product analytics +await segment`user-123 used feature advanced-reporting` +await segment`user-123 invited teammate to workspace` +await segment`user-123 hit usage limit on API calls` + +// Churn prediction +await segment`users inactive 14 days with active subscription` +``` + +### Marketing + +```typescript +// Campaign attribution +await segment`users from utm_source=google who converted` +await segment`compare conversion Facebook vs Google Q1` + +// Audience sync +await segment`sync high-value users to Facebook Custom Audience` +await segment`export churned users to email re-engagement` ``` -## Pricing Comparison +## GDPR Compliance -| Feature | Segment | segment.do | -|---------|---------|------------| -| 10K MTU | $120/mo | ~$5/mo | -| 100K MTU | $1,200/mo | ~$20/mo | -| 1M MTU | Custom ($25K+) | ~$150/mo | -| Identity Resolution | Add-on | Included | -| Warehouse Sync | Add-on | Included | -| First-Party Tracking | Proxy setup | Native | +```typescript +// Right to access +await segment`export all data for user-123` + +// Right to erasure +await segment`delete user-123` + +// Data stays in your region +// No cross-border transfers +// You control the infrastructure +``` + +## Deployment Options + +### Cloudflare Workers (Recommended) + +```bash +npx create-dotdo segment +# Deploys to your Cloudflare account +``` + +### Self-Hosted + +```bash +# Docker +docker run -p 8787:8787 dotdo/segment + +# Kubernetes +kubectl apply -f segment-do.yaml +``` ## The Rewrites Ecosystem @@ -317,8 +389,8 @@ segment.do is part of the rewrites family: | [gitx.do](https://gitx.do) | git | Version control for AI | | [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | | **segment.do** | Segment | CDP for AI | -| kafka.do | Kafka | Event streaming for AI | -| mongo.do | MongoDB | Document database for AI | +| [kafka.do](https://kafka.do) | Kafka | Event streaming for AI | +| [mongo.do](https://mongo.do) | MongoDB | Document database for AI | ## The workers.do Platform @@ -326,28 +398,107 @@ segment.do is a core service of [workers.do](https://workers.do) - the platform ```typescript import { priya, ralph, tom, mark } from 'agents.do' -import { Analytics } from 'segment.do' +import { segment } from 'segment.do' // AI agents with full analytics -const startup = { - product: priya, - engineering: ralph, - tech: tom, - marketing: mark, -} - -// Each agent tracks their own events -for (const [role, agent] of Object.entries(startup)) { - const analytics = Analytics.for(agent) - await analytics.track('Agent Started', { - role, - timestamp: new Date() - }) -} +await segment`priya started Q1 roadmap planning` +await segment`ralph deployed authentication service` +await segment`tom approved PR 456 for merge` +await segment`mark published launch announcement` + +// Query agent activity +await segment`agent activity this week` +await segment`most productive agent today` ``` Both kinds of workers. Working for you. +## Why Open Source for CDP? + +### 1. Data Sovereignty + +Your customer data is your most valuable asset: +- Process events in your infrastructure +- No vendor access to your data +- Full GDPR compliance by design +- Export anytime, no lock-in + +### 2. Cost Transparency + +MTU pricing punishes growth: +- Pay for compute, not users +- No enterprise sales calls +- No surprise tier jumps +- Scale predictably + +### 3. Real Interoperability + +Segment destinations are vendor agreements: +- Build your own destinations +- Modify existing connectors +- No API rate limit surprises +- Community-driven integrations + +### 4. Privacy First + +First-party tracking should be default: +- Same-domain collection +- No third-party cookies +- Ad blocker resistant +- User trust preserved + +## Roadmap + +### Core CDP +- [x] Event tracking (track, identify, page, group) +- [x] Identity resolution +- [x] Destination routing +- [x] First-party tracking +- [ ] Computed traits +- [ ] Predictive audiences +- [ ] Journey orchestration + +### Destinations +- [x] Google Analytics 4 +- [x] Mixpanel +- [x] Amplitude +- [x] Webhooks +- [ ] Facebook Pixel +- [ ] Google Ads +- [ ] Salesforce +- [ ] HubSpot + +### Warehouse +- [x] R2 export +- [x] Parquet format +- [ ] Iceberg tables +- [ ] BigQuery sync +- [ ] Snowflake sync + +## Contributing + +segment.do is open source under the MIT license. + +```bash +git clone https://github.com/dotdo/segment.do +cd segment.do +pnpm install +pnpm test +``` + ## License -MIT +MIT License - Track everything. Own everything. + +--- + +

+ The $3.2B acquisition ends here. +
+ Edge-native. Privacy-first. User-owned. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/sentry/README.md b/rewrites/sentry/README.md index 713500f5..b23df728 100644 --- a/rewrites/sentry/README.md +++ b/rewrites/sentry/README.md @@ -1,104 +1,241 @@ -# @dotdo/sentry +# sentry.do -Sentry-compatible error monitoring on Cloudflare Durable Objects - edge-native error tracking for AI agents and humans. +> Error Monitoring. Edge-Native. AI-First. Open by Default. + +Sentry charges $26/member/month for Team, $80/member for Business. Reserved capacity. Per-event pricing. Usage spikes punish you. Self-hosting requires 10+ services and a full-time SRE. Meanwhile, your AI agents throw millions of errors and your bill explodes. + +**sentry.do** is the open-source alternative. Sub-50ms ingestion. Deploys in minutes. AI agents that debug themselves. + +## AI-Native API + +```typescript +import { sentry } from 'sentry.do' // Full SDK +import { sentry } from 'sentry.do/tiny' // Minimal client +import { sentry } from 'sentry.do/compat' // Drop-in Sentry SDK replacement +``` + +Natural language for error monitoring: + +```typescript +import { sentry } from 'sentry.do' + +// Talk to it like a colleague +const issues = await sentry`top errors this week` +const checkout = await sentry`errors affecting checkout` +const spikes = await sentry`error spikes in the last hour` + +// Chain like sentences +await sentry`unresolved errors assigned to me` + .notify(`@channel heads up on these`) + +// Errors that debug themselves +await sentry`database connection failed in checkout` + .investigate() // AI analyzes root cause + .suggest() // proposes fix + .assign('ralph') // routes to dev agent +``` ## The Problem -Error monitoring platforms were built for centralized infrastructure: -- High latency from edge to central ingestion -- Expensive per-event pricing at scale -- Complex self-hosting requirements -- Limited integration with AI workflows +Sentry dominates error monitoring but: + +| What Sentry Charges | The Reality | +|---------------------|-------------| +| **Per-Event Pricing** | $0.00029/event adds up fast | +| **Reserved Capacity** | Pay for 500K even if you use 100K | +| **Overage Fees** | Surprise bills when traffic spikes | +| **Self-Hosting** | 10+ services (Kafka, Redis, Postgres, ClickHouse...) | +| **AI Features** | "Autofix" is premium only | +| **Team Seats** | $26-80/member/month | + +### The Edge Problem + +Every error travels: +``` +Your Edge Worker -> Internet -> Sentry US/EU -> Processing -> Your Dashboard +``` + +That's 50-200ms latency per error. For AI agents making thousands of decisions per second, this is unacceptable. -AI agents need error monitoring that: -- Ingests at the edge with minimal latency -- Scales to millions of isolated projects -- Integrates with MCP for AI-native debugging -- Provides drop-in Sentry SDK compatibility +### The Self-Hosting Tax -## The Vision +Want to run your own Sentry? -Every AI agent gets their own error monitoring instance. +``` +Kafka (3 nodes) +Redis Cluster +PostgreSQL +ClickHouse Cluster +Snuba +Symbolicator +Relay +Web Workers +Cron Workers +``` + +Full-time SRE required. $5K+/month in infrastructure. Updates break things. + +### The AI Gap + +AI agents need to understand and fix their own errors. Current error monitoring: +- Alerts humans (who are asleep) +- Groups by stack trace (missing semantic context) +- Suggests fixes (premium only, often wrong) +- No MCP integration + +## The Solution + +**sentry.do** reimagines error monitoring for the edge and AI: + +``` +Sentry sentry.do +----------------------------------------------------------------- +50-200ms latency Sub-50ms edge ingestion +$0.00029/event + overages Flat pricing, unlimited events +10+ services to self-host One Durable Object +Premium-only AI features AI-native, Quinn investigates +Human-centric alerting Agent-first error handling +Stack trace grouping Semantic grouping with AI +US/EU regions only Global edge, your account +``` + +## One-Click Deploy + +```bash +npx create-dotdo sentry +``` + +Edge-native error monitoring. Running on infrastructure you control. ```typescript -import * as Sentry from '@dotdo/sentry' +import { Sentry } from 'sentry.do' -// Drop-in replacement - just change the DSN host -Sentry.init({ - dsn: 'https://key@errors.do/123', // That's it! +export default Sentry({ + name: 'my-startup', + domain: 'errors.my-startup.com', }) - -// All existing Sentry code works unchanged -Sentry.captureException(new Error('Something went wrong')) -Sentry.setUser({ id: 'user-123' }) -Sentry.addBreadcrumb({ category: 'ui', message: 'Button clicked' }) ``` ## Features -- **Sentry Protocol Compatible** - Drop-in replacement via DSN change -- **Edge-Native Ingestion** - Sub-50ms p50 latency -- **Intelligent Grouping** - Fingerprinting with optional ML-based semantic matching -- **Source Map Support** - Upload, storage, and symbolication -- **Real-Time Alerting** - Webhooks, Slack, Discord, PagerDuty -- **MCP Tools** - AI-native error investigation -- **Per-Project Isolation** - Each agent/project gets dedicated Durable Object +### Error Capture -## Architecture +```typescript +// Report errors naturally +await sentry`database connection failed in checkout` +await sentry`payment timeout for order ${orderId}` +await sentry`user ${userId} hit rate limit on /api/search` + +// AI infers severity, tags, and context +await sentry`critical: payment processing down` // severity: fatal +await sentry`checkout button not responding` // severity: error, tag: checkout +await sentry`slow query on user lookup` // severity: warning +``` +### Error Investigation + +```typescript +// Query your errors naturally +const recent = await sentry`errors in the last hour` +const critical = await sentry`unresolved critical errors` +const checkout = await sentry`errors affecting checkout this week` + +// AI-powered investigation +await sentry`investigate error abc123` + .explain() // what happened and why + .suggest() // how to fix it + .similar() // related errors ``` - +-----------------------+ - | errors.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | ErrorIngestionDO | | IssueGroupingDO | | SymbolicationDO | - | Rate limiting | | Fingerprinting | | Source maps | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +-------------------+ - | D1 / R2 / KV | - +-------------------+ + +### Error Resolution + +```typescript +// Fix workflows with AI agents +await sentry`error abc123` + .investigate() + .map(findings => ralph`fix this: ${findings}`) + .map(fix => tom`review fix for ${fix}`) + +// Or let Quinn (QA) handle the whole thing +await quinn`investigate and fix error abc123` ``` -## Installation +### Issue Management -```bash -npm install @dotdo/sentry +```typescript +// Manage issues naturally +await sentry`mark abc123 resolved` +await sentry`assign xyz789 to ralph` +await sentry`ignore errors from /health endpoint` +await sentry`merge abc123 and def456` + +// Bulk operations +await sentry`resolve all checkout errors from yesterday` +await sentry`assign authentication errors to tom` +``` + +### Alerting + +```typescript +// Configure alerts naturally +await sentry`alert slack #engineering on critical errors` +await sentry`page on-call when error rate spikes 50%` +await sentry`notify ralph when checkout errors exceed 10/minute` + +// Smart alerting (AI groups related issues) +await sentry`alert on new issue types only` +``` + +### Source Maps + +```typescript +// Upload source maps naturally +await sentry`upload sourcemaps from ./dist for release 1.0.0` + +// Query with symbolication +await sentry`errors in checkout.ts line 42` ``` -## Quick Start +## Promise Pipelining -### Basic Error Capture +Chain operations without round trips: ```typescript -import * as Sentry from '@dotdo/sentry' +// One network call, multiple operations +await sentry`critical errors this week` + .map(error => quinn`investigate ${error}`) + .map(findings => ralph`fix ${findings}`) + .map(fix => [tom, priya].map(r => r`review ${fix}`)) + +// AI agent workflows +const resolved = await sentry`unresolved errors in payments` + .each(error => error.investigate().suggest()) + .filter(e => e.confidence > 0.8) + .each(error => error.assign('ralph')) +``` -Sentry.init({ - dsn: 'https://key@errors.do/123', - release: '1.0.0', - environment: 'production', -}) +## Sentry SDK Compatibility + +Drop-in replacement - just change the DSN: -// Auto-capture unhandled errors -Sentry.captureException(new Error('Database connection failed')) +```typescript +import * as Sentry from 'sentry.do/compat' -// Capture with context -Sentry.captureException(error, { - tags: { feature: 'checkout' }, - extra: { orderId: '12345' }, +// Works with existing code +Sentry.init({ + dsn: 'https://key@errors.do/123', // Point to your instance }) + +// All existing Sentry SDK code works +Sentry.captureException(new Error('Something went wrong')) +Sentry.setUser({ id: 'user-123' }) +Sentry.addBreadcrumb({ category: 'ui', message: 'Button clicked' }) ``` -### Cloudflare Workers Integration +## Cloudflare Workers Integration ```typescript -import { withSentry } from '@dotdo/sentry/cloudflare' +import { withSentry } from 'sentry.do/cloudflare' export default withSentry({ dsn: 'https://key@errors.do/123', @@ -111,110 +248,176 @@ export default withSentry({ }) ``` -### MCP Tools for AI Agents +## AI Agent Integration ```typescript -import { quinn } from 'agents.do' +import { quinn, ralph, tom } from 'agents.do' -// Quinn (QA agent) investigates errors -quinn`what are the top unresolved errors this week?` +// Quinn (QA) investigates errors +await quinn`what are the top unresolved errors this week?` +await quinn`investigate the checkout error spike` -// Ralph (Dev agent) debugs specific issues -ralph`investigate error sentry-abc123 and suggest a fix` +// Ralph (Dev) debugs and fixes +await ralph`investigate error sentry-abc123 and suggest a fix` +await ralph`implement fix for ${findings}` + +// Full workflow: detect -> investigate -> fix -> review +await sentry`new critical errors` + .map(e => quinn`investigate ${e}`) + .map(findings => ralph`fix ${findings}`) + .map(fix => tom`review ${fix}`) ``` -## API Reference +## Architecture -### Initialization +### Edge-Native Design -```typescript -Sentry.init({ - dsn: string, // Required: errors.do DSN - release?: string, // App version for source maps - environment?: string, // production, staging, etc. - sampleRate?: number, // 0.0 to 1.0 (default: 1.0) - beforeSend?: (event) => event | null, // Filter/modify events -}) +``` + +-----------------------+ + | errors.do | + | (Cloudflare Worker) | + +-----------------------+ + | + +---------------+---------------+ + | | | + +------------------+ +------------------+ +------------------+ + | ErrorIngestionDO | | IssueGroupingDO | | SymbolicationDO | + | Edge capture | | AI-powered | | Source maps | + | <10ms p50 | | fingerprinting | | symbolication | + +------------------+ +------------------+ +------------------+ + | | | + +---------------+---------------+ + | + +-------------------+ + | SQLite (hot) | + | R2 (warm/cold) | + +-------------------+ ``` -### Error Capture +### Storage Tiers -```typescript -Sentry.captureException(error, options?) -Sentry.captureMessage(message, level?) -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active issues, recent events | <10ms | +| **Warm** | R2 + SQLite Index | Historical events (30-90 days) | <100ms | +| **Cold** | R2 Archive | Compliance retention | <1s | -### Context +### Per-Project Isolation -```typescript -Sentry.setUser({ id, email, username }) -Sentry.setTag(key, value) -Sentry.setExtra(key, value) -Sentry.addBreadcrumb({ category, message, level, data }) -``` +Each project gets its own Durable Object. Complete data isolation. No noisy neighbor problems. -### Scope +## vs Sentry -```typescript -Sentry.withScope((scope) => { - scope.setTag('transaction', 'checkout') - Sentry.captureException(error) -}) -``` +| Feature | Sentry | sentry.do | +|---------|--------|-----------| +| **Latency** | 50-200ms | <10ms edge | +| **Pricing** | Per-event + overages | Flat rate | +| **Self-Hosting** | 10+ services | One Durable Object | +| **AI Features** | Premium only | Built-in, open | +| **Agent Integration** | None | MCP native | +| **Data Location** | US/EU only | Your Cloudflare account | +| **Semantic Grouping** | Stack trace only | AI-powered | +| **Auto-Fix** | Premium, limited | Quinn + Ralph | -## Source Maps +## Use Cases -Upload source maps during your build: +### AI Agent Monitoring -```bash -# Using CLI -npx @dotdo/sentry-cli releases new 1.0.0 -npx @dotdo/sentry-cli releases files 1.0.0 upload-sourcemaps ./dist +```typescript +// Every agent gets their own error context +const ralphErrors = await sentry`errors from agent ralph today` +const quinnErrors = await sentry`errors from agent quinn today` -# Or via API -curl -X POST https://errors.do/api/123/sourcemaps \ - -H "Authorization: Bearer $API_KEY" \ - -F "release=1.0.0" \ - -F "file=@dist/app.js.map" +// Agents can monitor themselves +await ralph`check my recent errors and investigate any critical ones` ``` -## Alerting +### Multi-Tenant Applications + +```typescript +// Errors by tenant +await sentry`errors for tenant acme-corp this week` +await sentry`which tenants have the most errors?` + +// Isolation by design +// Each tenant can have their own error.do instance +``` -Configure alerts in the dashboard or via API: +### CI/CD Integration ```typescript -// Webhook notification -{ - "type": "new_issue", - "channel": "webhook", - "url": "https://your-app.com/webhook", - "conditions": { - "level": ["error", "fatal"] - } -} +// Block deploys on error spikes +await sentry`error rate for release 1.2.3` +await sentry`regressions in the last deploy` -// Slack notification -{ - "type": "error_spike", - "channel": "slack", - "webhook_url": "https://hooks.slack.com/...", - "threshold": { "increase_percent": 50, "window_minutes": 5 } -} +// Release tracking +await sentry`compare errors between 1.2.2 and 1.2.3` ``` -## The Rewrites Ecosystem +### On-Call Workflows -@dotdo/sentry is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare Durable Objects: +```typescript +// Smart on-call +await sentry`critical unresolved errors I should know about` +await sentry`errors affecting > 100 users` + +// AI triage +await sentry`prioritize tonight's errors by impact` + .each(e => e.investigate()) + .notify('#on-call') +``` -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| **@dotdo/sentry** | Sentry | Error monitoring for AI | -| mongo.do | MongoDB | Document database for AI | -| kafka.do | Kafka | Event streaming for AI | +## Roadmap + +### Core +- [x] Error ingestion (<10ms p50) +- [x] Issue grouping (fingerprinting) +- [x] Source map support +- [x] Sentry SDK compatibility +- [x] Natural language API +- [ ] Performance monitoring (traces) +- [ ] Session replay +- [ ] Cron monitoring + +### AI +- [x] Semantic grouping +- [x] AI investigation (Quinn) +- [x] Fix suggestions (Ralph) +- [x] MCP tools +- [ ] Auto-fix with review (experimental) +- [ ] Predictive alerting +- [ ] Root cause analysis + +### Integrations +- [x] Slack alerts +- [x] Discord alerts +- [x] Webhook notifications +- [ ] PagerDuty +- [ ] Opsgenie +- [ ] Linear/Jira issue creation + +## Contributing + +```bash +git clone https://github.com/dotdo/sentry.do +cd sentry.do +pnpm install +pnpm test +``` ## License -MIT +MIT License + +--- + +

+ Error monitoring for the edge age. +
+ Sub-50ms ingestion. AI-native debugging. Your infrastructure. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/servicenow/README.md b/rewrites/servicenow/README.md index 4acde650..361d90aa 100644 --- a/rewrites/servicenow/README.md +++ b/rewrites/servicenow/README.md @@ -6,27 +6,77 @@ ServiceNow built a $150B+ company charging $100+ per user per month for IT Servi **servicenow.do** is the open-source alternative. Deploy your own instance in one click. AI agents are first-class citizens. Table API compatible - your existing integrations just work. +## AI-Native API + +```typescript +import { servicenow } from 'servicenow.do' // Full SDK +import { servicenow } from 'servicenow.do/tiny' // Minimal client +import { servicenow } from 'servicenow.do/table' // Table API only +``` + +Natural language for IT service management: + +```typescript +import { servicenow } from 'servicenow.do' + +// Talk to it like a service desk +const incident = await servicenow`email server down for networking team` +const p1s = await servicenow`P1 incidents this week` +const changes = await servicenow`pending changes for CAB review` + +// Chain like sentences +await servicenow`P1 incidents this week` + .map(inc => servicenow`root cause for ${inc}`) + +// Incidents that resolve themselves +await servicenow`email server down` + .assign('Network Operations') + .resolve('Restarted mail service') +``` + ## The Problem ServiceNow revolutionized enterprise IT. But the model is broken: -- **$100+ per user/month** - A 1,000-person company pays $1.2M+ annually -- **6-18 month implementations** - Consultants cost more than the software -- **Shared tenants** - Your data lives with everyone else's -- **AI as afterthought** - "Now Assist" bolted onto 20-year-old architecture -- **Vendor lock-in** - Leaving means losing years of configuration +| What ServiceNow Charges | The Reality | +|------------------------|-------------| +| **Per User Licensing** | $100+/user/month ($1.2M+ annually for 1,000 users) | +| **Implementation** | 6-18 months, consultants cost more than software | +| **Shared Tenants** | Your data lives with everyone else's | +| **AI Integration** | "Now Assist" bolted onto 20-year-old architecture | +| **Vendor Lock-in** | Leaving means losing years of configuration | -Meanwhile, AI agents are the new workforce. They need to file incidents, resolve tickets, run change management. ServiceNow wasn't built for them. +### The Now Tax + +Since going public: + +- Aggressive upselling to higher tiers +- Complexity drives consultant dependency +- Slow innovation on core platform +- AI features require premium licenses +- Data trapped in proprietary formats + +IT departments are hostage to a workflow company that sees ITSM as a subscription vehicle. + +### The AI Gap + +AI agents are the new workforce. They need to file incidents, resolve tickets, run change management. ServiceNow wasn't built for them. ## The Solution **servicenow.do** reimagines ITSM for the AI era: -- **One-click deploy** - Your own instance, not a shared tenant -- **Edge-native** - Runs on Cloudflare's global network, not legacy data centers -- **AI-native** - AI agents can file incidents, resolve tickets, manage changes -- **Open source** - MIT licensed, no vendor lock-in -- **Table API compatible** - `/api/now/table/*` works exactly the same +``` +ServiceNow servicenow.do +----------------------------------------------------------------- +$100+/user/month Deploy for free +6-18 month implementation Deploy in minutes +Shared tenant Your own instance +AI as afterthought AI-native design +Oracle/Azure data centers Your Cloudflare account +$$$ consultants Code it yourself +Vendor lock-in Open source, MIT licensed +``` ## One-Click Deploy @@ -36,8 +86,6 @@ npx create-dotdo servicenow That's it. Your own ServiceNow instance running on Cloudflare's edge. -Or deploy to your workers.do workspace: - ```typescript import { ServiceNow } from 'servicenow.do' @@ -51,133 +99,104 @@ export default ServiceNow({ ### Incident Management -Full incident lifecycle - create, assign, escalate, resolve, close. - ```typescript -// Create an incident -const incident = await snow.incidents.create({ - short_description: 'Email server down', - urgency: 1, - impact: 1, - category: 'Network', -}) - -// Assign to a group -await incident.assign({ group: 'Network Operations' }) - -// Resolve -await incident.resolve({ - close_notes: 'Restarted mail service', - close_code: 'Solved (Permanently)', -}) +// Just say it +const incident = await servicenow`email server down for networking team` + +// Full lifecycle in natural language +await servicenow`email server down` + .assign('Network Operations') + .escalate('P1') + .resolve('Restarted mail service') + +// Query incidents naturally +await servicenow`open incidents for networking team` +await servicenow`P1 incidents this week` +await servicenow`incidents affecting prod-web-01` + +// Batch operations read like commands +await servicenow`reassign all John Smith incidents to Jane Doe` +await servicenow`close resolved incidents older than 30 days` ``` ### Problem Management -Root cause analysis and permanent fixes. - ```typescript -// Create problem from related incidents -const problem = await snow.problems.create({ - short_description: 'Recurring mail server crashes', - related_incidents: [inc001, inc002, inc003], -}) - -// Document root cause -await problem.rootCause({ - cause: 'Memory leak in mail daemon', - workaround: 'Scheduled restart every 24h', -}) - -// Implement permanent fix -await problem.implement({ - change: change001, - resolution: 'Upgraded mail server to v2.1.0', -}) +// Create problems from patterns +await servicenow`problem from recurring mail server crashes` + +// Root cause analysis +await servicenow`root cause for PRB0001234` + .document('Memory leak in mail daemon') + .workaround('Scheduled restart every 24h') + .fix('Upgrade to v2.1.0') + +// Find problems naturally +await servicenow`problems without root cause` +await servicenow`problems linked to prod-db-01` ``` ### Change Management -Standard, normal, and emergency changes with approval workflows. - ```typescript -// Create a change request -const change = await snow.changes.create({ - type: 'normal', - short_description: 'Upgrade production database', - risk: 'moderate', - impact_analysis: 'Brief downtime during maintenance window', -}) - -// Submit for approval -await change.submit() - -// Approve (via workflow or AI agent) -await change.approve({ approver: 'CAB' }) - -// Implement -await change.implement() +// Request changes naturally +const change = await servicenow`change to upgrade production database` + .risk('moderate') + .window('Sunday 2am-6am') + +// Approval workflow in plain English +await servicenow`submit CHG0001234 for CAB approval` +await servicenow`approve CHG0001234` +await servicenow`implement CHG0001234` + +// Query changes naturally +await servicenow`pending changes for CAB review` +await servicenow`emergency changes this month` +await servicenow`failed changes in Q4` ``` ### Service Catalog -Self-service requests for users and AI agents. - ```typescript -// Define a catalog item -const laptopRequest = await snow.catalog.create({ - name: 'Request New Laptop', - category: 'Hardware', - workflow: 'hardware-fulfillment', - variables: [ - { name: 'laptop_type', type: 'choice', choices: ['MacBook Pro', 'ThinkPad'] }, - { name: 'accessories', type: 'multiselect', choices: ['Monitor', 'Keyboard', 'Mouse'] }, - ], -}) - -// Submit a request -await snow.requests.create({ - catalog_item: laptopRequest.sys_id, - variables: { - laptop_type: 'MacBook Pro', - accessories: ['Monitor', 'Keyboard'], - }, -}) +// Request things naturally +await servicenow`request MacBook Pro with monitor and keyboard` +await servicenow`request access to production database` +await servicenow`request new hire onboarding for Jane Doe` + +// Check request status +await servicenow`my pending requests` +await servicenow`hardware requests awaiting approval` ``` ### Knowledge Base -Searchable documentation for humans and AI. - ```typescript -// Create an article -await snow.knowledge.create({ - title: 'How to reset your password', - category: 'Self-Service', - content: '...', - keywords: ['password', 'reset', 'login'], -}) - -// Search (used by AI agents to resolve tickets) -const articles = await snow.knowledge.search('email not working') +// Search knowledge naturally +await servicenow`how to reset VPN password` +await servicenow`email troubleshooting steps` + +// Create articles naturally +await servicenow`article: how to connect to VPN from home` + .category('Self-Service') + .keywords('vpn', 'remote', 'work from home') + +// AI uses knowledge to resolve tickets +await servicenow`unresolved incidents` + .map(inc => servicenow`knowledge article for ${inc}`) + .map(article => inc.suggest(article)) ``` ### CMDB -Configuration management database for your entire infrastructure. - ```typescript -// Register a CI -await snow.cmdb.create({ - sys_class_name: 'cmdb_ci_server', - name: 'prod-web-01', - ip_address: '10.0.1.100', - environment: 'production', - owner: 'platform-team', -}) - -// Query relationships -const dependencies = await snow.cmdb.relationships('prod-web-01') +// Register assets naturally +await servicenow`server prod-web-01 at 10.0.1.100 in production` +await servicenow`database prod-db-01 depends on prod-storage-01` + +// Query the CMDB naturally +await servicenow`what depends on prod-web-01` +await servicenow`production servers in Austin data center` +await servicenow`assets owned by platform team` ``` ## Table API Compatible @@ -201,63 +220,66 @@ curl -X PATCH https://your-instance.servicenow.do/api/now/table/incident/INC0001 All `/api/now/table/*` endpoints are supported. GlideRecord queries work. Scripted REST APIs work. Your existing integrations just work. -## AI-Native +## AI-Native ITSM -AI agents are first-class citizens in servicenow.do: +### Incidents That Triage Themselves + +```typescript +// AI determines urgency and impact automatically +await servicenow`checkout broken, users see 500 error, started 10 min ago` +// -> P1 incident, assigned to Critical Response Team, notifications sent + +// Or be explicit +await servicenow`email slow for marketing team` + .priority('P3') + .assign('Email Team') +``` -### AI Files Tickets +### Root Cause in One Line ```typescript -import { quinn } from 'agents.do' - -// QA agent found a bug -quinn` - The checkout flow is broken in production. - Users see a 500 error when clicking "Place Order". - Affects all users. Started 10 minutes ago. -` -// Quinn automatically files an incident with correct urgency/impact +// AI analyzes patterns and suggests root cause +await servicenow`P1 incidents this week` + .map(inc => servicenow`root cause for ${inc}`) + .map(rca => servicenow`problem from ${rca}`) + +// Or just ask +await servicenow`why did prod-db-01 fail yesterday` ``` -### AI Resolves Tickets +### Change Impact Analysis ```typescript -import { ralph } from 'agents.do' - -// Developer agent resolves incidents -ralph` - INC0012345 is caused by a database connection pool exhaustion. - Deployed fix in PR #789. Connection limit increased from 50 to 200. - Monitoring shows errors have stopped. -` -// Ralph updates the incident, links the change, and resolves +// AI evaluates change risk automatically +await servicenow`change to upgrade production database` +// -> Analyzes CMDB, identifies dependencies, suggests maintenance window + +// Review pending changes +await servicenow`pending changes` + .map(chg => servicenow`impact analysis for ${chg}`) ``` -### AI Runs Changes +### Knowledge That Learns ```typescript -import { tom } from 'agents.do' - -// Tech lead manages change approval -tom` - Review CHG0001234 for the database upgrade. - Verify the rollback plan is complete. - Approve if risk assessment is acceptable. -` -// Tom reviews the change, verifies docs, and approves via CAB workflow +// AI creates articles from resolved incidents +await servicenow`resolved incidents this month` + .map(inc => servicenow`knowledge article from ${inc}`) + +// Articles improve resolution times +await servicenow`new incidents` + .map(inc => servicenow`suggest resolution for ${inc}`) ``` -### AI Searches Knowledge +### SLA Management ```typescript -import { priya } from 'agents.do' - -// Product agent answers questions -priya` - A customer is asking about our SLA for P1 incidents. - Check the knowledge base and respond with our policy. -` -// Priya searches KB, finds the article, and responds accurately +// Monitor SLAs naturally +await servicenow`incidents breaching SLA` +await servicenow`P1 incidents close to breach` + +// AI escalates automatically +// -> Notifications sent before breach, not after ``` ## Architecture @@ -291,62 +313,54 @@ servicenow.do is built on Cloudflare's edge infrastructure: ### Dynamic Table Engine -ServiceNow's power is its table-driven architecture. servicenow.do implements this as a dynamic schema engine: +ServiceNow's power is its table-driven architecture. servicenow.do implements this naturally: ```typescript -// Tables are defined at runtime -await snow.tables.create({ - name: 'u_custom_asset', - extends: 'cmdb_ci', - columns: [ - { name: 'u_asset_tag', type: 'string', maxLength: 40 }, - { name: 'u_purchase_date', type: 'date' }, - { name: 'u_cost', type: 'currency' }, - ], -}) - -// Queries work immediately -const assets = await snow.table('u_custom_asset') - .where('u_cost', '>', 10000) - .orderBy('u_purchase_date', 'desc') - .limit(100) +// Create tables with natural language +await servicenow`table custom_asset extending cmdb_ci with asset_tag, purchase_date, cost` + +// Or be explicit about types +await servicenow`table custom_asset` + .extends('cmdb_ci') + .field('asset_tag', 'string') + .field('purchase_date', 'date') + .field('cost', 'currency') + +// Query naturally +await servicenow`custom assets over $10,000 by purchase date` +await servicenow`custom assets purchased this year` ``` ### Business Rules Engine -React to data changes with server-side logic: +React to data changes with natural language: ```typescript -// Business rule: auto-assign P1 incidents -await snow.rules.create({ - table: 'incident', - when: 'before', - operation: 'insert', - condition: 'priority == 1', - script: async (current) => { - current.assignment_group = 'Critical Response Team' - current.notify = 'manager' - }, -}) +// Auto-assign P1 incidents +await servicenow`when P1 incident created assign to Critical Response Team and notify manager` + +// Auto-escalate breaching SLAs +await servicenow`when incident SLA at 80% escalate to supervisor` + +// Close stale incidents +await servicenow`when resolved incident untouched for 7 days close automatically` ``` ### Workflow Engine -Visual workflows for approvals, tasks, and automation: +Define workflows naturally: ```typescript // Change approval workflow -await snow.workflows.create({ - name: 'Normal Change Approval', - table: 'change_request', - stages: [ - { name: 'Submit', action: 'validate_fields' }, - { name: 'Review', action: 'assign_reviewer' }, - { name: 'CAB Approval', action: 'cab_vote', quorum: 3 }, - { name: 'Implementation', action: 'schedule_window' }, - { name: 'Post-Implementation', action: 'verify_success' }, - ], -}) +await servicenow`workflow: normal change approval` + .step('submit and validate') + .step('assign reviewer') + .step('CAB approval with 3 votes') + .step('schedule implementation window') + .step('verify success') + +// Or describe it +await servicenow`create workflow for normal changes requiring CAB approval` ``` ### Durable Object per Instance @@ -362,32 +376,98 @@ Each servicenow.do deployment is a single Durable Object: | Feature | ServiceNow | servicenow.do | |---------|------------|---------------| -| Pricing | $100+/user/month | Free (open source) | -| Deployment | 6-18 months | 1 click | -| Architecture | Shared tenant | Your own instance | -| AI Integration | Bolted on | Native | -| Table API | Proprietary | Compatible | -| Data Location | Their data centers | Your choice (edge) | -| Customization | $$$$ consultants | Code it yourself | -| Lock-in | Years of migration | MIT licensed | +| **Pricing** | $100+/user/month | Free (open source) | +| **Deployment** | 6-18 months | Deploy in minutes | +| **Architecture** | Shared tenant | Your own instance | +| **AI** | Bolted on (Now Assist) | AI-first design | +| **Table API** | Proprietary | Compatible | +| **Data Location** | Their data centers | Your Cloudflare account | +| **Customization** | $$$$ consultants | Code it yourself | +| **Lock-in** | Years of migration | MIT licensed | + +## Use Cases + +### IT Operations + +```typescript +// Daily standup for ops team +await servicenow`P1 and P2 incidents from overnight` +await servicenow`changes scheduled for today` +await servicenow`SLAs at risk` + +// Incident management at scale +await servicenow`open incidents` + .map(inc => servicenow`suggested resolution for ${inc}`) + .map((inc, suggestion) => inc.tryResolve(suggestion)) +``` + +### Service Desk + +```typescript +// First-line support +await servicenow`new tickets for Service Desk` + .map(ticket => servicenow`auto-resolve if knowledge exists for ${ticket}`) + +// Escalation +await servicenow`tickets open more than 4 hours` + .map(ticket => ticket.escalate()) +``` + +### Change Management + +```typescript +// CAB meeting prep +await servicenow`changes pending CAB approval` + .map(chg => servicenow`risk assessment for ${chg}`) + +// Post-implementation review +await servicenow`changes implemented this week` + .map(chg => servicenow`success rate for ${chg}`) +``` + +### Reporting + +```typescript +// Executive dashboard +await servicenow`MTTR this month vs last month` +await servicenow`incident volume by category` +await servicenow`SLA compliance by team` + +// Export for analysis +await servicenow`export P1 incidents for 2024` +``` ## Roadmap +### Core ITSM - [x] Incident Management - [x] Problem Management - [x] Change Management - [x] Service Catalog - [x] Knowledge Base - [x] CMDB + +### Platform - [x] Table API compatibility - [x] Business Rules Engine - [x] Workflow Engine -- [ ] Service Level Management +- [x] SLA Management - [ ] Asset Management - [ ] Project Portfolio Management + +### Enterprise - [ ] Discovery & Service Mapping - [ ] Security Operations - [ ] HR Service Delivery +- [ ] Customer Service Management + +### AI +- [x] Natural language queries +- [x] Auto-triage incidents +- [x] Root cause suggestions +- [x] Knowledge article generation +- [ ] Predictive incident prevention +- [ ] Automated resolution ## Contributing @@ -403,3 +483,16 @@ pnpm test ## License MIT + +--- + +

+ The $150B tax ends here. +
+ Edge-native. AI-first. Your instance. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/servicetitan/README.md b/rewrites/servicetitan/README.md index f48ca511..3b765c6d 100644 --- a/rewrites/servicetitan/README.md +++ b/rewrites/servicetitan/README.md @@ -6,14 +6,47 @@ ServiceTitan charges $300-500+/month. That prices out 90% of contractors. Your neighborhood plumber deserves the same tools as a $10B franchise. +## AI-Native API + +```typescript +import { servicetitan } from 'servicetitan.do' // Full SDK +import { servicetitan } from 'servicetitan.do/tiny' // Minimal client +import { servicetitan } from 'servicetitan.do/mobile' // Tech mobile app +``` + +Natural language for field service: + +```typescript +import { servicetitan } from 'servicetitan.do' + +// Talk to it like you're on the phone with dispatch +const job = await servicetitan`AC repair 123 Main St, Johnson Residence, same-day` +await servicetitan`assign ${job} to Mike` +await servicetitan`optimize routes for today` + +// Chain like a conversation +await servicetitan`jobs scheduled for tomorrow` + .map(job => servicetitan`send ETA reminder to ${job.customer}`) + +// The whole call-to-cash cycle +await servicetitan`water heater install 456 Oak Ave, urgent` + .assign() // AI picks best tech + .estimate() // good-better-best options + .complete() // tech marks done + .invoice() // auto-generate invoice + .collect() // tap to pay on-site +``` + ## The Problem Field service management software has become a gatekeeper: -- **ServiceTitan**: $300-500+/month minimum, enterprise sales process -- **Housecall Pro**: $65-200/month, AI features locked behind premium tiers -- **Jobber**: $49-199/month, dispatch optimization costs extra -- **FieldEdge**: Enterprise pricing, 12-month contracts +| What They Charge | The Reality | +|------------------|-------------| +| **ServiceTitan** | $300-500+/month minimum, enterprise sales process | +| **Housecall Pro** | $65-200/month, AI features locked behind premium tiers | +| **Jobber** | $49-199/month, dispatch optimization costs extra | +| **FieldEdge** | Enterprise pricing, 12-month contracts | Meanwhile, your local HVAC tech runs their business from sticky notes and a whiteboard. @@ -23,13 +56,16 @@ The result? Small contractors work harder, earn less, and lose jobs to companies **servicetitan.do** is open-source field service management that deploys in one click. -Every plumber, electrician, and HVAC technician gets: -- Intelligent scheduling and dispatch -- Real-time GPS tracking -- Professional estimates and invoices -- Inventory management -- Customer history and communication -- AI that actually helps (not upsells) +``` +ServiceTitan servicetitan.do +----------------------------------------------------------------- +$300-500+/month $0 (self-host) +Enterprise sales process Deploy in 5 minutes +AI features cost extra AI-first, included +12-month contracts No contracts +Proprietary data Your data, your servers +$500/hour customization Open source, modify anything +``` Running on Cloudflare's edge. Costing pennies. Owned by you. @@ -41,172 +77,158 @@ npx create-dotdo servicetitan That's it. Your HVAC company now has its own scheduling, dispatch, and invoicing system. -Or deploy instantly to Cloudflare: +```typescript +import { ServiceTitan } from 'servicetitan.do' -[![Deploy to Cloudflare Workers](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/dotdo/servicetitan.do) +export default ServiceTitan({ + name: 'johnson-hvac', + domain: 'dispatch.johnsonhvac.com', + timezone: 'America/Chicago', +}) +``` ## Features ### Jobs & Work Orders ```typescript -import { fsm } from 'servicetitan.do' - -// Create a job from a customer call -const job = await fsm.jobs.create({ - customer: 'Johnson Residence', - address: '123 Main St', - type: 'hvac-repair', - urgency: 'same-day', - description: 'AC not cooling, thermostat reads 85F' -}) - -// AI suggests the best technician -const assignment = await fsm.dispatch.recommend(job) -// -> { technician: 'Mike', eta: '2:30 PM', confidence: 0.94 } +// Create jobs like you're talking to dispatch +const job = await servicetitan`AC not cooling at Johnson Residence, 123 Main St` +const urgent = await servicetitan`emergency water leak at 456 Oak Ave` +const scheduled = await servicetitan`tune-up for Maria next Tuesday morning` + +// AI infers what you need +await servicetitan`Johnson Residence` // returns customer +await servicetitan`jobs at Johnson Residence` // returns job history +await servicetitan`Johnson Residence equipment` // returns installed units ``` -### Intelligent Dispatch Board +### Intelligent Dispatch -Real-time dispatch with WebSocket updates: +```typescript +// Dispatch like you're talking to the board +await servicetitan`assign ${job} to best tech for HVAC` +await servicetitan`who's closest to 123 Main St?` +await servicetitan`Mike's schedule today` + +// AI handles the complexity +await servicetitan`assign ${job}` +// -> Considers: skills, location, traffic, workload, customer history +// -> Returns: { tech: 'Mike', eta: '2:30 PM', confidence: 0.94 } + +// Bulk dispatch just works +await servicetitan`jobs needing assignment` + .each(job => servicetitan`assign ${job}`) +``` + +### Route Optimization ```typescript -// Subscribe to dispatch board changes -fsm.dispatch.subscribe(board => { - // Live updates as jobs move, techs report status - console.log(board.unassigned) // Jobs needing techs - console.log(board.enroute) // Techs heading to jobs - console.log(board.onsite) // Active jobs -}) +// One line to optimize the day +await servicetitan`optimize routes for today` +await servicetitan`optimize routes to maximize revenue` +await servicetitan`optimize routes to minimize drive time` + +// Re-optimize on the fly +await servicetitan`shuffle today's routes, emergency came in` ``` -### Pricebook & Estimates +### Estimates & Pricebook ```typescript -// Generate professional estimates with your pricebook -const estimate = await fsm.estimates.create({ - job: job.id, - options: [ - { name: 'Repair', items: ['capacitor-replacement', 'refrigerant-recharge'] }, - { name: 'Replace', items: ['ac-unit-3ton', 'installation-labor'] } - ] -}) +// Generate estimates naturally +await servicetitan`estimate for ${job}: capacitor replacement or full AC swap` +await servicetitan`good-better-best for water heater install` -// AI-generated good-better-best options -const aiEstimate = await fsm.estimates.generate({ - job: job.id, - diagnosis: 'Compressor failing, unit is 15 years old' -}) +// AI builds from diagnosis +await servicetitan`compressor failing, unit is 15 years old` + .estimate() // generates repair vs replace options + +// Present and close on-site +await servicetitan`customer approved the replacement option` ``` ### Invoicing & Payments ```typescript -// Convert completed job to invoice -const invoice = await fsm.invoices.create({ - job: job.id, - items: estimate.selectedOption.items, - labor: { hours: 2.5, rate: 125 } -}) +// Job complete? Invoice it +await servicetitan`invoice ${job}` -// Accept payment on-site -await fsm.payments.charge({ - invoice: invoice.id, - method: 'card-present', // Tap to pay - amount: invoice.total -}) +// Or let the workflow handle it +await servicetitan`complete ${job}: replaced capacitor, 2.5 hours` + .invoice() // auto-generates from parts + labor + .collect() // tap to pay on-site + +// Check payment status +await servicetitan`unpaid invoices this week` +await servicetitan`Johnson Residence balance` ``` -### GPS Fleet Tracking +### Fleet Tracking ```typescript -// Real-time technician locations -fsm.fleet.track(technicianId, location => { - // Updates every 30 seconds (configurable) - map.updateMarker(technicianId, location) -}) +// Real-time location +await servicetitan`where is Mike?` +await servicetitan`techs near downtown` +await servicetitan`Mike's route progress` -// Route optimization for the day -const routes = await fsm.routes.optimize({ - technicians: ['mike', 'sarah', 'dave'], - jobs: todaysJobs, - constraints: { - maxDriveTime: 45, // minutes between jobs - lunchWindow: ['12:00', '13:00'] - } -}) +// Customers get Uber-style tracking +await servicetitan`send ETA to Johnson Residence` +// -> "Mike is 10 minutes away!" ``` ### Inventory Management ```typescript -// Track parts on trucks and in warehouse -await fsm.inventory.use({ - technician: 'mike', - part: 'capacitor-45/5', - job: job.id -}) +// Track parts naturally +await servicetitan`Mike used a 45/5 capacitor on ${job}` +await servicetitan`parts low on Mike's truck` +await servicetitan`warehouse inventory for capacitors` -// Auto-reorder when low -fsm.inventory.setThreshold('capacitor-45/5', { - warehouse: 20, - truck: 3, - supplier: 'grainger', - autoOrder: true -}) +// Auto-reorder just works +await servicetitan`reorder parts running low` ``` -## AI-Powered Dispatch +## AI-Powered Operations This isn't "AI" as a marketing checkbox. It's intelligence that makes contractors money. ### Smart Assignment ```typescript -// AI considers everything: -// - Technician skills and certifications -// - Current location and traffic -// - Job complexity and history -// - Customer preferences ("Sarah was great last time") -// - Profitability (senior tech for premium jobs) - -const assignment = await fsm.dispatch.recommend(job) +// AI considers everything, you just ask +await servicetitan`assign ${job}` + +// Factors: skills, certs, location, traffic, workload, +// customer history ("Sarah was great last time"), profitability ``` -### Route Optimization +### Duration Prediction ```typescript -// Minimize drive time, maximize billable hours -await fsm.routes.optimize({ - objective: 'maximize-revenue', // or 'minimize-fuel', 'balance-workload' - constraints: { - technicianHours: 8, - priorityJobs: ['emergency', 'callback'] - } -}) +// AI learns from your data +await servicetitan`how long for a water heater install at ${job.address}?` +// -> "3.5 hours - older home, basement access adds time" ``` -### Job Duration Prediction +### Automatic Scheduling ```typescript -// AI learns from your historical data -const prediction = await fsm.jobs.predictDuration({ - type: 'water-heater-install', - equipment: 'rheem-50gal-gas', - location: job.address -}) -// -> { estimate: 3.5, confidence: 0.87, factors: ['older home', 'basement access'] } +// Customer self-books, AI slots them perfectly +await servicetitan`enable online booking for tune-ups and estimates` + +// AI manages capacity, prevents overbooking +await servicetitan`can we fit an emergency today?` ``` -### Automatic Scheduling +### Revenue Insights ```typescript -// Customer self-books, AI slots them perfectly -fsm.scheduling.enableSelfBook({ - types: ['tune-up', 'estimate', 'maintenance'], - availability: 'auto', // AI manages the calendar - confirmationChannel: 'sms' -}) +// Ask about your business +await servicetitan`revenue this week` +await servicetitan`top performing tech this month` +await servicetitan`average ticket by job type` +await servicetitan`callbacks this quarter` ``` ## Offline-First @@ -216,82 +238,48 @@ Contractors work in basements. Rural areas. Job sites with zero signal. **servicetitan.do** works offline. Period. ```typescript -// Durable Objects maintain local state -// Changes sync automatically when connected - -// On the job site (offline): -await fsm.jobs.update(job.id, { - status: 'completed', - notes: 'Replaced capacitor, tested operation', - parts: ['capacitor-45/5'] -}) +// On the job site (no signal): +await servicetitan`complete ${job}: replaced capacitor, tested operation` // -> Stored locally, queued for sync // Back in the truck (online): // -> Syncs automatically, dispatch board updates ``` -### How It Works - -Each technician's device maintains a local Durable Object replica: - -``` -Phone/Tablet - | - v -[Local DO] <-- Works offline - | - v (when connected) -[Edge DO] --> [Central DO] -``` - -No "sync failed" errors. No lost work orders. No excuses. +Each technician's device maintains a local Durable Object replica. No "sync failed" errors. No lost work orders. No excuses. ## Real-Time -Everything updates instantly across all devices: +Everything updates instantly across all devices. -### WebSocket Dispatch Board +### Dispatch Board ```typescript -// Office sees real-time status -fsm.dispatch.subscribe(update => { - switch (update.type) { - case 'tech-location': - map.moveMarker(update.techId, update.coords) - break - case 'job-status': - board.updateJob(update.jobId, update.status) - break - case 'new-job': - board.addJob(update.job) - playAlert('new-call') - break - } -}) +// Office sees everything live +await servicetitan`watch dispatch board` +// -> Real-time: job status, tech locations, new calls + +// Techs see their queue update +await servicetitan`watch my jobs` ``` ### Customer Updates ```typescript -// Customers get Uber-style tracking -fsm.notifications.enable(job.id, { - channels: ['sms', 'email'], - events: ['tech-assigned', 'on-the-way', 'arriving-soon', 'completed'] -}) +// Customers get Uber-style tracking automatically +await servicetitan`notify ${job.customer} when tech is on the way` -// "Mike is 10 minutes away!" +// Or enable for all jobs +await servicetitan`enable customer notifications for all jobs` +// -> "Mike is 10 minutes away!" ``` ### Team Communication ```typescript -// Instant messaging between office and field -fsm.chat.send({ - to: 'mike', - message: 'Customer called - they found the leak under the sink', - job: job.id -}) +// Message between office and field +await servicetitan`tell Mike: customer found the leak under the sink` +await servicetitan`message all techs: office closing early today` ``` ## Architecture @@ -369,59 +357,17 @@ npm test npm run deploy ``` -### Configuration - -```typescript -// wrangler.toml -name = "my-hvac-company" -compatibility_date = "2024-01-01" - -[vars] -COMPANY_NAME = "Johnson HVAC" -TIMEZONE = "America/Chicago" - -[[durable_objects.bindings]] -name = "JOBS" -class_name = "JobDO" - -[[durable_objects.bindings]] -name = "DISPATCH" -class_name = "DispatchDO" - -[[durable_objects.bindings]] -name = "TECHNICIANS" -class_name = "TechnicianDO" -``` - ## Integrations -### Payments - ```typescript -// Stripe Connect (payments.do) -fsm.payments.configure({ - provider: 'stripe', - accountId: 'acct_xxx' -}) -``` - -### Communication - -```typescript -// Twilio for SMS -fsm.notifications.configure({ - sms: { provider: 'twilio', accountSid: 'xxx' } -}) -``` - -### Accounting - -```typescript -// QuickBooks sync -fsm.accounting.connect({ - provider: 'quickbooks', - realmId: 'xxx' -}) +// Connect to everything naturally +await servicetitan`connect Stripe for payments` +await servicetitan`connect QuickBooks for accounting` +await servicetitan`connect Twilio for customer SMS` + +// Then just use them +await servicetitan`sync today's invoices to QuickBooks` +await servicetitan`text Johnson Residence their appointment reminder` ``` ## Pricing @@ -466,15 +412,36 @@ See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. ## Roadmap -- [x] Core job management -- [x] Dispatch board with WebSocket -- [x] Basic scheduling -- [ ] AI route optimization -- [ ] Pricebook builder -- [ ] QuickBooks integration -- [ ] Mobile apps (React Native) -- [ ] Membership/service agreement management -- [ ] Marketing automation +### Core +- [x] Jobs & Work Orders +- [x] Intelligent Dispatch +- [x] Route Optimization +- [x] Estimates & Invoicing +- [x] Payments (Stripe) +- [x] Fleet Tracking +- [x] Inventory Management +- [ ] Memberships & Service Agreements +- [ ] Marketing Automation + +### AI +- [x] Smart Tech Assignment +- [x] Route Optimization +- [x] Duration Prediction +- [ ] Demand Forecasting +- [ ] Automated Follow-ups +- [ ] Voice Dispatch + +### Integrations +- [x] Stripe Payments +- [ ] QuickBooks +- [ ] Twilio SMS +- [ ] Google Calendar +- [ ] Zapier + +### Mobile +- [ ] Tech Mobile App (React Native) +- [ ] Customer Booking App +- [ ] Offline-First Sync ## License @@ -484,10 +451,13 @@ Your business software shouldn't hold your business hostage. --- -
- -**Built with [workers.do](https://workers.do)** - -*Workers work for you.* - -
+

+ Your neighborhood plumber deserves enterprise tools. +
+ AI-first. Offline-ready. Contractor-owned. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/shopify/README.md b/rewrites/shopify/README.md index 5b00e22d..2f5018c4 100644 --- a/rewrites/shopify/README.md +++ b/rewrites/shopify/README.md @@ -6,40 +6,33 @@ Shopify powers millions of stores, taking a cut of every transaction. 2.9% + $0. **shopify.do** is the open-source alternative. Deploy your own e-commerce platform. Choose your payment processor. AI that actually sells for you. -## The workers.do Way - -You built a brand people love. You've got product-market fit. But every sale, Shopify takes nearly 3%. At scale, that's hundreds of thousands of dollars - money that should go into inventory, marketing, or your own pocket. - -**workers.do** gives you AI that actually sells: - -```typescript -import { shopify, mark } from 'workers.do' - -// Natural language for commerce -const orders = await shopify`find orders from ${campaign} this week` -const inventory = await shopify`which SKUs are below reorder point` -const customers = await shopify`show VIP customers who haven't ordered in 60 days` -``` - -Promise pipelining for order fulfillment - one network round trip: +## AI-Native API ```typescript -// Campaign to customer delight -const delighted = await shopify`find orders from ${campaign}` - .map(order => shopify`fulfill ${order} with expedited shipping`) - .map(fulfilled => mark`send thank you to ${fulfilled.customer}`) - .map(thanked => shopify`add ${thanked.customer} to VIP segment`) +import { shopify } from 'shopify.do' // Full SDK +import { shopify } from 'shopify.do/tiny' // Minimal client +import { shopify } from 'shopify.do/storefront' // Storefront-only ``` -AI agents that grow your store: +Natural language for commerce: ```typescript -import { priya, ralph, sally } from 'agents.do' - -// E-commerce intelligence -await priya`analyze cart abandonment and recommend recovery flow` -await ralph`optimize product page for ${sku} based on conversion data` -await sally`draft win-back email sequence for churned subscribers` +import { shopify } from 'shopify.do' + +// Talk to it like a colleague +const sales = await shopify`best selling products this month` +const inventory = await shopify`SKUs below reorder point` +const vips = await shopify`VIP customers who haven't ordered in 60 days` + +// Chain like sentences +await shopify`orders from Black Friday campaign` + .map(order => shopify`fulfill ${order} with tracking`) + .map(fulfilled => shopify`send thank you to ${fulfilled.customer}`) + +// Products document themselves +await shopify`create product "Premium Headphones" - Black $299, White $299` +await shopify`add to Audio collection` +await shopify`feature on homepage` ``` ## The Problem @@ -81,284 +74,112 @@ Your own e-commerce platform. With AI that actually sells. ## Features -### Storefront - -Beautiful, fast, customizable: - -```typescript -import { store } from 'shopify.do' - -// Configure your store -await store.configure({ - name: 'Acme Co', - domain: 'shop.acme.com', - currency: 'USD', - theme: 'minimal', // or 'bold', 'classic', or custom - features: { - multiCurrency: true, - multilingual: ['en', 'es', 'fr'], - subscriptions: true, - b2b: false, - }, -}) -``` - ### Products -Rich catalog management: - ```typescript -// Create product -await store.products.create({ - title: 'Premium Wireless Headphones', - handle: 'premium-wireless-headphones', - description: 'Crystal-clear audio with 30-hour battery life...', - vendor: 'Acme Audio', - type: 'Electronics', - tags: ['wireless', 'bluetooth', 'audio', 'premium'], - variants: [ - { - title: 'Midnight Black', - sku: 'AWH-001-BLK', - price: 299.99, - compareAtPrice: 349.99, - inventory: 150, - weight: 0.5, - weightUnit: 'lb', - }, - { - title: 'Pearl White', - sku: 'AWH-001-WHT', - price: 299.99, - inventory: 75, - weight: 0.5, - weightUnit: 'lb', - }, - ], - images: [ - { url: 'https://...', alt: 'Headphones front view' }, - { url: 'https://...', alt: 'Headphones side view' }, - ], - seo: { - title: 'Premium Wireless Headphones | Acme Co', - description: 'Experience crystal-clear audio...', - }, -}) - -// Create collection -await store.collections.create({ - title: 'Audio Equipment', - handle: 'audio', - rules: [ - { field: 'type', relation: 'equals', value: 'Electronics' }, - { field: 'tags', relation: 'contains', value: 'audio' }, - ], - sortOrder: 'best-selling', -}) +// Create products naturally +await shopify`create product "Premium Headphones" - Black $299, White $299` +await shopify`add "Coffee Subscription" - 12oz monthly $24.99, weekly $19.99` + +// AI infers what you need +await shopify`headphones` // returns product +await shopify`headphones inventory` // returns stock levels +await shopify`headphones sales this month` // returns analytics + +// Collections just work +await shopify`create "Audio Equipment" collection from electronics tagged audio` +await shopify`best sellers this quarter` +await shopify`products under $50 with low stock` ``` ### Checkout -Conversion-optimized, flexible: - ```typescript -// Create checkout -const checkout = await store.checkout.create({ - lineItems: [ - { variantId: 'AWH-001-BLK', quantity: 1 }, - { variantId: 'CASE-001', quantity: 1 }, - ], - email: 'customer@example.com', -}) - -// Apply discount -await checkout.applyDiscount('SAVE20') - -// Calculate shipping -const rates = await checkout.shippingRates({ - address: { - country: 'US', - state: 'CA', - zip: '94102', - }, -}) - -// Complete with any processor -await checkout.complete({ - paymentMethod: 'stripe', - token: 'tok_...', - shippingRate: rates[0].id, -}) +// Checkout is just a sentence +await shopify`checkout ${cart} with SAVE20 discount` +await shopify`complete order for sarah@example.com express shipping` + +// Or let AI handle it +await shopify`process abandoned carts from today` + .map(cart => shopify`send recovery email to ${cart.customer}`) ``` ### Orders -Full lifecycle management: - ```typescript -// Order created automatically from checkout -const order = await store.orders.get('ORD-1001') - -// Fulfill order -await order.fulfill({ - lineItems: [{ id: 'line-1', quantity: 1 }], - tracking: { - company: 'UPS', - number: '1Z999AA10123456784', - url: 'https://ups.com/track/...', - }, - notifyCustomer: true, -}) - -// Handle return -await order.return({ - lineItems: [{ id: 'line-1', quantity: 1, reason: 'Defective' }], - refund: { - amount: 299.99, - method: 'original', // or 'store_credit' - }, -}) +// Natural order management +await shopify`orders from Black Friday` +await shopify`unfulfilled orders over $500` +await shopify`Sarah Chen's order history` + +// Fulfill in bulk with pipelining +await shopify`orders ready to ship` + .map(order => shopify`fulfill ${order} with tracking`) + .map(fulfilled => shopify`notify ${fulfilled.customer}`) + +// Returns are just as simple +await shopify`return headphones from order 1001 - defective` +await shopify`refund $299 to original payment method` ``` ### Subscriptions -Recurring revenue built-in: - ```typescript -// Create subscription product -await store.products.create({ - title: 'Coffee Subscription', - type: 'Subscription', - variants: [ - { - title: '12oz Monthly', - price: 24.99, - subscription: { - interval: 'month', - intervalCount: 1, - }, - }, - { - title: '12oz Weekly', - price: 19.99, - subscription: { - interval: 'week', - intervalCount: 1, - }, - }, - ], -}) - -// Manage subscription -await store.subscriptions.pause('SUB-1001') -await store.subscriptions.changeFrequency('SUB-1001', 'week', 2) -await store.subscriptions.cancel('SUB-1001', { reason: 'Too much coffee' }) +// Manage subscriptions naturally +await shopify`pause Sarah's coffee subscription` +await shopify`change John to bi-weekly delivery` +await shopify`cancel subscription 1001 - too much coffee` + +// Subscription analytics +await shopify`churned subscribers this month` +await shopify`subscriptions renewing tomorrow` +await shopify`MRR trend last 6 months` ``` ### Inventory -Real-time stock management: - ```typescript -// Configure locations -await store.inventory.locations([ - { name: 'Main Warehouse', type: 'warehouse', address: {...} }, - { name: 'NYC Store', type: 'retail', address: {...} }, - { name: 'LA Store', type: 'retail', address: {...} }, -]) - -// Set inventory by location -await store.inventory.set({ - sku: 'AWH-001-BLK', - location: 'Main Warehouse', - quantity: 500, -}) - -// Reserve for order -await store.inventory.reserve({ - sku: 'AWH-001-BLK', - location: 'Main Warehouse', - quantity: 1, - orderId: 'ORD-1001', -}) - -// Transfer between locations -await store.inventory.transfer({ - sku: 'AWH-001-BLK', - from: 'Main Warehouse', - to: 'NYC Store', - quantity: 50, -}) +// Query inventory naturally +await shopify`headphones stock across all locations` +await shopify`what's below reorder point` +await shopify`NYC store inventory` + +// Manage stock +await shopify`add 500 black headphones to warehouse` +await shopify`transfer 50 headphones from warehouse to NYC store` +await shopify`reserve 10 units for order 1001` + +// AI inventory alerts +await shopify`SKUs that will stockout this week` + .map(sku => shopify`create PO for ${sku}`) ``` ### Discounts & Promotions -Flexible pricing rules: - ```typescript -// Percentage discount -await store.discounts.create({ - code: 'SAVE20', - type: 'percentage', - value: 20, - appliesTo: 'all', - usageLimit: 1000, - startsAt: '2025-01-01', - endsAt: '2025-01-31', -}) - -// Buy X get Y -await store.discounts.create({ - type: 'buyXgetY', - rules: { - buy: { quantity: 2, collection: 'shirts' }, - get: { quantity: 1, collection: 'shirts', discount: 100 }, - }, -}) - -// Automatic discount -await store.discounts.create({ - type: 'automatic', - name: 'Free shipping over $100', - rules: { - cartMinimum: 100, - discount: { type: 'shipping', value: 100 }, - }, -}) +// Create discounts naturally +await shopify`create SAVE20 for 20% off everything, limit 1000 uses, expires Jan 31` +await shopify`buy 2 shirts get 1 free` +await shopify`free shipping over $100` + +// Query promotions +await shopify`active discount codes` +await shopify`most used promos this quarter` +await shopify`revenue impact of Black Friday sale` ``` ### Shipping -Carrier-agnostic rate calculation: - ```typescript -// Configure shipping zones -await store.shipping.zones([ - { - name: 'US Domestic', - countries: ['US'], - rates: [ - { name: 'Standard', price: 7.99, deliveryDays: '5-7' }, - { name: 'Express', price: 14.99, deliveryDays: '2-3' }, - { name: 'Overnight', price: 29.99, deliveryDays: '1' }, - ], - }, - { - name: 'International', - countries: ['*'], - exclude: ['US'], - rates: [ - { name: 'International', price: 24.99, deliveryDays: '7-14' }, - ], - }, -]) - -// Or use carrier rates -await store.shipping.carriers({ - ups: { accountNumber: '...', enabled: true }, - fedex: { accountNumber: '...', enabled: true }, - usps: { enabled: true }, -}) +// Shipping rates in plain English +await shopify`standard shipping $7.99 for US, 5-7 days` +await shopify`express $14.99, 2-3 days` +await shopify`international $24.99, 7-14 days` + +// Track shipments +await shopify`where is order 1001` +await shopify`orders stuck in transit` +await shopify`late deliveries this week` ``` ## AI-Native Commerce @@ -370,27 +191,13 @@ This is where shopify.do transforms e-commerce. ```typescript import { mark } from 'agents.do' -// Generate product descriptions -await mark` - Write a compelling product description for these headphones: - - 30-hour battery - - Active noise cancellation - - Bluetooth 5.2 - - Memory foam ear cushions - - Make it benefit-focused, not feature-focused. - Target audience: work-from-home professionals. -` +// Generate product descriptions naturally +await mark`write description for headphones - 30hr battery, ANC, for remote workers` -// Generate entire product listing +// Generate entire listings await mark` - Create a complete product listing for a new coffee subscription: - - Origin: Ethiopia Yirgacheffe - - Roast: Medium - - Process: Washed - - Flavor notes: Blueberry, jasmine, citrus - - Include: title, description, SEO metadata, suggested pricing. + create listing for Ethiopian coffee subscription: + medium roast, washed, notes of blueberry and jasmine ` ``` @@ -399,24 +206,14 @@ await mark` ```typescript import { sally } from 'agents.do' -// Customer chatbot that actually sells -await store.ai.configure({ - assistant: sally, - capabilities: [ - 'productRecommendation', - 'sizeAdvice', - 'orderTracking', - 'returnInitiation', - 'discountApplication', - ], -}) - +// Sally handles the conversation // Customer: "I need running shoes for marathon training" -// Sally: "For marathon training, you'll want shoes with great cushioning -// and durability. Based on your previous orders (size 10, neutral -// gait), I'd recommend our CloudStride Pro at $159. It's our -// best seller for long-distance. Would you like me to add it -// to your cart with free shipping?" +// Sally: "For marathon training, you'll want shoes with great cushioning. +// Based on your previous orders (size 10), I'd recommend our +// CloudStride Pro at $159. Want me to add it to your cart?" + +await sally`help ${customer} find running shoes for marathons` +await sally`recommend products based on ${customer}'s purchase history` ``` ### AI Merchandising @@ -424,48 +221,25 @@ await store.ai.configure({ ```typescript import { priya } from 'agents.do' -// Optimize product placement -await priya` - Analyze our store performance: - 1. Which products should be featured on homepage? - 2. What collections need reordering by conversion rate? - 3. Which products are underperforming vs inventory level? - - Recommend merchandising changes to maximize revenue. -` +// Optimize your store with one line +await priya`which products should be featured on homepage` +await priya`reorder collections by conversion rate` +await priya`find underperforming products with high inventory` -// Priya analyzes and acts: -// "Recommending: -// 1. Feature 'Premium Headphones' (high margin, low visibility, good reviews) -// 2. Move 'Accessories' collection up - 3.2% conversion vs 1.8% average -// 3. Discount 'Vintage Speaker' - 180 days inventory, declining views -// -// Shall I implement these changes?" +// Chain analysis to action +await priya`products with declining views but good reviews` + .map(product => shopify`promote ${product} in email campaign`) ``` ### AI Dynamic Pricing ```typescript -import { ada } from 'shopify.do/agents' - -// Configure dynamic pricing -await store.pricing.ai({ - mode: 'optimize-margin', // or 'maximize-volume', 'competitive' - constraints: { - minMargin: 30, // Never go below 30% margin - maxDiscount: 40, // Never discount more than 40% - priceChangeFrequency: 'daily', - }, - factors: [ - 'inventoryLevel', - 'competitorPricing', - 'demandForecast', - 'customerSegment', - 'timeOfYear', - ], -}) - -// Prices adjust automatically based on: +// Pricing in plain English +await shopify`optimize prices for margin, min 30%, max 40% discount` +await shopify`price headphones competitively vs Amazon` +await shopify`increase price on low-stock items` + +// AI adjusts automatically based on: // - Stock levels (low stock = higher price) // - Competitor monitoring // - Purchase intent signals @@ -477,85 +251,49 @@ await store.pricing.ai({ ```typescript import { mark } from 'agents.do' -// Generate abandoned cart emails -await mark` - Write an abandoned cart email sequence (3 emails): - - Email 1: 1 hour after abandonment (gentle reminder) - - Email 2: 24 hours (address objections) - - Email 3: 72 hours (urgency/discount) +// Generate campaigns naturally +await mark`write abandoned cart sequence - friendly but premium voice` +await mark`win-back email for customers inactive 60 days` - Include personalization placeholders. - Our brand voice is friendly but premium. -` +// Automate with pipelining +await shopify`customers who abandoned cart today` + .map(customer => mark`send recovery email to ${customer}`) -// Auto-send personalized emails -await store.email.configure({ - automation: { - abandonedCart: { enabled: true, ai: mark }, - postPurchase: { enabled: true, ai: mark }, - winback: { enabled: true, ai: mark, inactiveDays: 60 }, - }, -}) +await shopify`VIPs who haven't ordered in 30 days` + .map(vip => mark`send personalized win-back to ${vip}`) ``` ### AI Customer Service ```typescript -// 24/7 AI support that actually resolves issues -await store.support.configure({ - ai: { - enabled: true, - agent: sally, - canPerformActions: [ - 'lookupOrder', - 'trackShipment', - 'initiateReturn', - 'applyDiscount', // Up to 15% - 'escalateToHuman', - ], - escalationRules: { - refundOver: 100, // Escalate refunds > $100 - angryCustomer: true, // Escalate heated conversations - complexIssue: true, // Escalate multi-issue tickets - }, - }, -}) +import { sally } from 'agents.do' +// Sally handles support naturally // Customer: "Where's my order? It was supposed to arrive yesterday!" -// Sally: "I see your order ORD-1001 is currently in transit with UPS. -// It looks like there was a weather delay in Memphis. Current -// estimated delivery is tomorrow by 5pm. I apologize for the -// inconvenience - would you like a 15% discount code for your -// next order?" +// Sally: "I see your order is in transit with UPS - weather delay in Memphis. +// Estimated delivery tomorrow by 5pm. Would you like a 15% discount +// code for the inconvenience?" + +await sally`help ${customer} track order 1001` +await sally`process return request for ${customer}` +await sally`resolve ${ticket} or escalate if needed` ``` ## Storefront API -Build any frontend: +Build any frontend with the same natural syntax: ```typescript -import { createStorefrontClient } from 'shopify.do/client' - -const client = createStorefrontClient({ - domain: 'shop.acme.com', - storefrontToken: 'sf_...', -}) - -// Fetch products (GraphQL-compatible) -const products = await client.products.list({ - first: 10, - filters: { collection: 'audio' }, -}) - -// Add to cart -const cart = await client.cart.create({ - lines: [{ variantId: 'gid://shopify/ProductVariant/123', quantity: 1 }], -}) - -// Checkout -const checkout = await client.checkout.create({ - cartId: cart.id, -}) +import { shopify } from 'shopify.do' + +// Fetch products naturally +const products = await shopify`audio products, first 10` +const featured = await shopify`featured collection` + +// Cart operations +await shopify`add headphones to cart` +await shopify`apply SAVE20 to cart` +await shopify`checkout cart with express shipping` ``` ### Headless Commerce @@ -564,22 +302,19 @@ Use any frontend framework: ```typescript // Next.js -// app/products/[handle]/page.tsx export default async function ProductPage({ params }) { - const product = await shopify.products.getByHandle(params.handle) + const product = await shopify`product ${params.handle}` return } // Remix -// app/routes/products.$handle.tsx export async function loader({ params }) { - const product = await shopify.products.getByHandle(params.handle) + const product = await shopify`product ${params.handle}` return json({ product }) } // Astro -// src/pages/products/[handle].astro -const product = await shopify.products.getByHandle(Astro.params.handle) +const product = await shopify`product ${Astro.params.handle}` ``` ## Architecture @@ -610,40 +345,15 @@ StoreDO (config, settings, theme) ### Edge Commerce -``` -Customer Browser Cloudflare Edge Origin - | | | - |---[Product Page]---------->| | - | [Cache HIT] | - |<--[Instant Response]-------| | - | | | - |---[Add to Cart]----------->| | - | [Edge DO] | - |<--[Updated Cart]-----------| | - | | | - |---[Checkout]-------------->| | - | [Edge DO]---[Payment]--------->| - |<--[Confirmation]-----------|<--[Webhook]------------| -``` +Product pages cached at edge. Cart and checkout run in Durable Objects. Payment processing via any provider. Sub-50ms response times globally. ### Payment Abstraction ```typescript -// Unified payment interface -interface PaymentProcessor { - createPaymentIntent(amount: number, currency: string): Promise - confirmPayment(intentId: string, token: string): Promise - refund(chargeId: string, amount?: number): Promise -} - -// Implementations for every major processor -const stripe = new StripeProcessor({ apiKey: '...' }) -const adyen = new AdyenProcessor({ apiKey: '...', merchantAccount: '...' }) -const square = new SquareProcessor({ accessToken: '...' }) -const paypal = new PayPalProcessor({ clientId: '...', clientSecret: '...' }) - -// Use any processor -await store.payments.configure({ processor: stripe }) +// Use any processor - just say it +await shopify`use Stripe for payments` +await shopify`enable PayPal checkout` +await shopify`add Apple Pay and Google Pay` ``` ## Why Open Source E-commerce? @@ -695,10 +405,7 @@ kubectl apply -f shopify-do-deployment.yaml ```typescript // Edge for storefront, origin for admin -await store.config.hybrid({ - edge: ['storefront', 'cart', 'checkout'], - origin: ['admin', 'reporting', 'inventory'], -}) +await shopify`run storefront at edge, admin at origin` ``` ## Roadmap diff --git a/rewrites/snowflake/README.md b/rewrites/snowflake/README.md index b09fc595..e1eb6177 100644 --- a/rewrites/snowflake/README.md +++ b/rewrites/snowflake/README.md @@ -35,32 +35,38 @@ Iceberg = enterprise only Iceberg-native from day one Managed service only Self-host or managed ``` -## One-Click Deploy +## AI-Native API -```bash -npx create-dotdo snowflake +```typescript +import { snowflake } from 'snowflake.do' // Full SDK +import { snowflake } from 'snowflake.do/tiny' // Minimal client +import { snowflake } from 'snowflake.do/iceberg' // Iceberg-only operations ``` -Your own data warehouse. Running on Cloudflare. Predictable costs. - -## Natural Language to SQL - -Skip the SQL. Just ask: +Natural language for data warehousing: ```typescript import { snowflake } from 'snowflake.do' -// Natural language queries -const revenue = await snowflake`what was our total revenue last quarter?` -const trends = await snowflake`show me customer churn trends by cohort` -const anomaly = await snowflake`why did costs spike on Tuesday?` +// Talk to it like a data analyst +const revenue = await snowflake`total revenue last quarter` +const trends = await snowflake`customer churn by cohort` +const anomaly = await snowflake`why did costs spike Tuesday?` -// Returns data, SQL, and narrative -console.log(revenue.data) // Query results -console.log(revenue.sql) // Generated SQL -console.log(revenue.insight) // AI-generated explanation +// Chain like sentences +await snowflake`create warehouse ANALYTICS medium auto-suspend 60s` +await snowflake`clone database PRODUCTION to DEV_TESTING` +await snowflake`stream order changes from ORDERS, merge to summary every 5 minutes` ``` +## One-Click Deploy + +```bash +npx create-dotdo snowflake +``` + +Your own data warehouse. Running on Cloudflare. Predictable costs. + ## Promise Pipelining: Data Pipelines in One Round Trip Chain transformations without waiting: @@ -110,413 +116,195 @@ const analysis = await snowflake`query customer_360 for enterprise accounts` ### Virtual Warehouses -Create isolated compute clusters that scale independently: - ```typescript -import { Warehouse } from 'snowflake.do' - -// Create warehouses for different workloads -const analytics = Warehouse.create({ - name: 'ANALYTICS_WH', - size: 'medium', - autoSuspend: 60, // Suspend after 60s idle - autoResume: true, - minCluster: 1, - maxCluster: 4, // Auto-scale to 4 clusters - scalingPolicy: 'economy', -}) - -const etl = Warehouse.create({ - name: 'ETL_WH', - size: 'xlarge', - autoSuspend: 120, - resourceMonitor: 'ETL_BUDGET', - maxCredits: 1000, -}) +// Just say it +const analytics = await snowflake`create warehouse ANALYTICS medium auto-suspend 60s` +const etl = await snowflake`create warehouse ETL xlarge with 1000 credit limit` + +// Scale like talking to an ops engineer +await snowflake`scale ANALYTICS to 4 clusters` +await snowflake`suspend ETL warehouse` +await snowflake`resume ANALYTICS when queries waiting` // Run queries on specific warehouses -await snowflake.use(analytics)`select * from sales` -await snowflake.use(etl)`copy into events from @stage` +await snowflake`on ANALYTICS: top customers by revenue last quarter` +await snowflake`on ETL: copy events from @stage` ``` ### Zero-Copy Cloning -Clone databases, schemas, or tables instantly without copying data: - ```typescript -import { Database, Schema, Table } from 'snowflake.do' - -// Clone entire database for testing -const prodClone = await Database.clone({ - source: 'PRODUCTION', - target: 'DEV_TESTING', - // Zero storage cost - shares underlying data -}) - -// Clone schema for experimentation -const schemaClone = await Schema.clone({ - source: 'PRODUCTION.ANALYTICS', - target: 'PRODUCTION.ANALYTICS_EXPERIMENT', -}) - -// Clone table at a point in time -const tableClone = await Table.clone({ - source: 'ORDERS', - target: 'ORDERS_BACKUP', - at: { timestamp: '2024-01-15 10:30:00' }, -}) +// Clone like you'd ask a DBA +await snowflake`clone database PRODUCTION to DEV_TESTING` +await snowflake`clone schema PRODUCTION.ANALYTICS for experimentation` +await snowflake`clone ORDERS table from yesterday at 10:30am` // Test against production data safely -await snowflake.use(prodClone)` - -- Safe to run destructive queries - delete from customers where status = 'test' -` +await snowflake`on DEV_TESTING: delete test customers` +// Zero storage cost - shares underlying data ``` ### Time Travel -Query data as it existed at any point in the past: - ```typescript -import { TimeTravel } from 'snowflake.do' - -// Query historical data -const yesterday = await snowflake` - select * from orders - at (timestamp => '2024-01-14 00:00:00'::timestamp) -` - -// Recover deleted data -const deleted = await snowflake` - select * from customers - before (statement => '${deleteStatementId}') -` - -// Undo a bad update -await TimeTravel.restore({ - table: 'PRODUCTS', - to: { timestamp: '2024-01-14 23:59:59' }, -}) - -// Compare data across time -const diff = await TimeTravel.diff({ - table: 'INVENTORY', - from: { offset: '-24 hours' }, - to: { timestamp: 'current' }, -}) +// Query the past naturally +const yesterday = await snowflake`orders from yesterday` +const lastWeek = await snowflake`customer count as of last Monday` +const preIncident = await snowflake`inventory before the bad update` + +// Recover and restore +await snowflake`restore PRODUCTS table to yesterday 11:59pm` +await snowflake`undo that delete on customers` + +// Compare across time +const diff = await snowflake`diff INVENTORY last 24 hours` +await snowflake`what changed in orders since Tuesday?` ``` ### Semi-Structured Data (JSON/Avro/Parquet) -Query JSON, Avro, and Parquet natively: - ```typescript -import { Variant } from 'snowflake.do' - -// Load JSON directly -await snowflake` - copy into events - from @json_stage - file_format = (type = JSON) -` - -// Query nested JSON with dot notation -const events = await snowflake` - select - raw:user.id::string as user_id, - raw:event.type::string as event_type, - raw:properties.amount::number as amount, - raw:metadata.tags[0]::string as first_tag - from events - where raw:event.timestamp::timestamp > dateadd(hour, -24, current_timestamp()) -` - -// Flatten arrays -const items = await snowflake` - select - order_id, - f.value:product_id::string as product_id, - f.value:quantity::number as quantity, - f.value:price::number as price - from orders, - lateral flatten(input => order_data:items) f -` - -// Load Parquet with schema inference -await snowflake` - create table logs using template ( - select array_agg(object_construct(*)) - from table(infer_schema( - location => '@parquet_stage', - file_format => 'parquet_format' - )) - ) -` +// Load any format - AI infers the schema +await snowflake`load events from @json_stage` +await snowflake`load logs from s3://bucket/parquet/` +await snowflake`ingest avro files from @kafka_stage` + +// Query nested data naturally +const events = await snowflake`user events last 24 hours with properties.amount > 100` +const items = await snowflake`flatten order line items with product and quantity` + +// Just ask for what you need +await snowflake`event types by user from the nested JSON` +await snowflake`extract all tags from metadata arrays` ``` ### Data Sharing & Marketplace -Share data securely without copying: - ```typescript -import { Share, DataExchange } from 'snowflake.do' - -// Create a share -const share = await Share.create({ - name: 'CUSTOMER_INSIGHTS', - database: 'ANALYTICS', - schemas: ['PUBLIC'], - tables: ['CUSTOMER_METRICS', 'PRODUCT_USAGE'], -}) - -// Add consumers -await share.addAccount('PARTNER_ACCOUNT_123') -await share.addAccount('PARTNER_ACCOUNT_456') - -// Create a listing for the marketplace -const listing = await DataExchange.createListing({ - share: 'CUSTOMER_INSIGHTS', - title: 'Real-Time Customer Analytics', - description: 'Live customer metrics updated hourly', - pricing: { type: 'free' }, // or { type: 'paid', perQuery: 0.01 } - categories: ['Analytics', 'Marketing'], -}) - -// Consume shared data (as a consumer) -const sharedDb = await DataExchange.mount({ - listing: 'provider.CUSTOMER_INSIGHTS', - database: 'PARTNER_DATA', -}) - -await snowflake`select * from PARTNER_DATA.PUBLIC.CUSTOMER_METRICS` +// Share data like sending a file +await snowflake`share CUSTOMER_METRICS and PRODUCT_USAGE with partner ABC` +await snowflake`share analytics schema with our reseller network` + +// Publish to marketplace +await snowflake`list "Real-Time Customer Analytics" for free in marketplace` +await snowflake`list weather data at $0.01 per query` + +// Consume shared data +await snowflake`mount partner.CUSTOMER_INSIGHTS as PARTNER_DATA` +const metrics = await snowflake`customer metrics from PARTNER_DATA` ``` ### Snowpipe: Continuous Data Ingestion -Stream data in real-time: - ```typescript -import { Snowpipe, Stage } from 'snowflake.do' - -// Create a stage pointing to S3 -const stage = await Stage.create({ - name: 'EVENTS_STAGE', - url: 's3://my-bucket/events/', - credentials: { - awsKeyId: env.AWS_KEY_ID, - awsSecretKey: env.AWS_SECRET_KEY, - }, - fileFormat: { type: 'JSON' }, -}) - -// Create a Snowpipe for automatic ingestion -const pipe = await Snowpipe.create({ - name: 'EVENTS_PIPE', - stage: 'EVENTS_STAGE', - table: 'RAW_EVENTS', - autoIngest: true, - // Files loaded within seconds of arrival -}) - -// Monitor pipe status -const status = await pipe.status() -console.log(status.pendingFileCount) -console.log(status.lastIngestTimestamp) - -// Manual refresh if needed -await pipe.refresh({ prefix: 'events/2024/01/' }) +// Set up streaming ingestion naturally +await snowflake`stage s3://my-bucket/events/ as EVENTS_STAGE` +await snowflake`pipe events from EVENTS_STAGE into RAW_EVENTS automatically` + +// Monitor like asking a colleague +const status = await snowflake`how many files pending in EVENTS_PIPE?` +await snowflake`when did EVENTS_PIPE last run?` + +// Manual operations when needed +await snowflake`refresh EVENTS_PIPE for January 2024 files` +await snowflake`pause the events pipe` +await snowflake`resume all pipes` ``` ### Streams & Tasks: Change Data Capture -Track changes and automate processing: - ```typescript -import { Stream, Task } from 'snowflake.do' - -// Create a stream to capture changes -const stream = await Stream.create({ - name: 'ORDERS_CHANGES', - table: 'ORDERS', - appendOnly: false, // Capture inserts, updates, and deletes -}) - -// Query the stream for changes -const changes = await snowflake` - select - METADATA$ACTION as action, - METADATA$ISUPDATE as is_update, - * - from ORDERS_CHANGES - where METADATA$ACTION = 'INSERT' -` - -// Create a task to process changes automatically -const task = await Task.create({ - name: 'PROCESS_ORDER_CHANGES', - warehouse: 'ANALYTICS_WH', - schedule: 'USING CRON 0/5 * * * * UTC', // Every 5 minutes - statement: ` - merge into ORDER_SUMMARY s - using (select * from ORDERS_CHANGES) c - on s.order_date = c.order_date - when matched then update set total = s.total + c.amount - when not matched then insert (order_date, total) values (c.order_date, c.amount) - `, - when: 'SYSTEM$STREAM_HAS_DATA(\'ORDERS_CHANGES\')', -}) - -// Task dependencies (DAG) -await Task.create({ - name: 'DOWNSTREAM_ANALYTICS', - after: ['PROCESS_ORDER_CHANGES'], // Runs after parent completes - statement: `call refresh_dashboards()`, -}) +// Track changes naturally +await snowflake`stream order changes from ORDERS` +const newOrders = await snowflake`new inserts from ORDERS_CHANGES` +const allChanges = await snowflake`what changed in ORDERS since last check?` + +// Automate processing like scheduling a meeting +await snowflake`every 5 minutes: merge order changes into ORDER_SUMMARY` +await snowflake`when orders change: update the daily totals` + +// Build task DAGs naturally +await snowflake`after order processing: refresh dashboards` +await snowflake`chain: ingest -> transform -> aggregate -> publish` + +// Monitor tasks +await snowflake`show me failed tasks today` +await snowflake`why did ORDER_SUMMARY task fail?` ``` ### Iceberg Tables -Native Apache Iceberg support for open table formats: - ```typescript -import { IcebergTable, ExternalVolume } from 'snowflake.do' - -// Create external volume pointing to your object storage -const volume = await ExternalVolume.create({ - name: 'ICEBERG_STORAGE', - storageLocations: [{ - name: 's3_location', - storageBaseUrl: 's3://my-iceberg-bucket/', - storageProvider: 'S3', - storageAwsRoleArn: env.AWS_ROLE_ARN, - }], -}) - -// Create Iceberg table (Snowflake-managed) -const table = await IcebergTable.create({ - name: 'CUSTOMER_EVENTS', - externalVolume: 'ICEBERG_STORAGE', - catalog: 'SNOWFLAKE', // Snowflake manages the catalog - baseLocation: 'customer_events/', - columns: [ - { name: 'event_id', type: 'STRING' }, - { name: 'customer_id', type: 'STRING' }, - { name: 'event_type', type: 'STRING' }, - { name: 'timestamp', type: 'TIMESTAMP' }, - { name: 'properties', type: 'OBJECT' }, - ], -}) - -// Query Iceberg table like any other table -await snowflake` - insert into CUSTOMER_EVENTS - select * from RAW_EVENTS - where event_date = current_date() -` +// Create Iceberg tables naturally +await snowflake`store CUSTOMER_EVENTS as iceberg in s3://my-iceberg-bucket/` +await snowflake`create iceberg table EVENTS with event_id, customer_id, timestamp` -// Access from external engines (Spark, Trino, etc.) -const metadata = await table.getIcebergMetadata() -console.log(metadata.metadataLocation) // s3://my-iceberg-bucket/customer_events/metadata/... - -// Read existing Iceberg tables from external catalogs -const externalTable = await IcebergTable.mount({ - name: 'EXTERNAL_EVENTS', - externalVolume: 'ICEBERG_STORAGE', - catalogIntegration: 'GLUE_CATALOG', - catalogTableName: 'analytics.events', -}) +// Query like any table +await snowflake`today's events into CUSTOMER_EVENTS` +await snowflake`customers by event count from CUSTOMER_EVENTS` + +// Open format = use anywhere +const metadata = await snowflake`iceberg metadata for CUSTOMER_EVENTS` +// Access from Spark, Trino, Presto - it's just Iceberg + +// Mount external Iceberg tables +await snowflake`mount iceberg table analytics.events from Glue catalog` +await snowflake`query the Databricks events table` ``` ## AI-Native Features ### Natural Language Queries -Ask questions in plain English: - ```typescript -import { askSnowflake } from 'snowflake.do' - -// Simple queries -const answer = await askSnowflake('How many orders did we have last month?') -// Returns: { value: 15234, sql: "SELECT COUNT(*) FROM orders WHERE...", chart: } - -// Complex analytics -const analysis = await askSnowflake(` - What's driving the increase in customer churn? - Break it down by acquisition channel and product line. -`) - -// Predictive -const forecast = await askSnowflake(` - Predict our revenue for next quarter based on historical trends - and seasonality patterns. -`) +// Just ask - it's all the same syntax +const orders = await snowflake`how many orders last month?` +const churn = await snowflake`what's driving customer churn by channel?` +const forecast = await snowflake`predict Q2 revenue from historical trends` + +// Returns data, SQL, and narrative +console.log(orders.data) // Query results +console.log(orders.sql) // Generated SQL +console.log(orders.insight) // AI-generated explanation + +// Complex analytics in plain English +await snowflake` + why did west coast revenue drop? + compare to last quarter by product line +` ``` ### AI Agents as Data Engineers -Let AI agents manage your data warehouse: - ```typescript import { priya, tom, ralph, quinn } from 'agents.do' -import { snowflake } from 'snowflake.do' - -// Product manager explores data for roadmap -const insights = await priya` - analyze our user engagement data and identify - the top 3 features driving retention -` -// Tech lead reviews data architecture -const review = await tom` - review our data warehouse schema and suggest - optimizations for our growing query load -` - -// Developer builds data pipeline -const pipeline = await ralph` - build an ETL pipeline that ingests events from Kafka, - transforms them for analytics, and loads into snowflake -` - -// QA validates data quality -const quality = await quinn` - create data quality tests for the customer_360 table - checking for nulls, duplicates, and referential integrity -` +// Agents talk to the warehouse like teammates +const insights = await priya`what features drive retention from user engagement data?` +const review = await tom`review our warehouse schema for the growing query load` +const pipeline = await ralph`build ETL from Kafka through transform to snowflake` +const quality = await quinn`test customer_360 for nulls, dupes, and broken refs` -// Chain it all together -const dataProduct = await priya`define requirements for customer analytics` +// Chain agents and data operations together +const dataProduct = await priya`define customer analytics requirements` .map(spec => tom`design the data model for ${spec}`) - .map(model => ralph`implement ${model} in snowflake.do`) - .map(impl => quinn`validate data quality of ${impl}`) + .map(model => ralph`implement ${model} in snowflake`) + .map(impl => quinn`validate ${impl}`) .map(validated => priya`create dashboard from ${validated}`) ``` ### Automated Optimization -AI-powered query and cost optimization: - ```typescript -import { optimize } from 'snowflake.do' - -// Analyze and optimize slow queries -const recommendations = await optimize.queries({ - timeRange: 'last_7_days', - minExecutionTime: '30s', -}) - -// Get warehouse sizing recommendations -const warehouseAdvice = await optimize.warehouses({ - analyzeUsage: true, - suggestConsolidation: true, -}) - -// Automatic clustering recommendations -const clusteringAdvice = await optimize.clustering({ - table: 'EVENTS', - sampleQueries: true, -}) +// Ask for optimization advice naturally +await snowflake`why are my queries slow this week?` +await snowflake`which warehouses should I consolidate?` +await snowflake`how should I cluster the EVENTS table?` + +// Or just let it optimize +await snowflake`optimize slow queries from last 7 days` +await snowflake`right-size all warehouses` +await snowflake`auto-cluster EVENTS based on query patterns` ``` ## Architecture @@ -597,32 +385,16 @@ Every capability exposed as AI tools: ```typescript import { snowflakeTools } from 'snowflake.do/mcp' -// Available tools -snowflakeTools.map(t => t.name) -// [ -// 'query', -// 'create_table', -// 'create_warehouse', -// 'clone_database', -// 'time_travel_query', -// 'create_stream', -// 'create_task', -// 'create_pipe', -// 'create_share', -// 'load_data', -// 'optimize_query', -// 'explain_plan', -// ] - -// AI can manage the entire warehouse -await invokeTool('create_table', { - name: 'EVENTS', - columns: [ - { name: 'event_id', type: 'VARCHAR' }, - { name: 'payload', type: 'VARIANT' }, - ], - clusterBy: ['event_date'], -}) +// AI agents can manage the entire warehouse through natural language +// The MCP server translates to appropriate operations + +// Tools available: +// - query, create_table, create_warehouse, clone_database +// - time_travel, stream, task, pipe, share, load_data +// - optimize, explain, monitor + +// AI just talks naturally - tools are invoked automatically +await snowflake`create EVENTS table with event_id and payload, cluster by date` ``` ## Deployment Options diff --git a/rewrites/splunk/README.md b/rewrites/splunk/README.md index 50fae957..9342c2fe 100644 --- a/rewrites/splunk/README.md +++ b/rewrites/splunk/README.md @@ -6,6 +6,35 @@ Splunk dominates enterprise log analytics and SIEM. But at $150/GB/day for cloud **splunk.do** reimagines log analytics for the AI era. Full SPL compatibility. Unlimited data. Zero per-GB pricing. +## AI-Native API + +```typescript +import { splunk } from 'splunk.do' // Full SDK +import { splunk } from 'splunk.do/tiny' // Minimal client +import { splunk } from 'splunk.do/spl' // SPL-only operations +``` + +Natural language for log analytics: + +```typescript +import { splunk } from 'splunk.do' + +// Talk to it like a colleague +const errors = await splunk`errors in checkout last hour` +const attacks = await splunk`brute force attempts today` +const anomalies = await splunk`what's unusual in network traffic?` + +// Chain like sentences +await splunk`failed logins followed by success` + .map(events => splunk`correlate with data exfiltration`) + .alert(`Potential account compromise detected`) + +// Ingestion that documents itself +await splunk`ingest nginx access_combined into main` +await splunk`stream kafka logs from events topic` +await splunk`tail /var/log/*.log as syslog` +``` + ## The Problem Cisco acquired Splunk for $28B and built a data empire on: @@ -19,30 +48,6 @@ Cisco acquired Splunk for $28B and built a data empire on: 100 GB/day? **$15,000/day**. That's **$5.5 million/year**. Just for logs. -## The workers.do Way - -Your security team is drowning. Logs are pouring in from everywhere. Every query in SPL requires a specialist. Every GB stored is another $150. And when a breach happens, you need answers in minutes, not hours of query tuning. - -What if log analysis was a conversation? - -```typescript -import { splunk, tom } from 'workers.do' - -// Natural language log analytics -const errors = await splunk`search errors from ${service} last hour` -const attack = await splunk`find brute force attempts against ${system}` -const anomaly = await splunk`what's unusual in network traffic today?` - -// Chain security investigation into response -const incident = await splunk`show failed logins followed by successful access` - .map(events => splunk`correlate with data exfiltration patterns`) - .map(findings => tom`assess severity and recommend response for ${findings}`) -``` - -One import. Natural language. AI-powered security operations. - -That's log analytics that works for you. - ## The Solution **splunk.do** is Splunk reimagined: @@ -66,47 +71,84 @@ npx create-dotdo splunk Your own Splunk instance. Running on Cloudflare. Zero per-GB fees. -## Log Analytics Without Limits +```typescript +import { Splunk } from 'splunk.do' + +export default Splunk({ + name: 'acme-soc', + domain: 'logs.acme.com', + retention: { + hot: '7d', + warm: '30d', + cold: '365d', + }, +}) +``` + +## Features -Ingest, search, and analyze unlimited log data: +### Log Ingestion ```typescript -import { splunk } from 'splunk.do' +// Ingest in plain English +await splunk`ingest nginx access_combined into main` +await splunk`stream cloudflare logs into web` +await splunk`tail /var/log/auth.log as linux:auth` -// Ingest logs -await splunk.ingest({ - source: 'nginx', - sourcetype: 'access_combined', - index: 'main', - events: [ - { _raw: '10.0.0.1 - - [01/Jan/2024:00:00:00 +0000] "GET /api/users HTTP/1.1" 200 1234' }, - ], -}) +// AI infers what you need +await splunk`ingest these events` // auto-detects format +await splunk`ingest from S3 bucket logs/` // configures input +await splunk`ingest kafka events topic` // sets up consumer +``` -// Search with SPL -const results = await splunk.search({ - query: 'index=main sourcetype=access_combined status>=500 | stats count by status', - earliest: '-24h', - latest: 'now', -}) +### Search -// Natural language search -const insights = await splunk`show me all errors in the last hour with their root causes` +```typescript +// Just ask +await splunk`errors in checkout last hour` +await splunk`500 errors by endpoint today` +await splunk`slow queries over 500ms` + +// AI generates SPL underneath +await splunk`errors in checkout` // -> index=main level=error source=checkout earliest=-1h +await splunk`top error sources` // -> index=main level=error | stats count by source | sort -count ``` -## Features +### Alerting -### SPL (Search Processing Language) +```typescript +// Natural as talking to a SOC analyst +await splunk`alert me when errors spike in production` +await splunk`page on-call if auth failures exceed 100/min` +await splunk`slack #security on brute force detection` -Full SPL compatibility: +// Chain alerts into investigation +await splunk`failed logins > 10 per IP` + .alert(`Brute force detected`) + .block() // auto-block the IP +``` -```spl -// Basic search -index=main sourcetype=access_combined status>=400 +### SPL Compatibility -// Field extraction -index=main | rex field=_raw "user=(?\w+)" +Full SPL underneath - natural language generates it: +```typescript +// Your query +await splunk`errors in checkout last hour` + +// Generated SPL +// index=main level=error source=checkout earliest=-1h + +// Your query +await splunk`top 10 slowest endpoints today` + +// Generated SPL +// index=main | stats avg(response_time) as avg_time by endpoint | sort -avg_time | head 10 +``` + +All SPL commands work: + +```spl // Statistics index=main | stats count, avg(response_time) by endpoint @@ -116,666 +158,455 @@ index=main | timechart span=1h count by status // Transactions index=main | transaction session_id maxspan=30m -// Lookups -index=main | lookup users.csv user_id OUTPUT user_name, department - -// Subsearches -index=main [search index=alerts level=critical | fields src_ip] - -// Eval expressions -index=main | eval response_category=case( - status<300, "success", - status<400, "redirect", - status<500, "client_error", - true(), "server_error" -) - -// Join -index=main | join user_id [search index=users | fields user_id, email] - -// Machine learning (MLTK equivalent) +// Machine learning index=main | fit DensityFunction response_time by endpoint ``` ### Indexing -High-performance log indexing: - ```typescript -import { Index } from 'splunk.do/indexing' - -// Create index with configuration -const mainIndex = Index({ - name: 'main', - maxDataSize: 'auto', - frozenTimePeriod: '365d', - homePath: 'r2://indexes/main/hot', - coldPath: 'r2://indexes/main/cold', - frozenPath: 'r2://indexes/main/frozen', -}) +// Create indexes naturally +await splunk`create index main frozen in 365d` +await splunk`create index security retention 90d` +await splunk`create index metrics with rollups hourly daily` // Configure field extractions -mainIndex.props({ - TRANSFORMS: [ - { regex: /user_id=(\w+)/, field: 'user_id' }, - { regex: /duration=(\d+)ms/, field: 'duration_ms', type: 'number' }, - ], - TIME_FORMAT: '%Y-%m-%dT%H:%M:%S.%f%z', - TZ: 'UTC', -}) +await splunk`extract user_id from pattern user=(\w+)` +await splunk`extract duration_ms as number from duration=(\d+)ms` -// Source types -const nginxSourcetype = Sourcetype({ - name: 'nginx:access', - EXTRACT: [ - { regex: /"(?\w+)\s+(?[^\s]+)\s+(?[^"]+)"/, format: 'fields' }, - { regex: /(?\d{3})/, format: 'number' }, - ], - TIME_PREFIX: '\\[', - TIME_FORMAT: '%d/%b/%Y:%H:%M:%S %z', -}) +// Source types are inferred +await splunk`ingest nginx` // knows nginx format +await splunk`ingest json` // knows JSON format +await splunk`ingest syslog` // knows syslog format ``` ### Data Inputs -Multiple ingestion methods: - ```typescript -import { Inputs } from 'splunk.do/inputs' - -// HTTP Event Collector (HEC) -const hec = Inputs.hec({ - token: 'your-token', - ssl: true, - acknowledgement: true, -}) - -// Send events via HEC -await hec.send({ - event: { message: 'User logged in', user: 'alice' }, - sourcetype: 'app:auth', - index: 'main', -}) +// Just say where logs come from +await splunk`collect from HEC token abc123` +await splunk`collect from syslog port 514` +await splunk`collect from kafka brokers kafka:9092 topic logs` +await splunk`collect from S3 bucket my-logs prefix logs/` +await splunk`collect from cloudwatch log-group /aws/lambda/my-function` -// Syslog -const syslog = Inputs.syslog({ - port: 514, - protocol: 'tcp', - sourcetype: 'syslog', -}) - -// File monitoring (via forwarder) -const files = Inputs.monitor({ - path: '/var/log/*.log', - sourcetype: 'linux:syslog', - recursive: true, -}) - -// Kafka integration -const kafka = Inputs.kafka({ - brokers: ['kafka:9092'], - topics: ['logs', 'events'], - consumerGroup: 'splunk-consumer', -}) - -// AWS (S3, CloudWatch, etc.) -const s3 = Inputs.s3({ - bucket: 'my-logs-bucket', - region: 'us-east-1', - prefix: 'logs/', - sourcetype: 'aws:s3:accesslogs', -}) +// Forwarders +await splunk`forward /var/log/*.log to main` +await splunk`forward docker containers to containers index` ``` ### Dashboards -Build operational dashboards: - ```typescript -import { Dashboard, Panel } from 'splunk.do/dashboard' - -const securityDashboard = Dashboard({ - title: 'Security Operations Center', - refresh: '5m', - panels: [ - Panel.singleValue({ - title: 'Critical Alerts', - search: 'index=alerts level=critical | stats count', - colorBy: 'value', - thresholds: { warn: 10, critical: 50 }, - }), - Panel.timechart({ - title: 'Security Events Over Time', - search: 'index=security | timechart span=1h count by event_type', - stackMode: 'stacked', - }), - Panel.map({ - title: 'Attack Sources', - search: 'index=security event_type=attack | iplocation src_ip | geostats count by Country', - }), - Panel.table({ - title: 'Recent Alerts', - search: 'index=alerts | head 100 | table _time, level, message, src_ip', - drilldown: { - search: 'index=alerts src_ip=$row.src_ip$', - }, - }), - Panel.choropleth({ - title: 'Events by Country', - search: 'index=security | iplocation src_ip | stats count by Country', - }), - ], - inputs: [ - { type: 'time', token: 'time', default: '-24h' }, - { type: 'dropdown', token: 'severity', options: ['critical', 'high', 'medium', 'low'] }, - ], -}) +// Describe what you want to see +await splunk`dashboard security ops` + .add(`critical alerts count`) + .add(`security events over time`) + .add(`attack sources on map`) + .add(`recent alerts table`) + .refresh(`5m`) + +// Or one-liners +await splunk`dashboard api health: errors, latency p99, throughput` +await splunk`dashboard user activity: logins by hour, active users, failed attempts` ``` ### Alerts -Proactive monitoring: - ```typescript -import { Alert } from 'splunk.do/alerts' - -// Scheduled alert -const errorSpike = Alert({ - name: 'Error Rate Spike', - search: 'index=main level=error | stats count | where count > 100', - schedule: '*/5 * * * *', - trigger: { - condition: 'number of results > 0', - throttle: '10m', - }, - actions: [ - { type: 'email', to: 'ops@company.com' }, - { type: 'slack', channel: '#alerts' }, - { type: 'webhook', url: 'https://api.pagerduty.com/events' }, - ], -}) +// Natural alerting +await splunk`alert errors > 100 in 5 minutes email ops@company.com` +await splunk`alert brute force detected slack #security` +await splunk`alert payment failures page on-call` -// Real-time alert -const securityAlert = Alert({ - name: 'Brute Force Detection', - search: 'index=auth event_type=login_failed | stats count by src_ip | where count > 10', - type: 'realtime', - window: '5m', - trigger: { - condition: 'number of results > 0', - }, - actions: [ - { type: 'notable', severity: 'high' }, - { type: 'risk', score: 50, object: 'src_ip' }, - ], -}) +// Chain detection into response +await splunk`failed logins > 10 from same IP` + .alert(`Brute force detected`) + .block() // auto-block at firewall + .notify() // tell security team -// Correlation search -const correlationAlert = Alert({ - name: 'Multi-Stage Attack', - search: ` - | from datamodel:Network_Traffic - | tstats count from datamodel:Authentication where Authentication.action=failure - | join src_ip [| from datamodel:Malware] - `, - trigger: { - condition: 'number of results > 0', - }, -}) +// Correlation as a sentence +await splunk`alert failed login then successful from different IP` + .severity(`high`) + .investigate() // open incident automatically ``` ### Reports -Scheduled reports and exports: - ```typescript -import { Report } from 'splunk.do/reports' - -const weeklyReport = Report({ - name: 'Weekly Security Summary', - search: ` - index=security earliest=-7d - | stats count by event_type, severity - | sort -count - `, - schedule: '0 9 * * MON', - export: { - format: 'pdf', - recipients: ['security-team@company.com'], - subject: 'Weekly Security Report - ${now}', - }, -}) +// Scheduled reports in plain English +await splunk`weekly security summary email security-team@company.com` +await splunk`daily error report to #engineering at 9am` +await splunk`monthly compliance export to S3 bucket reports/` -// Pivot reports -const pivotReport = Report({ - name: 'Traffic Analysis', - datamodel: 'Network_Traffic', - pivot: { - rows: ['src_ip', 'dest_port'], - columns: ['action'], - values: [{ field: 'bytes', aggregation: 'sum' }], - }, -}) +// Ad-hoc exports +await splunk`export last week errors as csv` +await splunk`export incidents since January as pdf` ``` ## SIEM & Security ### Enterprise Security -Full SIEM capabilities: - ```typescript -import { ES } from 'splunk.do/security' - -// Notable events -const notable = ES.notable({ - search: 'index=proxy dest_category=malware', - severity: 'critical', - rule_name: 'Malware Communication Detected', - drilldown: 'index=proxy src_ip=$src_ip$ earliest=-24h', -}) +// Notable events - just describe the threat +await splunk`detect malware communication as critical` +await splunk`detect lateral movement as high severity` +await splunk`detect data exfiltration pattern` -// Risk scoring -const riskRule = ES.risk({ - search: 'index=auth event_type=login_failed | stats count by user | where count > 5', - risk_object_field: 'user', - risk_object_type: 'user', - risk_score: 20, - risk_message: 'Multiple failed logins for user $user$', -}) +// Risk scoring in plain English +await splunk`add risk 20 to users with 5+ failed logins` +await splunk`add risk 50 to IPs hitting honeypots` +await splunk`add risk 100 to known bad actors` // Threat intelligence -const threatIntel = ES.threatIntel({ - search: 'index=firewall | lookup threat_intel.csv dest_ip OUTPUT threat_type, confidence', - feed: 'custom_threat_feed', -}) +await splunk`enrich firewall logs with threat intel` +await splunk`match traffic against IOC feeds` +await splunk`flag connections to known C2 servers` -// Investigation workbench -const investigation = await ES.investigate({ - startingPoint: { type: 'ip', value: '10.0.0.1' }, - timeRange: '-7d', - expand: ['dns', 'auth', 'proxy', 'firewall'], -}) +// Investigation +await splunk`investigate IP 10.0.0.1 last 7 days` + .expand(`dns, auth, proxy, firewall`) + .timeline() + .graph() ``` -### User Behavior Analytics (UBA) - -Detect anomalous behavior: +### User Behavior Analytics ```typescript -import { UBA } from 'splunk.do/uba' - -// Baseline user behavior -await UBA.baseline({ - entity: 'user', - activities: ['login', 'file_access', 'email_send', 'web_browse'], - timeframe: '30d', -}) +// Baseline in one line +await splunk`baseline user behavior over 30 days` -// Detect anomalies -const anomalies = await UBA.detect({ - entity: 'user', - sensitivity: 'medium', - timeRange: '-24h', -}) +// Detect anomalies naturally +await splunk`anomalous user activity today` +await splunk`users behaving unusually this week` +await splunk`alice@company.com doing anything weird?` -for (const anomaly of anomalies) { - console.log(anomaly.entity) // 'alice@company.com' - console.log(anomaly.activity) // 'file_access' - console.log(anomaly.deviation) // 3.5 standard deviations - console.log(anomaly.explanation) // 'Accessed 10x more files than usual' -} - -// Threat models -const insiderThreat = UBA.threatModel({ - name: 'Data Exfiltration', - indicators: [ - { activity: 'file_download', threshold: '5x baseline' }, - { activity: 'usb_insert', weight: 2 }, - { activity: 'after_hours_access', weight: 1.5 }, - ], - threshold: 75, -}) +// Threat models as sentences +await splunk`detect data exfiltration: downloads 5x normal, USB usage, after hours access` +await splunk`detect account takeover: login from new location then password change` +await splunk`detect insider threat: bulk file access before resignation` ``` -### SOAR Integration - -Security orchestration and response: +### SOAR ```typescript -import { SOAR } from 'splunk.do/soar' - -// Playbook -const phishingPlaybook = SOAR.playbook({ - name: 'Phishing Response', - trigger: { - type: 'notable', - rule: 'Phishing Email Detected', - }, - steps: [ - { - name: 'Extract Indicators', - action: 'extract_iocs', - input: { email: '$notable.email$' }, - }, - { - name: 'Check Reputation', - action: 'virustotal_lookup', - input: { url: '$indicators.urls$' }, - }, - { - name: 'Block Sender', - action: 'block_email_sender', - condition: '$reputation.malicious$', - input: { sender: '$notable.sender$' }, - }, - { - name: 'Notify SOC', - action: 'create_ticket', - input: { - title: 'Phishing Incident - $notable.subject$', - severity: '$notable.severity$', - }, - }, - ], -}) - -// Custom actions -SOAR.action({ - name: 'isolate_host', - parameters: ['hostname'], - script: async ({ hostname }) => { - await crowdstrike.containHost(hostname) - return { status: 'isolated', hostname } - }, -}) +// Playbooks as workflows +await splunk`on phishing detected` + .extract(`IOCs from email`) + .check(`reputation via VirusTotal`) + .block(`sender if malicious`) + .notify(`SOC with ticket`) + +// Response actions in natural language +await splunk`isolate host infected-laptop via CrowdStrike` +await splunk`block IP 1.2.3.4 at firewall` +await splunk`disable user alice@company.com` +await splunk`quarantine email from attacker@evil.com` + +// Chain detection to response +await splunk`ransomware indicators detected` + .isolate() // contain the host + .snapshot() // preserve evidence + .escalate() // page security team ``` ## AI-Native Features ### Natural Language Search -Skip SPL complexity: - ```typescript -import { ask } from 'splunk.do' +// Just ask - no SPL required +await splunk`errors in the last hour` +await splunk`failed logins followed by success from different IP` +await splunk`top sources of 500 errors today` +await splunk`why is the API slow right now?` -// Simple questions -const q1 = await ask('show me all errors in the last hour') -// Generates: index=main level=error earliest=-1h +// AI infers the right query +await splunk`errors in checkout` +// -> index=main level=error source=checkout -// Complex queries -const q2 = await ask('find all failed logins followed by successful login from different IP') -// Generates: complex transaction/correlation SPL - -// Analytical questions -const q3 = await ask('what are the top sources of 500 errors today?') -// Generates: index=main status=500 | stats count by source | sort -count | head 10 - -// Diagnostic questions -const q4 = await ask('why is the API slow right now?') -// Returns: analysis with correlated logs, traces, and metrics +await splunk`users logging in from multiple countries` +// -> index=auth | iplocation src_ip | stats dc(Country) by user | where dc > 1 ``` ### SPL Assistant -AI helps write and optimize SPL: - ```typescript -import { spl } from 'splunk.do' - -// Generate SPL from description -const query = await spl.generate({ - description: 'Find users who logged in from multiple countries in the last 24 hours', - index: 'auth', -}) -// Returns: index=auth event_type=login | iplocation src_ip | stats dc(Country) as countries values(Country) by user | where countries > 1 +// Explain SPL in plain English +await splunk`explain: index=main | tstats count by _time span=1h | timewrap 1d` -// Optimize existing SPL -const optimized = await spl.optimize({ - query: 'index=main | search status=500 | stats count by endpoint', -}) -// Returns: index=main status=500 | stats count by endpoint -// Explanation: "Moved status filter before search command for better performance" +// Optimize SPL automatically +await splunk`optimize: index=main | search status=500 | stats count` +// Returns: index=main status=500 | stats count +// "Moved filter before search for 10x speedup" -// Explain SPL -const explanation = await spl.explain({ - query: 'index=main | tstats count where index=main by _time span=1h | timewrap 1d', -}) -// Returns: detailed explanation of each command +// Generate SPL from description +await splunk`SPL for: users with login from new device after password reset` ``` ### Anomaly Detection -AI-powered anomaly detection: - ```typescript -import { ML } from 'splunk.do/ml' +// Detect anomalies naturally +await splunk`anomalies in API latency this week` +await splunk`unusual traffic patterns today` +await splunk`outliers in request volume by IP` -// Anomaly detection -const anomalies = await ML.detectAnomalies({ - search: 'index=main | timechart span=1h count by endpoint', - sensitivity: 0.8, - timeRange: '-7d', -}) - -// Forecasting -const forecast = await ML.forecast({ - search: 'index=main | timechart span=1h count', - horizon: '24h', - confidence: 0.95, -}) - -// Clustering -const clusters = await ML.cluster({ - search: 'index=main level=error | table message', - algorithm: 'kmeans', - k: 10, -}) -// Groups similar error messages together +// Forecast +await splunk`predict error rate next 24 hours` +await splunk`forecast disk usage next week` -// Outlier detection -const outliers = await ML.outliers({ - search: 'index=main | stats count by src_ip', - field: 'count', - method: 'mad', // median absolute deviation -}) +// Cluster similar events +await splunk`group similar error messages` +await splunk`cluster security events by pattern` ``` ### AI Agents for Security -AI agents for SOC operations: - ```typescript import { tom, quinn } from 'agents.do' import { splunk } from 'splunk.do' -// Tech lead investigates incident -const investigation = await tom` - investigate the security incident from alert ID 12345 - correlate all related events and identify attack timeline -` +// Chain investigation with agents +await splunk`security incident alert 12345` + .map(events => tom`correlate and build attack timeline`) + .map(timeline => tom`identify root cause and impact`) + .map(analysis => quinn`validate findings and check for gaps`) -// QA validates detection rules -const validation = await quinn` - test the brute force detection rule against last month's data - identify false positives and suggest tuning -` +// Tune detection rules +await splunk`brute force detections last month` + .map(detections => quinn`identify false positives`) + .map(fps => tom`suggest rule improvements`) ``` ## Architecture -### Indexing Pipeline - -``` -Log Sources --> Universal Forwarder --> Heavy Forwarder --> Indexer - | | | - Raw Data Parsing/Routing Indexing - Compression Field Extraction Storage -``` - ### Durable Objects ``` - +------------------------+ - | splunk.do Worker | - | (API + Search Head) | - +------------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | IndexerDO | | SearchHeadDO | | ForwarderDO | - | (Index + Search) | | (Distributed) | | (Collection) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +--------------------+--------------------+ - | | | - +----------+ +-----------+ +------------+ - | D1 | | R2 | | Analytics | - | (Meta) | | (Buckets) | | Engine | - +----------+ +-----------+ +------------+ + SplunkDO (config, users, roles, inputs) + | + +-- IndexerDO (hot storage, real-time search) + | |-- SQLite: Hot buckets (encrypted) + | +-- R2: Warm/Cold buckets (compressed) + | + +-- SearchHeadDO (distributed search, federation) + | |-- SQLite: Job state, saved searches + | +-- Query optimization + | + +-- ForwarderDO (collection, parsing) + | |-- SQLite: Input state, checkpoints + | +-- R2: Raw data staging + | + +-- AlertDO (detection, response) + |-- SQLite: Alert state, throttling + +-- Webhook integration ``` ### Storage Tiers -Splunk's bucket-based storage model: - -``` -Hot Buckets (SQLite) Warm Buckets (R2) Cold Buckets (R2 IA) - | | | -Real-time search Recent historical Long-term archive -High write rate Read-optimized Cost-optimized -Edge locations Regional R2 Archive R2 -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Real-time, last 7 days | <10ms | +| **Warm** | R2 Standard | Historical, 30-90 days | <100ms | +| **Cold** | R2 IA | Compliance, 1+ years | <1s | ### Distributed Search +Queries automatically fan out and merge: + ```typescript -// Search head distributes to indexers -SearchHead - | - +-- IndexerDO[0] (US-East) - +-- IndexerDO[1] (US-West) - +-- IndexerDO[2] (EU-West) - | -Merge Results - | -Return to Client +// Your query +await splunk`errors in checkout last week` + +// Under the hood +// SearchHead -> IndexerDO[US-East, US-West, EU-West] +// Parallel execution, results merge, dedupe, sort ``` ## Data Onboarding -### Universal Forwarder - ```bash # Install forwarder curl -sL https://splunk.do/forwarder | bash -# Configure outputs -splunk set deploy-poll your-org.splunk.do:8089 +# One-line setup +splunk forward /var/log/*.log to main at logs.acme.com +``` -# Add inputs -splunk add monitor /var/log/syslog -sourcetype syslog -splunk add monitor /var/log/nginx/*.log -sourcetype nginx +```typescript +// Or configure in code +await splunk`forward syslog port 514 to syslog index` +await splunk`forward kafka events topic to main` +await splunk`forward S3 bucket logs/ to archive` ``` -### Heavy Forwarder +## Migration from Splunk -For parsing and routing: +### One-Line Migration -```typescript -// Heavy forwarder configuration -HeavyForwarder({ - inputs: [ - { type: 'tcp', port: 514, sourcetype: 'syslog' }, - { type: 'udp', port: 514, sourcetype: 'syslog' }, - ], - outputs: [ - { server: 'indexer1.splunk.do:9997' }, - { server: 'indexer2.splunk.do:9997' }, - ], - routing: { - 'sourcetype=syslog': 'index=syslog', - 'sourcetype=nginx': 'index=web', - 'default': 'index=main', - }, -}) +```bash +npx splunk-migrate from splunk.company.com ``` -## Migration from Splunk +Migrates: +- Saved searches +- Dashboards +- Alerts +- Inputs configuration +- Field extractions ### SPL Compatibility -Full SPL command support: +Full SPL command support. All existing queries work unchanged: + +| Category | Commands | +|----------|----------| +| **Search** | search, where, regex, rex | +| **Stats** | stats, eventstats, streamstats, chart, timechart | +| **Transform** | top, rare, contingency, cluster | +| **Report** | table, fields, rename, convert | +| **Join** | join, append, union, multisearch | +| **ML** | anomalydetection, predict, fit | + +### API Compatibility + +All Splunk REST endpoints supported: ``` -Category Commands ------------------------------------------------------------------ -Search search, where, regex, rex -Stats stats, eventstats, streamstats, chart, timechart -Transforming top, rare, contingency, cluster -Reporting table, fields, rename, convert -Join join, append, union, multisearch -Transaction transaction, associate -Machine Learning anomalydetection, predict, fit +POST /services/search/jobs +GET /services/search/jobs/{id} +POST /services/receivers/simple +POST /services/collector/event (HEC) +GET /services/data/indexes ``` -### Configuration Migration +## vs Splunk Enterprise / Splunk Cloud -```bash -# Export Splunk configuration -splunk export server-config --output ./config +| Feature | Splunk | splunk.do | +|---------|--------|-----------| +| **Pricing** | $150/GB/day | $0 - run your own | +| **Implementation** | Weeks to months | Deploy in minutes | +| **Annual Cost** | $5M+ at scale | ~$100/month | +| **Architecture** | On-prem/Cloud | Edge-native, global | +| **Query Language** | SPL only | SPL + natural language | +| **AI** | Splunk AI Assistant | AI-first design | +| **SIEM** | Enterprise Security ($$$) | Included | +| **SOAR** | Splunk SOAR ($$$) | Included | +| **UBA** | Splunk UBA ($$$) | Included | +| **Lock-in** | Cisco lock-in | MIT licensed | -# Import to splunk.do -npx splunk-migrate import ./config +## Use Cases -# Migrate saved searches -npx splunk-migrate searches --source splunk.company.com +### Security Operations -# Migrate dashboards -npx splunk-migrate dashboards --source splunk.company.com +```typescript +// SOC in a few lines +await splunk`detect threats across all sources` +await splunk`alert on brute force, malware, exfiltration` +await splunk`investigate incidents automatically` + .correlate() + .timeline() + .respond() ``` -### API Compatibility +### DevOps Observability +```typescript +// Full observability +await splunk`errors in production with stack traces` +await splunk`latency p99 by service` +await splunk`deployment impact on error rate` ``` -Endpoint Status ------------------------------------------------------------------ -POST /services/search/jobs Supported -GET /services/search/jobs/{id} Supported -POST /services/receivers/simple Supported -POST /services/collector/event Supported (HEC) -GET /services/data/indexes Supported + +### Compliance + +```typescript +// Audit and retain +await splunk`retain security logs 7 years` +await splunk`export SOC 2 evidence for Q4` +await splunk`PCI audit trail for card transactions` ``` ## Roadmap +### Core Platform - [x] SPL search engine -- [x] Indexing and storage +- [x] Indexing and storage (hot/warm/cold) - [x] Dashboards and visualizations -- [x] Alerting +- [x] Alerting (scheduled + real-time) - [x] Natural language search - [x] HEC data collection -- [ ] Enterprise Security (ES) -- [ ] SOAR integration -- [ ] User Behavior Analytics -- [ ] ITSI -- [ ] Data models -- [ ] Pivot +- [ ] Metrics store +- [ ] Trace ingestion + +### Security +- [x] Enterprise Security (ES) +- [x] SOAR integration +- [x] User Behavior Analytics +- [ ] Threat intelligence platform +- [ ] Automated response actions +- [ ] Compliance reporting + +### AI +- [x] Natural language queries +- [x] Anomaly detection +- [x] Agent integration +- [ ] Predictive alerting +- [ ] Automated investigation +- [ ] Root cause analysis ## Why Open Source? -Log analytics shouldn't cost $5M/year: +### 1. Cost Liberation + +$150/GB/day is a tax on observability. Open source means: +- Zero per-GB fees +- No capacity licensing +- No add-on pricing +- Run on your infrastructure + +### 2. No Lock-in -1. **Your logs** - Machine data is critical operational intelligence -2. **Your searches** - SPL knowledge shouldn't be vendor-locked -3. **Your security** - SIEM capabilities shouldn't require enterprise pricing -4. **Your retention** - Historical data shouldn't cost per-GB forever +Your logs, your SPL, your dashboards. Open source enables: +- Export everything, anytime +- Switch providers freely +- Customize without consultants +- Integrate with anything -Splunk showed the world what log analytics could be. **splunk.do** makes it accessible to everyone. +### 3. AI Enablement + +Closed platforms control your AI options. Open source means: +- Integrate any LLM +- Build custom detection +- Train on your data +- Natural language everything + +### 4. Community Innovation + +Security moves faster than vendor roadmaps. Open source enables: +- Detection rules from the community +- Shared threat intelligence +- Faster vulnerability response +- Collective defense + +## Contributing + +splunk.do is open source under the MIT license. + +We especially welcome contributions from: +- Security engineers and SOC analysts +- DevOps and SRE teams +- Detection engineers +- Data engineers + +```bash +git clone https://github.com/dotdo/splunk.do +cd splunk.do +pnpm install +pnpm test +``` ## License @@ -784,7 +615,12 @@ MIT License - Ingest everything. Search anything. Detect all threats. ---

- splunk.do is part of the dotdo platform. + The $28B acquisition ends here.
- Website | Docs | Discord + SPL-compatible. AI-native. Zero per-GB fees. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/studio/README.md b/rewrites/studio/README.md index 778f72ce..3677dcf9 100644 --- a/rewrites/studio/README.md +++ b/rewrites/studio/README.md @@ -1,89 +1,207 @@ # studio.do -Database Studio on Cloudflare Durable Objects - A database management UI for every AI agent. +> Database Studio. Edge-Native. AI-First. Every Agent Gets Their Own. -## The workers.do Way +Drizzle Studio is a centralized dashboard. TablePlus is a desktop app. pgAdmin is from the 2000s. They're all built for humans clicking through GUIs. But when Tom needs to understand why queries are slow, or Ralph needs to implement a migration, they shouldn't navigate interfaces designed for human fingers. -Your AI agents need to see the data. Existing tools are built for humans at dashboards. But when Tom needs to understand why queries are slow, or Ralph needs to implement a migration, they shouldn't have to navigate a GUI designed for human fingers. +**studio.do** gives every AI agent their own Database Studio. Persistent state. Query history. Schema introspection. All through natural language. -**Natural Language First** +## AI-Native API + +```typescript +import { studio } from 'studio.do' // Full SDK +import { studio } from 'studio.do/tiny' // Minimal client +import { studio } from 'studio.do/mcp' // MCP tools only +``` + +Natural language for database operations: ```typescript -import { tom, ralph } from 'agents.do' import { studio } from 'studio.do' -// Tagged template literals - talk to your agents -const schema = await tom`show me the database schema` -const slow = await tom`find slow queries from today` -const migration = await studio`generate migration for ${changes}` +// Talk to it like a colleague +const schema = await studio`schema for users` +const slow = await studio`slow queries from today` +const admins = await studio`active admins created this month in users` + +// Chain like sentences +await studio`tables with no indexes` + .map(table => studio`add index suggestions for ${table}`) + +// Connections are one line +await studio`connect to Turso at libsql://my-db.turso.io` +await studio`connect to D1 database MAIN_DB` +await studio`connect to Supabase project xyz` ``` -**Promise Pipelining** +## The Problem -Chain operations without `Promise.all`. One network round trip: +Drizzle Studio dominates database tooling: + +| What Drizzle Studio Requires | The Reality | +|------------------------------|-------------| +| **Config Files** | `drizzle.config.ts` required | +| **Centralized** | Single server, not edge-native | +| **Proprietary** | Free tier, paid cloud | +| **Human-Only** | No MCP/AI integration | +| **Shared State** | Everyone sees the same thing | + +### The Database Tax + +Every database interaction requires: +- Opening a GUI application +- Navigating menus and buttons +- Writing SQL in text boxes +- Copy-pasting results + +AI agents can't do any of this. They need programmatic access. + +### The Isolation Problem + +When Tom analyzes slow queries, he shouldn't see Ralph's query history. When Ralph runs migrations, he shouldn't affect Tom's connections. Shared dashboards break isolation. + +## The Solution + +**studio.do** reimagines database tooling for AI: + +``` +Drizzle Studio studio.do +----------------------------------------------------------------- +Config files required Runtime introspection +Centralized server Durable Object per agent +Proprietary cloud Open source (MIT) +Human GUI Natural language API +Shared dashboard Isolated per agent +No MCP MCP tools native +``` + +## One-Click Deploy + +```bash +npx create-dotdo studio +``` + +A Database Studio for every AI agent. Running on infrastructure you control. ```typescript -const optimized = await studio`introspect ${database}` - .map(table => tom`analyze ${table} for optimization`) - .map(suggestion => ralph`implement ${suggestion}`) +import { Studio } from 'studio.do' + +export default Studio({ + name: 'my-studio', + domain: 'db.my-app.com', +}) ``` -**Migrations and Schema Management** +## Features + +### Schema Introspection ```typescript -// Migrations -await studio.migrate.generate() // Generate from schema diff -await studio.migrate.push() // Apply pending migrations -await studio.migrate.status() // Check migration state - -// Schema management -await studio.schema.diff(a, b) // Compare schemas -await studio.schema.snapshot() // Save current state +// Natural as asking a colleague +await studio`schema` // all tables +await studio`schema for users` // single table +await studio`columns in orders` // column definitions +await studio`indexes on products` // index definitions +await studio`foreign keys for invoices` // relationships + +// AI infers what you need +await studio`users` // returns schema +await studio`users data` // returns rows +await studio`users structure` // returns columns ``` -## The Problem +### Browsing Data + +```typescript +// Just describe what you want +await studio`active admins in users` +await studio`orders from last week` +await studio`products under $50` + +// Sorting and limiting read naturally +await studio`top 10 customers by revenue` +await studio`oldest users created before 2020` +await studio`recent errors limit 100` +``` -AI agents need to inspect and manage databases. But existing tools like Drizzle Studio are: -- Centralized (not edge-native) -- Proprietary (not open source) -- Config-driven (require `drizzle.config.ts`) -- Human-focused (no MCP/AI integration) +### Executing Queries -## The Vision +```typescript +// Raw SQL when you need it +await studio`run SELECT * FROM users WHERE id = 123` +await studio`execute DROP TABLE temp_data` + +// Query history +await studio`my recent queries` +await studio`slow queries from today` +await studio`failed queries this week` +``` -Every AI agent gets their own Database Studio instance. +### Connections ```typescript -import { tom, ralph } from 'agents.do' -import { Turso } from 'turso.do' -import { Studio } from 'studio.do' +// Connect with one line +await studio`connect to Turso at libsql://my-db.turso.io` +await studio`connect to D1 database MAIN_DB` +await studio`connect to Supabase project xyz` +await studio`connect to Postgres at postgres://localhost/mydb` + +// Manage connections +await studio`list connections` +await studio`disconnect from Turso` +await studio`switch to production` +``` -// Each agent has their own database -const tomDb = Turso.for(tom) -const ralphDb = Turso.for(ralph) +### Migrations + +```typescript +// Schema changes are sentences +await studio`add email column to users` +await studio`create index on orders.customer_id` +await studio`rename column username to name in users` + +// Migration management +await studio`pending migrations` +await studio`apply migrations` +await studio`rollback last migration` + +// Generate from diff +await studio`generate migration from schema changes` +``` -// And their own Studio to manage it -const tomStudio = Studio.for(tom) -await tomStudio.connect(tomDb) +### Schema Visualization -// AI can now introspect and modify data structures -await tom`check my database schema and suggest optimizations` -// Uses MCP tools: studio_introspect, studio_query +```typescript +// ER diagrams on demand +await studio`diagram for orders` +await studio`visualize relationships` +await studio`entity diagram for billing tables` ``` -Not a shared admin panel. Each agent has isolated, persistent Studio state. +### Promise Pipelining -## Features +Chain operations without `Promise.all`. One network round trip: -- **Runtime Introspection** - No config files needed, introspects live databases -- **Multi-Database Support** - Turso, Supabase.do, Cloudflare D1, any libSQL -- **MCP Tools** - AI-native database operations via Model Context Protocol -- **Per-Agent Isolation** - Each agent/tenant gets their own Studio instance -- **Query History** - Stored in DO SQLite, persists across sessions -- **Schema Visualization** - ER diagrams via Mermaid +```typescript +// Analyze and optimize +await studio`tables with no indexes` + .map(table => studio`suggest indexes for ${table}`) + .map(suggestion => studio`apply ${suggestion}`) + +// Bulk introspection +await studio`all tables` + .map(table => studio`analyze ${table} for optimization`) + +// Migration pipeline +await studio`schema diff from last week` + .map(change => studio`generate migration for ${change}`) + .map(migration => studio`review ${migration}`) +``` ## Architecture +### Durable Object per Agent + ``` +-----------------------+ | studio.do | @@ -106,175 +224,119 @@ Not a shared admin panel. Each agent has isolated, persistent Studio state. **Key insight**: Durable Objects provide per-agent state. Each Studio instance stores its own connection info, query history, and preferences in DO SQLite. -## Installation +### Multi-Database Support -```bash -npm install studio.do -``` +| Database | Connection | +|----------|------------| +| **Turso** | `connect to Turso at libsql://...` | +| **D1** | `connect to D1 database BINDING` | +| **Supabase** | `connect to Supabase project ID` | +| **Postgres** | `connect to Postgres at postgres://...` | +| **SQLite** | `connect to SQLite at /path/to/db` | -## Quick Start +### MCP Tools -### Connect to a Database +AI agents interact via Model Context Protocol: -```typescript -import { Studio } from 'studio.do' +| Tool | Natural Language Equivalent | +|------|----------------------------| +| `studio_connect` | `connect to Turso at...` | +| `studio_introspect` | `schema for users` | +| `studio_browse` | `active admins in users` | +| `studio_query` | `run SELECT * FROM...` | +| `studio_alter` | `add email column to users` | +| `studio_visualize` | `diagram for orders` | -const studio = new Studio(env.STUDIO) +## Agent Integration -// Connect to Turso -await studio.connect('turso', { - url: 'libsql://my-db-org.turso.io', - authToken: env.TURSO_TOKEN -}) +```typescript +import { tom, ralph } from 'agents.do' +import { studio } from 'studio.do' -// Or connect to D1 -await studio.connect('d1', { - binding: env.D1_DATABASE -}) +// Each agent has their own Studio +await tom`connect to production database` +await ralph`connect to staging database` -// Or connect to Supabase.do -await studio.connect('supabase', { - binding: env.SUPABASE_DO -}) +// They don't see each other's connections +await tom`my connections` // [production] +await ralph`my connections` // [staging] + +// Agents can analyze and implement +await tom`find slow queries` + .map(query => tom`analyze ${query}`) + .map(issue => ralph`optimize ${issue}`) ``` -### Introspect Schema +## vs Drizzle Studio -```typescript -// Get full schema -const schema = await studio.introspect() -console.log(schema.tables) -// [{ name: 'users', columns: [...], indexes: [...], foreignKeys: [...] }, ...] +| Feature | Drizzle Studio | studio.do | +|---------|----------------|-----------| +| **Architecture** | Centralized server | Durable Object per agent | +| **Open Source** | No | Yes (MIT) | +| **Schema Source** | Config file (`drizzle.config.ts`) | Runtime introspection | +| **AI Integration** | None | MCP tools native | +| **Multi-tenancy** | Shared instance | Isolated per agent | +| **Offline** | No | Yes (DO state persists) | +| **Natural Language** | No | Yes | +| **ER Diagrams** | No | Yes (Mermaid) | -// Get single table -const usersTable = await studio.introspect('users') -``` +## Use Cases -### Browse Data +### Database Exploration ```typescript -// Simple browse -const users = await studio.browse('users', { limit: 50 }) - -// With filters -const activeUsers = await studio.browse('users', { - filter: { status: 'active', role: 'admin' }, - sort: { created_at: 'desc' }, - limit: 20, - offset: 0 -}) +// Discover schema +await studio`all tables` +await studio`relationships between tables` +await studio`tables with most rows` + +// Profile data +await studio`column statistics for users` +await studio`null values in orders` +await studio`duplicate emails in customers` ``` -### Execute Queries +### Performance Analysis ```typescript -// Run SQL -const result = await studio.query('SELECT * FROM users WHERE id = ?', [123]) - -// Get query history -const history = await studio.getQueryHistory({ limit: 10 }) +// Find problems +await studio`slow queries from last hour` +await studio`missing indexes` +await studio`table scan queries` + +// Fix them +await studio`suggest indexes for orders` + .map(suggestion => studio`create ${suggestion}`) ``` -### MCP Tools +### Data Migration ```typescript -import { studioTools, invokeTool } from 'studio.do/mcp' +// Export data +await studio`export users to CSV` +await studio`backup orders table` -// List available tools -console.log(studioTools.map(t => t.name)) -// ['studio_connect', 'studio_introspect', 'studio_browse', 'studio_query', ...] - -// Invoke a tool -const schema = await invokeTool('studio_introspect', { - database: 'main', - tables: ['users', 'posts'] -}) - -// Browse with natural language (AI feature) -const result = await invokeTool('studio_browse', { - table: 'users', - natural: 'active admins created this month' -}) +// Import data +await studio`import customers from CSV` +await studio`restore orders from backup` ``` -### Durable Object +### Schema Evolution ```typescript -import { StudioDO } from 'studio.do/do' - -// In your worker -export { StudioDO } - -export default { - async fetch(request, env) { - // Each agent gets their own Studio instance - const id = env.STUDIO.idFromName('agent-tom') - const stub = env.STUDIO.get(id) - return stub.fetch(request) - } -} +// Make changes +await studio`add phone column to customers` +await studio`drop legacy_id from orders` +await studio`rename email to email_address in users` + +// Track changes +await studio`migration history` +await studio`schema changes this month` ``` -## API Overview - -### Core (`studio.do`) - -**Connection Management** -- `connect(type, config)` - Connect to a database -- `disconnect(name?)` - Disconnect from database(s) -- `listConnections()` - List active connections - -**Schema Introspection** -- `introspect(table?)` - Get schema for all tables or specific table -- `getTables()` - List table names -- `getColumns(table)` - Get column definitions -- `getIndexes(table)` - Get index definitions -- `getForeignKeys(table)` - Get foreign key relationships - -**Data Operations** -- `browse(table, options?)` - Browse table data with filtering/pagination -- `query(sql, params?)` - Execute arbitrary SQL -- `insert(table, data)` - Insert row(s) -- `update(table, data, where)` - Update row(s) -- `delete(table, where)` - Delete row(s) - -**Query Management** -- `getQueryHistory(options?)` - Get past queries -- `saveQuery(name, sql)` - Save a named query -- `getSavedQueries()` - List saved queries - -### MCP Tools (`studio.do/mcp`) - -| Tool | Description | -|------|-------------| -| `studio_connect` | Connect to a database | -| `studio_introspect` | Get database schema | -| `studio_browse` | Browse table data with filters | -| `studio_query` | Execute SQL query | -| `studio_alter` | Modify schema (DDL) | -| `studio_visualize` | Generate ER diagram | - -### Durable Object (`studio.do/do`) - -- `StudioDO` - Main Durable Object class -- Handles all operations via fetch API -- Stores preferences and history in DO SQLite - -## Comparison with Drizzle Studio - -| Feature | Drizzle Studio | studio.do | -|---------|---------------|-----------| -| **Architecture** | Centralized server | Durable Object per user | -| **Open Source** | No | Yes (MIT) | -| **Schema Source** | Config file | Runtime introspection | -| **AI Integration** | None | MCP tools native | -| **Multi-tenancy** | Shared instance | Isolated per agent | -| **Offline** | No | Yes (DO state persists) | -| **ER Diagrams** | No | Yes (Mermaid) | - ## The Rewrites Ecosystem -studio.do completes the AI infrastructure stack: +studio.do completes the AI database infrastructure: | Package | Purpose | |---------|---------| @@ -286,3 +348,16 @@ studio.do completes the AI infrastructure stack: ## License MIT + +--- + +

+ Drizzle Studio is for humans. studio.do is for AI. +
+ Natural language. Per-agent isolation. Edge-native. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/tableau/README.md b/rewrites/tableau/README.md index 4c33e074..d02440f3 100644 --- a/rewrites/tableau/README.md +++ b/rewrites/tableau/README.md @@ -6,62 +6,86 @@ Tableau revolutionized how businesses see data. But at $70/user/month for Creato **tableau.do** reimagines data visualization for the AI era. Natural language to charts. One-click deploy. No per-seat licensing. -## The Problem +## AI-Native API -Salesforce acquired Tableau for $15.7B and built an empire on: +```typescript +import { tableau } from 'tableau.do' // Full SDK +import { tableau } from 'tableau.do/tiny' // Minimal client +import { tableau } from 'tableau.do/viz' // Visualization-only +``` -- **Per-seat pricing** - Creator ($70), Explorer ($42), Viewer ($15) per user/month -- **Role-based paywalls** - Want to make a chart? Pay for Creator. Want to explore? Pay for Explorer. -- **AI as premium upsell** - "Ask Data" and AI features require additional licensing -- **Server complexity** - Tableau Server requires dedicated infrastructure and admins -- **Data extracts** - Performance requires pre-computing extracts, not live queries -- **Salesforce lock-in** - Einstein integration ties you deeper into the ecosystem +Natural language for data visualization: -A 50-person analytics team? **$50k+/year** before you connect a single data source. +```typescript +import { tableau } from 'tableau.do' -## The workers.do Way +// Talk to it like a colleague +const sales = await tableau`sales by region this quarter` +const trend = await tableau`revenue trend last 12 months` +const compare = await tableau`compare Q3 vs Q4 by product` + +// Chain like sentences +await tableau`customers at risk of churn` + .visualize(`heatmap by segment and tenure`) + +// Dashboards that build themselves +await tableau`executive dashboard` + .add(`revenue this quarter`) + .add(`top customers by lifetime value`) + .add(`churn risk by segment`) + .publish() +``` -Your investors want dashboards. Your team needs metrics. Tableau wants $50k/year and a dedicated admin just to show bar charts. +## The Problem -**What if visualization was as simple as asking?** +Salesforce acquired Tableau for $15.7B and built an empire on: -```typescript -import { tableau } from 'tableau.do' -import { priya, mark } from 'agents.do' +| What Tableau Charges | The Reality | +|---------------------|-------------| +| **Creator** | $70/user/month to make charts | +| **Explorer** | $42/user/month to interact | +| **Viewer** | $15/user/month just to look | +| **AI Features** | Premium tier upsell | +| **Server** | Dedicated infrastructure + admin | +| **Lock-in** | Einstein ties you to Salesforce | -// Natural language to visualization -const chart = await tableau`show ${metric} by ${dimension} over time` -const dashboard = await tableau`build executive dashboard from ${salesModel}` -const insight = await tableau`what's driving the Q3 decline?` +### The Salesforce Tax -// AI agents as analysts -const analysis = await priya`analyze funnel conversion and visualize drop-off` -const report = await mark`create board deck with revenue trends` -``` +Since the acquisition: -**Promise pipelining** - chain operations without `Promise.all`: +- Per-seat pricing at every tier +- AI features locked behind premium +- Einstein integration push +- Server complexity unchanged +- Data extracts still required -```typescript -const deck = await tableau`query ${salesData}` - .map(data => tableau`visualize ${data} as ${chartType}`) - .map(viz => [priya, mark].map(r => r`review ${viz} for board`)) -``` +A 50-person analytics team? **$50k+/year** before you connect a single data source. + +### The Visualization Gap + +Tableau talks "self-service." But: + +- Creator license to make a chart +- Explorer license to interact +- Viewer license just to see it +- Premium for AI-powered insights +- Server for any collaboration -One network round trip. Natural language. Your data, visualized. +Everyone should see their data. Not just licensed "Creators." ## The Solution -**tableau.do** is Tableau reimagined: +**tableau.do** reimagines visualization for everyone: ``` Traditional Tableau tableau.do ----------------------------------------------------------------- -$15-70/user/month $0 - run your own +$70/user/month Creator $0 - run your own +$42/user/month Explorer Everyone explores +$15/user/month Viewer Everyone views "Ask Data" premium Natural language built-in Tableau Server Cloudflare Workers Data extracts Live queries at the edge -Desktop app required Browser-first, PWA optional -VizQL proprietary VizQL-compatible (open) ``` ## One-Click Deploy @@ -70,436 +94,240 @@ VizQL proprietary VizQL-compatible (open) npx create-dotdo tableau ``` -Your own Tableau Server. Running on Cloudflare. Zero per-seat fees. - -## Natural Language to Visualization - -Skip the drag-and-drop. Just ask: +Your own visualization platform. Running on Cloudflare. Zero per-seat fees. ```typescript -import { tableau } from 'tableau.do' - -// Natural language creates visualizations -const chart = await tableau`show me sales by region over time` -const dashboard = await tableau`create a dashboard showing customer metrics` -const insight = await tableau`what's driving the decline in Q3?` +import { Tableau } from 'tableau.do' -// Returns both visualization spec and rendered output -console.log(chart.spec) // VegaLite/VizQL spec -console.log(chart.svg) // Rendered SVG -console.log(chart.png) // Rendered PNG -console.log(chart.insight) // AI-generated narrative +export default Tableau({ + name: 'acme-analytics', + domain: 'viz.acme.com', + theme: 'brand', +}) ``` ## Features ### Data Connections -Connect to any data source: - -| Source | Status | Description | -|--------|--------|-------------| -| **D1/SQLite** | Built-in | Native Cloudflare integration | -| **PostgreSQL** | Supported | Via Hyperdrive connection pooling | -| **MySQL** | Supported | Via Hyperdrive | -| **Snowflake** | Supported | Direct query | -| **BigQuery** | Supported | Direct query | -| **REST APIs** | Supported | Any JSON/CSV endpoint | -| **R2/S3** | Supported | Parquet, CSV, JSON files | -| **Spreadsheets** | Supported | Excel, Google Sheets | - ```typescript -import { DataSource } from 'tableau.do' - -// Connect to your data -const sales = DataSource.postgres({ - connectionString: env.DATABASE_URL, - schema: 'public', - tables: ['orders', 'customers', 'products'], -}) - -// Or connect to files -const logs = DataSource.parquet({ - bucket: env.R2_BUCKET, - prefix: 'logs/', -}) +// Natural as describing your data +const sales = await tableau`connect postgres with orders, customers, products` +const logs = await tableau`connect parquet files in analytics bucket` +const sheets = await tableau`import Q4 forecast from Google Sheets` + +// AI infers what you need +await tableau`orders` // connects to orders table +await tableau`sales data` // finds relevant datasource +await tableau`last year revenue` // queries the right tables ``` -### Visualization Types - -All the charts you need: +### Visualizations ```typescript -import { Chart } from 'tableau.do' - -// Bar charts -Chart.bar({ x: 'region', y: 'sales', color: 'product' }) - -// Line charts with trends -Chart.line({ x: 'date', y: 'revenue', trendline: true }) - -// Scatter plots -Chart.scatter({ x: 'price', y: 'quantity', size: 'profit', color: 'category' }) - -// Geographic maps -Chart.map({ geo: 'state', value: 'customers', type: 'choropleth' }) - -// Treemaps -Chart.treemap({ path: ['category', 'subcategory'], value: 'sales' }) - -// Heat maps -Chart.heatmap({ x: 'hour', y: 'day', value: 'traffic' }) - -// Bullet charts -Chart.bullet({ measure: 'actual', target: 'goal', ranges: [0, 50, 75, 100] }) +// Just describe what you want to see +await tableau`sales by region as bar chart` +await tableau`revenue trend over 12 months` +await tableau`customers by state on a map` +await tableau`traffic by hour and day as heatmap` + +// AI picks the right chart type +await tableau`show me the distribution` // histogram +await tableau`compare categories` // bar chart +await tableau`show the trend` // line chart +await tableau`show relationships` // scatter plot ``` ### Dashboards -Compose visualizations into interactive dashboards: - ```typescript -import { Dashboard } from 'tableau.do' - -export const SalesDashboard = Dashboard({ - name: 'Sales Performance', - layout: 'responsive', - sections: [ - { - title: 'KPIs', - items: [ - { type: 'metric', field: 'total_revenue', format: 'currency' }, - { type: 'metric', field: 'order_count', format: 'number' }, - { type: 'metric', field: 'avg_order_value', format: 'currency' }, - ], - }, - { - title: 'Trends', - items: [ - { type: 'chart', chart: 'revenue_over_time', span: 2 }, - { type: 'chart', chart: 'orders_by_region', span: 1 }, - ], - }, - { - title: 'Details', - items: [ - { type: 'table', source: 'recent_orders', pageSize: 20 }, - ], - }, - ], - filters: [ - { field: 'date_range', type: 'dateRange', default: 'last30Days' }, - { field: 'region', type: 'multiSelect', source: 'regions' }, - { field: 'product', type: 'search', source: 'products' }, - ], -}) -``` - -### VizQL Compatibility +// Dashboards are just sentences +await tableau`sales dashboard` + .add(`revenue this quarter`) + .add(`orders by region`) + .add(`top 10 customers`) -Write VizQL queries directly: +// Or one-shot +await tableau`executive dashboard with revenue, orders, and customer metrics` -```typescript -import { vizql } from 'tableau.do' - -// VizQL query syntax -const result = await vizql` - SELECT - [Region], - SUM([Sales]) AS [Total Sales], - AVG([Profit]) AS [Avg Profit] - FROM [Orders] - WHERE [Order Date] >= #2024-01-01# - GROUP BY [Region] - ORDER BY [Total Sales] DESC -` - -// Renders to visualization spec -console.log(result.visualization) +// Interactive filters just work +await tableau`sales dashboard filtered by region and date` ``` ### Calculated Fields -Create calculations without writing SQL: - ```typescript -import { Field } from 'tableau.do' - -const profitMargin = Field.calculated({ - name: 'Profit Margin', - formula: '[Profit] / [Revenue]', - format: 'percent', -}) - -const customerSegment = Field.calculated({ - name: 'Customer Segment', - formula: ` - IF [Lifetime Value] > 10000 THEN 'Enterprise' - ELSEIF [Lifetime Value] > 1000 THEN 'Business' - ELSE 'Consumer' - END - `, -}) - -const rollingAvg = Field.calculated({ - name: '7-Day Moving Average', - formula: 'WINDOW_AVG(SUM([Revenue]), -6, 0)', - partitionBy: ['Region'], -}) +// Define calculations naturally +const margin = await tableau`profit margin as [Profit] / [Revenue] in percent` +const segment = await tableau`customer segment: Enterprise if LTV > 10k, Business if > 1k, else Consumer` +const rolling = await tableau`7-day moving average of revenue by region` + +// Or ask for them +await tableau`add year-over-year growth calculation` +await tableau`add running total to sales` +await tableau`rank customers by revenue within each region` ``` ### Table Calculations -Window functions and analytics: - ```typescript -// Rank by sales within each region -Field.calculation('Sales Rank', 'RANK(SUM([Sales]))', { - partitionBy: 'Region', - orderBy: 'Sales DESC' -}) - -// Running total -Field.calculation('Running Total', 'RUNNING_SUM(SUM([Revenue]))') - -// Percent of total -Field.calculation('% of Total', 'SUM([Sales]) / TOTAL(SUM([Sales]))') - -// Year-over-year growth -Field.calculation('YoY Growth', '(ZN(SUM([Sales])) - LOOKUP(ZN(SUM([Sales])), -1)) / ABS(LOOKUP(ZN(SUM([Sales])), -1))') +// Window functions as natural language +await tableau`rank by sales within each region` +await tableau`running total of revenue` +await tableau`percent of total by category` +await tableau`compare to previous year` + +// Complex analytics +await tableau`show month-over-month change` +await tableau`add trend line to revenue chart` +await tableau`forecast next 3 months based on trend` ``` -## AI-Native Features +## AI-Native Analytics ### Ask Data -Natural language queries that understand your data: - ```typescript -import { askData } from 'tableau.do' - -// Simple questions -const answer1 = await askData('What were total sales last month?') -// Returns: { value: 1234567, visualization: , narrative: "Total sales..." } - -// Complex analysis -const answer2 = await askData('Which products are underperforming compared to last year?') -// Returns: { data: [...], visualization: , narrative: "..." } - -// Predictive -const answer3 = await askData('What will sales be next quarter based on current trends?') -// Returns: { prediction: {...}, confidence: 0.85, visualization: } +// Just ask questions +await tableau`what were total sales last month?` +await tableau`which products are underperforming?` +await tableau`what will sales be next quarter?` + +// AI returns answers with visualizations +const answer = await tableau`why did churn spike in Q3?` +// Returns chart + narrative + contributing factors ``` ### Explain Data -AI-powered anomaly detection and explanation: - ```typescript -import { explainData } from 'tableau.do' +// Ask why things changed +await tableau`why did revenue drop in February?` +await tableau`what's driving the Q3 decline?` +await tableau`explain the spike in signups last week` + +// AI surfaces contributing factors automatically +// - Marketing spend decreased 40% +// - Key product out of stock +// - Seasonal trend +``` -// Why did this metric change? -const explanation = await explainData({ - metric: 'revenue', - period: { from: '2024-01-01', to: '2024-03-31' }, - question: 'Why did revenue drop in February?', -}) +### Auto-Insights -// Returns contributing factors with visualizations -console.log(explanation.factors) -// [ -// { factor: 'Marketing spend decreased 40%', impact: -0.35, viz: }, -// { factor: 'Key product out of stock', impact: -0.25, viz: }, -// { factor: 'Seasonal trend', impact: -0.15, viz: }, -// ] +```typescript +// Surface patterns automatically +await tableau`what's interesting in sales data?` +await tableau`anomalies in the last 30 days` +await tableau`trends I should know about` + +// AI proactively finds insights +// "Revenue up 23% in West region" +// "Churn correlated with support tickets" +// "Tuesday orders 40% higher than average" ``` -### Auto-Insights +### Promise Pipelining -Automatic discovery of interesting patterns: +Chain operations without `Promise.all`: ```typescript -import { autoInsights } from 'tableau.do' - -const insights = await autoInsights({ - datasource: 'sales', - focus: ['revenue', 'customers', 'churn'], -}) +const deck = await tableau`sales data` + .map(data => tableau`visualize ${data} by region`) + .map(viz => [priya, mark].map(r => r`review ${viz} for board`)) -// Returns ranked insights with visualizations -for (const insight of insights) { - console.log(insight.headline) // "Revenue up 23% in West region" - console.log(insight.significance) // 0.95 - console.log(insight.visualization) // - console.log(insight.narrative) // "The West region saw significant..." -} +// One network round trip. Natural language. Your data, visualized. ``` ### AI Agents as Analysts -AI agents can create and iterate on visualizations: - ```typescript import { priya, mark } from 'agents.do' -import { tableau } from 'tableau.do' // Product manager analyzes metrics -const analysis = await priya` - analyze our funnel conversion rates and create a dashboard - showing where users are dropping off -` +await priya`analyze funnel conversion and visualize drop-off` // Marketing creates campaign dashboard -const campaign = await mark` - build a real-time dashboard showing our launch campaign performance - with social metrics, signups, and revenue attribution -` +await mark`real-time dashboard for launch campaign with social, signups, revenue` + +// They iterate naturally +await priya`this chart is confusing, simplify it` +await mark`add comparison to last quarter` ``` ## Architecture -### Edge-First Rendering - -Visualizations render at the edge, close to your users: +### Durable Object per Dashboard ``` - +------------------------+ - | tableau.do Worker | - | (Query + Render) | - +------------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | DashboardDO | | DataSourceDO | | ChartDO | - | (Layout + State) | | (Connection) | | (Spec + Cache) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +--------------------+--------------------+ - | | | - +----------+ +-----------+ +------------+ - | D1 | | Hyperdrive | | R2 | - | (Metadata)| | (Postgres) | | (Exports) | - +----------+ +-----------+ +------------+ +TableauDO (config, theme, permissions) + | + +-- DashboardsDO (layouts, filters, state) + | |-- SQLite: Dashboard definitions + | +-- R2: Exported snapshots + | + +-- DataSourcesDO (connections, schemas) + | |-- SQLite: Connection metadata + | +-- Hyperdrive: Live queries + | + +-- ChartsDO (specs, cache, renders) + |-- SQLite: Chart specs + +-- KV: Rendered outputs (SVG/PNG) ``` ### Storage Tiers -- **Hot (SQLite/D1)** - Dashboard metadata, user preferences, recent queries -- **Warm (Hyperdrive)** - Live connections to external databases -- **Cold (R2)** - Exported data, snapshots, large datasets - -### Rendering Pipeline - -```typescript -// Query -> Spec -> Render -> Cache -NaturalLanguage | VizQL | ChartBuilder - ↓ - Query Engine (SQL generation) - ↓ - Data Fetch (Hyperdrive/D1/R2) - ↓ - VegaLite Spec Generation - ↓ - Server-Side Render (SVG/PNG/PDF) - ↓ - Edge Cache (KV) - ↓ - Client Delivery -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Dashboard metadata, preferences | <10ms | +| **Warm** | Hyperdrive | Live database queries | <100ms | +| **Cold** | R2 | Exports, snapshots, large datasets | <1s | ## Embedding -Embed visualizations anywhere: - -### iframe Embed - -```html - -``` - -### JavaScript SDK - ```typescript -import { TableauEmbed } from 'tableau.do/embed' - -const viz = new TableauEmbed({ - container: document.getElementById('viz'), - dashboard: 'sales-performance', - filters: { - region: ['West', 'East'], - date: { start: '2024-01-01', end: '2024-03-31' }, - }, - onSelect: (data) => console.log('Selected:', data), -}) +// Embed anywhere +await tableau`sales dashboard`.embed('div#chart') -// Programmatic interaction -await viz.applyFilter('product', ['Widget A', 'Widget B']) -const image = await viz.export('png') -const data = await viz.getData() -``` - -### React Component +// Or as iframe +await tableau`sales dashboard`.iframe({ width: '100%', height: 600 }) -```tsx +// React component import { Tableau } from 'tableau.do/react' -function AnalyticsDashboard() { - return ( - - ) -} + ``` -## MCP Tools - -Every visualization capability exposed as MCP tools: +### Interactive Features ```typescript -import { tableauTools } from 'tableau.do/mcp' - -// Available tools -tableauTools.map(t => t.name) -// [ -// 'create_chart', -// 'create_dashboard', -// 'query_data', -// 'ask_data', -// 'explain_data', -// 'export_visualization', -// 'list_datasources', -// 'connect_datasource', -// ] - -// AI can create visualizations -await invokeTool('create_chart', { - type: 'bar', - datasource: 'sales', - x: 'region', - y: 'sum(revenue)', - title: 'Revenue by Region', -}) +// Filters are natural language +await tableau`sales dashboard` + .filter(`West and East regions`) + .filter(`Q1 2024`) + +// Export anywhere +await tableau`executive dashboard`.export('pdf') +await tableau`revenue chart`.export('png') +await tableau`customer data`.export('csv') ``` +## vs Tableau + +| Feature | Tableau | tableau.do | +|---------|---------|-----------| +| **Pricing** | $15-70/user/month | $0 - run your own | +| **AI** | Premium tier | Built-in | +| **Architecture** | Server + Desktop | Edge-native | +| **Data** | Extracts required | Live queries | +| **Deploy** | Months | Minutes | +| **Lock-in** | Salesforce ecosystem | MIT licensed | + ## Deployment Options ### Cloudflare Workers (Recommended) ```bash npx create-dotdo tableau -# Deploys to your Cloudflare account ``` ### Docker @@ -512,73 +340,102 @@ docker run -p 8787:8787 dotdo/tableau ```bash git clone https://github.com/dotdo/tableau.do -cd tableau.do -npm install -npm run dev # Local development -npm run deploy # Production deployment +cd tableau.do && pnpm install && pnpm dev ``` ## Migration from Tableau -### Export Your Workbooks - -```bash -# Export from Tableau Server/Online -tableau export --server your-server.tableau.com --workbooks all +```typescript +// Import existing workbooks +await tableau`import workbooks from Tableau Server` -# Import to tableau.do -npx tableau-migrate import --source ./exports +// VizQL expressions work directly +await tableau`SUM([Sales]) by [Region]` // same syntax +await tableau`WINDOW_AVG(SUM([X]), -6, 0)` // same functions +await tableau`DATETRUNC('month', [Order Date])` // same date math ``` -### VizQL Compatibility +## Use Cases -Most VizQL expressions work directly: +### Executive Dashboards +```typescript +await tableau`executive dashboard` + .add(`revenue this quarter vs target`) + .add(`top 10 customers by ARR`) + .add(`churn by segment`) + .share(`board@company.com`) ``` -Tableau tableau.do ------------------------------------------------------------------ -[Sales] [Sales] (same) -SUM([Revenue]) SUM([Revenue]) (same) -DATETRUNC('month', [Date]) DATETRUNC('month', [Date]) (same) -WINDOW_AVG(SUM([X]), -6, 0) WINDOW_AVG(SUM([X]), -6, 0)(same) -ZN([Value]) ZN([Value]) (same) -LOOKUP(SUM([X]), -1) LOOKUP(SUM([X]), -1) (same) + +### Sales Analytics + +```typescript +await tableau`sales pipeline by stage and owner` +await tableau`conversion rates by lead source` +await tableau`forecast vs actual by quarter` +``` + +### Product Analytics + +```typescript +await tableau`user funnel from signup to paid` +await tableau`feature usage by cohort` +await tableau`retention curve by acquisition channel` ``` ## Roadmap -- [x] Core chart types (bar, line, scatter, map, etc.) -- [x] Dashboard builder -- [x] VizQL query engine +### Core - [x] Natural language queries +- [x] Chart types (bar, line, scatter, map) +- [x] Dashboard builder - [x] Calculated fields - [x] Table calculations - [ ] Storytelling (guided narratives) -- [ ] Prep (data transformation flows) -- [ ] Mobile app (PWA) +- [ ] Prep (data transformation) - [ ] Alerting and subscriptions -- [ ] Collaboration (comments, annotations) -- [ ] Data modeling layer + +### AI +- [x] Ask Data +- [x] Explain Data +- [x] Auto-Insights +- [ ] Predictive analytics +- [ ] Anomaly detection +- [ ] Natural language alerts ## Why Open Source? -Data visualization is too important to be locked behind per-seat pricing: +Data visualization shouldn't require a license for every viewer: 1. **Your insights** - Understanding your business shouldn't cost $70/user 2. **Your data** - Visualizations should run where your data lives -3. **Your AI** - Natural language analytics should be built-in, not premium -4. **Your team** - Everyone should be able to explore data, not just "Creators" +3. **Your AI** - Natural language should be built-in, not premium +4. **Your team** - Everyone should explore data, not just "Creators" Tableau showed the world what visual analytics could be. **tableau.do** makes it accessible to everyone. +## Contributing + +```bash +git clone https://github.com/dotdo/tableau.do +cd tableau.do +pnpm install +pnpm test +``` + ## License -MIT License - Use it however you want. Build dashboards. Embed in your product. Sell it to clients. +MIT License - Build dashboards. Embed in your product. Visualize anything. ---

- tableau.do is part of the dotdo platform. + The $15B visualization platform ends here.
- Website | Docs | Discord + Natural language. AI-native. Everyone's an analyst. +

+ Website | + Docs | + Discord | + GitHub

diff --git a/rewrites/temporal/README.md b/rewrites/temporal/README.md index 22a4a0f8..39bc28ae 100644 --- a/rewrites/temporal/README.md +++ b/rewrites/temporal/README.md @@ -1,486 +1,355 @@ # temporal.do -Durable workflows for startup founders. No servers. No Cassandra. Just code. +> Durable Workflows. Edge-Native. Zero Infrastructure. AI-First. -## The Problem - -You're building a startup. You need durable workflows: -- Send a welcome email, wait 7 days, send a follow-up -- Process an order with payment, inventory, and shipping -- Onboard a customer with approval gates and retries - -Traditional Temporal requires: -- Running Temporal server clusters -- Managing Cassandra or MySQL persistence -- Operating Elasticsearch for visibility -- Complex networking and service mesh -- **47 hours to production** (and a DevOps engineer you don't have) +Temporal charges enterprises millions. Clusters, Cassandra, Elasticsearch, DevOps teams, months of setup. All to run a workflow that sends an email, waits a week, and follows up. -**Every hour debugging infrastructure is an hour not building your product.** -**Every $500/month on servers is runway burned.** +**temporal.do** is the open-source alternative. Deploys in seconds. Runs on Cloudflare. Natural language workflows that read like business logic. -## The Vision - -Durable workflows in 47 seconds. +## AI-Native API ```typescript -import { temporal } from '@dotdo/temporal' - -temporal`process order ${order} with payment and shipping` -temporal`onboard ${user.email} with welcome email, wait 7 days, then follow up` -temporal`run daily at 9am: generate sales report` +import { temporal } from 'temporal.do' // Full SDK +import { temporal } from 'temporal.do/tiny' // Minimal client +import { temporal } from 'temporal.do/durable' // Durable-only operations ``` -Natural language. Tagged templates. Workflows as conversations. - -## Promise Pipelining - -Chain workflows without `Promise.all`. One network round trip: +Natural language for durable workflows: ```typescript -const result = await temporal`validate order ${order}` - .map(validated => temporal`charge ${validated.payment}`) - .map(charged => temporal`ship ${charged.items}`) - .map(shipped => temporal`email tracking to ${order.customer}`) -// One network round trip! -``` +import { temporal } from 'temporal.do' -Compose complex business processes: +// Talk to it like you're dictating +await temporal`onboard ${user.email}` + .send(`Welcome! Thanks for joining`) + .wait('7 days') + .notify(`How are things?`) -```typescript -const onboarding = await temporal`create account for ${email}` - .map(account => temporal`send welcome email to ${account.email}`) - .map(() => temporal`wait 7 days`) - .map(() => temporal`send follow-up to ${email}`) - .map(() => temporal`check if ${email} completed onboarding`) - .map(status => status.completed - ? temporal`mark ${email} as active` - : temporal`escalate ${email} to sales`) +// Approvals that wait forever +await temporal`request approval for ${request}` + .waitFor('approval') + .then(a => a.approved ? temporal`process ${request}` : temporal`reject ${request}`) + +// Schedules read like calendar entries +await temporal`run daily at 9am: generate sales report` ``` -## Agent Integration +## The Problem -Ask Ralph to build your workflows: +Traditional Temporal requires: -```typescript -import { ralph, tom, priya } from 'agents.do' +| What Temporal Needs | The Reality | +|---------------------|-------------| +| **Temporal Server** | Complex multi-service cluster | +| **Cassandra/MySQL** | Production database ops | +| **Elasticsearch** | Another cluster to manage | +| **Networking** | Service mesh complexity | +| **DevOps Team** | $150k/year minimum | +| **Time to Production** | 47 hours (optimistic) | -ralph`implement an order processing workflow with retries and compensation` -tom`review the workflow for failure handling edge cases` -priya`add monitoring for workflow SLA violations` -``` +Every hour debugging infrastructure is an hour not building your product. Every $500/month on servers is runway burned. -Workflows can invoke agents: +## The Solution -```typescript -const support = await temporal`customer ${customer.id} submitted ticket ${ticket}` - .map(ticket => priya`triage support ticket: ${ticket.description}`) - .map(priority => priority === 'urgent' - ? temporal`page on-call with ${ticket}` - : temporal`queue ${ticket} for next business day`) +``` +Traditional Temporal temporal.do +----------------------------------------------------------------- +47 hours to production 47 seconds +$500/month infrastructure $0 (included) +3+ services to manage 0 services +Cassandra expertise required Just TypeScript +DevOps team needed Solo founder ready +Complex Worker setup One line deploys ``` -## The Transformation - -| Before (Traditional) | After (temporal.do) | -|---------------------|---------------------| -| 47 hours to production | 47 seconds | -| $500/month infrastructure | $0 (included) | -| 3 services to manage | 0 services | -| Cassandra expertise required | Just TypeScript | -| DevOps team needed | Solo founder ready | +## One-Click Deploy ```bash -# Before -docker-compose up temporal cassandra elasticsearch -# 47 hours of configuration, debugging, and prayer - -# After -npm install @dotdo/temporal -# Ship your product +npx create-dotdo temporal ``` -## For Temporal Users (Familiar API) - -Already know Temporal? Use the structured API: +A durable workflow engine. Running on infrastructure you control. Natural language from day one. ```typescript -import { Temporal } from '@dotdo/temporal' - -const temporal = new Temporal({ namespace: 'my-app' }) - -// Define a workflow -const orderWorkflow = temporal.defineWorkflow( - 'order-workflow', - async (ctx, order: Order) => { - // Each activity is retried automatically - const validated = await ctx.activity('validate-order', () => - validateOrder(order) - ) - - // Timers without blocking workers - await ctx.sleep('wait-for-payment', '5m') +import { Temporal } from 'temporal.do' - // Wait for external signals - const approval = await ctx.waitForSignal('approval', { timeout: '24h' }) - - // Execute child workflow - const shipping = await ctx.executeChild(shippingWorkflow, validated) - - return { orderId: order.id, status: 'completed', shipping } - } -) - -// Start workflow execution -const handle = await temporal.startWorkflow(orderWorkflow, { - workflowId: `order-${orderId}`, - args: [orderData] +export default Temporal({ + name: 'my-startup', + domain: 'workflows.my-startup.com', }) - -// Query workflow state -const status = await handle.query('status') - -// Send signals -await handle.signal('approval', { approved: true }) ``` -No servers to manage. No databases to operate. Just durable workflows that work. - ## Features -- **Natural Language Workflows** - Define business logic in plain English -- **Promise Pipelining** - Chain workflows with `.map()` in one round trip -- **Activity Execution** - Automatic retries with exponential backoff -- **Signals and Queries** - Real-time workflow interaction -- **Child Workflows** - Compose workflows hierarchically -- **Timers and Scheduling** - Sleep, cron schedules, delayed execution -- **Event History Replay** - Deterministic replay for failure recovery -- **TypeScript First** - Full type safety matching Temporal SDK patterns -- **Edge Native** - Runs on Cloudflare's global network - -## Architecture +### Order Processing +```typescript +// Just say what you want +await temporal`process order ${orderId}` + +// AI infers the workflow +await temporal`process order ${orderId}` + .validate() + .charge() + .ship() + .notify() + +// Or chain explicitly +await temporal`validate order ${order}` + .map(validated => temporal`charge ${validated.payment}`) + .map(charged => temporal`ship ${charged.items}`) + .map(shipped => temporal`email tracking to ${order.customer}`) ``` - +----------------------+ - | temporal.do | - | (Cloudflare Worker) | - +----------------------+ - | - +---------------+---------------+---------------+ - | | | | - +------------------+ +------------------+ +------------------+ +------------------+ - | WorkflowDO | | ActivityDO | | TimerDO | | HistoryDO | - | (execution state)| | (retry/timeout) | | (schedules) | | (event sourcing) | - +------------------+ +------------------+ +------------------+ +------------------+ - | | | | - +---------------+---------------+---------------+ - | - +-------------------+ - | Cloudflare Queues | - | (task dispatch) | - +-------------------+ - | - +-------------------+ - | fsx.do / gitx.do | - | (AI-native state) | - +-------------------+ -``` - -**Key insight**: Durable Objects provide single-threaded, strongly consistent state per workflow execution. Each workflow gets its own WorkflowDO for deterministic execution. Activities run in separate ActivityDOs with retry isolation. Event history enables replay for recovery. -## Installation +### User Onboarding -```bash -npm install @dotdo/temporal +```typescript +// The canonical example - one readable chain +await temporal`create account for ${email}` + .send(`Welcome! Thanks for joining`) + .wait('7 days') + .send(`How are things going?`) + .check('completed onboarding') + .then(done => done + ? temporal`mark ${email} as active` + : temporal`escalate ${email} to sales`) ``` -## Quick Start +### Approvals -### Tagged Template Style (Recommended) +```typescript +// Wait for human input, forever if needed +await temporal`request budget approval for ${expense}` + .waitFor('approval', { timeout: '48h' }) + .then(result => result?.approved + ? temporal`reimburse ${expense}` + : temporal`notify ${expense.submitter} of rejection`) + +// Multi-level approvals +await temporal`purchase request ${request}` + .waitFor('manager approval') + .waitFor('finance approval') + .waitFor('legal approval') + .process() +``` + +### Scheduled Jobs ```typescript -import { temporal } from '@dotdo/temporal' +// Cron in plain English +await temporal`run daily at 9am: generate sales report` +await temporal`run weekly on Monday: sync CRM data` +await temporal`run monthly on the 1st: process billing` + +// Intervals +await temporal`run every 5 minutes: check queue depth` +await temporal`run hourly: aggregate metrics` +``` -// Simple workflow -await temporal`send welcome email to ${user.email}` +### Compensation (Sagas) -// With timing -await temporal`wait 7 days then send follow-up to ${user.email}` +```typescript +// When things go wrong, undo gracefully +await temporal`book trip ${booking}` + .reserve('flight') + .reserve('hotel') + .reserve('car') + .onFailure(step => temporal`cancel ${step}`) + +// AI handles compensation automatically +await temporal`order ${order} with compensation` +``` -// With conditions -await temporal` - if ${user.plan} is premium - then schedule onboarding call - else send self-service guide -` +### Long-Running Workflows -// Scheduled -await temporal`run daily at 9am: generate sales report and email to ${team}` +```typescript +// Workflows that span days, weeks, months +await temporal`subscription lifecycle for ${customer}` + .onStart(() => temporal`provision ${customer}`) + .onRenewal(() => temporal`charge ${customer}`) + .onCancel(() => temporal`offboard ${customer}`) + +// Insurance claims that take months +await temporal`process claim ${claimId}` + .waitFor('documentation') + .waitFor('adjuster review') + .waitFor('approval') + .payout() ``` -### Define Activities +## Promise Pipelining + +Chain workflows without `Promise.all`. One network round trip: ```typescript -import { Temporal } from '@dotdo/temporal' +// All of this is ONE network request +const result = await temporal`validate order ${order}` + .map(validated => temporal`charge ${validated.payment}`) + .map(charged => temporal`ship ${charged.items}`) + .map(shipped => temporal`email tracking to ${order.customer}`) -const temporal = new Temporal({ namespace: 'my-app' }) +// Branching logic +const support = await temporal`customer ticket ${ticket}` + .map(t => priya`triage: ${t.description}`) + .map(priority => priority === 'urgent' + ? temporal`page on-call with ${ticket}` + : temporal`queue ${ticket} for next business day`) +``` -// Define activities with automatic retry -const activities = temporal.defineActivities({ - fetchUser: async (userId: string) => { - return await db.users.findById(userId) - }, +## Agent Integration - sendEmail: async (to: string, subject: string, body: string) => { - return await emailService.send({ to, subject, body }) - }, +Ask Ralph to build your workflows: - chargePayment: async (amount: number, customerId: string) => { - return await stripe.charges.create({ amount, customer: customerId }) - } -}) +```typescript +import { ralph, tom, priya } from 'agents.do' + +ralph`implement order processing with retries and compensation` +tom`review the workflow for failure handling edge cases` +priya`add monitoring for workflow SLA violations` ``` -### Define Workflows +Workflows can invoke agents: ```typescript -const userOnboardingWorkflow = temporal.defineWorkflow( - 'user-onboarding', - async (ctx, userId: string) => { - // Fetch user data (auto-retried on failure) - const user = await ctx.activity('fetch-user', () => - activities.fetchUser(userId) - ) - - // Send welcome email - await ctx.activity('send-welcome', () => - activities.sendEmail(user.email, 'Welcome!', 'Thanks for joining') - ) - - // Wait 7 days - await ctx.sleep('onboarding-delay', '7d') - - // Send follow-up - await ctx.activity('send-followup', () => - activities.sendEmail(user.email, 'How are things?', 'Check in message') - ) - - return { userId, status: 'onboarded' } - } -) +await temporal`support ticket ${ticket}` + .map(t => priya`triage: ${t.description}`) + .map(priority => priority === 'urgent' + ? temporal`page on-call` + : temporal`queue for next business day`) ``` -### Start Workflows +## Signals and Queries ```typescript -// Start a workflow execution -const handle = await temporal.startWorkflow(userOnboardingWorkflow, { - workflowId: `onboard-${userId}`, - taskQueue: 'onboarding', - args: [userId] -}) +// Query any workflow's state +const status = await temporal`order ${orderId}`.query('status') -// Get the result (waits for completion) -const result = await handle.result() +// Send signals to running workflows +await temporal`order ${orderId}`.signal('approval', { approved: true }) -// Or get handle to existing workflow -const existing = temporal.getHandle('onboard-123') +// Wait for signals in workflows +await temporal`process refund ${refundId}` + .waitFor('customer confirmation') + .then(() => temporal`issue refund`) ``` -### Signals and Queries +## Child Workflows ```typescript -const approvalWorkflow = temporal.defineWorkflow( - 'approval-workflow', - async (ctx, request: ApprovalRequest) => { - // Set up queryable state - let status = 'pending' - ctx.setQueryHandler('status', () => status) - - // Wait for approval signal - const approval = await ctx.waitForSignal<{ approved: boolean }>('approval', { - timeout: '48h' - }) - - if (!approval || !approval.approved) { - status = 'rejected' - return { status: 'rejected' } - } - - status = 'approved' - await ctx.activity('process-approval', () => processApproval(request)) - - return { status: 'completed' } - } -) - -// Query workflow state -const handle = temporal.getHandle('approval-123') -const status = await handle.query('status') - -// Send signal -await handle.signal('approval', { approved: true }) +// Compose workflows naturally +await temporal`fulfill order ${orderId}` + .spawn(temporal`process payment ${payment}`) + .spawn(temporal`reserve inventory ${items}`) + .spawn(temporal`schedule shipping ${address}`) + .join() // wait for all + +// Or let AI compose them +await temporal`fulfill order ${orderId} with parallel payment and shipping` ``` -### Child Workflows +## Why temporal.do? -```typescript -const parentWorkflow = temporal.defineWorkflow( - 'parent-workflow', - async (ctx, orderId: string) => { - const order = await ctx.activity('fetch-order', () => - fetchOrder(orderId) - ) - - // Execute child workflows in parallel - const [payment, shipping] = await Promise.all([ - ctx.executeChild(paymentWorkflow, { - workflowId: `payment-${orderId}`, - args: [order.payment] - }), - ctx.executeChild(shippingWorkflow, { - workflowId: `shipping-${orderId}`, - args: [order.shipping] - }) - ]) - - return { orderId, payment, shipping } - } -) -``` +| Feature | Traditional Temporal | temporal.do | +|---------|---------------------|-------------| +| **Time to Production** | 47 hours | 47 seconds | +| **Infrastructure** | Temporal + Cassandra + Elasticsearch | Zero | +| **Monthly Cost** | $500+ (servers) | $0 (included) | +| **Deployment** | Multi-service cluster | One line | +| **Expertise Required** | DevOps team | Just TypeScript | +| **Cold Starts** | Yes | No (Durable Objects) | +| **Global Distribution** | Complex | Automatic (Edge) | -### Scheduled Workflows +## Architecture -```typescript -// Run workflow on a schedule -await temporal.scheduleWorkflow(dailyReportWorkflow, { - scheduleId: 'daily-report', - cron: '0 9 * * *', // Daily at 9am UTC - args: [] -}) +### Durable Object per Workflow -// Or with more control -await temporal.scheduleWorkflow(cleanupWorkflow, { - scheduleId: 'cleanup', - interval: '1h', // Every hour - jitter: '5m', // Random delay up to 5 minutes - args: [] -}) +``` +WorkflowEngineDO (routing, scheduling) + | + +-- WorkflowDO (execution state per workflow) + | |-- SQLite: Event history, state snapshots + | +-- Alarm: Timer management + | + +-- ActivityDO (retry isolation per activity) + | |-- SQLite: Attempt tracking + | +-- Queues: Task dispatch + | + +-- SchedulerDO (cron, intervals) + |-- SQLite: Schedule definitions + +-- Alarms: Next execution times ``` -## Retry Policies +**Key insight**: Durable Objects provide single-threaded, strongly consistent state per workflow execution. Each workflow gets its own DO for deterministic execution. Activities run in separate DOs with retry isolation. -```typescript -const robustWorkflow = temporal.defineWorkflow( - 'robust-workflow', - async (ctx, data: Data) => { - // Custom retry policy for specific activity - const result = await ctx.activity('risky-operation', () => - riskyOperation(data), - { - retry: { - initialInterval: '1s', - backoffCoefficient: 2, - maximumAttempts: 10, - maximumInterval: '1h', - nonRetryableErrors: ['ValidationError'] - }, - startToCloseTimeout: '10m', - heartbeatTimeout: '1m' - } - ) - - return result - } -) -``` +### Storage Tiers -Default retry policy: -- Initial interval: 1s -- Backoff coefficient: 2 -- Maximum attempts: unlimited -- Maximum interval: 100s +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active workflows, recent history | <10ms | +| **Warm** | R2 + SQLite Index | Completed workflows (30 days) | <100ms | +| **Cold** | R2 Archive | Compliance retention (years) | <1s | -## Worker Setup +## Retry Policies ```typescript -import { Worker } from '@dotdo/temporal' - -// Create worker to process workflows and activities -const worker = new Worker({ - namespace: 'my-app', - taskQueue: 'main', - workflows: [orderWorkflow, userOnboardingWorkflow], - activities -}) +// Automatic retries with sensible defaults +await temporal`charge customer ${customerId}` + .retry({ max: 10, backoff: 'exponential' }) + +// Or specify inline +await temporal`risky operation ${data}` + .retry({ max: 5, interval: '1s', maxInterval: '1h' }) -// Start the worker -export default { - async fetch(request: Request, env: Env) { - return worker.fetch(request, env) - } -} +// Non-retryable errors +await temporal`validate input ${data}` + .noRetry(['ValidationError', 'AuthError']) ``` -## MCP Tools Integration +Default policy: exponential backoff, 1s initial, 2x coefficient, unlimited attempts. -temporal.do exposes MCP tools for AI-native workflow management: +## MCP Tools ```typescript -// AI can start workflows -await mcp.invoke('temporal.startWorkflow', { - workflow: 'order-workflow', - workflowId: 'order-123', - args: [{ productId: 'abc', quantity: 2 }] -}) - -// AI can query state -const status = await mcp.invoke('temporal.query', { - workflowId: 'order-123', - queryType: 'status' -}) - -// AI can send signals -await mcp.invoke('temporal.signal', { - workflowId: 'order-123', - signalName: 'approval', - args: [{ approved: true }] -}) - -// AI can list workflows -const workflows = await mcp.invoke('temporal.list', { - namespace: 'my-app', - status: 'running' -}) +// AI agents can manage workflows directly +await mcp`start order workflow for ${orderId}` +await mcp`what's the status of order ${orderId}?` +await mcp`approve order ${orderId}` +await mcp`list running workflows` + +// Or use structured tools +await mcp.invoke('temporal.start', { workflow: 'order', args: [orderId] }) +await mcp.invoke('temporal.query', { workflowId: orderId, query: 'status' }) +await mcp.invoke('temporal.signal', { workflowId: orderId, signal: 'approve' }) ``` -Integration with fsx.do for workflow state persistence: -```typescript -// Workflow state automatically persisted to fsx.do -await fsx.read('/.temporal/workflows/order-123/history.json') +## vs Traditional Temporal + +```bash +# Before - Traditional Temporal +docker-compose up temporal cassandra elasticsearch +# 47 hours of configuration, debugging, and prayer +# $500/month minimum for servers +# DevOps team on call -// Version control with gitx.do -await gitx.log('/.temporal/workflows/order-123') +# After - temporal.do +npm install temporal.do +# Ship your product +# $0 infrastructure +# Sleep well ``` ## The Rewrites Ecosystem -temporal.do is part of the rewrites family - reimplementations of popular infrastructure on Cloudflare: +temporal.do is part of the rewrites family - popular infrastructure reimplemented on Cloudflare: -| Rewrite | Original | Purpose | +| Rewrite | Replaces | Purpose | |---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [inngest.do](https://inngest.do) | Inngest | Event-driven workflows | -| **temporal.do** | Temporal | Durable workflow orchestration | +| **temporal.do** | Temporal | Durable workflows | +| [inngest.do](https://inngest.do) | Inngest | Event-driven functions | | [kafka.do](https://kafka.do) | Kafka | Event streaming | | [nats.do](https://nats.do) | NATS | Messaging | - -Each rewrite follows the same pattern: -- Durable Objects for state -- SQLite for persistence -- Cloudflare Queues for messaging -- Compatible API with the original +| [fsx.do](https://fsx.do) | fs | Filesystem for AI | +| [gitx.do](https://gitx.do) | git | Version control for AI | ## Why Cloudflare? @@ -488,17 +357,60 @@ Each rewrite follows the same pattern: 2. **No Cold Starts** - Durable Objects stay warm 3. **Unlimited Duration** - No execution timeouts 4. **Built-in Queues** - Reliable task dispatch -5. **Single-Threaded DO** - No race conditions in workflow execution -6. **SQLite + R2** - Event history persistence with infinite retention - -## Related Domains +5. **Single-Threaded DO** - No race conditions +6. **SQLite + R2** - Infinite retention + +## Roadmap + +### Core Workflows +- [x] Natural Language Workflows +- [x] Promise Pipelining +- [x] Activity Retry with Backoff +- [x] Timers and Sleep +- [x] Signals and Queries +- [x] Child Workflows +- [x] Cron Scheduling +- [ ] Workflow Versioning +- [ ] Continue-as-new + +### Durability +- [x] Event History +- [x] Replay Recovery +- [x] Activity Heartbeats +- [ ] Sticky Execution +- [ ] Search Attributes + +### Observability +- [x] Workflow Status +- [x] Activity Tracking +- [ ] OpenTelemetry Export +- [ ] Metrics Dashboard +- [ ] Workflow Visualization + +## Contributing + +temporal.do is open source under the MIT license. -- **workflows.do** - Workflow orchestration -- **inngest.do** - Event-driven functions -- **jobs.do** - Background job queue -- **cron.do** - Scheduled tasks -- **triggers.do** - Event triggers +```bash +git clone https://github.com/dotdo/temporal.do +cd temporal.do +pnpm install +pnpm test +``` ## License MIT + +--- + +

+ 47 seconds beats 47 hours. +
+ Durable workflows. Zero infrastructure. Natural language. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/trigger/README.md b/rewrites/trigger/README.md index 26012842..3997b8b5 100644 --- a/rewrites/trigger/README.md +++ b/rewrites/trigger/README.md @@ -103,70 +103,92 @@ for await (const update of job.progress()) { } ``` -## When You Need Control +## Resilience That Reads Like English -For complex retry policies, concurrency limits, and precise configuration: +Configure retry and concurrency with fluent methods: ```typescript -import { task, schedules } from 'trigger.do' - -// Define a task with full control -export const processVideo = task({ - id: 'process-video', - retry: { maxAttempts: 3, backoff: 'exponential' }, - concurrency: { limit: 10, key: 'payload.userId' }, - run: async (payload: { videoId: string }) => { - const video = await downloadVideo(payload.videoId) - await checkpoint('downloaded', { size: video.size }) - - const encoded = await encodeVideo(video) - await checkpoint('encoded', { format: encoded.format }) - - const uploaded = await uploadToStorage(encoded) - return { url: uploaded.url } - } -}) - -// Scheduled job with cron -export const dailyCleanup = schedules.task({ - id: 'daily-cleanup', - cron: '0 3 * * *', // 3am daily - run: async () => { - await cleanupOldFiles() - await compactDatabase() - } -}) - -// Trigger programmatically -await processVideo.trigger({ videoId: 'abc123' }) +// Retry with exponential backoff +const job = trigger`process video ${videoId}` + .retry(10) + .backoff('exponential') + .jitter() + +// Concurrency per user +await trigger`sync data for ${userId}` + .concurrency(5, 'userId') + .rateLimit(100, '1m') + +// Chain them naturally +await trigger`download, encode, upload ${videoId}` + .retry(3) + .backoff('exponential') + .concurrency(10) + .checkpoint('each-stage') ``` -### Retry Policies +### Full Control, Still Natural ```typescript -task({ - id: 'resilient-task', - retry: { - maxAttempts: 10, // Max retry attempts - backoff: 'exponential', // 'exponential' | 'linear' | 'fixed' - initialDelay: '1s', // First retry delay - maxDelay: '1h', // Maximum delay between retries - factor: 2, // Backoff multiplier - jitter: true // Prevent thundering herd - }, - run: handler -}) +// Video processing with checkpoints +const processed = await trigger`process video ${videoId}` + .retry(3) + .backoff('exponential') + .concurrency(10) + .map(async (video) => { + const downloaded = await downloadVideo(video.id) + return { downloaded, size: downloaded.size } + }) + .checkpoint('downloaded') + .map(({ downloaded }) => encodeVideo(downloaded)) + .checkpoint('encoded') + .map(encoded => uploadToStorage(encoded)) + +// Scheduled cleanup - say it like you mean it +trigger`every day at 3am: cleanup old files, compact database` + +// Or with explicit cron +trigger`0 3 * * *: cleanup and compact` +``` + +### Retry Strategies + +```typescript +// Exponential backoff with jitter +trigger`call flaky API for ${customerId}` + .retry(10) + .backoff('exponential') + .initialDelay('1s') + .maxDelay('1h') + .jitter() + +// Linear backoff for rate limits +trigger`sync to third-party ${service}` + .retry(5) + .backoff('linear') + .initialDelay('30s') + +// Fixed delay for retries +trigger`send webhook to ${endpoint}` + .retry(3) + .backoff('fixed', '10s') ``` -### Concurrency Control +### Concurrency That Makes Sense ```typescript -task({ - id: 'rate-limited', - concurrency: { limit: 10, key: 'payload.orgId' }, - rateLimit: { limit: 100, period: '1m' }, - run: handler -}) +// Per-org concurrency +trigger`import data for ${orgId}` + .concurrency(10, 'orgId') + +// Global rate limit +trigger`call external API` + .rateLimit(100, '1m') + +// Both together +trigger`sync ${customerId} data to ${service}` + .concurrency(5, 'customerId') + .rateLimit(1000, '1h') ``` ## AI-Native Integration @@ -191,85 +213,32 @@ const review = await trigger`review PR ${prNumber} in ${repo}` ## MCP Tools -Full AI agent access to job management: +AI agents manage jobs with natural language: ```typescript -import { defineTool } from 'trigger.do/mcp' - -// List all registered tasks -export const listTasks = defineTool({ - name: 'list_tasks', - description: 'List all registered background tasks', - parameters: {}, - execute: async () => { - return await trigger.tasks.list() - } -}) - -// Get task run status -export const getTaskStatus = defineTool({ - name: 'get_task_status', - description: 'Get the status of a task run', - parameters: { - runId: { type: 'string', description: 'The run ID to check' } - }, - execute: async ({ runId }) => { - return await trigger.runs.get(runId) - } -}) - -// Cancel a running task -export const cancelTask = defineTool({ - name: 'cancel_task', - description: 'Cancel a running task', - parameters: { - runId: { type: 'string', description: 'The run ID to cancel' } - }, - execute: async ({ runId }) => { - return await trigger.runs.cancel(runId) - } -}) - -// List recent runs -export const listRuns = defineTool({ - name: 'list_runs', - description: 'List recent task runs with optional filtering', - parameters: { - taskId: { type: 'string', description: 'Filter by task ID', optional: true }, - status: { type: 'string', description: 'Filter by status', optional: true }, - limit: { type: 'number', description: 'Max results', optional: true } - }, - execute: async ({ taskId, status, limit }) => { - return await trigger.runs.list({ taskId, status, limit }) - } -}) - -// Get execution logs -export const getLogs = defineTool({ - name: 'get_logs', - description: 'Get logs for a task run', - parameters: { - runId: { type: 'string', description: 'The run ID' }, - tail: { type: 'number', description: 'Last N lines', optional: true } - }, - execute: async ({ runId, tail }) => { - return await trigger.runs.logs(runId, { tail }) - } -}) - -// Trigger a task -export const triggerTask = defineTool({ - name: 'trigger_task', - description: 'Trigger a background task with payload', - parameters: { - taskId: { type: 'string', description: 'Task identifier' }, - payload: { type: 'object', description: 'Task payload' } - }, - execute: async ({ taskId, payload }) => { - const handle = await trigger.tasks.trigger(taskId, payload) - return { runId: handle.id, status: 'triggered' } - } -}) +import { trigger } from 'trigger.do' + +// List jobs +await trigger`list all running jobs` +await trigger`show failed jobs from yesterday` +await trigger`what's queued for video processing?` + +// Monitor jobs +await trigger`status of job ${runId}` +await trigger`logs for ${runId}` +await trigger`why did ${runId} fail?` + +// Control jobs +await trigger`cancel ${runId}` +await trigger`retry all failed jobs from today` +await trigger`pause video processing queue` + +// Bulk operations +await trigger`failed jobs this week` + .map(job => trigger`retry ${job}`) + +await trigger`stuck jobs older than 1 hour` + .map(job => trigger`cancel and restart ${job}`) ``` ## Architecture diff --git a/rewrites/workday/README.md b/rewrites/workday/README.md index 9b904db8..7c75fb1e 100644 --- a/rewrites/workday/README.md +++ b/rewrites/workday/README.md @@ -1,72 +1,95 @@ # workday.do -> Enterprise HCM. AI-native. Deploy in minutes, not months. +> Enterprise HCM. Edge-Native. Open by Default. AI-First. -## The Problem +Workday charges enterprises millions for implementations, locks them into proprietary systems, and treats AI as an afterthought. Implementation takes 6-18 months. Licensing costs $100K+ annually. The system that manages your people requires its own team to manage. -Workday changed the world by moving HR to the cloud. But that was 2005. +**workday.do** is the open-source alternative. Deploy in minutes, not months. AI-native from day one. Your data, your infrastructure. -Today: +## AI-Native API -- **Implementation takes 6-18 months** - Consultants, configuration, data migration -- **Costs $100K+ annually** - Enterprise licensing, per-employee fees, support contracts -- **AI is a bolt-on** - "Coming soon" features, additional SKUs, integration headaches -- **Complexity as a moat** - The system that manages your people shouldn't need its own team to manage +```typescript +import { workday } from 'workday.do' // Full SDK +import { workday } from 'workday.do/tiny' // Minimal client +import { workday } from 'workday.do/payroll' // Payroll-only operations +``` -The irony? Workday was founded to be simpler than PeopleSoft. Twenty years later, it *is* PeopleSoft. +Natural language for HR workflows: -## The Solution +```typescript +import { workday } from 'workday.do' -**workday.do** is open-source Human Capital Management that runs on Cloudflare Workers. +// Talk to it like a colleague +await workday`hire Alex Chen as Software Engineer in Engineering at $150k starting Jan 15` +await workday`promote Alex to Senior Engineer at $165k effective March 1` +await workday`Alex's vacation balance` -- **Deploy in minutes** - Not months -- **AI-native from day one** - Not a feature flag -- **Your data, your infrastructure** - Not their cloud, their terms -- **Enterprise capabilities** - Without enterprise complexity +// Chain like sentences +await workday`engineers needing reviews` + .notify(`Your performance review is due`) -```bash -npx create-dotdo workday +// Requests that route themselves +await workday`request 40 hours vacation for Alex Chen Feb 17-21` + .route('manager-approval') + .onApprove(async () => await workday`block Alex's calendar`) ``` -That's it. You now have: +## The Problem + +Workday changed the world by moving HR to the cloud. But that was 2005. + +| What Workday Charges | The Reality | +|---------------------|-------------| +| **Implementation** | $500K-5M+ (6-18 month projects) | +| **Annual Licensing** | $100K-1M+ per year | +| **Per-Employee Fees** | $50-200 per employee per month | +| **AI Features** | Additional SKUs, integration headaches | +| **Customization** | $300/hour consultants | +| **Vendor Lock-in** | Decades of data trapped | + +### The Complexity Tax -- Employee records with effective dating -- Organizational hierarchies -- Compensation management -- Time off tracking -- Business process workflows -- AI HR assistant +The irony? Workday was founded to be simpler than PeopleSoft. Twenty years later, it *is* PeopleSoft: -All running on edge infrastructure you control. +- Implementation projects measured in years +- Dedicated "Workday admins" required +- Bolt-on AI sold separately +- Configuration complexity as a moat + +## The Solution + +**workday.do** reimagines HCM for the AI era: + +``` +Workday workday.do +----------------------------------------------------------------- +$500K-5M implementation Deploy in minutes +$100K+/year maintenance $0 - run your own +18-month implementations npm install && deploy +Bolt-on AI features AI-native from day one +Workday's cloud lock-in Your Cloudflare account +Consultants for everything Code-first, instant deploy +Per-employee licensing Open source, MIT licensed +``` ## One-Click Deploy ```bash -# Create your HCM instance npx create-dotdo workday - -# Or clone and customize -git clone https://github.com/dotdo/workday.do -cd workday.do -npm install -npm run deploy ``` -Your Workday alternative is live. Add your first employee: +An enterprise HCM. Running on infrastructure you control. AI-native from day one. ```typescript -import { hr } from 'workday.do' - -await hr.workers.hire({ - name: 'Alex Chen', - position: 'Software Engineer', - organization: 'Engineering', - startDate: '2025-01-15', - compensation: { - salary: 150000, - currency: 'USD', - frequency: 'annual' - } +import { Workday } from 'workday.do' + +export default Workday({ + name: 'acme-corp', + domain: 'hr.acme.com', + features: { + effectiveDating: true, + aiAssistant: true, + }, }) ``` @@ -74,153 +97,129 @@ await hr.workers.hire({ ### Workers -The heart of any HCM. Not "employees" - **workers**. Because the future includes contractors, AI agents, and work arrangements we haven't invented yet. - ```typescript -// Every worker has a complete history -const alex = await hr.workers.get('alex-chen') +// Find anyone +const alex = await workday`Alex Chen` +const engineers = await workday`engineers in Austin` +const managers = await workday`managers with more than 5 direct reports` + +// AI infers what you need +await workday`Alex Chen` // returns worker +await workday`Alex's compensation` // returns comp details +await workday`Alex's full history` // returns complete record +``` -// See their current state -alex.position // Software Engineer -alex.organization // Engineering -alex.manager // Sarah Kim -alex.compensation // $150,000/year +### Hiring -// Or as of any date -const alexLastYear = await hr.workers.get('alex-chen', { asOf: '2024-01-01' }) +```typescript +// Hire naturally +await workday`hire Alex Chen as Software Engineer in Engineering at $150k starting Jan 15` +await workday`hire Maria Santos as contractor, product design, $100/hour` +await workday`onboard Alex with standard engineering checklist` + +// Batch hiring reads like a roster +await workday` + hire for Engineering starting Feb 1: + - Sarah Kim, Senior Engineer, $180k + - James Liu, Staff Engineer, $220k + - Priya Patel, Engineering Manager, $200k +` ``` ### Positions -Jobs exist independent of people. Positions are the boxes on the org chart - workers fill them. - ```typescript -await hr.positions.create({ - title: 'Senior Software Engineer', - organization: 'Engineering', - level: 'IC4', - headcount: 3, // Budget for 3 - filled: 2, // 2 currently filled - compensationRange: { - min: 140000, - max: 200000, - currency: 'USD' - } -}) +// Create positions naturally +await workday`create Senior Software Engineer position in Engineering, level IC4, budget 3 headcount, $140k-200k range` +await workday`open positions in Engineering` +await workday`unfilled roles this quarter` ``` ### Organizations -Hierarchies that actually work. Teams within teams within teams - traversable, queryable, time-aware. - ```typescript -// Get the entire org tree -const company = await hr.orgs.tree() - -// Find all engineers (including sub-orgs) -const engineering = await hr.orgs.get('engineering') -const allEngineers = await engineering.allWorkers() // Recursive - -// See the org as it was -const orgLastQuarter = await hr.orgs.tree({ asOf: '2024-10-01' }) +// Query the org naturally +await workday`org chart` +await workday`Engineering team` +await workday`everyone in Engineering including sub-teams` +await workday`who reports to Sarah Kim` +await workday`org chart as of last quarter` ``` ### Compensation -Total rewards, not just salary. Base, bonus, equity, benefits - all versioned, all auditable. - ```typescript -await hr.compensation.adjust({ - worker: 'alex-chen', - effectiveDate: '2025-03-01', - changes: { - baseSalary: 165000, // Promotion raise - bonus: { target: 15 }, // 15% target bonus - equity: { - grant: 5000, // RSU grant - vestingSchedule: '4-year-1-cliff' - } - }, - reason: 'promotion', - approvedBy: 'sarah-kim' -}) +// Adjust comp naturally +await workday`give Alex a raise to $165k effective March 1` +await workday`Alex promotion to Senior with 15% bonus target and 5000 RSUs` +await workday`market adjustment for all engineers, 5% increase July 1` + +// Review compensation +await workday`Alex's total comp` +await workday`engineering comp by level` +await workday`who's below market in Austin` ``` ### Time Off -Accruals, balances, requests, approvals. No spreadsheets required. - ```typescript -// Check balance -const pto = await hr.timeOff.balance('alex-chen', 'vacation') -// { accrued: 120, used: 40, available: 80, unit: 'hours' } +// Check balance naturally +await workday`how much PTO does Alex have` +await workday`Alex's vacation balance` // Request time off -await hr.timeOff.request({ - worker: 'alex-chen', - type: 'vacation', - start: '2025-02-17', - end: '2025-02-21', - hours: 40, - notes: 'Family vacation' -}) -// Automatically routes to manager for approval +await workday`request vacation for Alex Feb 17-21` + .route('manager-approval') + .onApprove(async () => await workday`block Alex's calendar`) + +// Manage leave +await workday`who's out next week` +await workday`Engineering PTO calendar for March` ``` ### Recruiting -Open requisitions, candidates, interview workflows. The pipeline before the payroll. - ```typescript -await hr.recruiting.openReq({ - position: 'senior-software-engineer-001', - hiringManager: 'sarah-kim', - targetStartDate: '2025-04-01', - interviewPlan: ['recruiter-screen', 'tech-phone', 'onsite', 'offer'] -}) +// Open reqs naturally +await workday`open Senior Engineer req for Sarah's team, start April 1` +await workday`open positions in Engineering` +await workday`candidates in pipeline for senior engineer` + +// Move candidates through +await workday`schedule Alex Chen for onsite with Sarah's team` +await workday`extend offer to Alex Chen for Senior Engineer at $175k` ``` ### Performance -Goals, reviews, feedback. Continuous performance management, not annual paperwork. - ```typescript -await hr.performance.setGoals('alex-chen', { - period: '2025-H1', - goals: [ - { objective: 'Ship authentication system', keyResults: ['...'], weight: 40 }, - { objective: 'Mentor two junior engineers', keyResults: ['...'], weight: 30 }, - { objective: 'Reduce API latency by 50%', keyResults: ['...'], weight: 30 } - ] -}) +// Set goals naturally +await workday`Alex's goals for H1: ship auth system, mentor 2 junior engineers, reduce API latency 50%` +await workday`who's missing goals for this quarter` + +// Reviews +await workday`start Q1 performance reviews for Engineering` +await workday`Alex's review history` +await workday`outstanding reviews for Sarah's team` ``` ## Effective Dating -This is the superpower most HR systems lack (or charge enterprise prices for). - -**Every change in workday.do is versioned with an effective date.** Not just "when it was entered" - when it takes effect. +Every change is versioned. Not just "when it was entered" - when it takes effect. ```typescript -// Schedule a future promotion -await hr.workers.update('alex-chen', { - effectiveDate: '2025-03-01', // Takes effect March 1st - position: 'Senior Software Engineer', - compensation: { salary: 165000 } -}) - -// The change is recorded now, but... -const alexToday = await hr.workers.get('alex-chen') -alexToday.position // 'Software Engineer' (still) - -const alexMarch = await hr.workers.get('alex-chen', { asOf: '2025-03-01' }) -alexMarch.position // 'Senior Software Engineer' (future state) - -// Time travel through your entire org -const orgHistory = await hr.orgs.history('engineering', { - from: '2024-01-01', - to: '2025-12-31' -}) +// Schedule future changes +await workday`promote Alex to Senior Engineer at $165k effective March 1` + +// Query point-in-time +await workday`Alex's position` // Software Engineer (now) +await workday`Alex's position as of March 1` // Senior Software Engineer (future) +await workday`who was Alex's manager last January` + +// Time travel through your org +await workday`Engineering headcount over the last year` +await workday`org changes between Q1 and Q3 2024` +await workday`who reported to Sarah on March 15, 2024` ``` **Why this matters:** @@ -230,49 +229,58 @@ const orgHistory = await hr.orgs.history('engineering', { - **Compliance** - Answer "who reported to whom on March 15th, 2024?" - **Analytics** - Accurate point-in-time headcount, compensation, structure -## AI HR Assistant +## AI-Native HR Your HR team gets an AI colleague on day one. ```typescript -import { ada } from 'workday.do/agents' - // Employees ask questions naturally -await ada`How much PTO do I have left?` -// "You have 80 hours of vacation remaining. Your next accrual of 6.67 hours is on February 1st." +await workday`how much PTO do I have` +// "You have 80 hours of vacation remaining. Your next accrual is February 1st." + +await workday`I need next Friday off` +// Creates time-off request, routes to manager for approval -await ada`I need to take next Friday off` -// "I've created a time-off request for Friday, January 17th (8 hours vacation). -// This has been sent to Sarah Kim for approval." +await workday`what's the process for referring a candidate` +// Returns process with links, offers to start a referral -await ada`What's the process for referring a candidate?` -// "To refer a candidate: [detailed process with links] -// Would you like me to start a referral for a specific person?" +await workday`explain my benefits` +// Personalized benefits summary based on their elections ``` -**AI-powered workflows:** +### AI-Powered Workflows + +```typescript +// Onboarding that handles itself +await workday`onboard Alex Chen starting Jan 15` + .guide() // AI walks them through paperwork + .introduce() // schedules meet-the-team + .setup() // IT provisioning, badge, etc. + +// Offboarding that misses nothing +await workday`offboard Sarah Kim last day Feb 28` + .checklist() // ensures complete handoff + .exit() // schedules exit interview + .process() // handles all systems access + +// Manager support +await workday`help me write Alex's performance review` +await workday`how do I handle a difficult conversation about performance` +await workday`attrition trends in Engineering over the last year` +``` -- **Onboarding** - Ada guides new hires through paperwork, introductions, setup -- **Offboarding** - Ensures nothing falls through the cracks -- **Policy questions** - Instant answers, not support tickets -- **Manager support** - Helps with reviews, promotions, difficult conversations -- **Analytics** - "Show me attrition trends in Engineering over the last year" +### Population Health for HR ```typescript -// Configure your AI assistant -await hr.config.ai({ - assistant: { - name: 'Ada', - personality: 'helpful, professional, slightly warm', - knowledgeSources: ['handbook', 'policies', 'benefits-guide'], - escalateTo: 'hr-team@company.com', - capabilities: { - createTimeOffRequests: true, - answerCompensationQuestions: false, // Sensitive - human only - scheduleOnboarding: true - } - } -}) +// Query your workforce like a database +await workday`flight risk in Engineering` +await workday`employees with no manager meeting in 30 days` +await workday`contractors approaching 18-month limit` + +// Close HR gaps at scale +await workday`employees missing emergency contacts` + .outreach() // personalized reminders + .track() // compliance reporting ``` ## Enterprise Grade @@ -281,21 +289,14 @@ Open-source doesn't mean toy. ### Audit Trails -Every action is logged. Who changed what, when, why. - ```typescript -const history = await hr.audit.query({ - entity: 'worker', - entityId: 'alex-chen', - from: '2024-01-01' -}) -// [{ -// timestamp: '2024-03-01T00:00:00Z', -// action: 'compensation.adjust', -// actor: 'sarah-kim', -// changes: { baseSalary: { from: 140000, to: 165000 } }, -// reason: 'promotion' -// }, ...] +// Query audit naturally +await workday`who changed Alex's compensation and when` +await workday`all changes to Engineering org this quarter` +await workday`audit trail for Alex since January` + +// AI surfaces anomalies +await workday`unusual changes this month` ``` ### Role-Based Security @@ -303,16 +304,10 @@ const history = await hr.audit.query({ Fine-grained permissions. Managers see their teams. HR sees everyone. Employees see themselves. ```typescript -await hr.security.defineRole('manager', { - workers: { - read: 'direct-reports', // Only their team - update: 'direct-reports', // Can update their reports - compensation: 'view-only' // Can see but not change - }, - timeOff: { - approve: 'direct-reports' // Approve their team's requests - } -}) +// Query access naturally +await workday`what can Sarah see` +await workday`who has access to compensation data` +await workday`break glass access log this month` ``` ### Business Process Flows @@ -320,16 +315,17 @@ await hr.security.defineRole('manager', { Complex approvals made simple. Promotions, transfers, terminations - all with proper routing. ```typescript -await hr.workflows.define('promotion', { - trigger: 'compensation.adjust where reason = promotion', - steps: [ - { actor: 'manager', action: 'initiate' }, - { actor: 'hr-partner', action: 'review' }, - { actor: 'comp-team', action: 'approve', condition: 'change > 20%' }, - { actor: 'vp', action: 'approve', condition: 'new-level >= director' } - ], - onComplete: 'notify worker, update systems' -}) +// Approvals route themselves +await workday`promote Alex to Senior at $165k` + .route('manager') // Sarah initiates + .route('hr-partner') // HR reviews + .route('comp-team') // comp approves (if >20% change) + .route('vp') // VP approves (if director+) + .onComplete(async () => await workday`notify Alex`) + +// Check approval status +await workday`pending approvals for Sarah` +await workday`where is Alex's promotion in the workflow` ``` ### Compliance Ready @@ -343,60 +339,164 @@ Built for the regulatory reality of HR. ## Architecture -workday.do is built on Cloudflare Durable Objects - the same technology powering real-time collaboration at scale. +### Durable Object per Organization ``` -WorkerDO - Individual employee record - | Bi-temporal data (effective date + transaction time) - | Complete employment history +CompanyDO (config, branding, policies) | -PositionDO - Job definition - | Headcount, compensation bands, requirements + +-- WorkersDO (demographics, employment) + | |-- SQLite: Worker records (encrypted) + | +-- R2: Documents, photos (encrypted) | -OrganizationDO - Org unit (team, department, division) - | Hierarchy traversal, effective-dated structure + +-- OrgsDO (structure, hierarchy) + | |-- SQLite: Org units, positions + | +-- Effective-dated changes | -CompensationDO - Compensation record - | All components, versioned + +-- CompensationDO (payroll, benefits) + | |-- SQLite: Comp data (encrypted) | -TimeOffDO - Leave balances and requests - | Accrual rules, approval workflows + +-- TimeOffDO (accruals, balances, requests) + | |-- SQLite: Leave records | -WorkflowDO - Business process instance - Multi-step approvals, routing + +-- WorkflowsDO (approvals, routing) + |-- SQLite: Process instances + +-- State machines ``` -**Bi-temporal data model:** +### Bi-Temporal Data Model ```typescript // Every record has two time dimensions -{ - effectiveDate: '2025-03-01', // When it takes effect in reality - transactionTime: '2025-01-15', // When it was recorded in the system +// "What was true on March 1st?" (as-of query) +await workday`Alex's manager as of March 1` + +// "What did we know on Jan 15th?" (as-at query) +await workday`Alex's record as we knew it on Jan 15` + +// "What did we think on Jan 15th would be true on March 1st?" (bi-temporal) +await workday`Alex's planned promotion as recorded Jan 15` +``` + +### Storage Tiers + +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active employees, recent history | <10ms | +| **Warm** | R2 + SQLite Index | Terminated employees (2-7 years) | <100ms | +| **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | + +## vs Workday + +| Feature | Workday | workday.do | +|---------|---------|-----------| +| **Implementation** | $500K-5M+ | Deploy in minutes | +| **Annual Cost** | $100K-1M+ | ~$100/month | +| **Timeline** | 6-18 months | Same day | +| **AI** | Bolt-on, additional SKU | AI-native foundation | +| **Data Location** | Workday's cloud | Your Cloudflare account | +| **Customization** | $300/hour consultants | Code it yourself | +| **Effective Dating** | Enterprise tier | Included | +| **Updates** | Bi-annual releases | Continuous deployment | +| **Lock-in** | Decades of migration | MIT licensed | + +## Use Cases + +### Self-Service Portal + +```typescript +// Employees help themselves +await workday`my PTO balance` +await workday`request Friday off` +await workday`update my address to 123 Main St` +await workday`my pay stubs` +``` + +### Manager Dashboard + +```typescript +// Managers manage +await workday`my team` +await workday`pending approvals` +await workday`who's out this week` +await workday`compensation review for my team` +``` - // This enables: - // - "What was true on March 1st?" (as-of query) - // - "What did we think was true on Jan 15th?" (as-at query) - // - "What did we think on Jan 15th would be true on March 1st?" (bi-temporal) -} +### HR Analytics + +```typescript +// HR gets insights +await workday`headcount by department` +await workday`attrition trends this year` +await workday`time to fill by role type` +await workday`diversity metrics for Engineering` ``` -**Storage tiers:** +### Payroll Integration + +```typescript +// Seamless payroll prep +await workday`payroll changes this period` +await workday`export to ADP` +await workday`employees with tax form changes` +``` + +## Why Open Source for HCM? + +### 1. Cost Liberation -- **Hot (SQLite in DO)** - Active employees, recent history -- **Warm (R2)** - Terminated employees, older history -- **Cold (R2 Archive)** - Compliance retention, rarely accessed +$500K-5M implementations are resources diverted from your people. Open source means: +- Minutes to deploy, not months +- No implementation consultants +- No per-employee licensing +- No vendor lock-in -## Why This Exists +### 2. AI Enablement -We believe: +Closed HCM systems control what AI you can use. Open source means: +- Integrate any LLM +- Build custom HR automation +- Reduce administrative burden +- Natural language for everything -1. **HR software should be accessible to all companies** - Not just those with $100K+ budgets -2. **AI should be native, not an add-on** - The future of HR is AI-assisted -3. **Your people data is yours** - Not locked in a vendor's cloud -4. **Open source wins** - The best software is built in the open +### 3. Your People Data is Yours -Workday (the company) did something important - they proved cloud HR could work. Now it's time for the next evolution: open, AI-native, deployable anywhere. +HR data is sensitive. Open source enables: +- True data portability +- Deploy where you need (sovereignty) +- No vendor data mining +- Full audit control + +### 4. Innovation Velocity + +HCM moves slowly because vendors profit from complexity. Open source enables: +- HR teams to influence development +- Startups to integrate without approval +- Organizations to customize for their needs + +## Deployment Options + +### Cloudflare Workers + +```bash +npx create-dotdo workday +# Deploys to your Cloudflare account +``` + +### Private Cloud + +```bash +docker run -p 8787:8787 dotdo/workday +# Or Kubernetes +kubectl apply -f workday-do.yaml +``` + +### On-Premises + +For organizations requiring complete control: + +```bash +./workday-do-install.sh --on-premises --employee-count=500 +``` ## Contributing @@ -405,16 +505,23 @@ workday.do is open source under the MIT license. ```bash git clone https://github.com/dotdo/workday.do cd workday.do -npm install -npm test +pnpm install +pnpm test ``` -See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - ## License -MIT - Use it, fork it, build on it. +MIT License - For the people who manage people. --- -**Workers work for you.** Even the ones managing your workers. +

+ The enterprise HCM monopoly ends here. +
+ AI-native. Effective-dated. Your data. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/zapier/README.md b/rewrites/zapier/README.md index a7cfa145..3d0a8143 100644 --- a/rewrites/zapier/README.md +++ b/rewrites/zapier/README.md @@ -1,28 +1,92 @@ # zapier.do -Automation that speaks your language. Natural language triggers, actions, and workflows on Cloudflare. +> Automation that speaks your language. Natural language triggers, actions, and workflows on Cloudflare. -## The Hero +Zapier charges $600/month for 50,000 tasks. Your startup scales, your automation bill scales faster. Every webhook, every sync, every notification - metered and monetized. You're paying a tax on your own success. -**For startup founders who need automation without the $600/month Zapier bill.** +**zapier.do** is the open-source alternative. Unlimited automations. Edge-native. AI-first. Just tell it what you want. -You're building a startup. You need to connect your apps - Stripe to Slack, GitHub to Notion, webhooks to everywhere. Zapier wants $600/month for 50,000 tasks. You want to focus on your product, not your automation bill. +## AI-Native API -**zapier.do gives you unlimited automation at edge speed.** No per-task pricing. No vendor lock-in. Just tell it what you want. +```typescript +import { zapier } from 'zapier.do' // Full SDK +import { zapier } from 'zapier.do/tiny' // Minimal client +import { zapier } from 'zapier.do/triggers' // Trigger-only operations +``` -## Natural Language Automation +Natural language for automation workflows: ```typescript import { zapier } from 'zapier.do' -zapier`when new user signs up, add to Salesforce and send welcome email` -zapier`every morning at 9am, sync Stripe charges to Google Sheet` -zapier`when GitHub issue closes, post to Slack #shipped` +// Talk to it like a colleague +zapier`when Stripe payment: add to HubSpot, notify #sales` +zapier`every morning at 9am: sync charges to Google Sheet` +zapier`when GitHub issue closes: post to #shipped` + +// Chain like sentences +await zapier`new signup` + .map(user => salesforce`create contact ${user}`) + .map(contact => email`send welcome to ${contact}`) ``` -That's it. No drag-and-drop UI. No JSON configs. Just say what you need. +## The Problem + +Zapier dominates automation, but at what cost? + +| What Zapier Charges | The Reality | +|---------------------|-------------| +| **Free Tier** | 100 tasks/month (gone in a day) | +| **Starter** | $29/month for 750 tasks | +| **Professional** | $99/month for 2,000 tasks | +| **Team** | $399/month for 50,000 tasks | +| **Company** | $799+/month for unlimited | +| **Per-Task Overages** | $0.01-0.05 per task beyond limit | + +### The Task Tax + +Every automation run is a "task." One Zap with 5 steps = 5 tasks. A 1000-customer webhook = 1000 tasks. Your cost scales with your success. + +### The Lock-in Problem + +- Proprietary WYSIWYG interface - no code, no version control +- Can't export your Zap logic +- Complex workflows become unmaintainable spaghetti +- No AI agents - humans click forever + +### The Latency Problem + +- Cold starts on every trigger +- Multi-step Zaps add latency +- No edge distribution +- Enterprise features behind paywalls + +## The Solution + +**zapier.do** reimagines automation for developers and AI: + +``` +Zapier zapier.do +----------------------------------------------------------------- +$600/month for 50k tasks $0 - unlimited on Cloudflare +Proprietary WYSIWYG Code-first, version controlled +Cold starts every run Edge-native, no cold starts +Human-only clicks AI-native from day one +No customization Full TypeScript control +Vendor lock-in Open source, MIT licensed +``` -## Promise Pipelining +## One-Click Deploy + +```bash +npx create-dotdo zapier +``` + +Unlimited automations. Running on your Cloudflare account. AI-native from day one. + +## Features + +### Promise Pipelining Chain automations with `.map()` - one network round trip: @@ -46,7 +110,7 @@ const onboarding = await zapier`user signed up` .map(results => analytics`track onboarding_started ${results}`) ``` -## Agent Integration +### Agent Integration Let your AI agents set up automation for you: @@ -63,150 +127,106 @@ priya`create automation for our lead qualification pipeline` ralph`when deal closes in Pipedrive, trigger celebration in Slack and update forecast sheet` ``` -## AI-Native (MCP Tools) +### AI-Native (MCP Tools) -zapier.do is built for AI agents, not just humans. Full MCP integration means any AI can: +zapier.do is built for AI agents, not just humans. Full MCP integration - AI speaks natural language too: ```typescript -// AI agents can discover and execute automations -const tools = await zapier.mcp.listTools() - -// Create automations programmatically -await zapier.mcp.call('createAutomation', { - trigger: 'webhook:order/created', - actions: ['stripe:charge', 'email:confirm', 'slack:notify'] -}) - -// Execute with context -await zapier.mcp.call('runZap', { - name: 'orderFlow', - context: { orderId: '123', customer: 'jane@startup.com' } -}) +// AI agents create automations the same way +await zapier`when order created: charge Stripe, confirm email, notify #orders` + +// AI can discover what's available +await zapier`list my automations` +await zapier`show triggers for Stripe` +await zapier`what apps are connected?` + +// AI can manage automations naturally +await zapier`pause the order processing automation` +await zapier`enable all Slack notifications` +await zapier`delete the old lead routing zap` ``` Works with fsx.do and gitx.do for file and version control operations: ```typescript -zapier`when PR merges, write changelog to /docs/changelog.md and commit` -zapier`daily at midnight, export analytics to /reports/daily.jsonl` +zapier`when PR merges: write changelog to /docs/changelog.md, commit` +zapier`daily at midnight: export analytics to /reports/daily.jsonl` +zapier`when deploy succeeds: update /status/health.json, notify #ops` ``` -## The Stakes +### Triggers -**Without zapier.do:** -- $600/month Zapier bill that scales with your success -- Vendor lock-in to proprietary WYSIWYG interface -- Cold starts and latency in critical workflows -- No customization beyond drag-and-drop -- AI agents can't automate for you +Natural language triggers - just say when: -**With zapier.do:** -- Unlimited automations on Cloudflare's free tier -- Define in code, version control, deploy anywhere -- Global edge execution, no cold starts -- Full programmatic control -- AI-native from day one +```typescript +// Webhooks - instant events +zapier`when order created: charge Stripe, email confirmation, notify #orders` +zapier`when lead submitted: add to HubSpot, start drip sequence` +zapier`when payment failed: notify #billing, email customer` + +// Schedules - say the time +zapier`every morning at 9am: sync Stripe to Google Sheet` +zapier`every Friday at 5pm: send weekly metrics to #team` +zapier`first Monday of month: generate MRR report` + +// Polling - periodic checks +zapier`check every 5 minutes: new items from API, post to Slack if any` +``` -## When You Need Control +### Filters and Routing -For complex conditional logic, use the structured API: +Natural language conditions - if you can say it, it works: ```typescript -import { Zapier } from 'zapier.do' - -const zapier = new Zapier({ id: 'my-automation' }) - -export const orderProcessing = zapier.createZap({ - name: 'Order Processing', - trigger: { - app: 'webhook', - event: 'order/created', - filters: [{ field: 'data.total', condition: 'greater_than', value: 100 }] - }, - actions: [ - { - name: 'chargePayment', - app: 'stripe', - action: 'createCharge', - inputs: { - amount: '{{trigger.data.total}}', - currency: 'usd' - } - }, - { - name: 'sendConfirmation', - app: 'email', - action: 'send', - inputs: { - to: '{{trigger.data.email}}', - template: 'order-confirmation' - } - }, - { - name: 'notifyTeam', - app: 'slack', - action: 'postMessage', - inputs: { - channel: '#orders', - text: 'New order: ${{trigger.data.total}} from {{trigger.data.email}}' - } - } - ] -}) +// Conditional routing with natural language +zapier`when lead created: if score > 50, route to #sales-enterprise` +zapier`when lead created: if company size = startup, route to #sales-startup` + +// Multi-path routing +zapier`when deal updated: + if stage = won, celebrate in #wins and update forecast + if stage = lost, log reason and notify manager` + +// Value-based filtering +zapier`when order received: if total > $100, process premium else standard` +zapier`when signup: if enterprise plan, assign to account team` ``` -### Triggers +### Order Processing ```typescript -// Webhook - instant -{ app: 'webhook', event: 'incoming', config: { path: '/hooks/orders' } } - -// Schedule - cron syntax -{ app: 'schedule', event: 'cron', config: { expression: '0 9 * * *' } } - -// Polling - check periodically -{ app: 'poll', event: 'new-item', config: { url: 'https://api.example.com/items', interval: '5m' } } +// The dictation test: a no-code builder could say this naturally +zapier`when order > $100: charge Stripe, send confirmation, notify #orders` + +// Multi-step with context +zapier`when new order: + charge customer + send receipt email + update inventory + if total > $500, notify #big-orders` ``` -### Filters and Paths +### Lead Qualification ```typescript -export const leadRouting = zapier.createZap({ - name: 'Lead Routing', - trigger: { app: 'webhook', event: 'lead/created' }, - actions: [ - { - name: 'filterHighValue', - app: 'filter', - action: 'only_continue_if', - inputs: { - conditions: [{ field: 'trigger.data.score', condition: 'greater_than', value: 50 }] - } - }, - { - name: 'routeBySize', - app: 'path', - action: 'switch', - inputs: { - field: 'trigger.data.companySize', - cases: { - 'enterprise': [{ app: 'slack', action: 'postMessage', inputs: { channel: '#sales-enterprise' } }], - 'startup': [{ app: 'slack', action: 'postMessage', inputs: { channel: '#sales-startup' } }] - } - } - } - ] -}) +// Route leads by score and size +zapier`when lead created: if score > 50, route enterprise to #sales-enterprise` +zapier`when lead created: if score <= 50, add to nurture sequence` + +// Company-based routing +zapier`when lead: + if Fortune 500, assign to enterprise team + if startup, assign to SMB team + else assign to general queue` ``` -### Template Expressions +### Data Sync ```typescript -'{{trigger.data.email}}' // Access trigger data -'{{steps.chargePayment.chargeId}}' // Previous step output -'{{formatDate(trigger.timestamp, "YYYY-MM-DD")}}' // Date formatting -'{{json(trigger.data)}}' // JSON serialization -'{{math(trigger.data.price * 0.9)}}' // Calculations +// Cross-app sync in plain English +zapier`when Stripe payment: add customer to HubSpot, log in Google Sheet` +zapier`when HubSpot deal closes: update Salesforce opportunity, notify Slack` +zapier`every hour: sync new Notion tasks to Linear` ``` ## Architecture @@ -273,6 +293,21 @@ zapier.do is part of the rewrites family - popular services reimplemented on Clo | **zapier.do** | Zapier | Automation for AI | | [kafka.do](https://kafka.do) | Kafka | Event streaming for AI | +## vs Zapier + +| Feature | Zapier | zapier.do | +|---------|--------|-----------| +| **Pricing** | $29-799/month, per-task | $0 - unlimited | +| **Free Tier** | 100 tasks/month | Unlimited | +| **Interface** | Drag-and-drop only | Natural language + code | +| **Version Control** | None | Git-native | +| **Cold Starts** | Yes, every trigger | None (Durable Objects) | +| **AI Integration** | None | AI-native from day one | +| **Customization** | Limited templates | Full TypeScript control | +| **Data Location** | Zapier's servers | Your Cloudflare account | +| **Export** | Not supported | Open source, MIT licensed | +| **Edge Distribution** | US-only | Global (300+ locations) | + ## Related Domains - **workflows.do** - Workflow orchestration @@ -282,4 +317,17 @@ zapier.do is part of the rewrites family - popular services reimplemented on Clo ## License -MIT +MIT License - Automate without limits. + +--- + +

+ The $600/month bill ends here. +
+ Edge-native. AI-first. Code-first. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/zendesk/README.md b/rewrites/zendesk/README.md index 689c06be..1fd1760b 100644 --- a/rewrites/zendesk/README.md +++ b/rewrites/zendesk/README.md @@ -4,6 +4,34 @@ Your own Zendesk. One click. Unlimited agents. AI that actually resolves tickets. +## AI-Native API + +```typescript +import { zendesk } from 'zendesk.do' // Full SDK +import { zendesk } from 'zendesk.do/tiny' // Minimal client +import { zendesk } from 'zendesk.do/triggers' // Automation-only +``` + +Natural language for support operations: + +```typescript +import { zendesk } from 'zendesk.do' + +// Talk to it like a support manager +const urgent = await zendesk`urgent tickets from enterprise customers` +const stale = await zendesk`pending tickets not updated in 72 hours` +const angry = await zendesk`tickets with negative sentiment` + +// Chain like sentences +await zendesk`pending tickets not updated in 72 hours` + .email('follow-up') + .tag('follow-up-sent') + +// Tickets resolve themselves +await zendesk`auto-resolve tickets with 85% confidence` +await zendesk`when ticket from vip is new: priority urgent, route to vip-support` +``` + ## The Problem Zendesk changed the game with modern help desk software. Then they locked it behind per-agent pricing: @@ -39,9 +67,17 @@ npx create-dotdo zendesk Your help desk. Running on Cloudflare. Ready for tickets. -```bash -# Or deploy to an existing workers.do project -npx dotdo add zendesk +```typescript +import { Zendesk } from 'zendesk.do' + +export default Zendesk({ + name: 'acme-support', + domain: 'support.acme.com', + ai: { + autoResolve: true, + confidence: 0.85, + }, +}) ``` ## Features @@ -50,18 +86,39 @@ Everything you expect from a modern help desk: ### Tickets & Views -- Unified inbox across all channels -- Custom views with saved filters -- Ticket fields, tags, priorities -- Collision detection for concurrent edits -- Satisfaction surveys (CSAT) +```typescript +// Find tickets naturally +const vip = await zendesk`tickets from enterprise customers` +const urgent = await zendesk`urgent unassigned tickets` +const mine = await zendesk`my open tickets` + +// AI infers what you need +await zendesk`ticket 12345` // returns ticket +await zendesk`comments on ticket 12345` // returns conversation +await zendesk`similar tickets to 12345` // returns related issues +``` -### Triggers & Automations +### Triggers & Rules -- **Triggers**: Fire on ticket create/update -- **Automations**: Time-based actions (SLA breaches, follow-ups) -- **Macros**: One-click response templates -- **Conditional logic**: Complex business rules +```typescript +// Rules read like policies +zendesk`when ticket from vip is new: priority urgent, route to vip-support` +zendesk`when ticket mentions refund: tag refund-request, assign to billing` +zendesk`when ticket sentiment negative and priority low: escalate` + +// SLAs in plain English +zendesk`enterprise tickets: first response 1 hour, resolution 4 hours` +zendesk`standard tickets: first response 4 hours, resolution 24 hours` +``` + +### Automations + +```typescript +// Time-based actions +zendesk`pending tickets not updated in 72 hours`.email('follow-up').tag('follow-up-sent') +zendesk`solved tickets after 48 hours`.close() +zendesk`tickets breaching SLA`.notify('on-call').escalate() +``` ### Service Level Agreements @@ -95,28 +152,15 @@ Here's what makes zendesk.do different: **AI isn't bolted on. It's the foundatio ### L1 Auto-Resolution ```typescript -// AI agent handles common tickets automatically -export const ticket = Ticket({ - triggers: { - onCreate: async (ticket) => { - const resolution = await ai.resolve(ticket, { - confidence: 0.85, // Only auto-resolve when confident - knowledgeBase: true, - ticketHistory: true, - }) - - if (resolution.autoResolved) { - return ticket.close({ - response: resolution.message, - tags: ['ai-resolved'] - }) - } - - // Low confidence? Suggest to human agent - ticket.addInternalNote(resolution.suggestion) - } - } -}) +// One line: AI handles L1 automatically +await zendesk`auto-resolve tickets with 85% confidence` + +// Or be specific +await zendesk`auto-resolve password reset tickets` +await zendesk`auto-resolve how-to questions from knowledge base` + +// Low confidence? Route with context +await zendesk`tickets AI cannot resolve`.assign('tier-2').note('AI suggestions attached') ``` ### How It Works @@ -129,12 +173,26 @@ export const ticket = Ticket({ ### AI Capabilities -- **Intent classification** - What does the customer need? -- **Sentiment detection** - Is this urgent? Is someone upset? -- **Similar ticket matching** - How was this solved before? -- **KB article suggestion** - Which help article answers this? -- **Response drafting** - Write human-like replies -- **Auto-tagging** - Categorize without manual work +```typescript +// Intent classification +await zendesk`billing questions today` +await zendesk`feature requests this week` + +// Sentiment detection +await zendesk`frustrated customers` +await zendesk`tickets with negative sentiment` + +// Similar ticket matching +await zendesk`tickets like 12345` +await zendesk`how was this solved before?` + +// Response drafting +await zendesk`draft response for ticket 12345` + .review() // human approves + .send() + +// Auto-tagging happens automatically +``` ### The Numbers @@ -152,60 +210,40 @@ Your human agents focus on complex issues. AI handles the rest. ## Full Workflow Engine -The same workflow power as Zendesk, defined in code: +The same workflow power as Zendesk, in natural language: ### Triggers ```typescript -// Trigger: Auto-assign VIP tickets -export const vipTrigger = Trigger({ - conditions: { - all: [ - { field: 'requester.tags', contains: 'vip' }, - { field: 'status', equals: 'new' } - ] - }, - actions: [ - { field: 'priority', value: 'urgent' }, - { field: 'group', value: 'vip-support' }, - { field: 'tags', add: 'vip-escalation' } - ] -}) +// Trigger rules read like policies you'd speak aloud +zendesk`when ticket from vip is new: priority urgent, route to vip-support` +zendesk`when ticket mentions billing: assign to billing-team` +zendesk`when ticket has attachment: tag has-attachment` +zendesk`when requester is enterprise: SLA enterprise, priority high` ``` ### Automations ```typescript -// Automation: Follow up on pending tickets -export const followUp = Automation({ - conditions: { - all: [ - { field: 'status', equals: 'pending' }, - { field: 'updated_at', before: '72 hours ago' } - ] - }, - actions: [ - { type: 'email', template: 'follow-up' }, - { field: 'tags', add: 'follow-up-sent' } - ] -}) +// Time-based automations +zendesk`pending tickets not updated in 72 hours`.email('follow-up').tag('follow-up-sent') +zendesk`solved tickets after 7 days`.close() +zendesk`unassigned urgent tickets after 15 minutes`.notify('on-call') +zendesk`SLA breach in 30 minutes`.warn('assigned-agent') ``` ### Macros ```typescript -// Macro: Common refund response -export const refundMacro = Macro({ - title: 'Process Refund', - actions: [ - { field: 'status', value: 'solved' }, - { field: 'tags', add: 'refund-processed' }, - { - type: 'comment', - value: 'Your refund has been processed and will appear in 3-5 business days.' - } - ] -}) +// Common responses +await zendesk`refund for ticket 12345` + .respond('Your refund has been processed and will appear in 3-5 business days.') + .tag('refund-processed') + .solve() + +// Or define reusable macros +zendesk`macro 'refund': respond with refund confirmation, tag refund-processed, solve` +zendesk`macro 'needs-info': respond asking for details, set pending, tag awaiting-customer` ``` ## API Compatible @@ -213,57 +251,42 @@ export const refundMacro = Macro({ Drop-in compatibility with Zendesk's API. Your existing integrations work. ```typescript -// Standard Zendesk API endpoints +// Same natural syntax for API operations +await zendesk`create ticket: billing issue from john@acme.com` +await zendesk`update ticket 12345: priority urgent` +await zendesk`add comment to 12345: We're looking into this` + +// Or use traditional REST GET /api/v2/tickets POST /api/v2/tickets GET /api/v2/tickets/{id} PUT /api/v2/tickets/{id} -DELETE /api/v2/tickets/{id} - -GET /api/v2/users -GET /api/v2/organizations -GET /api/v2/groups -GET /api/v2/views -GET /api/v2/triggers -GET /api/v2/automations -GET /api/v2/macros ``` Existing Zendesk integrations, webhooks, and scripts work with minimal changes. -```typescript -// Your existing code -const zendesk = new ZendeskClient({ - subdomain: 'yourcompany', // Change to your zendesk.do URL - // ... rest stays the same -}) -``` - ## Architecture Built on Cloudflare Durable Objects for global, real-time support: ``` -zendesk/ - src/ - durable-objects/ - TicketDO.ts # Individual ticket state - OrganizationDO.ts # Customer organization - ViewDO.ts # Real-time view updates - TriggerEngine.ts # Business rule execution - - ai/ - resolver.ts # AI ticket resolution - classifier.ts # Intent & sentiment - suggester.ts # KB & response suggestions - - api/ - v2/ # Zendesk-compatible API - - channels/ - email.ts # Inbound/outbound email - chat.ts # Live chat - form.ts # Contact forms +HelpDeskDO (config, agents, groups, SLAs) + | + +-- TicketsDO (ticket state, conversations) + | |-- SQLite: Active tickets (encrypted) + | +-- R2: Attachments (encrypted) + | + +-- ViewsDO (real-time ticket views) + | |-- SQLite: View definitions + | +-- WebSocket: Live updates + | + +-- TriggersDO (business rules engine) + | |-- SQLite: Rule definitions + | +-- Atomic execution + | + +-- KnowledgeDO (help center, articles) + |-- SQLite: Article content + +-- Vector: Semantic search ``` ### Why Durable Objects? @@ -275,26 +298,23 @@ zendesk/ ### Storage Tiers -```typescript -// Hot: SQLite in Durable Object (< 30 days) -// Active tickets, recent history - -// Warm: R2 object storage (30-365 days) -// Archived tickets, searchable - -// Cold: R2 archive (> 365 days) -// Compliance retention, compressed -``` +| Tier | Storage | Use Case | Query Speed | +|------|---------|----------|-------------| +| **Hot** | SQLite | Active tickets, recent history | <10ms | +| **Warm** | R2 + Index | Archived tickets (30-365 days) | <100ms | +| **Cold** | R2 Archive | Compliance retention (365+ days) | <1s | -## Self-Hosted vs Managed +## vs Zendesk -| | Self-Hosted | Managed (coming) | -|---|-------------|------------------| -| Price | Free + infrastructure | Usage-based | -| Deploy | Your Cloudflare account | One-click | -| Data | Your control | Encrypted, isolated | -| AI | Your LLM API key | Included | -| Support | Community | Priority | +| Feature | Zendesk Suite Professional | zendesk.do | +|---------|---------------------------|------------| +| **Price** | $115/agent/month | $0 + infrastructure | +| **AI Resolution** | Premium add-on | Built-in | +| **Agents** | Per-seat licensing | Unlimited | +| **Deploy** | SaaS only | Your Cloudflare account | +| **Customization** | Configuration UI | Code-first | +| **Data** | Zendesk's servers | Your control | +| **Lock-in** | Years of migration | MIT licensed | ## Getting Started @@ -304,59 +324,94 @@ zendesk/ npx create-dotdo zendesk ``` -### 2. Configure Email +### 2. Configure ```typescript -// wrangler.toml -[vars] -SUPPORT_EMAIL = "support@yourcompany.com" - -[[email]] -name = "inbound" -forward = "TicketDO" +// Just talk to it +zendesk`support email is support@acme.com` +zendesk`business hours 9am to 6pm Pacific weekdays` +zendesk`auto-resolve with 85% confidence` ``` -### 3. Add Your First Agent +### 3. Add Agents -```bash -curl -X POST https://your-zendesk.do/api/v2/users \ - -d '{"user": {"name": "Support Agent", "email": "agent@yourcompany.com", "role": "agent"}}' +```typescript +await zendesk`add agent Sarah Chen sarah@acme.com` +await zendesk`add agent Mike Park mike@acme.com to billing-team` +await zendesk`Sarah is admin` ``` -### 4. Enable AI Resolution +### 4. Define Rules ```typescript -// src/config.ts -export const config = { - ai: { - enabled: true, - provider: 'llm.do', // or 'openai', 'anthropic' - autoResolve: { - enabled: true, - confidenceThreshold: 0.85 - } - } -} +// Routing +zendesk`when ticket mentions billing: assign to billing-team` +zendesk`when ticket from enterprise: priority high, SLA enterprise` + +// SLAs +zendesk`enterprise: respond in 1 hour, resolve in 4 hours` +zendesk`standard: respond in 4 hours, resolve in 24 hours` + +// Automations +zendesk`pending tickets after 72 hours: follow up and tag follow-up-sent` ``` ## Migrate from Zendesk -One-time import of your existing tickets, users, and configuration: +```typescript +// One line migration +await zendesk`migrate from yourcompany.zendesk.com` + +// Or step by step +await zendesk`import tickets from yourcompany.zendesk.com` +await zendesk`import users from yourcompany.zendesk.com` +await zendesk`import triggers from yourcompany.zendesk.com` +await zendesk`import knowledge base from yourcompany.zendesk.com` +``` -```bash -npx zendesk-do migrate \ - --subdomain=yourcompany \ - --email=admin@yourcompany.com \ - --token=your_api_token +Imports tickets, users, organizations, groups, views, triggers, automations, macros, and knowledge base articles. + +## Use Cases + +### Customer Support Teams + +```typescript +// Daily operations +await zendesk`my open tickets` +await zendesk`urgent tickets needing response` +await zendesk`customers waiting more than 4 hours` + +// Team management +await zendesk`team performance this week` +await zendesk`who has capacity?` +await zendesk`redistribute Sarah's tickets to team` +``` + +### E-commerce Support + +```typescript +// Order issues +await zendesk`tickets about order 12345` +await zendesk`refund requests today` +await zendesk`shipping complaints this week` + +// Proactive support +await zendesk`customers with delayed orders`.notify('shipping update') ``` -Imports: -- Tickets & comments -- Users & organizations -- Groups & views -- Triggers & automations -- Macros -- Knowledge base articles +### SaaS Support + +```typescript +// Technical issues +await zendesk`bug reports this week` +await zendesk`tickets mentioning API errors` +await zendesk`enterprise customers with outage reports` + +// Success metrics +await zendesk`CSAT scores by agent this month` +await zendesk`resolution time trend` +await zendesk`top 10 ticket categories` +``` ## Contributing @@ -375,10 +430,13 @@ MIT - Use it however you want. Build a business on it. Fork it. Improve it. --- -
- -**Stop paying per agent. Start resolving with AI.** - -[Deploy Now](https://zendesk.do/deploy) | [Documentation](https://zendesk.do/docs) | [Discord](https://discord.gg/dotdo) - -
+

+ Stop paying per agent. Start resolving with AI. +
+ Unlimited agents. AI-first. Your data. +

+ Website | + Docs | + Discord | + GitHub +

diff --git a/rewrites/zoho/README.md b/rewrites/zoho/README.md index ee67338a..f71f972d 100644 --- a/rewrites/zoho/README.md +++ b/rewrites/zoho/README.md @@ -1,58 +1,40 @@ # zoho.do -> You're a startup founder. You need CRM, projects, support, invoicing - the whole stack. Zoho offers 50+ apps but they're all separate databases. Your data is fragmented. Your AI can't see the full picture. +> The Everything Suite. Unified. AI-Native. Actually Affordable. -

- The Everything Suite. Unified. AI-Native. Actually Affordable. -

- -

- npm version - npm downloads - license -

- ---- +Zoho built an empire as the "affordable" alternative. 50+ apps. 90 million users. But "affordable" still means $52-65/user/month for their full suite. Each app is its own database. Your data is fragmented. Your AI can't see the full picture. -## The workers.do Way +**zoho.do** is the unified suite. CRM, Projects, Desk, Campaigns, Books - all in one Durable Object. One database. One API. AI that sees everything. -Talk to your unified suite like a colleague: +## AI-Native API ```typescript -import { zoho, sally, priya, tom, ralph } from 'workers.do' - -// Natural language queries across all apps -const health = await zoho`how is the Acme Corp relationship?` -const pipeline = await zoho`show deals closing this quarter` -const capacity = await zoho`what's engineering capacity for Q1?` - -// AI agents work your entire suite -await sally`qualify leads and create deals for hot prospects` -await priya`check project status for enterprise customers` +import { zoho } from 'zoho.do' // Full SDK +import { zoho } from 'zoho.do/tiny' // Minimal client +import { zoho } from 'zoho.do/crm' // CRM-only operations ``` -### Promise Pipelining - -Chain operations across apps without waiting. One network round trip: +Natural language for business workflows: ```typescript -// Deal won -> project -> invoice -> support - all pipelined -const onboarded = await zoho`find deals closed this week` - .map(deal => ralph`create implementation project for ${deal}`) - .map(project => zoho`create invoice for ${project.deal}`) - .map(invoice => priya`setup support account for ${invoice.customer}`) - -// Cross-app customer health -const atRisk = await zoho`find customers with open tickets` - .map(customer => priya`analyze churn risk for ${customer}`) - .map(risk => [sally, tom].map(r => r`review ${risk} and create action plan`)) -``` - ---- - -Zoho built an empire by being the "affordable" alternative. 50+ apps. 90 million users. But "affordable" still means $52-65/user/month for their full suite. And each app is its own silo with its own data. +import { zoho } from 'zoho.do' -**zoho.do** is the unified suite. CRM, Projects, Desk, Campaigns, Books - all in one Durable Object. One database. One API. AI that sees everything. Deploy in 60 seconds. +// Talk to it like a colleague +const health = await zoho`how is Acme Corp relationship?` +const deals = await zoho`deals closing this quarter` +const atRisk = await zoho`customers with open tickets no contact 30 days` + +// Chain like sentences +await zoho`leads from website this week` + .map(lead => zoho`create deal for ${lead}`) + .map(deal => zoho`assign to sales rep nearest ${deal.location}`) + +// Cross-app operations document themselves +await zoho`deal won Acme Enterprise` + .project(`Acme Implementation Q1 2025`) + .invoice() + .notify(`Welcome aboard`) +``` ## The Problem @@ -140,49 +122,34 @@ import { Zoho } from 'zoho.do' export default Zoho({ name: 'my-company', domain: 'work.my-company.com', - apps: ['crm', 'projects', 'desk', 'campaigns', 'books'], // Enable what you need + apps: ['crm', 'projects', 'desk', 'campaigns', 'books'], }) ``` --- -## The Unified Suite - -### CRM (Zoho CRM) +## Features -Full-featured CRM with cross-app visibility: +### CRM ```typescript -import { zoho } from 'zoho.do' - -// Create a lead -const lead = await zoho.crm.leads.create({ - firstName: 'Alice', - lastName: 'Chen', - email: 'alice@acme.com', - company: 'Acme Corporation', - phone: '+1-555-0123', - source: 'Website', - industry: 'Technology', -}) - -// Convert to contact + account + deal -const conversion = await zoho.crm.leads.convert(lead.id, { - createDeal: true, - dealName: 'Acme Corp - Enterprise License', - dealValue: 75000, -}) - -// AI scoring (unified across all apps) -const score = await zoho.ai.score(lead.id) -// { -// score: 85, -// factors: [ -// { source: 'crm', factor: 'Company size matches ICP' }, -// { source: 'campaigns', factor: 'Opened 5 marketing emails' }, -// { source: 'desk', factor: 'No support tickets (good sign)' }, -// ], -// } +// Find anyone +const alice = await zoho`Alice Chen from Acme` +const hot = await zoho`leads score > 80` +const closing = await zoho`deals closing this month` + +// AI infers what you need +await zoho`Alice Chen` // returns contact +await zoho`Alice deals` // returns her deals +await zoho`Acme history` // returns full account timeline + +// Create naturally +const lead = await zoho`create lead Alice Chen from Acme, web source` +await zoho`convert ${lead} to deal Enterprise License 75k` + +// AI scoring sees everything +await zoho`score ${lead}` +// 85: Company matches ICP, opened 5 emails, no support tickets ``` ### Contacts Are Universal @@ -190,404 +157,153 @@ const score = await zoho.ai.score(lead.id) One contact record, visible everywhere: ```typescript -// Create a contact once -const contact = await zoho.contacts.create({ - firstName: 'Alice', - lastName: 'Chen', - email: 'alice@acme.com', - company: 'Acme Corporation', -}) +// Create once, appears everywhere +await zoho`create contact Alice Chen alice@acme.com` -// Same contact appears in: -await zoho.crm.contacts.get(contact.id) // CRM view -await zoho.campaigns.contacts.get(contact.id) // Marketing view -await zoho.desk.customers.get(contact.id) // Support view -await zoho.books.customers.get(contact.id) // Billing view +// Same person in every view +await zoho`Alice CRM view` // deals, activities +await zoho`Alice campaigns` // email engagement +await zoho`Alice tickets` // support history +await zoho`Alice invoices` // billing // No sync. No duplicates. One record. ``` -### Projects (Zoho Projects) - -Project management with CRM integration: +### Projects ```typescript -// Create a project (auto-linked to deal) -const project = await zoho.projects.create({ - name: 'Acme Corp Implementation', - description: 'Q1 enterprise rollout', - deal_id: deal.id, // Cross-app link - template: 'enterprise-implementation', - start_date: '2025-02-01', - end_date: '2025-04-30', -}) - -// Tasks with dependencies -await zoho.projects.tasks.create({ - project_id: project.id, - name: 'Technical discovery', - assignee: 'tom@company.com', - due_date: '2025-02-07', - priority: 'high', -}) - -await zoho.projects.tasks.create({ - project_id: project.id, - name: 'Environment setup', - assignee: 'ralph@company.com', - due_date: '2025-02-14', - depends_on: ['technical-discovery'], -}) - -// Milestones -await zoho.projects.milestones.create({ - project_id: project.id, - name: 'Go Live', - due_date: '2025-04-15', - linked_tasks: ['training', 'data-migration', 'testing'], -}) +// Create with tasks in one expression +const project = await zoho`create project Acme Implementation Q1 2025` + .task('discovery', { assignee: tom, due: 'Feb 7' }) + .task('setup', { assignee: ralph, due: 'Feb 14', after: 'discovery' }) + .task('training', { assignee: priya, due: 'Feb 21' }) + .milestone('Go Live', { due: 'Apr 15' }) + +// Natural queries +await zoho`Acme project status` +await zoho`tom tasks this week` +await zoho`overdue tasks engineering` // Time tracking -await zoho.projects.timesheet.log({ - task_id: task.id, - user: 'tom@company.com', - hours: 4, - date: '2025-02-05', - billable: true, - notes: 'Technical discovery call with client', -}) +await zoho`log 4 hours discovery call with Acme billable` ``` -### Desk (Zoho Desk) - -Help desk with full customer context: +### Desk ```typescript -// Create a ticket -const ticket = await zoho.desk.tickets.create({ - subject: 'Cannot access dashboard', - description: 'Getting 403 error when trying to login...', - contact_id: contact.id, // Same contact as CRM - priority: 'high', - channel: 'email', -}) - -// Agent sees EVERYTHING about this customer -const context = await zoho.desk.tickets.context(ticket.id) -// { -// contact: { ... }, -// company: { ... }, -// crmDeals: [{ name: 'Enterprise License', value: 75000, stage: 'Implementation' }], -// projects: [{ name: 'Acme Implementation', progress: 45 }], -// invoices: [{ number: 'INV-001', amount: 25000, status: 'paid' }], -// previousTickets: [{ subject: 'Setup question', resolved: true }], -// campaigns: [{ name: 'Product Launch', opened: true, clicked: true }], -// } - -// AI resolution (knows context from all apps) -const resolution = await zoho.ai.resolveTicket(ticket.id) -// AI knows: -// - They're a paying customer (from Books) -// - On an enterprise plan (from CRM deal) -// - In active implementation (from Projects) -// - Should be prioritized +// Tickets with full context +await zoho`ticket from Alice: cannot access dashboard, 403 error` + +// Agent sees everything about this customer +await zoho`Alice context` +// Enterprise deal $75k in implementation +// Project 45% complete, on schedule +// Paid $25k invoice on time +// Opened last 3 marketing emails +// One previous ticket (resolved) + +// AI resolution knows the full picture +await zoho`resolve Alice ticket` +// Knows: paying customer, enterprise plan, active implementation +// Prioritizes accordingly ``` -### Campaigns (Zoho Campaigns) - -Email marketing with unified contacts: +### Campaigns ```typescript -// Create a campaign -const campaign = await zoho.campaigns.create({ - name: 'Q1 Product Launch', - type: 'email', - subject: 'Introducing our new features', - content: '

Big news!

...', - segment: { - // Query across ALL apps - query: ` - contacts.lifecycle_stage = 'customer' - AND books.total_revenue > 10000 - AND desk.open_tickets = 0 - `, - }, -}) +// Target with cross-app queries +await zoho`campaign Q1 Launch to customers revenue > 10k no open tickets` -// Schedule -await zoho.campaigns.schedule(campaign.id, { - send_at: '2025-01-15T09:00:00Z', - timezone: 'America/New_York', -}) +// Schedule naturally +await zoho`send Q1 Launch Tuesday 9am Eastern` -// Track across apps -campaign.on('opened', async (contact) => { - // Update CRM engagement score - await zoho.crm.contacts.update(contact.id, { - last_marketing_engagement: new Date(), - engagement_score: contact.engagement_score + 5, - }) -}) - -campaign.on('clicked', async (contact, link) => { - if (link.type === 'pricing') { - // Create CRM task for sales - await zoho.crm.tasks.create({ - subject: `Follow up with ${contact.name} - clicked pricing`, - contact_id: contact.id, - due_date: 'tomorrow', - }) - } -}) +// Engagement flows back to CRM automatically +// Opens -> engagement score updates +// Pricing clicks -> sales task created ``` -### Books (Zoho Books) - -Accounting with complete context: +### Books ```typescript -// Create an invoice (linked to CRM deal) -const invoice = await zoho.books.invoices.create({ - customer_id: contact.id, // Same contact everywhere - deal_id: deal.id, // Links to CRM - items: [ - { product: 'Enterprise License', quantity: 50, rate: 150 }, - { product: 'Implementation Services', quantity: 40, rate: 200, unit: 'hours' }, - ], - due_date: '2025-03-01', - terms: 'Net 30', -}) +// Invoices linked to everything +await zoho`invoice Acme: 50 licenses at 150, 40 hours implementation at 200` -// When invoice is paid, everything updates -invoice.on('paid', async () => { - // CRM: Update deal to Closed Won - await zoho.crm.deals.update(deal.id, { - stage: 'Closed Won', - closed_date: new Date(), - }) - - // Projects: Start the project - await zoho.projects.update(project.id, { - status: 'active', - }) - - // Desk: Note on customer record - await zoho.desk.customers.addNote(contact.id, { - content: `Invoice ${invoice.number} paid. Implementation can begin.`, - }) -}) +// Payment triggers cascade +await zoho`Acme paid` +// Deal -> Closed Won +// Project -> Active +// Support -> Note added -// Financial reports -const revenue = await zoho.books.reports.revenue({ - period: 'Q1', - by: 'customer', - include_crm_data: true, // See deal source, campaign attribution -}) +// Reports with attribution +await zoho`Q1 revenue by customer with deal source` ``` -### People (Zoho People) - -HR with organizational context: +### People ```typescript -// Employee onboarding -const employee = await zoho.people.employees.create({ - firstName: 'Bob', - lastName: 'Smith', - email: 'bob@company.com', - department: 'Engineering', - role: 'Senior Developer', - manager: 'tom@company.com', - start_date: '2025-02-01', -}) - -// Auto-provisioning -employee.on('created', async () => { - // Create user in all apps - await zoho.createUser({ - email: employee.email, - name: `${employee.firstName} ${employee.lastName}`, - apps: ['crm', 'projects', 'desk'], - role: 'employee', - }) - - // Assign to projects - await zoho.projects.teams.add(employee.id, 'engineering') - - // Create onboarding project - await zoho.projects.create({ - name: `Onboarding: ${employee.firstName}`, - template: 'employee-onboarding', - assignee: employee.manager, - }) -}) - -// Time off (syncs with Projects) -await zoho.people.timeOff.request({ - employee_id: employee.id, - type: 'vacation', - start_date: '2025-03-15', - end_date: '2025-03-22', - reason: 'Spring break', -}) -// Auto-updates project timelines, reassigns tasks +// Onboarding is one line +await zoho`hire Bob Smith engineering senior dev reports to Tom Feb 1` +// Creates user in all apps +// Adds to engineering team +// Creates onboarding project + +// Time off syncs with projects +await zoho`Bob vacation Mar 15-22` +// Auto-updates timelines, reassigns tasks ``` --- ## AI That Sees Everything -### Unified AI (Zia Reimagined) - -AI that works across all apps: - ```typescript -import { zoho } from 'zoho.do' - // Ask anything -const answer = await zoho.ai.ask(` - How is the Acme Corp relationship going? -`) - -// AI sees: -// - CRM: Deal value, stage, activities -// - Projects: Implementation progress, blockers -// - Desk: Support tickets, satisfaction -// - Books: Payment history, outstanding invoices -// - Campaigns: Email engagement - -console.log(answer) -// "Acme Corp is a $75,000 enterprise deal currently in implementation. -// Project is 45% complete, on schedule for April go-live. -// They had one support ticket last week (resolved). -// Invoice #001 ($25,000) was paid on time. -// Contact Alice Chen opened your last 3 marketing emails. -// Recommendation: Schedule mid-implementation check-in call." +await zoho`how is Acme Corp relationship?` +// "$75k deal in implementation, 45% complete, on schedule. +// One ticket last week (resolved). Invoice paid on time. +// Alice opened last 3 emails. Schedule mid-implementation check-in." + +// Predictions use all apps +await zoho`churn risk Acme` +// 72% risk: 5 open tickets, unsubscribed newsletter, +// no meeting in 45 days, late on invoice + +// Forecasting +await zoho`Q2 revenue forecast` +// New: $450k, Renewals: $850k, Expansions: $120k +// Total: $1.42M (+23% YoY) + +// Capacity planning +await zoho`engineering capacity Q1` +// Available: 2400h, Committed: 1800h, Pipeline: 1200h +// Gap: 600h - hire or push BigCorp to Q2 ``` ### Cross-App Automation ```typescript -// Automation that spans apps -await zoho.automations.create({ - name: 'Enterprise Customer Journey', - trigger: { - app: 'crm', - event: 'deal.stage_changed', - condition: { stage: 'Closed Won', value: { gte: 50000 } }, - }, - actions: [ - // CRM: Update record - { app: 'crm', action: 'update_contact', data: { lifecycle_stage: 'customer' } }, - - // Projects: Create implementation project - { app: 'projects', action: 'create_project', data: { template: 'enterprise-implementation' } }, - - // Desk: Create customer record with context - { app: 'desk', action: 'create_customer', data: { priority: 'high' } }, - - // Books: Create initial invoice - { app: 'books', action: 'create_invoice', data: { template: 'enterprise-onboarding' } }, - - // Campaigns: Add to customer segment - { app: 'campaigns', action: 'add_to_segment', data: { segment: 'enterprise-customers' } }, - - // People: Notify account team - { app: 'people', action: 'notify', data: { team: 'customer-success' } }, - ], -}) -``` - -### Predictive Intelligence - -```typescript -// Churn prediction (uses all apps) -const churnRisk = await zoho.ai.predictChurn(contact.id) -// { -// risk: 0.72, // 72% likely to churn -// signals: [ -// { source: 'desk', signal: '5 open tickets, avg resolution 3 days' }, -// { source: 'campaigns', signal: 'Unsubscribed from newsletter' }, -// { source: 'crm', signal: 'No meeting in 45 days' }, -// { source: 'books', signal: 'Late on last invoice' }, -// ], -// recommendation: 'Schedule urgent executive check-in', -// } - -// Revenue forecasting (CRM + Books + Projects) -const forecast = await zoho.ai.forecast({ - period: 'Q2', - include: ['new_deals', 'renewals', 'expansions'], -}) -// { -// newDeals: { predicted: 450000, confidence: 0.75 }, -// renewals: { predicted: 850000, confidence: 0.92 }, // Based on Books data -// expansions: { predicted: 120000, confidence: 0.68 }, -// total: 1420000, -// vsLastYear: +23, -// } - -// Resource planning (Projects + People) -const capacity = await zoho.ai.planCapacity({ - period: 'Q1', - team: 'engineering', -}) -// { -// available: 2400, // hours -// committed: 1800, // from Projects -// pipeline: 1200, // from CRM deals -// gap: 600, // need to hire or delay -// recommendation: 'Hire 1 engineer or push BigCorp project to Q2', -// } +// Enterprise customer journey in one expression +await zoho`when deal closes > 50k` + .then(deal => zoho`${deal.contact} is now customer`) + .then(deal => zoho`create project for ${deal} from enterprise template`) + .then(deal => zoho`invoice ${deal}`) + .then(deal => zoho`add ${deal.contact} to enterprise segment`) + .then(deal => zoho`notify customer success about ${deal}`) ``` --- -## API Compatible - -Drop-in replacement for Zoho APIs: +## Universal Search ```typescript -// Before: Zoho Cloud -import { ZCRMRestClient } from '@zohocrm/nodejs-sdk-2.0' - -// After: zoho.do (same SDK, different endpoint) -const config = { - baseUrl: 'https://your-instance.zoho.do', - // ... rest of config -} -``` - -### Unified API (New) - -One API for everything: - -```typescript -import { zoho } from 'zoho.do' - -// Universal entity operations -const entities = await zoho.search({ - query: 'acme', - apps: ['crm', 'projects', 'desk', 'books'], -}) -// Returns contacts, deals, projects, tickets, invoices matching "acme" - -// Cross-app queries -const results = await zoho.query(` - SELECT - c.name as contact_name, - d.name as deal_name, - d.value as deal_value, - p.name as project_name, - p.progress, - i.total as invoice_total, - i.status as invoice_status - FROM crm.contacts c - JOIN crm.deals d ON c.id = d.contact_id - LEFT JOIN projects.projects p ON d.id = p.deal_id - LEFT JOIN books.invoices i ON c.id = i.customer_id - WHERE c.lifecycle_stage = 'customer' -`) +// Search across all apps naturally +await zoho`find acme` +// Returns: contacts, deals, projects, tickets, invoices + +// Contextual inference +await zoho`acme` // full picture +await zoho`acme deals` // just deals +await zoho`acme this quarter` // time-scoped ``` --- @@ -597,150 +313,78 @@ const results = await zoho.query(` ### One Durable Object = One Company ``` -┌─────────────────────────────────────────────────────────────────────┐ -│ zoho.do Worker │ -├─────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌─────────────────────────────────────────────────────────────────┐│ -│ │ CompanyDO (per organization) ││ -│ ├─────────────────────────────────────────────────────────────────┤│ -│ │ ││ -│ │ ┌─────────────────────────────────────────────────────────────┐││ -│ │ │ Unified Entity Store (SQLite) │││ -│ │ │ │││ -│ │ │ contacts │ deals │ projects │ tickets │ invoices │││ -│ │ │ All entities share foreign keys - no sync needed │││ -│ │ └─────────────────────────────────────────────────────────────┘││ -│ │ ││ -│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││ -│ │ │ CRM │ │ Projects│ │ Desk │ │ Campaigns│ ││ -│ │ │ Module │ │ Module │ │ Module │ │ Module │ ││ -│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ ││ -│ │ ││ -│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ││ -│ │ │ Books │ │ People │ │ Forms │ │ Analytics│ ││ -│ │ │ Module │ │ Module │ │ Module │ │ Module │ ││ -│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ ││ -│ │ ││ -│ │ ┌─────────────────────────────────────────────────────────────┐││ -│ │ │ Unified AI Layer │││ -│ │ │ Cross-app intelligence, predictions, automation │││ -│ │ └─────────────────────────────────────────────────────────────┘││ -│ │ ││ -│ └─────────────────────────────────────────────────────────────────┘│ -│ │ -├─────────────────────────────────────────────────────────────────────┤ -│ SQLite (all data) │ R2 (files) │ Vectorize (AI) │ KV (cache)│ -└─────────────────────────────────────────────────────────────────────┘ +CompanyDO (per organization) + | + +-- SQLite: contacts, deals, projects, tickets, invoices + | (All entities share foreign keys - no sync needed) + | + +-- R2: files, attachments, exports + | + +-- Modules: CRM, Projects, Desk, Campaigns, Books, People + | + +-- AI Layer: cross-app intelligence, predictions, automation ``` ### Why This Matters Traditional Zoho: ``` -App A Database <--sync--> App B Database <--sync--> App C Database - \ | / - \ | / - '----------> Data Warehouse <------------------' - | - Analytics/AI - (delayed, incomplete) +App A <--sync--> App B <--sync--> App C + | | | + '-------> Data Warehouse <-----------' + | + Analytics/AI (delayed) ``` zoho.do: ``` - SQLite (One Database) - | - +--------+--------+--------+--------+--------+ - | | | | | | - CRM Projects Desk Campaigns Books People - | | | | | | - +--------+--------+--------+--------+--------+ - | - AI Layer (Real-time) + SQLite (One Database) + | + CRM - Projects - Desk - Campaigns - Books - People + | + AI Layer (Real-time) ``` ---- - -## Pricing Comparison - -### Zoho One Pricing (2025) - -| Plan | Per User/Month | 50 Users/Year | -|------|---------------|---------------| -| Zoho One (Flexible) | $45 | $27,000 | -| Zoho One (All Employee) | $90 | $54,000 | - -Plus: -- Storage: +$4/5GB/month -- Zoho Flow: +$10-60/month -- Premium support: +$125-500/month - -### zoho.do Pricing - -| Resource | Cost | Notes | -|----------|------|-------| -| Durable Object | $0.15/million requests | Your company | -| SQLite Storage | $0.20/GB/month | ALL your data | -| R2 Storage | $0.015/GB/month | Files | -| Workers | Free tier: 100k/day | API calls | -| AI | ~$0.01/query | Via llm.do | +## vs Zoho One -**Example: 50 users, 100k entities, 500k operations/month** - -| | Zoho One | zoho.do | -|-|----------|---------| -| Software | $27,000-54,000/year | ~$50/month | -| Add-ons | $2,000+/year | $0 (included) | -| Sync conflicts | Constant | None | -| AI | Fragmented | Unified | -| **Total** | **$29,000-56,000/year** | **~$600/year** | - -**Savings: 97-99%** - ---- +| Feature | Zoho One | zoho.do | +|---------|----------|---------| +| **Cost (50 users)** | $27,000-54,000/year | ~$600/year | +| **Apps** | 50+ separate databases | One unified database | +| **Sync** | Constant conflicts | No sync needed | +| **AI** | Per-app, fragmented | Cross-app, unified | +| **Deploy** | SaaS only | Your infrastructure | ## Migration ```bash npx zoho.do migrate --from-zoho - -# Migrates from: -# - Zoho CRM -# - Zoho Projects -# - Zoho Desk -# - Zoho Campaigns -# - Zoho Books -# - Zoho People -# - Zoho Forms -# - Custom apps - -# Unifies into single database -# Deduplicates contacts -# Preserves relationships -# Converts automations +# Migrates all apps, unifies database, deduplicates contacts ``` ---- - -## Available Modules - -| Module | Zoho Equivalent | Status | -|--------|-----------------|--------| -| CRM | Zoho CRM | Complete | -| Projects | Zoho Projects | Complete | -| Desk | Zoho Desk | Complete | -| Campaigns | Zoho Campaigns | Complete | -| Books | Zoho Books | Complete | -| People | Zoho People | In Progress | -| Forms | Zoho Forms | Complete | -| Analytics | Zoho Analytics | Complete | -| Survey | Zoho Survey | Planned | -| Sign | Zoho Sign | Planned | -| Inventory | Zoho Inventory | Planned | -| Subscriptions | Zoho Subscriptions | Planned | - ---- +## Roadmap + +### Apps +- [x] CRM +- [x] Projects +- [x] Desk +- [x] Campaigns +- [x] Books +- [x] Forms +- [x] Analytics +- [ ] People (in progress) +- [ ] Survey +- [ ] Sign +- [ ] Inventory +- [ ] Subscriptions + +### AI +- [x] Natural Language Queries +- [x] Cross-App Intelligence +- [x] Churn Prediction +- [x] Revenue Forecasting +- [ ] Capacity Planning +- [ ] Automated Workflows ## Contributing @@ -751,11 +395,8 @@ git clone https://github.com/dotdo/zoho.do cd zoho.do pnpm install pnpm test -pnpm dev ``` ---- - ## License MIT License @@ -763,7 +404,7 @@ MIT License ---

- One suite. One database. One AI. One deployment. + One suite. One database. One AI.
The unified business platform Zoho could have been.

From 80b8dd8abecdcceb0e0ec159bad2cdf149523da5 Mon Sep 17 00:00:00 2001 From: Nathan Clevenger <4130910+nathanclevenger@users.noreply.github.com> Date: Wed, 7 Jan 2026 19:20:20 -0600 Subject: [PATCH 02/60] feat: add Claude Code plugin configuration to all submodules - Add .claude/settings.json to 15 submodules enabling: - beads@beads-marketplace (issue tracking) - ralph-loop@claude-plugins-official (implementation loops) - superpowers@superpowers-marketplace (enhanced capabilities) - dev-loop@workers-do-marketplace (full dev lifecycle) - Initialize beads in rewrites/excel (was missing) - Add Claude Code Cloud setup documentation Submodules configured: - primitives, mdxui - rewrites: gitx, mongo, redis, neo4j, excel, firebase, convex, kafka, nats, turso, fsx - packages: esm, claude Co-Authored-By: Claude Opus 4.5 --- docs/CLAUDE-CODE-CLOUD-SETUP.md | 166 +++++++++++++++++++++++++++++ mdxui | 2 +- packages/claude | 2 +- packages/esm | 2 +- primitives | 2 +- rewrites/convex | 2 +- rewrites/excel | 2 +- rewrites/firebase | 2 +- rewrites/fsx/.claude/settings.json | 8 ++ rewrites/gitx | 2 +- rewrites/kafka | 2 +- rewrites/mongo | 2 +- rewrites/nats | 2 +- rewrites/neo4j | 2 +- rewrites/redis | 2 +- rewrites/turso | 2 +- 16 files changed, 188 insertions(+), 14 deletions(-) create mode 100644 docs/CLAUDE-CODE-CLOUD-SETUP.md create mode 100644 rewrites/fsx/.claude/settings.json diff --git a/docs/CLAUDE-CODE-CLOUD-SETUP.md b/docs/CLAUDE-CODE-CLOUD-SETUP.md new file mode 100644 index 00000000..0880218d --- /dev/null +++ b/docs/CLAUDE-CODE-CLOUD-SETUP.md @@ -0,0 +1,166 @@ +# Claude Code Cloud Setup for Submodules + +This document explains how to set up Claude Code plugins (including beads and dev-loop) for the workers.do submodules, both locally and in Claude Code Cloud. + +## Overview + +Each submodule has been configured with `.claude/settings.json` to enable: +- **beads** - Issue tracking and TDD workflow +- **ralph-loop** - Autonomous implementation loops +- **superpowers** - Enhanced capabilities +- **dev-loop** - Full development lifecycle (brainstorm → plan → implement → review) + +## Local Setup + +### 1. Register the workers-do-marketplace + +The dev-loop plugin is hosted in the workers-do-marketplace. Register it: + +```bash +# Option A: Use Claude Code CLI +/plugin marketplace add dot-do/workers + +# Option B: Manual registration (already done if you ran setup script) +# The marketplace is at: github.com/dot-do/workers +``` + +### 2. Install Plugins (if not auto-installed) + +```bash +/plugin install beads@beads-marketplace +/plugin install ralph-loop@claude-plugins-official +/plugin install superpowers@superpowers-marketplace +/plugin install dev-loop@workers-do-marketplace +``` + +### 3. Verify Setup + +In any submodule directory: +```bash +# Check beads is working +bd ready + +# Check dev-loop is available +/dev-loop --help +``` + +## Claude Code Cloud Setup + +For Claude Code Cloud environments, the plugins are enabled via the `.claude/settings.json` file in each repository. + +### Required Marketplaces + +Claude Code Cloud needs access to these marketplaces: + +| Marketplace | GitHub Repo | Plugins | +|-------------|-------------|---------| +| beads-marketplace | steveyegge/beads | beads | +| claude-plugins-official | anthropics/claude-plugins-official | ralph-loop | +| superpowers-marketplace | obra/superpowers-marketplace | superpowers | +| workers-do-marketplace | dot-do/workers | dev-loop | + +### Configuration File + +Each submodule contains `.claude/settings.json`: + +```json +{ + "enabledPlugins": { + "beads@beads-marketplace": true, + "ralph-loop@claude-plugins-official": true, + "superpowers@superpowers-marketplace": true, + "dev-loop@workers-do-marketplace": true + } +} +``` + +### Cloud-Specific Notes + +1. **Marketplace Registration**: Cloud environments may need marketplaces pre-registered in your Claude Code Cloud organization settings. + +2. **Beads Data**: The `.beads/` directory is git-tracked and syncs automatically. No cloud-specific storage needed. + +3. **Dev-Loop State**: Dev-loop stores state in `.claude/dev-loop.local.md` which is gitignored by default. + +## Submodule List + +All configured submodules: + +| Submodule | Prefix | Status | +|-----------|--------|--------| +| primitives | primitives- | Ready | +| rewrites/gitx | gitx- | Ready | +| rewrites/mongo | mongo- | Ready | +| rewrites/redis | redis- | Ready | +| rewrites/neo4j | neo4j- | Ready | +| rewrites/excel | excel- | Ready | +| rewrites/firebase | firebase- | Ready | +| rewrites/convex | convex- | Ready | +| rewrites/kafka | kafka- | Ready | +| rewrites/nats | nats- | Ready | +| rewrites/turso | turso- | Ready | +| rewrites/fsx | fsx- | Ready | +| packages/esm | esm- | Ready | +| packages/claude | claude- | Ready | +| mdxui | mdxui- | Ready | + +## Workflow in Submodules + +### Starting Work + +```bash +cd rewrites/kafka + +# Check available work +bd ready + +# Claim a task +bd update kafka-xxx --status=in_progress + +# Or start a new dev loop +/dev-loop "Implement consumer group management" +``` + +### Completing Work + +```bash +# Close completed issues +bd close kafka-xxx kafka-yyy + +# Sync beads data +bd sync + +# Commit changes +git add . +git commit -m "feat: implement consumer groups" +git push +``` + +## Troubleshooting + +### Plugin Not Found + +If a plugin isn't found: +1. Check marketplace is registered: `/plugin marketplace list` +2. Re-add marketplace: `/plugin marketplace add ` +3. Reinstall plugin: `/plugin install @` + +### Beads Not Working + +```bash +# Re-initialize if needed +bd init --prefix= + +# Run health check +bd doctor --fix +``` + +### Dev-Loop State Issues + +```bash +# Reset dev-loop state +rm .claude/dev-loop.local.md + +# Start fresh +/dev-loop "your task" +``` diff --git a/mdxui b/mdxui index 57f8c1e0..5446f27c 160000 --- a/mdxui +++ b/mdxui @@ -1 +1 @@ -Subproject commit 57f8c1e0439efaaf04e5081a69a0264f6fa50932 +Subproject commit 5446f27c1c00aa02f3c8b35ea7e2cb9ce0364f3d diff --git a/packages/claude b/packages/claude index a6fddad8..5669dc51 160000 --- a/packages/claude +++ b/packages/claude @@ -1 +1 @@ -Subproject commit a6fddad85359a89d76a079a4981f0608c8383dfd +Subproject commit 5669dc51c89678e19c03e562ff7279760fb3d73e diff --git a/packages/esm b/packages/esm index 5ebc9248..ef391ded 160000 --- a/packages/esm +++ b/packages/esm @@ -1 +1 @@ -Subproject commit 5ebc9248fbdd15b09b43bf9be39ec11fe26241e3 +Subproject commit ef391ded61c4933bc98a28d1d5f5e3b754e37874 diff --git a/primitives b/primitives index 3b9d1ae8..52b34696 160000 --- a/primitives +++ b/primitives @@ -1 +1 @@ -Subproject commit 3b9d1ae8188c8330231a3771fb5377328c4d5542 +Subproject commit 52b34696f97912902135b6bcab931a8c667ba314 diff --git a/rewrites/convex b/rewrites/convex index 9e55a991..e7ef5ee3 160000 --- a/rewrites/convex +++ b/rewrites/convex @@ -1 +1 @@ -Subproject commit 9e55a991394a3882d42873a5ca0fe1d347a3a2f1 +Subproject commit e7ef5ee3b75497410b8c8a005dfa3b089a9b00a6 diff --git a/rewrites/excel b/rewrites/excel index 5b933902..eb7d95b9 160000 --- a/rewrites/excel +++ b/rewrites/excel @@ -1 +1 @@ -Subproject commit 5b93390237485c5b870d23d3dd9bb755a4b66ad5 +Subproject commit eb7d95b91b24a05f6ab9fe11fa8db06205bdeb17 diff --git a/rewrites/firebase b/rewrites/firebase index 83f213ef..6ec2150c 160000 --- a/rewrites/firebase +++ b/rewrites/firebase @@ -1 +1 @@ -Subproject commit 83f213efeda9ab41434876f333f54d582e574855 +Subproject commit 6ec2150cd5fb395c3f1134477592fd68fe107e29 diff --git a/rewrites/fsx/.claude/settings.json b/rewrites/fsx/.claude/settings.json new file mode 100644 index 00000000..13ee6f2a --- /dev/null +++ b/rewrites/fsx/.claude/settings.json @@ -0,0 +1,8 @@ +{ + "enabledPlugins": { + "beads@beads-marketplace": true, + "ralph-loop@claude-plugins-official": true, + "superpowers@superpowers-marketplace": true, + "dev-loop@workers-do-marketplace": true + } +} diff --git a/rewrites/gitx b/rewrites/gitx index 6fce86f5..db9a5800 160000 --- a/rewrites/gitx +++ b/rewrites/gitx @@ -1 +1 @@ -Subproject commit 6fce86f5c84c6f3ee08161bbb4cd2f1fff5a50f4 +Subproject commit db9a58002bd033e1cb12cf3333fb4426ec8ce7b0 diff --git a/rewrites/kafka b/rewrites/kafka index 815bc445..b65e9e4d 160000 --- a/rewrites/kafka +++ b/rewrites/kafka @@ -1 +1 @@ -Subproject commit 815bc4452226cf8fe1c85af53cf51b0098a85d21 +Subproject commit b65e9e4d8c33fe8fadbed3551a7cc9f518f39e19 diff --git a/rewrites/mongo b/rewrites/mongo index eb2d2eef..12bfba52 160000 --- a/rewrites/mongo +++ b/rewrites/mongo @@ -1 +1 @@ -Subproject commit eb2d2eef67fb1b68fab9e6cb76572a1658686ed1 +Subproject commit 12bfba528648de3db30ee74c18cbb9d2f5e45cbf diff --git a/rewrites/nats b/rewrites/nats index 73732599..ce379116 160000 --- a/rewrites/nats +++ b/rewrites/nats @@ -1 +1 @@ -Subproject commit 737325991b00cb009dd66e6cd337d09d59817511 +Subproject commit ce379116bfd42d6a88fac6ea05cc1ddba9771b82 diff --git a/rewrites/neo4j b/rewrites/neo4j index 1bf6734a..e53cd24c 160000 --- a/rewrites/neo4j +++ b/rewrites/neo4j @@ -1 +1 @@ -Subproject commit 1bf6734a9c49aede6995a24ec5da3cf805f54cc2 +Subproject commit e53cd24c8ab6067c185677bd998f21cc35cd5f4f diff --git a/rewrites/redis b/rewrites/redis index 28f67621..21cf4b66 160000 --- a/rewrites/redis +++ b/rewrites/redis @@ -1 +1 @@ -Subproject commit 28f676219e4aec5023352d33bb57e38625b96eee +Subproject commit 21cf4b66272008d5568258435220d15e58e88729 diff --git a/rewrites/turso b/rewrites/turso index 6b3664a5..4cb89f4e 160000 --- a/rewrites/turso +++ b/rewrites/turso @@ -1 +1 @@ -Subproject commit 6b3664a599619d2eb5ea5f98392c958dd06c1f89 +Subproject commit 4cb89f4e16d3263a458f64f40488ab852609e253 From 2cf14d87a7c2352f670ae9c333d39dee4066cd29 Mon Sep 17 00:00:00 2001 From: Nathan Clevenger <4130910+nathanclevenger@users.noreply.github.com> Date: Thu, 8 Jan 2026 05:18:58 -0600 Subject: [PATCH 03/60] refactor: move 70 non-submodule rewrites to opensaas repo Keeping only submodule-based rewrites in this repo: - convex, excel, firebase, fsx, gitx, kafka, mongo, nats, neo4j, redis, supabase, turso Co-Authored-By: Claude Opus 4.5 --- rewrites/airbyte/.beads/.gitignore | 39 - rewrites/airbyte/.beads/README.md | 81 -- rewrites/airbyte/.beads/config.yaml | 62 - rewrites/airbyte/.beads/interactions.jsonl | 0 rewrites/airbyte/.beads/issues.jsonl | 0 rewrites/airbyte/.beads/metadata.json | 4 - rewrites/airbyte/.gitattributes | 3 - rewrites/airbyte/AGENTS.md | 208 --- rewrites/airbyte/README.md | 426 ------ rewrites/airbyte/package.json | 83 -- rewrites/airbyte/src/index.ts | 54 - rewrites/airbyte/src/sdk/airbyte.ts | 192 --- rewrites/airbyte/src/sdk/types.ts | 149 -- rewrites/airtable/.beads/.gitignore | 39 - rewrites/airtable/.beads/README.md | 81 -- rewrites/airtable/.beads/config.yaml | 62 - rewrites/airtable/.beads/interactions.jsonl | 0 rewrites/airtable/.beads/issues.jsonl | 0 rewrites/airtable/.beads/metadata.json | 4 - rewrites/airtable/.gitattributes | 3 - rewrites/airtable/AGENTS.md | 40 - rewrites/airtable/README.md | 358 ----- rewrites/algolia/.beads/.gitignore | 39 - rewrites/algolia/.beads/README.md | 81 -- rewrites/algolia/.beads/config.yaml | 62 - rewrites/algolia/.beads/interactions.jsonl | 0 rewrites/algolia/.beads/issues.jsonl | 0 rewrites/algolia/.beads/metadata.json | 4 - rewrites/algolia/.gitattributes | 3 - rewrites/algolia/AGENTS.md | 132 -- rewrites/algolia/README.md | 388 ----- rewrites/amplitude/.beads/.gitignore | 39 - rewrites/amplitude/.beads/README.md | 81 -- rewrites/amplitude/.beads/config.yaml | 62 - rewrites/amplitude/.beads/interactions.jsonl | 0 rewrites/amplitude/.beads/issues.jsonl | 0 rewrites/amplitude/.beads/metadata.json | 4 - rewrites/amplitude/.gitattributes | 3 - rewrites/amplitude/AGENTS.md | 40 - rewrites/amplitude/README.md | 486 ------- rewrites/asana/.beads/.gitignore | 39 - rewrites/asana/.beads/README.md | 81 -- rewrites/asana/.beads/config.yaml | 62 - rewrites/asana/.beads/interactions.jsonl | 0 rewrites/asana/.beads/issues.jsonl | 0 rewrites/asana/.beads/metadata.json | 4 - rewrites/asana/.gitattributes | 3 - rewrites/asana/AGENTS.md | 40 - rewrites/asana/README.md | 453 ------ rewrites/athena/.beads/.gitignore | 39 - rewrites/athena/.beads/README.md | 81 -- rewrites/athena/.beads/config.yaml | 62 - rewrites/athena/.beads/interactions.jsonl | 0 rewrites/athena/.beads/issues.jsonl | 0 rewrites/athena/.beads/metadata.json | 4 - rewrites/athena/.gitattributes | 3 - rewrites/athena/AGENTS.md | 40 - rewrites/athena/README.md | 270 ---- rewrites/bamboohr/.beads/.gitignore | 39 - rewrites/bamboohr/.beads/README.md | 81 -- rewrites/bamboohr/.beads/config.yaml | 62 - rewrites/bamboohr/.beads/interactions.jsonl | 0 rewrites/bamboohr/.beads/issues.jsonl | 0 rewrites/bamboohr/.beads/metadata.json | 4 - rewrites/bamboohr/.gitattributes | 3 - rewrites/bamboohr/AGENTS.md | 40 - rewrites/bamboohr/README.md | 390 ----- rewrites/cerner/.beads/.gitignore | 39 - rewrites/cerner/.beads/README.md | 81 -- rewrites/cerner/.beads/config.yaml | 62 - rewrites/cerner/.beads/interactions.jsonl | 0 rewrites/cerner/.beads/issues.jsonl | 0 rewrites/cerner/.beads/metadata.json | 4 - rewrites/cerner/.gitattributes | 3 - rewrites/cerner/AGENTS.md | 40 - rewrites/cerner/README.md | 616 -------- rewrites/clio/.beads/.gitignore | 39 - rewrites/clio/.beads/README.md | 81 -- rewrites/clio/.beads/config.yaml | 62 - rewrites/clio/.beads/interactions.jsonl | 0 rewrites/clio/.beads/issues.jsonl | 25 - rewrites/clio/.beads/metadata.json | 4 - rewrites/clio/.gitattributes | 3 - rewrites/clio/AGENTS.md | 40 - rewrites/clio/README.md | 359 ----- rewrites/cloudera/.beads/.gitignore | 39 - rewrites/cloudera/.beads/README.md | 81 -- rewrites/cloudera/.beads/config.yaml | 62 - rewrites/cloudera/.beads/interactions.jsonl | 0 rewrites/cloudera/.beads/issues.jsonl | 0 rewrites/cloudera/.beads/metadata.json | 4 - rewrites/cloudera/.gitattributes | 3 - rewrites/cloudera/AGENTS.md | 40 - rewrites/composio/.beads/.gitignore | 39 - rewrites/composio/.beads/README.md | 81 -- rewrites/composio/.beads/config.yaml | 62 - rewrites/composio/.beads/interactions.jsonl | 0 rewrites/composio/.beads/issues.jsonl | 14 - rewrites/composio/.beads/metadata.json | 4 - rewrites/composio/.gitattributes | 3 - rewrites/composio/AGENTS.md | 212 --- rewrites/composio/README.md | 364 ----- rewrites/composio/package.json | 102 -- rewrites/composio/src/index.ts | 19 - rewrites/composio/src/sdk/composio.ts | 198 --- rewrites/composio/src/sdk/types.ts | 163 --- rewrites/confluence/.beads/.gitignore | 39 - rewrites/confluence/.beads/README.md | 81 -- rewrites/confluence/.beads/config.yaml | 62 - rewrites/confluence/.beads/interactions.jsonl | 0 rewrites/confluence/.beads/issues.jsonl | 0 rewrites/confluence/.beads/metadata.json | 4 - rewrites/confluence/.gitattributes | 3 - rewrites/confluence/AGENTS.md | 40 - rewrites/confluence/README.md | 475 ------ rewrites/coupa/.beads/.gitignore | 39 - rewrites/coupa/.beads/README.md | 81 -- rewrites/coupa/.beads/config.yaml | 62 - rewrites/coupa/.beads/interactions.jsonl | 0 rewrites/coupa/.beads/issues.jsonl | 0 rewrites/coupa/.beads/metadata.json | 4 - rewrites/coupa/.gitattributes | 3 - rewrites/coupa/AGENTS.md | 40 - rewrites/coupa/README.md | 520 ------- rewrites/customerio/.beads/.gitignore | 39 - rewrites/customerio/.beads/README.md | 81 -- rewrites/customerio/.beads/config.yaml | 62 - rewrites/customerio/.beads/interactions.jsonl | 0 rewrites/customerio/.beads/issues.jsonl | 0 rewrites/customerio/.beads/metadata.json | 4 - rewrites/customerio/.gitattributes | 3 - rewrites/customerio/AGENTS.md | 112 -- rewrites/customerio/README.md | 432 ------ rewrites/databricks/.beads/.gitignore | 39 - rewrites/databricks/.beads/README.md | 81 -- rewrites/databricks/.beads/config.yaml | 62 - rewrites/databricks/.beads/interactions.jsonl | 0 rewrites/databricks/.beads/metadata.json | 4 - rewrites/databricks/.gitattributes | 3 - rewrites/databricks/AGENTS.md | 40 - rewrites/databricks/README.md | 391 ----- rewrites/datadog/.beads/.gitignore | 39 - rewrites/datadog/.beads/README.md | 81 -- rewrites/datadog/.beads/config.yaml | 62 - rewrites/datadog/.beads/interactions.jsonl | 0 rewrites/datadog/.beads/issues.jsonl | 0 rewrites/datadog/.beads/metadata.json | 4 - rewrites/datadog/.gitattributes | 3 - rewrites/datadog/AGENTS.md | 40 - rewrites/datadog/README.md | 519 ------- rewrites/docs/research/analytics-cdp-scope.md | 755 ---------- rewrites/docs/research/auth-scope.md | 834 ----------- rewrites/docs/research/billing-scope.md | 1284 ----------------- rewrites/docs/research/errors-scope.md | 767 ---------- rewrites/docs/research/flags-scope.md | 890 ------------ rewrites/docs/research/notify-scope.md | 1120 -------------- rewrites/docs/research/search-scope.md | 540 ------- rewrites/docs/research/workflows-scope.md | 718 --------- rewrites/docs/reviews/architectural-review.md | 708 --------- rewrites/docs/reviews/general-code-review.md | 418 ------ .../docs/reviews/product-roadmap-review.md | 381 ----- rewrites/docs/reviews/typescript-review.md | 708 --------- rewrites/docusign/.beads/.gitignore | 39 - rewrites/docusign/.beads/README.md | 81 -- rewrites/docusign/.beads/config.yaml | 62 - rewrites/docusign/.beads/interactions.jsonl | 0 rewrites/docusign/.beads/issues.jsonl | 0 rewrites/docusign/.beads/metadata.json | 4 - rewrites/docusign/.gitattributes | 3 - rewrites/docusign/AGENTS.md | 40 - rewrites/docusign/README.md | 501 ------- rewrites/dynamics/.beads/.gitignore | 39 - rewrites/dynamics/.beads/README.md | 81 -- rewrites/dynamics/.beads/config.yaml | 62 - rewrites/dynamics/.beads/interactions.jsonl | 0 rewrites/dynamics/.beads/issues.jsonl | 0 rewrites/dynamics/.beads/metadata.json | 4 - rewrites/dynamics/.gitattributes | 3 - rewrites/dynamics/AGENTS.md | 40 - rewrites/dynamics/README.md | 547 ------- rewrites/epic/.beads/.gitignore | 39 - rewrites/epic/.beads/README.md | 81 -- rewrites/epic/.beads/config.yaml | 62 - rewrites/epic/.beads/interactions.jsonl | 0 rewrites/epic/.beads/issues.jsonl | 0 rewrites/epic/.beads/metadata.json | 4 - rewrites/epic/.gitattributes | 3 - rewrites/epic/AGENTS.md | 40 - rewrites/epic/README.md | 219 --- rewrites/firebolt/.beads/.gitignore | 39 - rewrites/firebolt/.beads/README.md | 81 -- rewrites/firebolt/.beads/config.yaml | 62 - rewrites/firebolt/.beads/interactions.jsonl | 0 rewrites/firebolt/.beads/issues.jsonl | 0 rewrites/firebolt/.beads/metadata.json | 4 - rewrites/firebolt/.gitattributes | 3 - rewrites/firebolt/AGENTS.md | 40 - rewrites/fivetran/.beads/.gitignore | 39 - rewrites/fivetran/.beads/README.md | 81 -- rewrites/fivetran/.beads/config.yaml | 62 - rewrites/fivetran/.beads/interactions.jsonl | 0 rewrites/fivetran/.beads/metadata.json | 4 - rewrites/fivetran/.gitattributes | 3 - rewrites/fivetran/AGENTS.md | 207 --- rewrites/fivetran/README.md | 449 ------ rewrites/fivetran/package.json | 53 - rewrites/greenhouse/.beads/.gitignore | 39 - rewrites/greenhouse/.beads/README.md | 81 -- rewrites/greenhouse/.beads/config.yaml | 62 - rewrites/greenhouse/.beads/interactions.jsonl | 0 rewrites/greenhouse/.beads/issues.jsonl | 100 -- rewrites/greenhouse/.beads/metadata.json | 4 - rewrites/greenhouse/.gitattributes | 3 - rewrites/greenhouse/AGENTS.md | 40 - rewrites/greenhouse/README.md | 111 -- rewrites/gusto/.beads/.gitignore | 39 - rewrites/gusto/.beads/README.md | 81 -- rewrites/gusto/.beads/config.yaml | 62 - rewrites/gusto/.beads/interactions.jsonl | 0 rewrites/gusto/.beads/issues.jsonl | 0 rewrites/gusto/.beads/metadata.json | 4 - rewrites/gusto/.gitattributes | 3 - rewrites/gusto/AGENTS.md | 40 - rewrites/gusto/README.md | 333 ----- rewrites/hana/.beads/.gitignore | 39 - rewrites/hana/.beads/README.md | 81 -- rewrites/hana/.beads/config.yaml | 62 - rewrites/hana/.beads/interactions.jsonl | 0 rewrites/hana/.beads/issues.jsonl | 0 rewrites/hana/.beads/metadata.json | 4 - rewrites/hana/.gitattributes | 3 - rewrites/hana/AGENTS.md | 40 - rewrites/hubspot/.beads/.gitignore | 39 - rewrites/hubspot/.beads/README.md | 81 -- rewrites/hubspot/.beads/config.yaml | 62 - rewrites/hubspot/.beads/interactions.jsonl | 0 rewrites/hubspot/.beads/issues.jsonl | 0 rewrites/hubspot/.beads/metadata.json | 4 - rewrites/hubspot/.gitattributes | 3 - rewrites/hubspot/AGENTS.md | 40 - rewrites/hubspot/README.md | 483 ------- rewrites/inngest/.beads/.gitignore | 39 - rewrites/inngest/.beads/README.md | 81 -- rewrites/inngest/.beads/config.yaml | 62 - rewrites/inngest/.beads/interactions.jsonl | 0 rewrites/inngest/.beads/issues.jsonl | 0 rewrites/inngest/.beads/metadata.json | 4 - rewrites/inngest/.gitattributes | 3 - rewrites/inngest/AGENTS.md | 178 --- rewrites/inngest/README.md | 415 ------ rewrites/intercom/.beads/.gitignore | 39 - rewrites/intercom/.beads/README.md | 81 -- rewrites/intercom/.beads/config.yaml | 62 - rewrites/intercom/.beads/interactions.jsonl | 0 rewrites/intercom/.beads/issues.jsonl | 0 rewrites/intercom/.beads/metadata.json | 4 - rewrites/intercom/.gitattributes | 3 - rewrites/intercom/AGENTS.md | 40 - rewrites/intercom/README.md | 389 ----- rewrites/jira/.beads/.gitignore | 39 - rewrites/jira/.beads/README.md | 81 -- rewrites/jira/.beads/config.yaml | 62 - rewrites/jira/.beads/interactions.jsonl | 0 rewrites/jira/.beads/issues.jsonl | 0 rewrites/jira/.beads/metadata.json | 4 - rewrites/jira/.gitattributes | 3 - rewrites/jira/AGENTS.md | 40 - rewrites/jira/README.md | 455 ------ rewrites/launchdarkly/.beads/.gitignore | 39 - rewrites/launchdarkly/.beads/README.md | 81 -- rewrites/launchdarkly/.beads/config.yaml | 62 - .../launchdarkly/.beads/interactions.jsonl | 0 rewrites/launchdarkly/.beads/issues.jsonl | 0 rewrites/launchdarkly/.beads/metadata.json | 4 - rewrites/launchdarkly/.gitattributes | 3 - rewrites/launchdarkly/AGENTS.md | 89 -- rewrites/launchdarkly/README.md | 395 ----- rewrites/linear/.beads/.gitignore | 39 - rewrites/linear/.beads/README.md | 81 -- rewrites/linear/.beads/config.yaml | 62 - rewrites/linear/.beads/interactions.jsonl | 0 rewrites/linear/.beads/issues.jsonl | 0 rewrites/linear/.beads/metadata.json | 4 - rewrites/linear/.gitattributes | 3 - rewrites/linear/AGENTS.md | 40 - rewrites/linear/README.md | 615 -------- rewrites/looker/.beads/.gitignore | 39 - rewrites/looker/.beads/README.md | 81 -- rewrites/looker/.beads/config.yaml | 62 - rewrites/looker/.beads/interactions.jsonl | 0 rewrites/looker/.beads/metadata.json | 4 - rewrites/looker/.gitattributes | 3 - rewrites/looker/AGENTS.md | 40 - rewrites/looker/README.md | 425 ------ rewrites/make/.beads/.gitignore | 39 - rewrites/make/.beads/README.md | 81 -- rewrites/make/.beads/config.yaml | 62 - rewrites/make/.beads/interactions.jsonl | 0 rewrites/make/.beads/issues.jsonl | 0 rewrites/make/.beads/metadata.json | 4 - rewrites/make/.gitattributes | 3 - rewrites/make/AGENTS.md | 217 --- rewrites/make/README.md | 437 ------ rewrites/make/package.json | 102 -- rewrites/mixpanel/.beads/.gitignore | 39 - rewrites/mixpanel/.beads/README.md | 81 -- rewrites/mixpanel/.beads/config.yaml | 62 - rewrites/mixpanel/.beads/interactions.jsonl | 0 rewrites/mixpanel/.beads/issues.jsonl | 0 rewrites/mixpanel/.beads/metadata.json | 4 - rewrites/mixpanel/.gitattributes | 3 - rewrites/mixpanel/AGENTS.md | 40 - rewrites/mixpanel/README.md | 431 ------ rewrites/monday/.beads/.gitignore | 39 - rewrites/monday/.beads/README.md | 81 -- rewrites/monday/.beads/config.yaml | 62 - rewrites/monday/.beads/interactions.jsonl | 0 rewrites/monday/.beads/issues.jsonl | 0 rewrites/monday/.beads/metadata.json | 4 - rewrites/monday/.gitattributes | 3 - rewrites/monday/AGENTS.md | 40 - rewrites/monday/README.md | 585 -------- rewrites/n8n/.beads/.gitignore | 39 - rewrites/n8n/.beads/README.md | 81 -- rewrites/n8n/.beads/config.yaml | 62 - rewrites/n8n/.beads/interactions.jsonl | 0 rewrites/n8n/.beads/issues.jsonl | 0 rewrites/n8n/.beads/metadata.json | 4 - rewrites/n8n/.gitattributes | 3 - rewrites/n8n/AGENTS.md | 209 --- rewrites/n8n/README.md | 387 ----- rewrites/n8n/package.json | 74 - .../n8n/src/durable-object/CredentialDO.ts | 295 ---- .../n8n/src/durable-object/ExecutionDO.ts | 285 ---- rewrites/n8n/src/durable-object/WorkflowDO.ts | 254 ---- rewrites/n8n/src/durable-object/index.ts | 7 - rewrites/n8n/src/index.ts | 13 - rewrites/n8n/src/mcp/index.ts | 6 - rewrites/n8n/src/mcp/tools.ts | 246 ---- rewrites/n8n/src/sdk/n8n.ts | 156 -- rewrites/n8n/src/sdk/types.ts | 198 --- rewrites/netsuite/.beads/.gitignore | 39 - rewrites/netsuite/.beads/README.md | 81 -- rewrites/netsuite/.beads/config.yaml | 62 - rewrites/netsuite/.beads/interactions.jsonl | 0 rewrites/netsuite/.beads/issues.jsonl | 0 rewrites/netsuite/.beads/metadata.json | 4 - rewrites/netsuite/.gitattributes | 3 - rewrites/netsuite/AGENTS.md | 40 - rewrites/netsuite/README.md | 698 --------- rewrites/notion/.beads/.gitignore | 39 - rewrites/notion/.beads/README.md | 81 -- rewrites/notion/.beads/config.yaml | 62 - rewrites/notion/.beads/interactions.jsonl | 0 rewrites/notion/.beads/issues.jsonl | 0 rewrites/notion/.beads/metadata.json | 4 - rewrites/notion/.gitattributes | 3 - rewrites/notion/AGENTS.md | 40 - rewrites/notion/README.md | 545 ------- rewrites/orb/.beads/.gitignore | 39 - rewrites/orb/.beads/README.md | 81 -- rewrites/orb/.beads/config.yaml | 62 - rewrites/orb/.beads/interactions.jsonl | 0 rewrites/orb/.beads/issues.jsonl | 0 rewrites/orb/.beads/metadata.json | 4 - rewrites/orb/.gitattributes | 3 - rewrites/orb/AGENTS.md | 139 -- rewrites/orb/README.md | 399 ----- rewrites/pipedrive/.beads/.gitignore | 39 - rewrites/pipedrive/.beads/README.md | 81 -- rewrites/pipedrive/.beads/config.yaml | 62 - rewrites/pipedrive/.beads/interactions.jsonl | 0 rewrites/pipedrive/.beads/issues.jsonl | 0 rewrites/pipedrive/.beads/metadata.json | 4 - rewrites/pipedrive/.gitattributes | 3 - rewrites/pipedrive/AGENTS.md | 40 - rewrites/pipedrive/README.md | 434 ------ rewrites/posthog/.beads/.gitignore | 39 - rewrites/posthog/.beads/README.md | 81 -- rewrites/posthog/.beads/config.yaml | 62 - rewrites/posthog/.beads/interactions.jsonl | 0 rewrites/posthog/.beads/issues.jsonl | 0 rewrites/posthog/.beads/metadata.json | 4 - rewrites/posthog/.gitattributes | 3 - rewrites/posthog/AGENTS.md | 103 -- rewrites/posthog/README.md | 325 ----- rewrites/powerbi/.beads/.gitignore | 39 - rewrites/powerbi/.beads/README.md | 81 -- rewrites/powerbi/.beads/config.yaml | 62 - rewrites/powerbi/.beads/interactions.jsonl | 0 rewrites/powerbi/.beads/metadata.json | 4 - rewrites/powerbi/.gitattributes | 3 - rewrites/powerbi/AGENTS.md | 40 - rewrites/powerbi/README.md | 573 -------- rewrites/procore/.beads/.gitignore | 39 - rewrites/procore/.beads/README.md | 81 -- rewrites/procore/.beads/config.yaml | 62 - rewrites/procore/.beads/interactions.jsonl | 0 rewrites/procore/.beads/issues.jsonl | 0 rewrites/procore/.beads/metadata.json | 4 - rewrites/procore/.gitattributes | 3 - rewrites/procore/AGENTS.md | 40 - rewrites/procore/README.md | 578 -------- rewrites/rippling/.beads/.gitignore | 39 - rewrites/rippling/.beads/README.md | 81 -- rewrites/rippling/.beads/config.yaml | 62 - rewrites/rippling/.beads/interactions.jsonl | 0 rewrites/rippling/.beads/issues.jsonl | 0 rewrites/rippling/.beads/metadata.json | 4 - rewrites/rippling/.gitattributes | 3 - rewrites/rippling/AGENTS.md | 40 - rewrites/rippling/README.md | 623 -------- rewrites/salesforce/.beads/.gitignore | 39 - rewrites/salesforce/.beads/README.md | 81 -- rewrites/salesforce/.beads/config.yaml | 62 - rewrites/salesforce/.beads/interactions.jsonl | 0 rewrites/salesforce/.beads/issues.jsonl | 23 - rewrites/salesforce/.beads/metadata.json | 4 - rewrites/salesforce/.gitattributes | 3 - rewrites/salesforce/AGENTS.md | 40 - rewrites/salesforce/README.md | 438 ------ rewrites/sap/.beads/.gitignore | 39 - rewrites/sap/.beads/README.md | 81 -- rewrites/sap/.beads/config.yaml | 62 - rewrites/sap/.beads/interactions.jsonl | 0 rewrites/sap/.beads/issues.jsonl | 0 rewrites/sap/.beads/metadata.json | 4 - rewrites/sap/.gitattributes | 3 - rewrites/sap/AGENTS.md | 40 - rewrites/sap/README.md | 568 -------- rewrites/segment/.beads/.gitignore | 39 - rewrites/segment/.beads/README.md | 81 -- rewrites/segment/.beads/config.yaml | 62 - rewrites/segment/.beads/interactions.jsonl | 0 rewrites/segment/.beads/issues.jsonl | 0 rewrites/segment/.beads/metadata.json | 4 - rewrites/segment/.gitattributes | 3 - rewrites/segment/AGENTS.md | 145 -- rewrites/segment/README.md | 504 ------- rewrites/sentry/.beads/.gitignore | 39 - rewrites/sentry/.beads/README.md | 81 -- rewrites/sentry/.beads/config.yaml | 62 - rewrites/sentry/.beads/interactions.jsonl | 0 rewrites/sentry/.beads/issues.jsonl | 0 rewrites/sentry/.beads/metadata.json | 4 - rewrites/sentry/.gitattributes | 3 - rewrites/sentry/AGENTS.md | 78 - rewrites/sentry/README.md | 423 ------ rewrites/servicenow/.beads/.gitignore | 39 - rewrites/servicenow/.beads/README.md | 81 -- rewrites/servicenow/.beads/config.yaml | 62 - rewrites/servicenow/.beads/interactions.jsonl | 0 rewrites/servicenow/.beads/issues.jsonl | 0 rewrites/servicenow/.beads/metadata.json | 4 - rewrites/servicenow/.gitattributes | 3 - rewrites/servicenow/AGENTS.md | 40 - rewrites/servicenow/README.md | 498 ------- rewrites/servicetitan/.beads/.gitignore | 39 - rewrites/servicetitan/.beads/README.md | 81 -- rewrites/servicetitan/.beads/config.yaml | 62 - .../servicetitan/.beads/interactions.jsonl | 0 rewrites/servicetitan/.beads/issues.jsonl | 0 rewrites/servicetitan/.beads/metadata.json | 4 - rewrites/servicetitan/.gitattributes | 3 - rewrites/servicetitan/AGENTS.md | 40 - rewrites/servicetitan/README.md | 463 ------ rewrites/shopify/.beads/.gitignore | 39 - rewrites/shopify/.beads/README.md | 81 -- rewrites/shopify/.beads/config.yaml | 62 - rewrites/shopify/.beads/interactions.jsonl | 0 rewrites/shopify/.beads/issues.jsonl | 0 rewrites/shopify/.beads/metadata.json | 4 - rewrites/shopify/.gitattributes | 3 - rewrites/shopify/AGENTS.md | 40 - rewrites/shopify/README.md | 474 ------ rewrites/sierra/.beads/.gitignore | 39 - rewrites/sierra/.beads/README.md | 81 -- rewrites/sierra/.beads/config.yaml | 62 - rewrites/sierra/.beads/interactions.jsonl | 0 rewrites/sierra/.beads/issues.jsonl | 0 rewrites/sierra/.beads/metadata.json | 4 - rewrites/sierra/.gitattributes | 3 - rewrites/sierra/AGENTS.md | 40 - rewrites/snowflake/.beads/.gitignore | 39 - rewrites/snowflake/.beads/README.md | 81 -- rewrites/snowflake/.beads/config.yaml | 62 - rewrites/snowflake/.beads/interactions.jsonl | 0 rewrites/snowflake/.beads/metadata.json | 4 - rewrites/snowflake/.gitattributes | 3 - rewrites/snowflake/AGENTS.md | 40 - rewrites/snowflake/README.md | 517 ------- rewrites/splunk/.beads/.gitignore | 39 - rewrites/splunk/.beads/README.md | 81 -- rewrites/splunk/.beads/config.yaml | 62 - rewrites/splunk/.beads/interactions.jsonl | 0 rewrites/splunk/.beads/issues.jsonl | 0 rewrites/splunk/.beads/metadata.json | 4 - rewrites/splunk/.gitattributes | 3 - rewrites/splunk/AGENTS.md | 40 - rewrites/splunk/README.md | 626 -------- rewrites/studio/.beads/.gitignore | 39 - rewrites/studio/.beads/README.md | 81 -- rewrites/studio/.beads/config.yaml | 62 - rewrites/studio/.beads/interactions.jsonl | 0 rewrites/studio/.beads/issues.jsonl | 0 rewrites/studio/.beads/metadata.json | 4 - rewrites/studio/.gitattributes | 3 - rewrites/studio/AGENTS.md | 40 - rewrites/studio/README.md | 363 ----- rewrites/tableau/.beads/.gitignore | 39 - rewrites/tableau/.beads/README.md | 81 -- rewrites/tableau/.beads/config.yaml | 62 - rewrites/tableau/.beads/interactions.jsonl | 0 rewrites/tableau/.beads/metadata.json | 4 - rewrites/tableau/.gitattributes | 3 - rewrites/tableau/AGENTS.md | 40 - rewrites/tableau/README.md | 441 ------ rewrites/temporal/.beads/.gitignore | 39 - rewrites/temporal/.beads/README.md | 81 -- rewrites/temporal/.beads/config.yaml | 62 - rewrites/temporal/.beads/interactions.jsonl | 0 rewrites/temporal/.beads/issues.jsonl | 0 rewrites/temporal/.beads/metadata.json | 4 - rewrites/temporal/.gitattributes | 3 - rewrites/temporal/AGENTS.md | 206 --- rewrites/temporal/README.md | 416 ------ rewrites/temporal/package.json | 80 - rewrites/temporal/src/.gitkeep | 0 rewrites/teradata/.beads/.gitignore | 39 - rewrites/teradata/.beads/README.md | 81 -- rewrites/teradata/.beads/config.yaml | 62 - rewrites/teradata/.beads/interactions.jsonl | 0 rewrites/teradata/.beads/issues.jsonl | 0 rewrites/teradata/.beads/metadata.json | 4 - rewrites/teradata/.gitattributes | 3 - rewrites/teradata/AGENTS.md | 40 - rewrites/teradata/TDD-BREAKDOWN.md | 875 ----------- rewrites/toast/.beads/.gitignore | 39 - rewrites/toast/.beads/README.md | 81 -- rewrites/toast/.beads/config.yaml | 62 - rewrites/toast/.beads/interactions.jsonl | 0 rewrites/toast/.beads/issues.jsonl | 28 - rewrites/toast/.beads/metadata.json | 4 - rewrites/toast/.gitattributes | 3 - rewrites/toast/AGENTS.md | 40 - rewrites/toast/README.md | 472 ------ rewrites/trigger/.beads/.gitignore | 39 - rewrites/trigger/.beads/README.md | 81 -- rewrites/trigger/.beads/config.yaml | 62 - rewrites/trigger/.beads/interactions.jsonl | 0 rewrites/trigger/.beads/issues.jsonl | 0 rewrites/trigger/.beads/metadata.json | 4 - rewrites/trigger/.gitattributes | 3 - rewrites/trigger/AGENTS.md | 224 --- rewrites/trigger/README.md | 302 ---- rewrites/trigger/package.json | 94 -- rewrites/trigger/src/core/index.ts | 7 - rewrites/trigger/src/core/schedules.ts | 54 - rewrites/trigger/src/core/task.ts | 68 - rewrites/trigger/src/core/trigger.ts | 52 - rewrites/trigger/src/index.ts | 15 - rewrites/trigger/src/types.ts | 195 --- rewrites/veeva/.beads/.gitignore | 39 - rewrites/veeva/.beads/README.md | 81 -- rewrites/veeva/.beads/config.yaml | 62 - rewrites/veeva/.beads/interactions.jsonl | 0 rewrites/veeva/.beads/issues.jsonl | 111 -- rewrites/veeva/.beads/metadata.json | 4 - rewrites/veeva/.gitattributes | 3 - rewrites/veeva/AGENTS.md | 40 - rewrites/veeva/README.md | 112 -- rewrites/workday/.beads/.gitignore | 39 - rewrites/workday/.beads/README.md | 81 -- rewrites/workday/.beads/config.yaml | 62 - rewrites/workday/.beads/interactions.jsonl | 0 rewrites/workday/.beads/issues.jsonl | 22 - rewrites/workday/.beads/metadata.json | 4 - rewrites/workday/.gitattributes | 3 - rewrites/workday/AGENTS.md | 40 - rewrites/workday/README.md | 527 ------- rewrites/yardi/.beads/.gitignore | 39 - rewrites/yardi/.beads/README.md | 81 -- rewrites/yardi/.beads/config.yaml | 62 - rewrites/yardi/.beads/interactions.jsonl | 0 rewrites/yardi/.beads/issues.jsonl | 0 rewrites/yardi/.beads/metadata.json | 4 - rewrites/yardi/.gitattributes | 3 - rewrites/yardi/AGENTS.md | 40 - rewrites/yardi/README.md | 496 ------- rewrites/zapier/.beads/.gitignore | 39 - rewrites/zapier/.beads/README.md | 81 -- rewrites/zapier/.beads/config.yaml | 62 - rewrites/zapier/.beads/interactions.jsonl | 0 rewrites/zapier/.beads/issues.jsonl | 14 - rewrites/zapier/.beads/metadata.json | 4 - rewrites/zapier/.gitattributes | 4 - rewrites/zapier/AGENTS.md | 194 --- rewrites/zapier/README.md | 333 ----- rewrites/zapier/package.json | 82 -- rewrites/zendesk/.beads/.gitignore | 39 - rewrites/zendesk/.beads/README.md | 81 -- rewrites/zendesk/.beads/config.yaml | 62 - rewrites/zendesk/.beads/interactions.jsonl | 0 rewrites/zendesk/.beads/issues.jsonl | 0 rewrites/zendesk/.beads/metadata.json | 4 - rewrites/zendesk/.gitattributes | 3 - rewrites/zendesk/AGENTS.md | 40 - rewrites/zendesk/README.md | 442 ------ rewrites/zoho/.beads/.gitignore | 39 - rewrites/zoho/.beads/README.md | 81 -- rewrites/zoho/.beads/config.yaml | 62 - rewrites/zoho/.beads/interactions.jsonl | 0 rewrites/zoho/.beads/issues.jsonl | 0 rewrites/zoho/.beads/metadata.json | 4 - rewrites/zoho/.gitattributes | 3 - rewrites/zoho/AGENTS.md | 40 - rewrites/zoho/README.md | 415 ------ 617 files changed, 57159 deletions(-) delete mode 100644 rewrites/airbyte/.beads/.gitignore delete mode 100644 rewrites/airbyte/.beads/README.md delete mode 100644 rewrites/airbyte/.beads/config.yaml delete mode 100644 rewrites/airbyte/.beads/interactions.jsonl delete mode 100644 rewrites/airbyte/.beads/issues.jsonl delete mode 100644 rewrites/airbyte/.beads/metadata.json delete mode 100644 rewrites/airbyte/.gitattributes delete mode 100644 rewrites/airbyte/AGENTS.md delete mode 100644 rewrites/airbyte/README.md delete mode 100644 rewrites/airbyte/package.json delete mode 100644 rewrites/airbyte/src/index.ts delete mode 100644 rewrites/airbyte/src/sdk/airbyte.ts delete mode 100644 rewrites/airbyte/src/sdk/types.ts delete mode 100644 rewrites/airtable/.beads/.gitignore delete mode 100644 rewrites/airtable/.beads/README.md delete mode 100644 rewrites/airtable/.beads/config.yaml delete mode 100644 rewrites/airtable/.beads/interactions.jsonl delete mode 100644 rewrites/airtable/.beads/issues.jsonl delete mode 100644 rewrites/airtable/.beads/metadata.json delete mode 100644 rewrites/airtable/.gitattributes delete mode 100644 rewrites/airtable/AGENTS.md delete mode 100644 rewrites/airtable/README.md delete mode 100644 rewrites/algolia/.beads/.gitignore delete mode 100644 rewrites/algolia/.beads/README.md delete mode 100644 rewrites/algolia/.beads/config.yaml delete mode 100644 rewrites/algolia/.beads/interactions.jsonl delete mode 100644 rewrites/algolia/.beads/issues.jsonl delete mode 100644 rewrites/algolia/.beads/metadata.json delete mode 100644 rewrites/algolia/.gitattributes delete mode 100644 rewrites/algolia/AGENTS.md delete mode 100644 rewrites/algolia/README.md delete mode 100644 rewrites/amplitude/.beads/.gitignore delete mode 100644 rewrites/amplitude/.beads/README.md delete mode 100644 rewrites/amplitude/.beads/config.yaml delete mode 100644 rewrites/amplitude/.beads/interactions.jsonl delete mode 100644 rewrites/amplitude/.beads/issues.jsonl delete mode 100644 rewrites/amplitude/.beads/metadata.json delete mode 100644 rewrites/amplitude/.gitattributes delete mode 100644 rewrites/amplitude/AGENTS.md delete mode 100644 rewrites/amplitude/README.md delete mode 100644 rewrites/asana/.beads/.gitignore delete mode 100644 rewrites/asana/.beads/README.md delete mode 100644 rewrites/asana/.beads/config.yaml delete mode 100644 rewrites/asana/.beads/interactions.jsonl delete mode 100644 rewrites/asana/.beads/issues.jsonl delete mode 100644 rewrites/asana/.beads/metadata.json delete mode 100644 rewrites/asana/.gitattributes delete mode 100644 rewrites/asana/AGENTS.md delete mode 100644 rewrites/asana/README.md delete mode 100644 rewrites/athena/.beads/.gitignore delete mode 100644 rewrites/athena/.beads/README.md delete mode 100644 rewrites/athena/.beads/config.yaml delete mode 100644 rewrites/athena/.beads/interactions.jsonl delete mode 100644 rewrites/athena/.beads/issues.jsonl delete mode 100644 rewrites/athena/.beads/metadata.json delete mode 100644 rewrites/athena/.gitattributes delete mode 100644 rewrites/athena/AGENTS.md delete mode 100644 rewrites/athena/README.md delete mode 100644 rewrites/bamboohr/.beads/.gitignore delete mode 100644 rewrites/bamboohr/.beads/README.md delete mode 100644 rewrites/bamboohr/.beads/config.yaml delete mode 100644 rewrites/bamboohr/.beads/interactions.jsonl delete mode 100644 rewrites/bamboohr/.beads/issues.jsonl delete mode 100644 rewrites/bamboohr/.beads/metadata.json delete mode 100644 rewrites/bamboohr/.gitattributes delete mode 100644 rewrites/bamboohr/AGENTS.md delete mode 100644 rewrites/bamboohr/README.md delete mode 100644 rewrites/cerner/.beads/.gitignore delete mode 100644 rewrites/cerner/.beads/README.md delete mode 100644 rewrites/cerner/.beads/config.yaml delete mode 100644 rewrites/cerner/.beads/interactions.jsonl delete mode 100644 rewrites/cerner/.beads/issues.jsonl delete mode 100644 rewrites/cerner/.beads/metadata.json delete mode 100644 rewrites/cerner/.gitattributes delete mode 100644 rewrites/cerner/AGENTS.md delete mode 100644 rewrites/cerner/README.md delete mode 100644 rewrites/clio/.beads/.gitignore delete mode 100644 rewrites/clio/.beads/README.md delete mode 100644 rewrites/clio/.beads/config.yaml delete mode 100644 rewrites/clio/.beads/interactions.jsonl delete mode 100644 rewrites/clio/.beads/issues.jsonl delete mode 100644 rewrites/clio/.beads/metadata.json delete mode 100644 rewrites/clio/.gitattributes delete mode 100644 rewrites/clio/AGENTS.md delete mode 100644 rewrites/clio/README.md delete mode 100644 rewrites/cloudera/.beads/.gitignore delete mode 100644 rewrites/cloudera/.beads/README.md delete mode 100644 rewrites/cloudera/.beads/config.yaml delete mode 100644 rewrites/cloudera/.beads/interactions.jsonl delete mode 100644 rewrites/cloudera/.beads/issues.jsonl delete mode 100644 rewrites/cloudera/.beads/metadata.json delete mode 100644 rewrites/cloudera/.gitattributes delete mode 100644 rewrites/cloudera/AGENTS.md delete mode 100644 rewrites/composio/.beads/.gitignore delete mode 100644 rewrites/composio/.beads/README.md delete mode 100644 rewrites/composio/.beads/config.yaml delete mode 100644 rewrites/composio/.beads/interactions.jsonl delete mode 100644 rewrites/composio/.beads/issues.jsonl delete mode 100644 rewrites/composio/.beads/metadata.json delete mode 100644 rewrites/composio/.gitattributes delete mode 100644 rewrites/composio/AGENTS.md delete mode 100644 rewrites/composio/README.md delete mode 100644 rewrites/composio/package.json delete mode 100644 rewrites/composio/src/index.ts delete mode 100644 rewrites/composio/src/sdk/composio.ts delete mode 100644 rewrites/composio/src/sdk/types.ts delete mode 100644 rewrites/confluence/.beads/.gitignore delete mode 100644 rewrites/confluence/.beads/README.md delete mode 100644 rewrites/confluence/.beads/config.yaml delete mode 100644 rewrites/confluence/.beads/interactions.jsonl delete mode 100644 rewrites/confluence/.beads/issues.jsonl delete mode 100644 rewrites/confluence/.beads/metadata.json delete mode 100644 rewrites/confluence/.gitattributes delete mode 100644 rewrites/confluence/AGENTS.md delete mode 100644 rewrites/confluence/README.md delete mode 100644 rewrites/coupa/.beads/.gitignore delete mode 100644 rewrites/coupa/.beads/README.md delete mode 100644 rewrites/coupa/.beads/config.yaml delete mode 100644 rewrites/coupa/.beads/interactions.jsonl delete mode 100644 rewrites/coupa/.beads/issues.jsonl delete mode 100644 rewrites/coupa/.beads/metadata.json delete mode 100644 rewrites/coupa/.gitattributes delete mode 100644 rewrites/coupa/AGENTS.md delete mode 100644 rewrites/coupa/README.md delete mode 100644 rewrites/customerio/.beads/.gitignore delete mode 100644 rewrites/customerio/.beads/README.md delete mode 100644 rewrites/customerio/.beads/config.yaml delete mode 100644 rewrites/customerio/.beads/interactions.jsonl delete mode 100644 rewrites/customerio/.beads/issues.jsonl delete mode 100644 rewrites/customerio/.beads/metadata.json delete mode 100644 rewrites/customerio/.gitattributes delete mode 100644 rewrites/customerio/AGENTS.md delete mode 100644 rewrites/customerio/README.md delete mode 100644 rewrites/databricks/.beads/.gitignore delete mode 100644 rewrites/databricks/.beads/README.md delete mode 100644 rewrites/databricks/.beads/config.yaml delete mode 100644 rewrites/databricks/.beads/interactions.jsonl delete mode 100644 rewrites/databricks/.beads/metadata.json delete mode 100644 rewrites/databricks/.gitattributes delete mode 100644 rewrites/databricks/AGENTS.md delete mode 100644 rewrites/databricks/README.md delete mode 100644 rewrites/datadog/.beads/.gitignore delete mode 100644 rewrites/datadog/.beads/README.md delete mode 100644 rewrites/datadog/.beads/config.yaml delete mode 100644 rewrites/datadog/.beads/interactions.jsonl delete mode 100644 rewrites/datadog/.beads/issues.jsonl delete mode 100644 rewrites/datadog/.beads/metadata.json delete mode 100644 rewrites/datadog/.gitattributes delete mode 100644 rewrites/datadog/AGENTS.md delete mode 100644 rewrites/datadog/README.md delete mode 100644 rewrites/docs/research/analytics-cdp-scope.md delete mode 100644 rewrites/docs/research/auth-scope.md delete mode 100644 rewrites/docs/research/billing-scope.md delete mode 100644 rewrites/docs/research/errors-scope.md delete mode 100644 rewrites/docs/research/flags-scope.md delete mode 100644 rewrites/docs/research/notify-scope.md delete mode 100644 rewrites/docs/research/search-scope.md delete mode 100644 rewrites/docs/research/workflows-scope.md delete mode 100644 rewrites/docs/reviews/architectural-review.md delete mode 100644 rewrites/docs/reviews/general-code-review.md delete mode 100644 rewrites/docs/reviews/product-roadmap-review.md delete mode 100644 rewrites/docs/reviews/typescript-review.md delete mode 100644 rewrites/docusign/.beads/.gitignore delete mode 100644 rewrites/docusign/.beads/README.md delete mode 100644 rewrites/docusign/.beads/config.yaml delete mode 100644 rewrites/docusign/.beads/interactions.jsonl delete mode 100644 rewrites/docusign/.beads/issues.jsonl delete mode 100644 rewrites/docusign/.beads/metadata.json delete mode 100644 rewrites/docusign/.gitattributes delete mode 100644 rewrites/docusign/AGENTS.md delete mode 100644 rewrites/docusign/README.md delete mode 100644 rewrites/dynamics/.beads/.gitignore delete mode 100644 rewrites/dynamics/.beads/README.md delete mode 100644 rewrites/dynamics/.beads/config.yaml delete mode 100644 rewrites/dynamics/.beads/interactions.jsonl delete mode 100644 rewrites/dynamics/.beads/issues.jsonl delete mode 100644 rewrites/dynamics/.beads/metadata.json delete mode 100644 rewrites/dynamics/.gitattributes delete mode 100644 rewrites/dynamics/AGENTS.md delete mode 100644 rewrites/dynamics/README.md delete mode 100644 rewrites/epic/.beads/.gitignore delete mode 100644 rewrites/epic/.beads/README.md delete mode 100644 rewrites/epic/.beads/config.yaml delete mode 100644 rewrites/epic/.beads/interactions.jsonl delete mode 100644 rewrites/epic/.beads/issues.jsonl delete mode 100644 rewrites/epic/.beads/metadata.json delete mode 100644 rewrites/epic/.gitattributes delete mode 100644 rewrites/epic/AGENTS.md delete mode 100644 rewrites/epic/README.md delete mode 100644 rewrites/firebolt/.beads/.gitignore delete mode 100644 rewrites/firebolt/.beads/README.md delete mode 100644 rewrites/firebolt/.beads/config.yaml delete mode 100644 rewrites/firebolt/.beads/interactions.jsonl delete mode 100644 rewrites/firebolt/.beads/issues.jsonl delete mode 100644 rewrites/firebolt/.beads/metadata.json delete mode 100644 rewrites/firebolt/.gitattributes delete mode 100644 rewrites/firebolt/AGENTS.md delete mode 100644 rewrites/fivetran/.beads/.gitignore delete mode 100644 rewrites/fivetran/.beads/README.md delete mode 100644 rewrites/fivetran/.beads/config.yaml delete mode 100644 rewrites/fivetran/.beads/interactions.jsonl delete mode 100644 rewrites/fivetran/.beads/metadata.json delete mode 100644 rewrites/fivetran/.gitattributes delete mode 100644 rewrites/fivetran/AGENTS.md delete mode 100644 rewrites/fivetran/README.md delete mode 100644 rewrites/fivetran/package.json delete mode 100644 rewrites/greenhouse/.beads/.gitignore delete mode 100644 rewrites/greenhouse/.beads/README.md delete mode 100644 rewrites/greenhouse/.beads/config.yaml delete mode 100644 rewrites/greenhouse/.beads/interactions.jsonl delete mode 100644 rewrites/greenhouse/.beads/issues.jsonl delete mode 100644 rewrites/greenhouse/.beads/metadata.json delete mode 100644 rewrites/greenhouse/.gitattributes delete mode 100644 rewrites/greenhouse/AGENTS.md delete mode 100644 rewrites/greenhouse/README.md delete mode 100644 rewrites/gusto/.beads/.gitignore delete mode 100644 rewrites/gusto/.beads/README.md delete mode 100644 rewrites/gusto/.beads/config.yaml delete mode 100644 rewrites/gusto/.beads/interactions.jsonl delete mode 100644 rewrites/gusto/.beads/issues.jsonl delete mode 100644 rewrites/gusto/.beads/metadata.json delete mode 100644 rewrites/gusto/.gitattributes delete mode 100644 rewrites/gusto/AGENTS.md delete mode 100644 rewrites/gusto/README.md delete mode 100644 rewrites/hana/.beads/.gitignore delete mode 100644 rewrites/hana/.beads/README.md delete mode 100644 rewrites/hana/.beads/config.yaml delete mode 100644 rewrites/hana/.beads/interactions.jsonl delete mode 100644 rewrites/hana/.beads/issues.jsonl delete mode 100644 rewrites/hana/.beads/metadata.json delete mode 100644 rewrites/hana/.gitattributes delete mode 100644 rewrites/hana/AGENTS.md delete mode 100644 rewrites/hubspot/.beads/.gitignore delete mode 100644 rewrites/hubspot/.beads/README.md delete mode 100644 rewrites/hubspot/.beads/config.yaml delete mode 100644 rewrites/hubspot/.beads/interactions.jsonl delete mode 100644 rewrites/hubspot/.beads/issues.jsonl delete mode 100644 rewrites/hubspot/.beads/metadata.json delete mode 100644 rewrites/hubspot/.gitattributes delete mode 100644 rewrites/hubspot/AGENTS.md delete mode 100644 rewrites/hubspot/README.md delete mode 100644 rewrites/inngest/.beads/.gitignore delete mode 100644 rewrites/inngest/.beads/README.md delete mode 100644 rewrites/inngest/.beads/config.yaml delete mode 100644 rewrites/inngest/.beads/interactions.jsonl delete mode 100644 rewrites/inngest/.beads/issues.jsonl delete mode 100644 rewrites/inngest/.beads/metadata.json delete mode 100644 rewrites/inngest/.gitattributes delete mode 100644 rewrites/inngest/AGENTS.md delete mode 100644 rewrites/inngest/README.md delete mode 100644 rewrites/intercom/.beads/.gitignore delete mode 100644 rewrites/intercom/.beads/README.md delete mode 100644 rewrites/intercom/.beads/config.yaml delete mode 100644 rewrites/intercom/.beads/interactions.jsonl delete mode 100644 rewrites/intercom/.beads/issues.jsonl delete mode 100644 rewrites/intercom/.beads/metadata.json delete mode 100644 rewrites/intercom/.gitattributes delete mode 100644 rewrites/intercom/AGENTS.md delete mode 100644 rewrites/intercom/README.md delete mode 100644 rewrites/jira/.beads/.gitignore delete mode 100644 rewrites/jira/.beads/README.md delete mode 100644 rewrites/jira/.beads/config.yaml delete mode 100644 rewrites/jira/.beads/interactions.jsonl delete mode 100644 rewrites/jira/.beads/issues.jsonl delete mode 100644 rewrites/jira/.beads/metadata.json delete mode 100644 rewrites/jira/.gitattributes delete mode 100644 rewrites/jira/AGENTS.md delete mode 100644 rewrites/jira/README.md delete mode 100644 rewrites/launchdarkly/.beads/.gitignore delete mode 100644 rewrites/launchdarkly/.beads/README.md delete mode 100644 rewrites/launchdarkly/.beads/config.yaml delete mode 100644 rewrites/launchdarkly/.beads/interactions.jsonl delete mode 100644 rewrites/launchdarkly/.beads/issues.jsonl delete mode 100644 rewrites/launchdarkly/.beads/metadata.json delete mode 100644 rewrites/launchdarkly/.gitattributes delete mode 100644 rewrites/launchdarkly/AGENTS.md delete mode 100644 rewrites/launchdarkly/README.md delete mode 100644 rewrites/linear/.beads/.gitignore delete mode 100644 rewrites/linear/.beads/README.md delete mode 100644 rewrites/linear/.beads/config.yaml delete mode 100644 rewrites/linear/.beads/interactions.jsonl delete mode 100644 rewrites/linear/.beads/issues.jsonl delete mode 100644 rewrites/linear/.beads/metadata.json delete mode 100644 rewrites/linear/.gitattributes delete mode 100644 rewrites/linear/AGENTS.md delete mode 100644 rewrites/linear/README.md delete mode 100644 rewrites/looker/.beads/.gitignore delete mode 100644 rewrites/looker/.beads/README.md delete mode 100644 rewrites/looker/.beads/config.yaml delete mode 100644 rewrites/looker/.beads/interactions.jsonl delete mode 100644 rewrites/looker/.beads/metadata.json delete mode 100644 rewrites/looker/.gitattributes delete mode 100644 rewrites/looker/AGENTS.md delete mode 100644 rewrites/looker/README.md delete mode 100644 rewrites/make/.beads/.gitignore delete mode 100644 rewrites/make/.beads/README.md delete mode 100644 rewrites/make/.beads/config.yaml delete mode 100644 rewrites/make/.beads/interactions.jsonl delete mode 100644 rewrites/make/.beads/issues.jsonl delete mode 100644 rewrites/make/.beads/metadata.json delete mode 100644 rewrites/make/.gitattributes delete mode 100644 rewrites/make/AGENTS.md delete mode 100644 rewrites/make/README.md delete mode 100644 rewrites/make/package.json delete mode 100644 rewrites/mixpanel/.beads/.gitignore delete mode 100644 rewrites/mixpanel/.beads/README.md delete mode 100644 rewrites/mixpanel/.beads/config.yaml delete mode 100644 rewrites/mixpanel/.beads/interactions.jsonl delete mode 100644 rewrites/mixpanel/.beads/issues.jsonl delete mode 100644 rewrites/mixpanel/.beads/metadata.json delete mode 100644 rewrites/mixpanel/.gitattributes delete mode 100644 rewrites/mixpanel/AGENTS.md delete mode 100644 rewrites/mixpanel/README.md delete mode 100644 rewrites/monday/.beads/.gitignore delete mode 100644 rewrites/monday/.beads/README.md delete mode 100644 rewrites/monday/.beads/config.yaml delete mode 100644 rewrites/monday/.beads/interactions.jsonl delete mode 100644 rewrites/monday/.beads/issues.jsonl delete mode 100644 rewrites/monday/.beads/metadata.json delete mode 100644 rewrites/monday/.gitattributes delete mode 100644 rewrites/monday/AGENTS.md delete mode 100644 rewrites/monday/README.md delete mode 100644 rewrites/n8n/.beads/.gitignore delete mode 100644 rewrites/n8n/.beads/README.md delete mode 100644 rewrites/n8n/.beads/config.yaml delete mode 100644 rewrites/n8n/.beads/interactions.jsonl delete mode 100644 rewrites/n8n/.beads/issues.jsonl delete mode 100644 rewrites/n8n/.beads/metadata.json delete mode 100644 rewrites/n8n/.gitattributes delete mode 100644 rewrites/n8n/AGENTS.md delete mode 100644 rewrites/n8n/README.md delete mode 100644 rewrites/n8n/package.json delete mode 100644 rewrites/n8n/src/durable-object/CredentialDO.ts delete mode 100644 rewrites/n8n/src/durable-object/ExecutionDO.ts delete mode 100644 rewrites/n8n/src/durable-object/WorkflowDO.ts delete mode 100644 rewrites/n8n/src/durable-object/index.ts delete mode 100644 rewrites/n8n/src/index.ts delete mode 100644 rewrites/n8n/src/mcp/index.ts delete mode 100644 rewrites/n8n/src/mcp/tools.ts delete mode 100644 rewrites/n8n/src/sdk/n8n.ts delete mode 100644 rewrites/n8n/src/sdk/types.ts delete mode 100644 rewrites/netsuite/.beads/.gitignore delete mode 100644 rewrites/netsuite/.beads/README.md delete mode 100644 rewrites/netsuite/.beads/config.yaml delete mode 100644 rewrites/netsuite/.beads/interactions.jsonl delete mode 100644 rewrites/netsuite/.beads/issues.jsonl delete mode 100644 rewrites/netsuite/.beads/metadata.json delete mode 100644 rewrites/netsuite/.gitattributes delete mode 100644 rewrites/netsuite/AGENTS.md delete mode 100644 rewrites/netsuite/README.md delete mode 100644 rewrites/notion/.beads/.gitignore delete mode 100644 rewrites/notion/.beads/README.md delete mode 100644 rewrites/notion/.beads/config.yaml delete mode 100644 rewrites/notion/.beads/interactions.jsonl delete mode 100644 rewrites/notion/.beads/issues.jsonl delete mode 100644 rewrites/notion/.beads/metadata.json delete mode 100644 rewrites/notion/.gitattributes delete mode 100644 rewrites/notion/AGENTS.md delete mode 100644 rewrites/notion/README.md delete mode 100644 rewrites/orb/.beads/.gitignore delete mode 100644 rewrites/orb/.beads/README.md delete mode 100644 rewrites/orb/.beads/config.yaml delete mode 100644 rewrites/orb/.beads/interactions.jsonl delete mode 100644 rewrites/orb/.beads/issues.jsonl delete mode 100644 rewrites/orb/.beads/metadata.json delete mode 100644 rewrites/orb/.gitattributes delete mode 100644 rewrites/orb/AGENTS.md delete mode 100644 rewrites/orb/README.md delete mode 100644 rewrites/pipedrive/.beads/.gitignore delete mode 100644 rewrites/pipedrive/.beads/README.md delete mode 100644 rewrites/pipedrive/.beads/config.yaml delete mode 100644 rewrites/pipedrive/.beads/interactions.jsonl delete mode 100644 rewrites/pipedrive/.beads/issues.jsonl delete mode 100644 rewrites/pipedrive/.beads/metadata.json delete mode 100644 rewrites/pipedrive/.gitattributes delete mode 100644 rewrites/pipedrive/AGENTS.md delete mode 100644 rewrites/pipedrive/README.md delete mode 100644 rewrites/posthog/.beads/.gitignore delete mode 100644 rewrites/posthog/.beads/README.md delete mode 100644 rewrites/posthog/.beads/config.yaml delete mode 100644 rewrites/posthog/.beads/interactions.jsonl delete mode 100644 rewrites/posthog/.beads/issues.jsonl delete mode 100644 rewrites/posthog/.beads/metadata.json delete mode 100644 rewrites/posthog/.gitattributes delete mode 100644 rewrites/posthog/AGENTS.md delete mode 100644 rewrites/posthog/README.md delete mode 100644 rewrites/powerbi/.beads/.gitignore delete mode 100644 rewrites/powerbi/.beads/README.md delete mode 100644 rewrites/powerbi/.beads/config.yaml delete mode 100644 rewrites/powerbi/.beads/interactions.jsonl delete mode 100644 rewrites/powerbi/.beads/metadata.json delete mode 100644 rewrites/powerbi/.gitattributes delete mode 100644 rewrites/powerbi/AGENTS.md delete mode 100644 rewrites/powerbi/README.md delete mode 100644 rewrites/procore/.beads/.gitignore delete mode 100644 rewrites/procore/.beads/README.md delete mode 100644 rewrites/procore/.beads/config.yaml delete mode 100644 rewrites/procore/.beads/interactions.jsonl delete mode 100644 rewrites/procore/.beads/issues.jsonl delete mode 100644 rewrites/procore/.beads/metadata.json delete mode 100644 rewrites/procore/.gitattributes delete mode 100644 rewrites/procore/AGENTS.md delete mode 100644 rewrites/procore/README.md delete mode 100644 rewrites/rippling/.beads/.gitignore delete mode 100644 rewrites/rippling/.beads/README.md delete mode 100644 rewrites/rippling/.beads/config.yaml delete mode 100644 rewrites/rippling/.beads/interactions.jsonl delete mode 100644 rewrites/rippling/.beads/issues.jsonl delete mode 100644 rewrites/rippling/.beads/metadata.json delete mode 100644 rewrites/rippling/.gitattributes delete mode 100644 rewrites/rippling/AGENTS.md delete mode 100644 rewrites/rippling/README.md delete mode 100644 rewrites/salesforce/.beads/.gitignore delete mode 100644 rewrites/salesforce/.beads/README.md delete mode 100644 rewrites/salesforce/.beads/config.yaml delete mode 100644 rewrites/salesforce/.beads/interactions.jsonl delete mode 100644 rewrites/salesforce/.beads/issues.jsonl delete mode 100644 rewrites/salesforce/.beads/metadata.json delete mode 100644 rewrites/salesforce/.gitattributes delete mode 100644 rewrites/salesforce/AGENTS.md delete mode 100644 rewrites/salesforce/README.md delete mode 100644 rewrites/sap/.beads/.gitignore delete mode 100644 rewrites/sap/.beads/README.md delete mode 100644 rewrites/sap/.beads/config.yaml delete mode 100644 rewrites/sap/.beads/interactions.jsonl delete mode 100644 rewrites/sap/.beads/issues.jsonl delete mode 100644 rewrites/sap/.beads/metadata.json delete mode 100644 rewrites/sap/.gitattributes delete mode 100644 rewrites/sap/AGENTS.md delete mode 100644 rewrites/sap/README.md delete mode 100644 rewrites/segment/.beads/.gitignore delete mode 100644 rewrites/segment/.beads/README.md delete mode 100644 rewrites/segment/.beads/config.yaml delete mode 100644 rewrites/segment/.beads/interactions.jsonl delete mode 100644 rewrites/segment/.beads/issues.jsonl delete mode 100644 rewrites/segment/.beads/metadata.json delete mode 100644 rewrites/segment/.gitattributes delete mode 100644 rewrites/segment/AGENTS.md delete mode 100644 rewrites/segment/README.md delete mode 100644 rewrites/sentry/.beads/.gitignore delete mode 100644 rewrites/sentry/.beads/README.md delete mode 100644 rewrites/sentry/.beads/config.yaml delete mode 100644 rewrites/sentry/.beads/interactions.jsonl delete mode 100644 rewrites/sentry/.beads/issues.jsonl delete mode 100644 rewrites/sentry/.beads/metadata.json delete mode 100644 rewrites/sentry/.gitattributes delete mode 100644 rewrites/sentry/AGENTS.md delete mode 100644 rewrites/sentry/README.md delete mode 100644 rewrites/servicenow/.beads/.gitignore delete mode 100644 rewrites/servicenow/.beads/README.md delete mode 100644 rewrites/servicenow/.beads/config.yaml delete mode 100644 rewrites/servicenow/.beads/interactions.jsonl delete mode 100644 rewrites/servicenow/.beads/issues.jsonl delete mode 100644 rewrites/servicenow/.beads/metadata.json delete mode 100644 rewrites/servicenow/.gitattributes delete mode 100644 rewrites/servicenow/AGENTS.md delete mode 100644 rewrites/servicenow/README.md delete mode 100644 rewrites/servicetitan/.beads/.gitignore delete mode 100644 rewrites/servicetitan/.beads/README.md delete mode 100644 rewrites/servicetitan/.beads/config.yaml delete mode 100644 rewrites/servicetitan/.beads/interactions.jsonl delete mode 100644 rewrites/servicetitan/.beads/issues.jsonl delete mode 100644 rewrites/servicetitan/.beads/metadata.json delete mode 100644 rewrites/servicetitan/.gitattributes delete mode 100644 rewrites/servicetitan/AGENTS.md delete mode 100644 rewrites/servicetitan/README.md delete mode 100644 rewrites/shopify/.beads/.gitignore delete mode 100644 rewrites/shopify/.beads/README.md delete mode 100644 rewrites/shopify/.beads/config.yaml delete mode 100644 rewrites/shopify/.beads/interactions.jsonl delete mode 100644 rewrites/shopify/.beads/issues.jsonl delete mode 100644 rewrites/shopify/.beads/metadata.json delete mode 100644 rewrites/shopify/.gitattributes delete mode 100644 rewrites/shopify/AGENTS.md delete mode 100644 rewrites/shopify/README.md delete mode 100644 rewrites/sierra/.beads/.gitignore delete mode 100644 rewrites/sierra/.beads/README.md delete mode 100644 rewrites/sierra/.beads/config.yaml delete mode 100644 rewrites/sierra/.beads/interactions.jsonl delete mode 100644 rewrites/sierra/.beads/issues.jsonl delete mode 100644 rewrites/sierra/.beads/metadata.json delete mode 100644 rewrites/sierra/.gitattributes delete mode 100644 rewrites/sierra/AGENTS.md delete mode 100644 rewrites/snowflake/.beads/.gitignore delete mode 100644 rewrites/snowflake/.beads/README.md delete mode 100644 rewrites/snowflake/.beads/config.yaml delete mode 100644 rewrites/snowflake/.beads/interactions.jsonl delete mode 100644 rewrites/snowflake/.beads/metadata.json delete mode 100644 rewrites/snowflake/.gitattributes delete mode 100644 rewrites/snowflake/AGENTS.md delete mode 100644 rewrites/snowflake/README.md delete mode 100644 rewrites/splunk/.beads/.gitignore delete mode 100644 rewrites/splunk/.beads/README.md delete mode 100644 rewrites/splunk/.beads/config.yaml delete mode 100644 rewrites/splunk/.beads/interactions.jsonl delete mode 100644 rewrites/splunk/.beads/issues.jsonl delete mode 100644 rewrites/splunk/.beads/metadata.json delete mode 100644 rewrites/splunk/.gitattributes delete mode 100644 rewrites/splunk/AGENTS.md delete mode 100644 rewrites/splunk/README.md delete mode 100644 rewrites/studio/.beads/.gitignore delete mode 100644 rewrites/studio/.beads/README.md delete mode 100644 rewrites/studio/.beads/config.yaml delete mode 100644 rewrites/studio/.beads/interactions.jsonl delete mode 100644 rewrites/studio/.beads/issues.jsonl delete mode 100644 rewrites/studio/.beads/metadata.json delete mode 100644 rewrites/studio/.gitattributes delete mode 100644 rewrites/studio/AGENTS.md delete mode 100644 rewrites/studio/README.md delete mode 100644 rewrites/tableau/.beads/.gitignore delete mode 100644 rewrites/tableau/.beads/README.md delete mode 100644 rewrites/tableau/.beads/config.yaml delete mode 100644 rewrites/tableau/.beads/interactions.jsonl delete mode 100644 rewrites/tableau/.beads/metadata.json delete mode 100644 rewrites/tableau/.gitattributes delete mode 100644 rewrites/tableau/AGENTS.md delete mode 100644 rewrites/tableau/README.md delete mode 100644 rewrites/temporal/.beads/.gitignore delete mode 100644 rewrites/temporal/.beads/README.md delete mode 100644 rewrites/temporal/.beads/config.yaml delete mode 100644 rewrites/temporal/.beads/interactions.jsonl delete mode 100644 rewrites/temporal/.beads/issues.jsonl delete mode 100644 rewrites/temporal/.beads/metadata.json delete mode 100644 rewrites/temporal/.gitattributes delete mode 100644 rewrites/temporal/AGENTS.md delete mode 100644 rewrites/temporal/README.md delete mode 100644 rewrites/temporal/package.json delete mode 100644 rewrites/temporal/src/.gitkeep delete mode 100644 rewrites/teradata/.beads/.gitignore delete mode 100644 rewrites/teradata/.beads/README.md delete mode 100644 rewrites/teradata/.beads/config.yaml delete mode 100644 rewrites/teradata/.beads/interactions.jsonl delete mode 100644 rewrites/teradata/.beads/issues.jsonl delete mode 100644 rewrites/teradata/.beads/metadata.json delete mode 100644 rewrites/teradata/.gitattributes delete mode 100644 rewrites/teradata/AGENTS.md delete mode 100644 rewrites/teradata/TDD-BREAKDOWN.md delete mode 100644 rewrites/toast/.beads/.gitignore delete mode 100644 rewrites/toast/.beads/README.md delete mode 100644 rewrites/toast/.beads/config.yaml delete mode 100644 rewrites/toast/.beads/interactions.jsonl delete mode 100644 rewrites/toast/.beads/issues.jsonl delete mode 100644 rewrites/toast/.beads/metadata.json delete mode 100644 rewrites/toast/.gitattributes delete mode 100644 rewrites/toast/AGENTS.md delete mode 100644 rewrites/toast/README.md delete mode 100644 rewrites/trigger/.beads/.gitignore delete mode 100644 rewrites/trigger/.beads/README.md delete mode 100644 rewrites/trigger/.beads/config.yaml delete mode 100644 rewrites/trigger/.beads/interactions.jsonl delete mode 100644 rewrites/trigger/.beads/issues.jsonl delete mode 100644 rewrites/trigger/.beads/metadata.json delete mode 100644 rewrites/trigger/.gitattributes delete mode 100644 rewrites/trigger/AGENTS.md delete mode 100644 rewrites/trigger/README.md delete mode 100644 rewrites/trigger/package.json delete mode 100644 rewrites/trigger/src/core/index.ts delete mode 100644 rewrites/trigger/src/core/schedules.ts delete mode 100644 rewrites/trigger/src/core/task.ts delete mode 100644 rewrites/trigger/src/core/trigger.ts delete mode 100644 rewrites/trigger/src/index.ts delete mode 100644 rewrites/trigger/src/types.ts delete mode 100644 rewrites/veeva/.beads/.gitignore delete mode 100644 rewrites/veeva/.beads/README.md delete mode 100644 rewrites/veeva/.beads/config.yaml delete mode 100644 rewrites/veeva/.beads/interactions.jsonl delete mode 100644 rewrites/veeva/.beads/issues.jsonl delete mode 100644 rewrites/veeva/.beads/metadata.json delete mode 100644 rewrites/veeva/.gitattributes delete mode 100644 rewrites/veeva/AGENTS.md delete mode 100644 rewrites/veeva/README.md delete mode 100644 rewrites/workday/.beads/.gitignore delete mode 100644 rewrites/workday/.beads/README.md delete mode 100644 rewrites/workday/.beads/config.yaml delete mode 100644 rewrites/workday/.beads/interactions.jsonl delete mode 100644 rewrites/workday/.beads/issues.jsonl delete mode 100644 rewrites/workday/.beads/metadata.json delete mode 100644 rewrites/workday/.gitattributes delete mode 100644 rewrites/workday/AGENTS.md delete mode 100644 rewrites/workday/README.md delete mode 100644 rewrites/yardi/.beads/.gitignore delete mode 100644 rewrites/yardi/.beads/README.md delete mode 100644 rewrites/yardi/.beads/config.yaml delete mode 100644 rewrites/yardi/.beads/interactions.jsonl delete mode 100644 rewrites/yardi/.beads/issues.jsonl delete mode 100644 rewrites/yardi/.beads/metadata.json delete mode 100644 rewrites/yardi/.gitattributes delete mode 100644 rewrites/yardi/AGENTS.md delete mode 100644 rewrites/yardi/README.md delete mode 100644 rewrites/zapier/.beads/.gitignore delete mode 100644 rewrites/zapier/.beads/README.md delete mode 100644 rewrites/zapier/.beads/config.yaml delete mode 100644 rewrites/zapier/.beads/interactions.jsonl delete mode 100644 rewrites/zapier/.beads/issues.jsonl delete mode 100644 rewrites/zapier/.beads/metadata.json delete mode 100644 rewrites/zapier/.gitattributes delete mode 100644 rewrites/zapier/AGENTS.md delete mode 100644 rewrites/zapier/README.md delete mode 100644 rewrites/zapier/package.json delete mode 100644 rewrites/zendesk/.beads/.gitignore delete mode 100644 rewrites/zendesk/.beads/README.md delete mode 100644 rewrites/zendesk/.beads/config.yaml delete mode 100644 rewrites/zendesk/.beads/interactions.jsonl delete mode 100644 rewrites/zendesk/.beads/issues.jsonl delete mode 100644 rewrites/zendesk/.beads/metadata.json delete mode 100644 rewrites/zendesk/.gitattributes delete mode 100644 rewrites/zendesk/AGENTS.md delete mode 100644 rewrites/zendesk/README.md delete mode 100644 rewrites/zoho/.beads/.gitignore delete mode 100644 rewrites/zoho/.beads/README.md delete mode 100644 rewrites/zoho/.beads/config.yaml delete mode 100644 rewrites/zoho/.beads/interactions.jsonl delete mode 100644 rewrites/zoho/.beads/issues.jsonl delete mode 100644 rewrites/zoho/.beads/metadata.json delete mode 100644 rewrites/zoho/.gitattributes delete mode 100644 rewrites/zoho/AGENTS.md delete mode 100644 rewrites/zoho/README.md diff --git a/rewrites/airbyte/.beads/.gitignore b/rewrites/airbyte/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/airbyte/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/airbyte/.beads/README.md b/rewrites/airbyte/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/airbyte/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/airbyte/.beads/config.yaml b/rewrites/airbyte/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/airbyte/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/airbyte/.beads/interactions.jsonl b/rewrites/airbyte/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/airbyte/.beads/issues.jsonl b/rewrites/airbyte/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/airbyte/.beads/metadata.json b/rewrites/airbyte/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/airbyte/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/airbyte/.gitattributes b/rewrites/airbyte/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/airbyte/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/airbyte/AGENTS.md b/rewrites/airbyte/AGENTS.md deleted file mode 100644 index c86184e1..00000000 --- a/rewrites/airbyte/AGENTS.md +++ /dev/null @@ -1,208 +0,0 @@ -# Airbyte Rewrite - AI Assistant Guidance - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Project Overview - -**Goal:** Build a Cloudflare-native Airbyte alternative - ELT data integration with 300+ connectors without managing infrastructure. - -**Package:** `@dotdo/airbyte` with domains `airbyte.do` / `etl.do` / `pipelines.do` - -**Core Primitives:** -- Durable Objects - Source, Destination, Connection, and Sync state -- Cloudflare Queues - Job scheduling and execution -- R2 Storage - Staging area for large data transfers -- MCP Tools - AI-native connector runtime (fsx.do, gitx.do) - -## Reference Documents - -1. **../workflows/SCOPE.md** - Complete workflows platform analysis -2. **../CLAUDE.md** - General rewrites guidance and TDD patterns -3. **../inngest/README.md** - Reference rewrite pattern - -## Key Airbyte Concepts - -### Sources -```typescript -const github = await airbyte.sources.create({ - name: 'github-source', - type: 'github', - config: { - credentials: { personal_access_token: env.GITHUB_TOKEN }, - repositories: ['myorg/myrepo'] - } -}) -``` - -### Destinations -```typescript -const snowflake = await airbyte.destinations.create({ - name: 'snowflake-dest', - type: 'snowflake', - config: { - host: 'account.snowflakecomputing.com', - database: 'analytics' - } -}) -``` - -### Connections (Sync Jobs) -```typescript -const connection = await airbyte.connections.create({ - name: 'github-to-snowflake', - source: github.id, - destination: snowflake.id, - streams: [ - { name: 'commits', syncMode: 'incremental', cursorField: 'date' } - ], - schedule: { cron: '0 */6 * * *' } -}) -``` - -### Sync Modes -- `full_refresh` - Replace all data each sync -- `incremental` - Only sync changed data (cursor-based) -- `cdc` - Change Data Capture for databases - -## Architecture - -``` -airbyte/ - src/ - core/ # Business logic - source-registry.ts # Source connector registry - destination-registry.ts # Destination connector registry - schema-discovery.ts # JSON Schema extraction - sync-engine.ts # Data extraction and loading - durable-object/ # DO implementations - SourceDO.ts # Source configuration and state - DestinationDO.ts # Destination configuration and state - ConnectionDO.ts # Connection orchestration - SyncDO.ts # Individual sync job execution - connectors/ # Connector implementations - sources/ # Source connectors - destinations/ # Destination connectors - mcp/ # MCP tool definitions - tools.ts # AI-accessible tools - sdk/ # Client SDK - airbyte.ts # Airbyte class - types.ts # TypeScript types - .beads/ # Issue tracking - AGENTS.md # This file - README.md # User documentation -``` - -## TDD Workflow - -Follow strict TDD for all implementation: - -```bash -# Find ready work (RED tests first) -bd ready - -# Claim work -bd update airbyte-xxx --status in_progress - -# After tests pass -bd close airbyte-xxx - -# Check what's unblocked -bd ready -``` - -### TDD Cycle Pattern -1. **[RED]** Write failing tests first -2. **[GREEN]** Implement minimum code to pass -3. **[REFACTOR]** Clean up without changing behavior - -## Implementation Priorities - -### Phase 1: Source Registry (airbyte-001) -1. [RED] Test source.create() API surface -2. [GREEN] Implement SourceDO with config storage -3. [REFACTOR] Schema validation - -### Phase 2: Destination Registry (airbyte-002) -1. [RED] Test destination.create() API surface -2. [GREEN] Implement DestinationDO with config storage -3. [REFACTOR] Credential encryption - -### Phase 3: Schema Discovery (airbyte-003) -1. [RED] Test sources.discover() catalog -2. [GREEN] Implement JSON Schema extraction -3. [REFACTOR] Stream metadata - -### Phase 4: Connection Orchestration (airbyte-004) -1. [RED] Test connections.create() with streams -2. [GREEN] Implement ConnectionDO with scheduling -3. [REFACTOR] Cron expression parsing - -### Phase 5: Sync Execution (airbyte-005) -1. [RED] Test connections.sync() job creation -2. [GREEN] Implement SyncDO with incremental state -3. [REFACTOR] Cursor management - -### Phase 6: Connector Runtime (airbyte-006) -1. [RED] Test connector spec/check/discover/read/write -2. [GREEN] Implement MCP-based connector protocol -3. [REFACTOR] fsx.do/gitx.do integration - -### Phase 7: Normalization (airbyte-007) -1. [RED] Test basic normalization transforms -2. [GREEN] Implement schema flattening -3. [REFACTOR] Type coercion - -### Phase 8: CDC Support (airbyte-008) -1. [RED] Test database change capture -2. [GREEN] Implement WAL/binlog readers -3. [REFACTOR] Replication slot management - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -bd dep tree # View dependency tree -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - -## Testing Commands - -```bash -# Run tests -npm test - -# Run with coverage -npm run test:coverage - -# Watch mode -npm run test:watch -``` diff --git a/rewrites/airbyte/README.md b/rewrites/airbyte/README.md deleted file mode 100644 index 5b8d84c2..00000000 --- a/rewrites/airbyte/README.md +++ /dev/null @@ -1,426 +0,0 @@ -# airbyte.do - -> Data Integration. Edge-Native. Natural Language First. - -Airbyte raised $181M to build "the open-source standard for data integration." Now they run a cloud service charging $1.50/credit, require Kubernetes for self-hosting, and make data engineers babysit YAML files and Docker containers. Moving data shouldn't require a DevOps team. - -**airbyte.do** is the serverless alternative. No Kubernetes. No Docker. No YAML. Natural language pipelines that deploy in seconds. - -## AI-Native API - -```typescript -import { airbyte } from 'airbyte.do' // Full SDK -import { airbyte } from 'airbyte.do/tiny' // Minimal client -import { airbyte } from 'airbyte.do/streaming' // Streaming ops -``` - -Natural language for data pipelines: - -```typescript -import { airbyte } from 'airbyte.do' - -// Talk to it like a colleague -const syncs = await airbyte`failing syncs this week` -const slow = await airbyte`connections taking > 1 hour` -const stale = await airbyte`sources not synced in 3 days` - -// Chain like sentences -await airbyte`stripe charges` - .sync(`to snowflake`) - -// Pipelines that build themselves -await airbyte`connect salesforce` - .discover() // find all objects - .sync(`bigquery`) // sync everything - .schedule(`hourly`) // keep it fresh -``` - -## The Problem - -Self-hosted Airbyte dominates open-source ELT: - -| What Airbyte Requires | The Reality | -|-----------------------|-------------| -| **Infrastructure** | 3-node Kubernetes cluster minimum | -| **Deployment** | 2+ hours to configure, test, deploy | -| **Monthly Compute** | $500-2000/month for mid-size workloads | -| **Expertise** | DevOps team or K8s knowledge required | -| **Configuration** | YAML files, Docker images, Helm charts | -| **Debugging** | kubectl logs, pod restarts, OOM errors | - -### The Kubernetes Tax - -Self-hosting Airbyte means: - -- Multi-node K8s cluster (EKS, GKE, or roll your own) -- Temporal for workflow orchestration -- PostgreSQL for metadata -- MinIO or S3 for staging -- Monitoring stack (Prometheus, Grafana) -- On-call rotation for connector failures - -Data engineers spend more time on infrastructure than data. - -### The Cloud Alternative Isn't Cheap - -Airbyte Cloud charges per credit: - -- $1.50/credit (rows synced) -- High-volume = high bills -- Unpredictable costs at scale -- Still debugging connector configs - -### The Configuration Complexity - -Every connector needs: - -```yaml -# This is what you're escaping -sourceDefinitionId: 778daa7c-feaf-4db6-96f3-70fd645acc77 -connectionConfiguration: - credentials: - auth_type: OAuth - client_id: ${SALESFORCE_CLIENT_ID} - client_secret: ${SALESFORCE_CLIENT_SECRET} - refresh_token: ${SALESFORCE_REFRESH_TOKEN} - start_date: "2024-01-01T00:00:00Z" - streams_criteria: - - criteria: starts with - value: Account -``` - -## The Solution - -**airbyte.do** reimagines data integration: - -``` -Self-Hosted Airbyte airbyte.do ------------------------------------------------------------------ -3-node K8s cluster Zero infrastructure -2 hours to deploy Deploy in seconds -$500/month compute Pay per sync -YAML configuration Natural language -Docker image management Managed connectors -kubectl debug \`airbyte\`why did it fail?\`\` -Temporal orchestration Durable Objects -Manual scaling Edge-native auto-scale -``` - -## One-Click Deploy - -```bash -npx create-dotdo airbyte -``` - -A full ELT platform. Running on your Cloudflare account. 300+ connectors ready. - -```typescript -import { Airbyte } from 'airbyte.do' - -export default Airbyte({ - name: 'my-data-platform', - domain: 'data.mycompany.com', -}) -``` - -## Features - -### Sources - -```typescript -// Connect sources naturally -await airbyte`connect to postgres at db.example.com` -await airbyte`add stripe as a source` -await airbyte`connect salesforce with oauth` - -// AI infers what you need -await airbyte`postgres tables` // lists available tables -await airbyte`stripe schema` // shows data structure -await airbyte`test salesforce connection` // verifies connectivity -``` - -### Destinations - -```typescript -// Warehouses are one line -await airbyte`send to snowflake analytics.raw` -await airbyte`connect bigquery my-project.raw_data` -await airbyte`add databricks lakehouse` - -// Or dictate directly -await airbyte`sync stripe to snowflake, bigquery, and redshift` -``` - -### Connections - -```typescript -// Just say it -await airbyte`sync postgres users to bigquery hourly` -await airbyte`github commits to snowflake every 6 hours` -await airbyte`stripe to databricks, incremental on updated_at` - -// AI configures sync modes automatically -await airbyte`what should I sync from salesforce?` - .sync(`to snowflake`) // submits with optimal settings - -// Batch connections read like a pipeline manifest -await airbyte` - postgres production: - - users incremental - - orders incremental - - products full refresh - all to bigquery hourly -` -``` - -### Sync Status - -```typescript -// View status naturally -await airbyte`sync status` -await airbyte`failed syncs today` -await airbyte`slowest connections this week` - -// AI surfaces what needs attention -``` - -### Schema Discovery - -```typescript -// Discover sources naturally -await airbyte`what tables are in postgres?` -await airbyte`salesforce objects with data` -await airbyte`stripe schema changes since last week` -``` - -### Troubleshooting - -```typescript -// Diagnose issues naturally -await airbyte`why did stripe sync fail?` -await airbyte`postgres connection timing out` -await airbyte`fix the salesforce oauth error` - -// AI chains diagnosis and fixes -await airbyte`connectors with errors` - .map(c => airbyte`diagnose ${c}`) - .map(fix => airbyte`apply ${fix}`) -``` - -## Promise Pipelining - -Chain operations without waiting. One network round trip. - -```typescript -// Discover and sync in one chain -await airbyte`postgres tables` - .map(table => airbyte`sync ${table} to bigquery incrementally`) - .map(sync => airbyte`verify ${sync}`) - -// Build complex pipelines that feel like talking -await airbyte`all stripe payment tables` - .map(table => airbyte`sync ${table} to snowflake with dedup`) - .map(sync => airbyte`transform cents to dollars in ${sync}`) - .map(result => airbyte`notify #data-team when ${result} completes`) -``` - -## Agent Integration - -```typescript -import { tom, priya } from 'agents.do' - -// Chain agents and tools -await priya`design customer analytics pipeline` - .map(spec => tom`implement ${spec} with airbyte.do`) - .map(pipeline => airbyte`deploy ${pipeline}`) -``` - -## Architecture - -### Durable Object per Connection - -``` -AirbyteDO (config, connectors, catalog) - | - +-- SourcesDO (source connectors) - | |-- SQLite: Source configs (encrypted) - | +-- Schema cache - | - +-- DestinationsDO (destination connectors) - | |-- SQLite: Destination configs (encrypted) - | - +-- ConnectionsDO (sync orchestration) - | |-- SQLite: Connection state - | +-- Sync history - | - +-- SyncsDO (job execution) - |-- SQLite: Job state, metrics - +-- R2: Staging data -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Active connections, recent syncs | <10ms | -| **Warm** | R2 + Index | Sync history (30 days) | <100ms | -| **Cold** | R2 Archive | Audit logs (1+ years) | <1s | - -## vs Airbyte - -| Feature | Airbyte (Self-Hosted) | Airbyte Cloud | airbyte.do | -|---------|----------------------|---------------|------------| -| **Infrastructure** | 3-node K8s cluster | None | None | -| **Deployment** | 2+ hours | Minutes | Seconds | -| **Monthly Cost** | $500-2000 compute | $1.50/credit | Pay per sync | -| **Configuration** | YAML files | UI forms | Natural language | -| **Debugging** | kubectl logs | Logs UI | `airbyte\`why did it fail?\`` | -| **Scaling** | Manual | Managed | Edge-native auto-scale | -| **Data Location** | Your K8s cluster | Airbyte's cloud | Your Cloudflare account | -| **Lock-in** | Open source | Proprietary cloud | MIT licensed | - -## Use Cases - -### Data Warehouse Loading - -Just sync and schedule. No infrastructure. - -### Real-Time Analytics - -CDC from databases, streaming to warehouses. - -### AI/ML Pipelines - -```typescript -// Sync to vector stores for RAG -await airbyte`sync notion pages to pinecone for search` - -// Embeddings included -await airbyte`zendesk tickets to weaviate with embeddings` -``` - -### Multi-Destination Fan-Out - -```typescript -// One source, many destinations -await airbyte`stripe to snowflake, bigquery, and redshift` -``` - -## Connectors - -### 300+ Sources Supported - -| Category | Examples | -|----------|----------| -| **Databases** | PostgreSQL, MySQL, MongoDB, SQL Server, Oracle | -| **Data Warehouses** | Snowflake, BigQuery, Redshift, Databricks | -| **SaaS** | Salesforce, HubSpot, Stripe, Shopify, Zendesk | -| **Files** | S3, GCS, SFTP, local files | -| **APIs** | REST, GraphQL, webhooks | - -### 50+ Destinations Supported - -| Category | Examples | -|----------|----------| -| **Warehouses** | Snowflake, BigQuery, Redshift, Databricks | -| **Lakes** | S3, GCS, Delta Lake, Iceberg | -| **Vector Stores** | Pinecone, Weaviate, Qdrant, Milvus | -| **Databases** | PostgreSQL, MySQL, MongoDB | - -## Deployment Options - -### Cloudflare Workers (Recommended) - -```bash -npx create-dotdo airbyte -# Deploys to your Cloudflare account -``` - -### Self-Hosted - -```bash -# Deploy to your infrastructure -docker run -p 8787:8787 dotdo/airbyte -``` - -## Why Cloudflare? - -### 1. Global Edge - -Sync jobs run close to data sources. Lower latency, faster syncs. - -### 2. No Cold Starts - -Durable Objects stay warm. No waiting for containers. - -### 3. Unlimited Duration - -Long-running syncs work naturally. No 30-second timeouts. - -### 4. Built-in Queues - -Reliable job scheduling. No external Temporal cluster. - -### 5. R2 Storage - -Staging area for large syncs. No S3 or MinIO setup. - -## Related Domains - -- **etl.do** - ETL pipeline orchestration -- **pipelines.do** - Data pipeline management -- **connectors.do** - Connector marketplace -- **catalog.do** - Data catalog and discovery - -## Roadmap - -### Core ELT -- [x] Source connectors (300+) -- [x] Destination connectors (50+) -- [x] Incremental sync -- [x] Full refresh -- [x] Schema discovery -- [ ] CDC support -- [ ] Custom transformations -- [ ] dbt integration - -### AI -- [x] Natural language configuration -- [x] Auto-troubleshooting -- [ ] Schema mapping suggestions -- [ ] Sync optimization -- [ ] Anomaly detection - -### Enterprise -- [x] Multi-workspace -- [ ] RBAC -- [ ] Audit logs -- [ ] SSO - -## Contributing - -airbyte.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/airbyte.do -cd airbyte.do -pnpm install -pnpm test -``` - -## License - -MIT License - Move data freely. - ---- - -

- No Kubernetes. No Docker. No YAML. -
- Natural language pipelines. Edge-native. Serverless. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/airbyte/package.json b/rewrites/airbyte/package.json deleted file mode 100644 index 330520be..00000000 --- a/rewrites/airbyte/package.json +++ /dev/null @@ -1,83 +0,0 @@ -{ - "name": "@dotdo/airbyte", - "version": "0.0.1", - "description": "Airbyte on Cloudflare Durable Objects - ELT data integration with 300+ connectors", - "type": "module", - "main": "dist/index.js", - "types": "dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js" - }, - "./core": { - "types": "./dist/core/index.d.ts", - "import": "./dist/core/index.js" - }, - "./do": { - "types": "./dist/durable-object/index.d.ts", - "import": "./dist/durable-object/index.js" - }, - "./mcp": { - "types": "./dist/mcp/index.d.ts", - "import": "./dist/mcp/index.js" - }, - "./connectors": { - "types": "./dist/connectors/index.d.ts", - "import": "./dist/connectors/index.js" - } - }, - "files": [ - "dist", - "LICENSE", - "README.md" - ], - "scripts": { - "build": "tsc", - "test": "vitest run", - "test:watch": "vitest", - "test:coverage": "vitest run --coverage", - "dev": "wrangler dev", - "deploy": "wrangler deploy", - "lint": "eslint src test --ext .ts", - "typecheck": "tsc --noEmit" - }, - "keywords": [ - "airbyte", - "etl", - "elt", - "data-integration", - "connectors", - "cloudflare", - "durable-objects", - "edge", - "mcp", - "ai", - "data-pipeline", - "sync" - ], - "author": "dot-do", - "license": "MIT", - "repository": { - "type": "git", - "url": "https://github.com/dot-do/airbyte" - }, - "homepage": "https://github.com/dot-do/airbyte#readme", - "bugs": { - "url": "https://github.com/dot-do/airbyte/issues" - }, - "engines": { - "node": ">=18.0.0" - }, - "devDependencies": { - "@cloudflare/workers-types": "^4.20241218.0", - "@types/node": "^22.10.2", - "typescript": "^5.7.2", - "vitest": "^2.1.8", - "wrangler": "^3.99.0" - }, - "dependencies": { - "hono": "^4.6.0", - "zod": "^3.23.0" - } -} diff --git a/rewrites/airbyte/src/index.ts b/rewrites/airbyte/src/index.ts deleted file mode 100644 index eef151d7..00000000 --- a/rewrites/airbyte/src/index.ts +++ /dev/null @@ -1,54 +0,0 @@ -/** - * @dotdo/airbyte - * - * Airbyte on Cloudflare - ELT data integration with 300+ connectors - * - * @example - * ```typescript - * import { Airbyte } from '@dotdo/airbyte' - * - * const airbyte = new Airbyte({ workspace: 'my-workspace' }) - * - * // Create source - * const github = await airbyte.sources.create({ - * name: 'github-source', - * type: 'github', - * config: { repositories: ['myorg/myrepo'] } - * }) - * - * // Create destination - * const snowflake = await airbyte.destinations.create({ - * name: 'snowflake-dest', - * type: 'snowflake', - * config: { database: 'analytics' } - * }) - * - * // Create connection and sync - * const connection = await airbyte.connections.create({ - * source: github.id, - * destination: snowflake.id, - * streams: [{ name: 'commits', syncMode: 'incremental' }] - * }) - * - * await airbyte.connections.sync(connection.id) - * ``` - */ - -export { Airbyte, type AirbyteOptions } from './sdk/airbyte' -export type { - Source, - SourceConfig, - Destination, - DestinationConfig, - Connection, - ConnectionConfig, - Stream, - StreamConfig, - SyncMode, - DestinationSyncMode, - SyncJob, - SyncStatus, - SchemaDiscoveryResult, - ConnectorSpec, - CheckResult -} from './sdk/types' diff --git a/rewrites/airbyte/src/sdk/airbyte.ts b/rewrites/airbyte/src/sdk/airbyte.ts deleted file mode 100644 index f9ac9319..00000000 --- a/rewrites/airbyte/src/sdk/airbyte.ts +++ /dev/null @@ -1,192 +0,0 @@ -/** - * Airbyte Client SDK - */ - -import type { - Source, - CreateSourceInput, - Destination, - CreateDestinationInput, - Connection, - ConnectionConfig, - SyncJob, - SyncStatus, - SchemaDiscoveryResult, - CheckResult, - SourceDefinition, - DestinationDefinition -} from './types' - -export interface AirbyteOptions { - workspace: string - baseUrl?: string - apiKey?: string -} - -export class Airbyte { - private workspace: string - private baseUrl: string - private apiKey?: string - - public readonly sources: SourcesAPI - public readonly destinations: DestinationsAPI - public readonly connections: ConnectionsAPI - public readonly catalog: CatalogAPI - - constructor(options: AirbyteOptions) { - this.workspace = options.workspace - this.baseUrl = options.baseUrl ?? 'https://airbyte.do' - this.apiKey = options.apiKey - - this.sources = new SourcesAPI(this) - this.destinations = new DestinationsAPI(this) - this.connections = new ConnectionsAPI(this) - this.catalog = new CatalogAPI(this) - } - - /** @internal */ - async _request(method: string, path: string, body?: unknown): Promise { - const url = `${this.baseUrl}/api/v1/${this.workspace}${path}` - const headers: Record = { - 'Content-Type': 'application/json' - } - if (this.apiKey) { - headers['Authorization'] = `Bearer ${this.apiKey}` - } - - const response = await fetch(url, { - method, - headers, - body: body ? JSON.stringify(body) : undefined - }) - - if (!response.ok) { - const error = await response.text() - throw new Error(`Airbyte API error: ${response.status} ${error}`) - } - - return response.json() as Promise - } -} - -class SourcesAPI { - constructor(private client: Airbyte) {} - - async create(input: CreateSourceInput): Promise { - return this.client._request('POST', '/sources', input) - } - - async get(id: string): Promise { - return this.client._request('GET', `/sources/${id}`) - } - - async list(): Promise { - return this.client._request('GET', '/sources') - } - - async update(id: string, input: Partial): Promise { - return this.client._request('PATCH', `/sources/${id}`, input) - } - - async delete(id: string): Promise { - await this.client._request('DELETE', `/sources/${id}`) - } - - async discover(id: string): Promise { - return this.client._request('POST', `/sources/${id}/discover`) - } - - async check(id: string): Promise { - return this.client._request('POST', `/sources/${id}/check`) - } -} - -class DestinationsAPI { - constructor(private client: Airbyte) {} - - async create(input: CreateDestinationInput): Promise { - return this.client._request('POST', '/destinations', input) - } - - async get(id: string): Promise { - return this.client._request('GET', `/destinations/${id}`) - } - - async list(): Promise { - return this.client._request('GET', '/destinations') - } - - async update(id: string, input: Partial): Promise { - return this.client._request('PATCH', `/destinations/${id}`, input) - } - - async delete(id: string): Promise { - await this.client._request('DELETE', `/destinations/${id}`) - } - - async check(id: string): Promise { - return this.client._request('POST', `/destinations/${id}/check`) - } -} - -class ConnectionsAPI { - constructor(private client: Airbyte) {} - - async create(input: ConnectionConfig): Promise { - return this.client._request('POST', '/connections', input) - } - - async get(id: string): Promise { - return this.client._request('GET', `/connections/${id}`) - } - - async list(): Promise { - return this.client._request('GET', '/connections') - } - - async update(id: string, input: Partial): Promise { - return this.client._request('PATCH', `/connections/${id}`, input) - } - - async delete(id: string): Promise { - await this.client._request('DELETE', `/connections/${id}`) - } - - async sync(id: string): Promise { - return this.client._request('POST', `/connections/${id}/sync`) - } - - async status(id: string): Promise<{ state: SyncStatus; progress: Record; started_at?: string }> { - return this.client._request('GET', `/connections/${id}/status`) - } - - async jobs(id: string): Promise { - return this.client._request('GET', `/connections/${id}/jobs`) - } - - async cancelJob(connectionId: string, jobId: string): Promise { - return this.client._request('POST', `/connections/${connectionId}/jobs/${jobId}/cancel`) - } -} - -class CatalogAPI { - constructor(private client: Airbyte) {} - - readonly sources = { - list: async (): Promise => { - return this.client._request('GET', '/catalog/sources') - }, - get: async (id: string): Promise => { - return this.client._request('GET', `/catalog/sources/${id}`) - } - } - - readonly destinations = { - list: async (): Promise => { - return this.client._request('GET', '/catalog/destinations') - }, - get: async (id: string): Promise => { - return this.client._request('GET', `/catalog/destinations/${id}`) - } - } -} diff --git a/rewrites/airbyte/src/sdk/types.ts b/rewrites/airbyte/src/sdk/types.ts deleted file mode 100644 index c2c7a13d..00000000 --- a/rewrites/airbyte/src/sdk/types.ts +++ /dev/null @@ -1,149 +0,0 @@ -/** - * Airbyte SDK Types - */ - -// Sync modes -export type SyncMode = 'full_refresh' | 'incremental' -export type DestinationSyncMode = 'overwrite' | 'append' | 'append_dedup' - -// Source types -export interface Source { - id: string - name: string - type: string - config: SourceConfig - createdAt: string - updatedAt: string -} - -export interface SourceConfig { - [key: string]: unknown -} - -export interface CreateSourceInput { - name: string - type: string - config: SourceConfig -} - -// Destination types -export interface Destination { - id: string - name: string - type: string - config: DestinationConfig - createdAt: string - updatedAt: string -} - -export interface DestinationConfig { - [key: string]: unknown -} - -export interface CreateDestinationInput { - name: string - type: string - config: DestinationConfig -} - -// Stream types -export interface Stream { - name: string - jsonSchema?: Record - supportedSyncModes?: SyncMode[] - sourceDefinedCursor?: boolean - defaultCursorField?: string[] - sourceDefinedPrimaryKey?: string[][] -} - -export interface StreamConfig { - name: string - syncMode: SyncMode - destinationSyncMode?: DestinationSyncMode - cursorField?: string - primaryKey?: string[][] -} - -// Connection types -export interface Connection { - id: string - name: string - sourceId: string - destinationId: string - streams: StreamConfig[] - schedule?: ConnectionSchedule - normalization?: NormalizationType - status: ConnectionStatus - createdAt: string - updatedAt: string -} - -export interface ConnectionConfig { - name: string - source: string - destination: string - streams: StreamConfig[] - schedule?: ConnectionSchedule - normalization?: NormalizationType -} - -export interface ConnectionSchedule { - cron?: string - interval?: string -} - -export type ConnectionStatus = 'active' | 'inactive' | 'deprecated' -export type NormalizationType = 'basic' | 'raw' | 'none' - -// Sync job types -export interface SyncJob { - id: string - connectionId: string - status: SyncStatus - progress: SyncProgress - startedAt: string - completedAt?: string - error?: string -} - -export type SyncStatus = 'pending' | 'running' | 'succeeded' | 'failed' | 'cancelled' - -export interface SyncProgress { - [streamName: string]: number -} - -// Schema discovery -export interface SchemaDiscoveryResult { - streams: Stream[] -} - -// Connector spec -export interface ConnectorSpec { - documentationUrl?: string - connectionSpecification: Record - supportsIncremental: boolean - supportedDestinationSyncModes?: DestinationSyncMode[] -} - -// Check result -export interface CheckResult { - status: 'succeeded' | 'failed' - message: string -} - -// Catalog types -export interface SourceDefinition { - id: string - name: string - dockerRepository?: string - documentationUrl?: string - icon?: string -} - -export interface DestinationDefinition { - id: string - name: string - dockerRepository?: string - documentationUrl?: string - icon?: string -} diff --git a/rewrites/airtable/.beads/.gitignore b/rewrites/airtable/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/airtable/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/airtable/.beads/README.md b/rewrites/airtable/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/airtable/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/airtable/.beads/config.yaml b/rewrites/airtable/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/airtable/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/airtable/.beads/interactions.jsonl b/rewrites/airtable/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/airtable/.beads/issues.jsonl b/rewrites/airtable/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/airtable/.beads/metadata.json b/rewrites/airtable/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/airtable/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/airtable/.gitattributes b/rewrites/airtable/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/airtable/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/airtable/AGENTS.md b/rewrites/airtable/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/airtable/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/airtable/README.md b/rewrites/airtable/README.md deleted file mode 100644 index cf5f4962..00000000 --- a/rewrites/airtable/README.md +++ /dev/null @@ -1,358 +0,0 @@ -# airtable.do - -> The spreadsheet-database. Edge-Native. Open by Default. AI-First. - -Airtable bridged the gap between spreadsheets and databases. But at $20-45/user/month with row limits, record caps, and AI locked behind enterprise pricing, it's become expensive for what it is. - -**airtable.do** is the spreadsheet-database reimagined. No row limits. No per-seat pricing. AI that builds apps for you. Own your data infrastructure. - -## AI-Native API - -```typescript -import { airtable } from 'airtable.do' // Full SDK -import { airtable } from 'airtable.do/tiny' // Minimal client -import { airtable } from 'airtable.do/sync' // Real-time sync -``` - -Natural language for data workflows: - -```typescript -import { airtable } from 'airtable.do' - -// Talk to it like a colleague -const deals = await airtable`deals over $50k closing this month` -const forecast = await airtable`projected revenue for Q2` -const churn = await airtable`customers at risk of churning` - -// Chain like sentences -await airtable`leads needing followup` - .notify(`Time to reach out`) - -// Apps that build themselves -await airtable`create project tracker with tasks, owners, and deadlines` - .addView('kanban by status') - .addView('calendar by deadline') -``` - -## The Problem - -Airtable's pricing creates artificial scarcity: - -| Plan | Price | Limits | -|------|-------|--------| -| Free | $0 | 1,000 records/base, 1GB attachments | -| Team | $20/user/month | 50,000 records/base, 20GB | -| Business | $45/user/month | 125,000 records/base, 100GB | -| Enterprise | Custom | 500,000 records, unlimited | - -**50-person team on Business?** That's **$27,000/year**. For a spreadsheet with a database backend. - -The real costs: -- **Record limits** - Hit 50k rows? Pay more or split your data -- **Per-seat pricing** - Every viewer costs money -- **Sync limits** - Real-time sync only on higher tiers -- **AI locked away** - AI features require Enterprise -- **Automation limits** - 25k-500k runs/month depending on tier -- **API rate limits** - 5 requests/second, seriously - -## The Solution - -**airtable.do** is what Airtable should be: - -``` -Traditional Airtable airtable.do ------------------------------------------------------------------ -$20-45/user/month $0 - run your own -Row limits (1k-500k) Unlimited rows -Per-seat pricing Use what you need -AI for Enterprise AI-native from start -5 req/sec API limit Unlimited API -Their servers Your Cloudflare account -Proprietary formulas Open formulas + TypeScript -``` - -## One-Click Deploy - -```bash -npx create-dotdo airtable -``` - -Your own Airtable. Running on Cloudflare. No limits. - -```typescript -import { Airtable } from 'airtable.do' - -export default Airtable({ - name: 'my-workspace', - domain: 'data.my-company.com', -}) -``` - -## Features - -### Bases & Tables - -```typescript -// Create bases naturally -await airtable`create Product Development base` -await airtable`add Features table with name, status, priority, owner, and due date` -await airtable`add Sprints table linked to Features` - -// AI infers the schema from your description -await airtable` - create CRM with: - - Companies with name, industry, size, and website - - Contacts linked to companies with name, email, role - - Deals linked to contacts with amount, stage, close date -` -``` - -### Records - -```typescript -// Create records naturally -await airtable`add task "Design homepage" for Sarah due Friday` -await airtable`new deal Acme Corp $50k in negotiation with John` -await airtable` - add contacts: - - Jane Smith, CEO at TechCo, jane@techco.com - - Bob Wilson, CTO at StartupX, bob@startupx.com -` - -// Query naturally -const tasks = await airtable`tasks assigned to Sarah` -const overdue = await airtable`overdue items` -const pipeline = await airtable`deals in negotiation over $25k` - -// Update naturally -await airtable`mark "Design homepage" complete` -await airtable`move Acme deal to closed won` -await airtable`assign all unassigned tasks to Mike` -``` - -### Field Types - -All the field types you'd expect - text, numbers, dates, dropdowns, linked records, formulas, attachments, and more. AI infers the right type from context. - -### Formulas - -```typescript -// Describe the calculation, AI writes the formula -await airtable`add formula "days until due" to Tasks` -await airtable`add field showing whether tasks are overdue` -await airtable`calculate total deal value per company` - -// Or express complex logic naturally -await airtable` - add priority score that's higher for: - - critical items (100 points) - - overdue items (1.5x multiplier) - - high value deals (extra 25 points if over $50k) -` -``` - -### Views - -```typescript -// Create views naturally -await airtable`show tasks as kanban by status` -await airtable`show deals as calendar by close date` -await airtable`show projects as timeline from start to due date` -await airtable`show designs as gallery with mockups` - -// Filtered views -await airtable`show my tasks sorted by priority` -await airtable`show overdue items grouped by owner` -await airtable`show Q1 deals over $50k as funnel by stage` -``` - -### Forms - -```typescript -// Create forms from tables -await airtable`create feedback form from Feedback table` -await airtable`create job application form with name, email, resume, and cover letter` - -// Configure with natural language -await airtable`make feedback form public with logo and custom thank you message` -await airtable`notify #product-feedback on Slack when form submitted` -``` - -## AI-Native Data Management - -AI doesn't just assist - it builds with you. - -### Schema Design - -```typescript -// Describe what you need, AI builds the schema -await airtable` - I need to track content marketing: - - blog posts with authors and topics - - draft, review, published workflow - - SEO metrics and social engagement -` -// AI creates Posts, Authors, Topics tables with proper relationships -``` - -### Data Entry - -```typescript -// Enter data naturally -await airtable`add post "AI in Project Management" by Sarah, topics AI and Productivity` -await airtable` - add contacts: - - John Smith, CEO at Acme, john@acme.com, met at conference - - Jane Doe, CTO at TechCo, jane@techco.com, inbound lead -` -``` - -### Data Cleanup - -```typescript -// Fix messy data with one command -await airtable`clean up Contacts: fix phone formats, merge duplicates, categorize companies` -await airtable`standardize country names in Leads table` -await airtable`split full names into first and last name` -``` - -### Insights - -```typescript -// Ask questions about your data -await airtable`which products are performing best this quarter?` -await airtable`who's exceeding quota?` -await airtable`show me churn risk by customer segment` - -// Get actionable recommendations -await airtable`what should we focus on to hit Q2 targets?` -``` - -## Automations - -```typescript -// Create automations naturally -await airtable`when new user signs up, send welcome email and notify #signups` -await airtable`when deal over $100k changes, notify sales director` -await airtable`every Monday at 9am, email pipeline report to sales team` - -// Chain automations with agents -await airtable`new leads` - .map(lead => priya`qualify ${lead}`) - .map(lead => airtable`update ${lead} with qualification score`) - .filter(lead => lead.score > 80) - .map(lead => sally`draft outreach for ${lead}`) - -// Bulk operations as pipelines -await airtable`overdue tasks` - .map(task => airtable`notify owner of ${task}`) - .map(task => airtable`escalate ${task} if over 7 days`) -``` - -## Interfaces (Apps) - -```typescript -// Build dashboards naturally -await airtable`create sales dashboard with pipeline value, deals by stage, and recent activity` -await airtable`add funnel chart showing deals by status` -await airtable`add team performance page with quotas and leaderboard` - -// Deploy as standalone apps -await airtable`publish sales dashboard at sales.my-company.com with SSO` -await airtable`create public status page from Projects table` -``` - -## API Compatible - -Full Airtable REST API compatibility. Existing Airtable SDK code works - just change the URL: - -```typescript -import Airtable from 'airtable' - -const base = new Airtable({ - apiKey: process.env.AIRTABLE_TOKEN, - endpointUrl: 'https://your-org.airtable.do', // Just change this -}).base('appXXXXXXXX') -``` - -## Architecture - -### Durable Object per Base - -``` -WorkspaceDO (bases, permissions) - | - +-- BaseDO:product-base - | +-- SQLite: tables, records, relations - | +-- Views, filters, formulas - | +-- WebSocket: real-time sync - | - +-- BaseDO:crm-base - +-- BaseDO:content-base - +-- AutomationDO (automation engine) -``` - -Each base is fully isolated. SQLite handles millions of rows efficiently. No artificial limits. - -## Migration from Airtable - -```bash -npx airtable-do migrate --token=your_pat --base=appXXXXXXXX -``` - -Imports everything: tables, records, views, linked records, formulas, automations. - -## Roadmap - -- [x] Tables with all field types -- [x] Linked records and rollups -- [x] Formulas (Airtable-compatible) -- [x] All view types -- [x] Forms -- [x] API compatibility -- [x] Automations (unlimited) -- [x] AI schema and formula generation -- [ ] Interfaces (custom apps) -- [ ] Extensions marketplace -- [ ] Synced tables -- [ ] Real-time collaboration -- [ ] Scripting (TypeScript) - -## Why Open Source? - -Your data infrastructure shouldn't have row limits: - -1. **Your data** - Operational data is too important for third parties -2. **Your schema** - How you model data is institutional knowledge -3. **Your automations** - Business logic lives in workflows -4. **Your AI** - Intelligence on your data should be yours - -Airtable showed the world what spreadsheet-database hybrids could be. **airtable.do** removes the limits and adds the AI. - -## Contributing - -We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - -Key areas: -- Field types and formula engine -- Views and visualization -- AI capabilities -- API compatibility -- Performance optimization - -## License - -MIT License - Build your business on it. - ---- - -

- The row limits end here. -
- No caps. No per-seat pricing. AI-native. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/algolia/.beads/.gitignore b/rewrites/algolia/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/algolia/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/algolia/.beads/README.md b/rewrites/algolia/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/algolia/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/algolia/.beads/config.yaml b/rewrites/algolia/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/algolia/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/algolia/.beads/interactions.jsonl b/rewrites/algolia/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/algolia/.beads/issues.jsonl b/rewrites/algolia/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/algolia/.beads/metadata.json b/rewrites/algolia/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/algolia/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/algolia/.gitattributes b/rewrites/algolia/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/algolia/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/algolia/AGENTS.md b/rewrites/algolia/AGENTS.md deleted file mode 100644 index ad76a256..00000000 --- a/rewrites/algolia/AGENTS.md +++ /dev/null @@ -1,132 +0,0 @@ -# Algolia Rewrite - Agent Instructions - -This project reimplements Algolia's search-as-a-service on Cloudflare Durable Objects. - -## Project Context - -**Package**: `@dotdo/algolia` or `searches.do` -**Location**: `/rewrites/algolia/` -**Pattern**: Follow the supabase.do rewrite architecture - -## Architecture Overview - -``` -algolia/ - src/ - core/ - indexing/ # Object indexing operations - search/ # Hybrid FTS5 + Vectorize search - faceting/ # Facet aggregation & caching - ranking/ # Custom ranking & ML re-ranking - durable-object/ - IndexDO # Per-index state (SQLite + Vectorize) - client/ - algoliasearch # SDK-compatible client - mcp/ # AI tool definitions -``` - -## Key Cloudflare Primitives - -| Primitive | Usage | -|-----------|-------| -| **D1 + FTS5** | Full-text keyword search | -| **Vectorize** | Semantic vector search | -| **Workers AI** | Embedding generation (bge models) | -| **KV** | Query/facet caching | -| **Durable Objects** | Per-index isolation | - -## TDD Workflow - -All work follows strict RED-GREEN-REFACTOR: - -1. **[RED]** Write failing tests first -2. **[GREEN]** Implement minimal code to pass -3. **[REFACTOR]** Optimize and clean up - -Check blocking dependencies: -```bash -bd blocked # Shows what's waiting on what -bd ready # Shows tasks ready to work on -``` - -## Beads Issue Tracking - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - -## API Reference - -### Algolia-Compatible Methods - -```typescript -// Indexing -index.saveObjects(objects) -index.saveObject(object) -index.partialUpdateObjects(objects) -index.deleteObjects(objectIDs) - -// Search -index.search(query, params) -index.searchForFacetValues(facetName, facetQuery) -client.multipleQueries(queries) - -// Settings -index.setSettings(settings) -index.getSettings() -``` - -### Search Parameters - -| Parameter | Description | -|-----------|-------------| -| `query` | Search query string | -| `filters` | Filter expression | -| `facetFilters` | Facet filter array | -| `numericFilters` | Numeric filter array | -| `hitsPerPage` | Results per page | -| `page` | Page number (0-indexed) | -| `facets` | Facets to retrieve | -| `attributesToRetrieve` | Fields to return | - -## Performance Targets - -- P50 search latency: < 10ms -- P99 search latency: < 50ms -- Indexing throughput: > 1000 docs/sec - -## Related Documentation - -- `/rewrites/search/docs/search-rewrite-scope.md` - Search research -- `/rewrites/supabase/README.md` - Rewrite pattern reference -- `/rewrites/CLAUDE.md` - Rewrite architecture guidelines diff --git a/rewrites/algolia/README.md b/rewrites/algolia/README.md deleted file mode 100644 index bc033cc9..00000000 --- a/rewrites/algolia/README.md +++ /dev/null @@ -1,388 +0,0 @@ -# algolia.do - -> Search-as-a-Service. Edge-Native. AI-First. Natural Language. - -Algolia charges $290/month for 100K documents. Typesense Cloud charges $60. algolia.do costs $1. Same features. 100x cheaper. Because search belongs at the edge. - -**algolia.do** is search reimagined for AI agents. Natural language queries. No configuration objects. Just say what you want to find. - -## AI-Native API - -```typescript -import { algolia } from 'algolia.do' // Full SDK -import { algolia } from 'algolia.do/tiny' // Minimal client -import { algolia } from 'algolia.do/instant' // InstantSearch compatible -``` - -Natural language for search: - -```typescript -import { algolia } from 'algolia.do' - -// Talk to it like a colleague -const headphones = await algolia`wireless headphones under $200` -const trending = await algolia`best selling products this week` -const similar = await algolia`products similar to AirPods Pro` - -// Chain like sentences -await algolia`products needing reviews` - .notify(`Your review would help other customers`) - -// Index without boilerplate -await algolia`index products from catalog` -await algolia`add SKU-12345 to products: Wireless Headphones $99` -await algolia`remove discontinued items from inventory` -``` - -## The Problem - -Algolia dominates search: - -| What Algolia Charges | The Reality | -|----------------------|-------------| -| **Search Operations** | $1.50 per 1,000 requests | -| **Records** | $0.40 per 1,000 records/month | -| **Index Replicas** | Each replica costs extra | -| **AI Features** | Premium tier only | -| **Support** | Enterprise pricing for priority | -| **Overages** | Surprise bills when you scale | - -### The Hidden Costs - -Beyond the pricing page: -- Vendor lock-in (proprietary query syntax) -- Limited semantic search -- AI features gated behind enterprise -- Cold start latency on shared infrastructure -- Per-index pricing kills multi-tenant apps - -### What AI Agents Need - -AI agents search differently: -- Natural language, not query DSL -- One index per agent or project -- Semantic understanding built-in -- Infinite indexes without infinite cost -- Edge latency, not datacenter latency - -## The Solution - -**algolia.do** reimagines search for AI: - -``` -Algolia algolia.do ------------------------------------------------------------------ -$290/month for 100K docs ~$1/month -Proprietary query syntax Natural language -Shared infrastructure Your Durable Object -Manual index management Just describe what you want -AI features = enterprise AI-first by default -Cold starts Sub-10ms edge latency -Per-index pricing Pay for compute, not indexes -``` - -## One-Click Deploy - -```bash -npx create-dotdo algolia -``` - -Search infrastructure on your Cloudflare account. Your data. Your control. - -```typescript -import { Algolia } from 'algolia.do' - -export default Algolia({ - name: 'my-search', - domain: 'search.myapp.com', - semantic: true, -}) -``` - -## Features - -### Searching - -```typescript -// Just say what you want -const results = await algolia`wireless headphones` -const filtered = await algolia`Sony headphones under $150` -const semantic = await algolia`comfortable audio for long flights` - -// AI infers what you need -await algolia`headphones` // returns products -await algolia`headphones by brand` // returns faceted results -await algolia`headphones trending` // returns sorted by popularity -``` - -### Indexing - -```typescript -// Index naturally -await algolia`index products from catalog` -await algolia`add to products: Wireless Earbuds $79 category:audio` -await algolia`update SKU-12345 price to $89` -await algolia`remove out-of-stock items` - -// Bulk operations read like instructions -await algolia` - products index: - - SKU-001: Wireless Headphones $99 - - SKU-002: Bluetooth Speaker $49 - - SKU-003: USB-C Cable $15 -` -``` - -### Faceting - -```typescript -// Natural facet queries -const brands = await algolia`headphones by brand` -const priceRanges = await algolia`headphones grouped by price range` -const categories = await algolia`all product categories with counts` - -// Filter like you'd say it -await algolia`Sony or Bose headphones under $200` -await algolia`4+ star products in electronics` -``` - -### Synonyms and Rules - -```typescript -// Configure with natural language -await algolia`synonyms: headphones = earbuds = earphones` -await algolia`synonyms: tv = television = flatscreen` -await algolia`boost: promoted products should rank higher` -await algolia`rule: searches for "cheap" should filter under $50` -``` - -### Hybrid Search - -```typescript -// Semantic search is natural -await algolia`good headphones for noisy offices` -await algolia`something to block out airplane noise` -await algolia`gift for someone who loves music` - -// AI understands intent, not just keywords -``` - -### Real-time Updates - -```typescript -// Live search that updates -const search = await algolia`wireless headphones`.live() - -search.on('update', (results) => { - // New products matching query -}) - -// Or with React -await algolia`trending products` - .subscribe(products => setProducts(products)) -``` - -## Pipeline Chains - -Chain operations without Promise.all: - -```typescript -// Find, analyze, act -await algolia`low stock products` - .map(product => algolia`suppliers for ${product.sku}`) - .map(suppliers => suppliers.notify(`Restock needed`)) - -// Search across multiple indexes -await algolia`search all indexes for "wireless"` - .map(result => result.boost()) - -// Batch operations -await algolia`products without images` - .map(product => product.generateImage()) - .map(product => product.save()) -``` - -## Architecture - -### Durable Object per Index - -``` - +-----------------------+ - | algolia.do | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | IndexDO (Tom) | | IndexDO (Ralph) | | IndexDO (...) | - | SQLite + FTS5 | | SQLite + FTS5 | | SQLite + FTS5 | - | + Vectorize | | + Vectorize | | + Vectorize | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +-------------------+ - | Vectorize | - | (semantic index) | - +-------------------+ -``` - -Each agent, project, or tenant gets their own Durable Object. SQLite FTS5 for keyword search. Vectorize for semantic search. No shared infrastructure. - -### Storage Tiers - -| Tier | Storage | Use Case | Latency | -|------|---------|----------|---------| -| **Hot** | SQLite FTS5 | Active indexes | <10ms | -| **Warm** | KV Cache | Frequent queries | <5ms | -| **Cold** | R2 | Index snapshots | <100ms | - -## vs Algolia - -| Feature | Algolia | algolia.do | -|---------|---------|------------| -| **100K docs** | ~$290/month | ~$1/month | -| **Query syntax** | Proprietary DSL | Natural language | -| **Semantic search** | Enterprise only | Built-in | -| **Latency** | 50-100ms | <10ms edge | -| **Multi-tenant** | Expensive | Free (per-DO) | -| **AI features** | Premium tier | Native | -| **Lock-in** | Proprietary | Open source | -| **InstantSearch** | Official | Compatible | - -## Use Cases - -### E-commerce - -```typescript -// Product search -await algolia`red running shoes size 10` -await algolia`gifts under $50 for runners` - -// Inventory management -await algolia`low stock items needing reorder` - .notify(`Restock alert`) -``` - -### Documentation - -```typescript -// Search docs naturally -await algolia`how to configure authentication` -await algolia`examples of rate limiting` -await algolia`errors related to permissions` -``` - -### AI Agents - -```typescript -import { tom, ralph, priya } from 'agents.do' - -// Each agent has isolated search -await algolia.for(tom)`my reviewed PRs` -await algolia.for(ralph)`components I built` -await algolia.for(priya)`features in the roadmap` -``` - -## InstantSearch Compatible - -```typescript -import { InstantSearch, SearchBox, Hits } from 'react-instantsearch' -import { algolia } from 'algolia.do' - -function App() { - return ( - - - - - ) -} -``` - -Works with Algolia's React, Vue, Angular, and vanilla JS libraries. - -## MCP Tools - -AI-native search through Model Context Protocol: - -```typescript -// Available as MCP tool -await mcp.search({ - index: 'products', - query: 'wireless headphones under $200' -}) - -// Or natural language -await mcp.algolia`find me noise cancelling headphones` -``` - -## The Rewrites Ecosystem - -algolia.do is part of the rewrites family: - -| Rewrite | Original | Purpose | -|---------|----------|---------| -| [fsx.do](https://fsx.do) | fs (Node.js) | Filesystem for AI | -| [gitx.do](https://gitx.do) | git | Version control for AI | -| [supabase.do](https://supabase.do) | Supabase | Postgres/BaaS for AI | -| **algolia.do** | Algolia | Search for AI | -| [mongo.do](https://mongo.do) | MongoDB | Document database for AI | -| [kafka.do](https://kafka.do) | Kafka | Event streaming for AI | - -## Roadmap - -### Search -- [x] Full-text search (FTS5) -- [x] Semantic search (Vectorize) -- [x] Hybrid search -- [x] Faceting -- [x] Typo tolerance -- [x] Synonyms -- [ ] Geo search -- [ ] Personalization - -### Features -- [x] Natural language queries -- [x] Real-time updates -- [x] InstantSearch compatible -- [x] MCP tools -- [ ] Query suggestions -- [ ] A/B testing -- [ ] Analytics - -### AI -- [x] Semantic understanding -- [x] Intent detection -- [ ] Query expansion -- [ ] Result re-ranking -- [ ] Conversational search - -## Contributing - -algolia.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/algolia.do -cd algolia.do -pnpm install -pnpm test -``` - -## License - -MIT License - Search belongs at the edge. - ---- - -

- 100x cheaper. Natural language. Edge-native. -
- Search reimagined for AI. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/amplitude/.beads/.gitignore b/rewrites/amplitude/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/amplitude/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/amplitude/.beads/README.md b/rewrites/amplitude/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/amplitude/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/amplitude/.beads/config.yaml b/rewrites/amplitude/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/amplitude/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/amplitude/.beads/interactions.jsonl b/rewrites/amplitude/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/amplitude/.beads/issues.jsonl b/rewrites/amplitude/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/amplitude/.beads/metadata.json b/rewrites/amplitude/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/amplitude/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/amplitude/.gitattributes b/rewrites/amplitude/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/amplitude/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/amplitude/AGENTS.md b/rewrites/amplitude/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/amplitude/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/amplitude/README.md b/rewrites/amplitude/README.md deleted file mode 100644 index db5325b0..00000000 --- a/rewrites/amplitude/README.md +++ /dev/null @@ -1,486 +0,0 @@ -# amplitude.do - -> Product Analytics. Edge-Native. Open by Default. AI-First. - -Amplitude built a $50k+/year product analytics empire. Per-MTU pricing that explodes at scale. Behavioral cohorts locked behind Growth tier. Experiments gated to Enterprise. Data retention capped. Your own data held hostage. - -**amplitude.do** is the open-source alternative. Zero MTU pricing. Unlimited events. AI-powered insights. Deploys in minutes, not sales calls. - -## AI-Native API - -```typescript -import { amplitude } from 'amplitude.do' // Full SDK -import { amplitude } from 'amplitude.do/tiny' // Minimal client -import { amplitude } from 'amplitude.do/analytics' // Analytics only -``` - -Natural language for product analytics: - -```typescript -import { amplitude } from 'amplitude.do' - -// Talk to it like a PM -const funnel = await amplitude`signup to purchase funnel this month` -const cohort = await amplitude`power users: 10+ sessions this week` -const drop = await amplitude`why did signups drop last Tuesday?` - -// Chain like sentences -await amplitude`users who signed up but never purchased` - .notify(`Complete your first purchase for 20% off`) - -// Events that capture themselves -await amplitude`user-123 clicked signup button` -await amplitude`sarah@acme.com upgraded to pro plan` -await amplitude`track pageview /dashboard for user-456` -``` - -## The Problem - -Amplitude dominates product analytics with aggressive pricing: - -| What Amplitude Charges | The Reality | -|------------------------|-------------| -| **Per-MTU Pricing** | Costs explode with success | -| **Growth Plan** | $50k+/year minimum | -| **Enterprise Plan** | $100k+/year+ | -| **Behavioral Cohorts** | Locked behind Growth tier | -| **Predictions** | Enterprise only | -| **Experiments** | Enterprise only | -| **Data Retention** | Capped by tier | - -### The MTU Tax - -10 million Monthly Tracked Users? You're paying $100k+/year. And that's before: - -- Behavioral cohorts -- Churn predictions -- A/B experiments -- Raw data export -- SSO/SAML - -Every successful feature launch increases your analytics bill. - -### The Feature Lock - -The most valuable analytics capabilities are paywalled: - -- **Behavioral cohorts** - Define users by what they do, not who they are -- **Predictions** - Churn risk, conversion likelihood, LTV -- **Experiments** - A/B testing with statistical rigor -- **Root cause** - AI explaining why metrics changed - -Growth companies need these most. Amplitude charges them extra. - -## The Solution - -**amplitude.do** reimagines product analytics: - -``` -Amplitude amplitude.do ------------------------------------------------------------------ -Per-MTU pricing Flat: pay for storage, not users -$50k/year Growth $0 - run your own -Behavioral cohorts (paid) Behavioral cohorts (free) -Limited data retention Unlimited (R2 storage) -Identity resolution (extra) Built-in identity graph -Raw data export (premium) Your data, always accessible -``` - -## One-Click Deploy - -```bash -npx create-dotdo amplitude -``` - -Your own Amplitude instance. Running on Cloudflare. Zero MTU fees. - -```typescript -import { Amplitude } from 'amplitude.do' - -export default Amplitude({ - name: 'my-product', - domain: 'analytics.my-product.com', -}) -``` - -## Features - -### Event Tracking - -```typescript -// Just say it -await amplitude`user-123 clicked signup button` -await amplitude`sarah@acme.com viewed pricing page` -await amplitude`user-456 completed checkout $99` - -// AI infers the event structure -await amplitude`user-123 clicked signup button` -// → { event: 'Button Clicked', properties: { button: 'signup' }, user_id: 'user-123' } - -// Batch events read like a log -await amplitude` - user-123: - - viewed landing page - - clicked signup - - completed registration - - started onboarding -` -``` - -### User Identification - -```typescript -// Identify naturally -await amplitude`user-123 is sarah@acme.com on pro plan at Acme Corp` -await amplitude`sarah works at Acme, 500 employees, Technology` - -// AI links identities -await amplitude`link user-123 to device-abc` -await amplitude`merge anonymous-456 into user-123` -``` - -### Funnels - -```typescript -// Funnels are one line -const funnel = await amplitude`signup to purchase funnel this month` -const mobile = await amplitude`iOS signup funnel last week` -const checkout = await amplitude`cart to checkout to payment funnel` - -// AI infers the steps -await amplitude`signup to purchase funnel` -// → Landing → Signup → Email Verified → First Purchase - -// Segment naturally -await amplitude`signup funnel by platform` -await amplitude`checkout funnel iOS vs Android` -await amplitude`onboarding funnel for enterprise users` - -// Find the leaks -await amplitude`where do users drop in checkout?` -await amplitude`biggest drop in signup funnel` -``` - -### Retention - -```typescript -// Retention curves naturally -await amplitude`day 1 7 30 retention this quarter` -await amplitude`retention for users who completed onboarding` -await amplitude`retention iOS vs Android` - -// Compare cohorts -await amplitude`retention: Jan signups vs Feb signups` -await amplitude`do power users retain better?` - -// Unbounded retention -await amplitude`users who ever purchased after signup` -``` - -### Cohorts - -```typescript -// Behavioral cohorts in plain English -const power = await amplitude`power users: 10+ sessions this week` -const dormant = await amplitude`users inactive for 30 days` -const churning = await amplitude`users likely to churn` - -// Combine conditions naturally -await amplitude`pro users who haven't used feature X` -await amplitude`signed up last month but never purchased` -await amplitude`mobile users with 5+ sessions this week` - -// Lifecycle cohorts just work -await amplitude`new users this week` -await amplitude`resurrected users` -await amplitude`users at risk of churning` - -// Act on cohorts -await amplitude`users who signed up but never purchased` - .notify(`Complete your first purchase for 20% off`) - .each(user => amplitude`flag ${user} for sales outreach`) -``` - -### User Journeys - -```typescript -// See how users flow -await amplitude`paths from signup to purchase` -await amplitude`what do users do after checkout?` -await amplitude`how do power users navigate?` - -// Find friction -await amplitude`where do users get stuck?` -await amplitude`dead ends in the product` -await amplitude`rage clicks this week` -``` - -### Experiments - -```typescript -// A/B tests are questions -await amplitude`new checkout vs control` -await amplitude`which pricing page converts better?` -await amplitude`is dark mode increasing engagement?` - -// Run experiments naturally -await amplitude`run experiment: blue button vs green button` - .on(`signup page visitors`) - .measure(`signups`) - -// Get results -await amplitude`checkout experiment results` -// → { winner: 'new_flow', lift: '+20%', confidence: 95% } - -// Feature flags -await amplitude`should user-123 see new feature?` -await amplitude`roll out dark mode to 10% of users` -``` - -## AI-Native Analytics - -### Root Cause Analysis - -```typescript -// Ask why, get answers -await amplitude`why did signups drop last Tuesday?` -// → { factors: ['ad campaign ended', 'homepage A/B test'], confidence: 0.85 } - -await amplitude`what changed when conversion improved?` -await amplitude`why is iOS retention higher than Android?` -await amplitude`what do power users do differently?` -``` - -### Predictive Analytics - -```typescript -// Predictions as questions -await amplitude`will user-123 churn?` -// → { probability: 0.72, factors: ['no login in 14 days', 'support ticket open'] } - -await amplitude`likelihood user-123 purchases this week` -await amplitude`predicted LTV for user-123` - -// Batch predictions -await amplitude`users likely to churn this month` - .each(user => amplitude`flag ${user} for retention outreach`) -``` - -### Auto-Instrumentation - -```typescript -// AI captures everything automatically -await amplitude`auto-track my-app.com` - -// AI names events semantically -// "Submit Button Clicked" not "click_btn_abc123" -// "Pricing Page Viewed" not "pageview_/pricing" - -// Rage click detection automatic -// Dead click detection automatic -// Session recording optional -``` - -### AI Agents as Analysts - -```typescript -import { priya, quinn } from 'agents.do' - -// PM analyzes user behavior -await priya`analyze our activation funnel and recommend changes` - .map(recommendations => amplitude`create issues for ${recommendations}`) - -// QA finds issues through event patterns -await quinn`find anomalies in our event data` - .map(anomalies => amplitude`alert on-call for ${anomalies}`) - -// Chain insights into action -await amplitude`weekly product insights` - .map(insights => priya`prioritize these for Q2`) - .map(priorities => amplitude`create dashboard for ${priorities}`) -``` - -## Architecture - -### Event Pipeline - -``` -Client SDK --> Edge Worker --> Event DO --> Storage Tiers - | - +------+------+ - | | - Validation Enrichment - (Schema) (GeoIP, UA) -``` - -### Durable Object per Product - -``` -ProductDO (config, users, events) - | - +-- UsersDO (identity graph) - | |-- SQLite: User profiles - | +-- Identity resolution - | - +-- EventsDO (event buffer) - | |-- SQLite: Recent events (7 days) - | +-- R2: Historical events (Parquet) - | - +-- AnalyticsDO (query engine) - | |-- Analytics Engine: Real-time aggregations - | +-- Cohort definitions - | - +-- ExperimentsDO (A/B testing) - |-- SQLite: Experiment configs - +-- Variant assignments -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Recent events (7 days) | <10ms | -| **Warm** | R2 Parquet | Historical events (months) | <100ms | -| **Cold** | R2 Archive | Long-term retention (years) | <1s | - -### Query Engine - -Real-time aggregations on Analytics Engine. Billions of events. Sub-second queries. - -## vs Amplitude - -| Feature | Amplitude | amplitude.do | -|---------|-----------|--------------| -| **Pricing** | Per-MTU ($50k+/year) | Flat storage cost | -| **Cohorts** | Growth tier only | Free | -| **Predictions** | Enterprise only | Free | -| **Experiments** | Enterprise only | Free | -| **Data Retention** | Tier-limited | Unlimited | -| **Raw Data Export** | Premium only | Your data, always | -| **Architecture** | Centralized | Edge-native, global | -| **Data Location** | Amplitude's cloud | Your Cloudflare account | -| **Lock-in** | Years of data trapped | MIT licensed | - -## Use Cases - -### Product Analytics - -```typescript -// Complete product analytics in natural language -await amplitude`daily active users this month` -await amplitude`feature adoption for new feature X` -await amplitude`power users vs casual users comparison` -``` - -### Growth Experiments - -```typescript -// Run experiments, ship winners -await amplitude`onboarding A vs B experiment results` - .map(result => amplitude`roll out ${result.winner} to 100%`) -``` - -### Churn Prevention - -```typescript -// Find and save at-risk users -await amplitude`users likely to churn this month` - .map(users => amplitude`create cohort at-risk from ${users}`) - .map(cohort => amplitude`notify ${cohort} with retention offer`) -``` - -### Product-Led Growth - -```typescript -// Activation insights -await amplitude`users who activated vs didn't - what's different?` - .map(insights => priya`recommend onboarding changes`) -``` - -## Migration from Amplitude - -```bash -# Export from Amplitude -amplitude export --project YOUR_PROJECT --start 2024-01-01 --end 2024-03-31 - -# Import to amplitude.do -npx amplitude-migrate import ./export.json -``` - -### API Compatibility - -Drop-in replacement for Amplitude HTTP API. All endpoints supported. - -## Roadmap - -### Core Analytics -- [x] Event Tracking -- [x] User Identification -- [x] Identity Resolution -- [x] Funnels -- [x] Retention -- [x] Cohorts -- [x] User Journeys -- [ ] Session Replay -- [ ] Heatmaps - -### AI -- [x] Natural Language Queries -- [x] Root Cause Analysis -- [x] Predictive Analytics -- [x] Auto-Instrumentation -- [ ] Anomaly Detection -- [ ] AI Insights Digest - -### Experiments -- [x] A/B Testing -- [x] Statistical Analysis -- [x] Feature Flags -- [ ] Holdout Groups -- [ ] Multi-Armed Bandits - -### Platform -- [x] Real-time Streaming -- [x] Batch Import -- [x] Data Export -- [ ] Warehouse Sync -- [ ] Reverse ETL - -## Why Open Source? - -Product analytics shouldn't cost more as you succeed: - -1. **Your events** - User behavior data is your product's lifeblood -2. **Your cohorts** - Behavioral segmentation drives growth -3. **Your experiments** - A/B testing shouldn't be premium -4. **Your scale** - Success shouldn't mean $100k+ analytics bills - -Amplitude showed the world what product analytics could be. **amplitude.do** makes it accessible to everyone. - -## Contributing - -amplitude.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/amplitude.do -cd amplitude.do -pnpm install -pnpm test -``` - -## License - -MIT License - Track all the events. Run all the experiments. - ---- - -

- The $50k/year tax ends here. -
- Zero MTU. Unlimited events. AI-first. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/asana/.beads/.gitignore b/rewrites/asana/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/asana/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/asana/.beads/README.md b/rewrites/asana/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/asana/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/asana/.beads/config.yaml b/rewrites/asana/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/asana/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/asana/.beads/interactions.jsonl b/rewrites/asana/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/asana/.beads/issues.jsonl b/rewrites/asana/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/asana/.beads/metadata.json b/rewrites/asana/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/asana/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/asana/.gitattributes b/rewrites/asana/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/asana/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/asana/AGENTS.md b/rewrites/asana/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/asana/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/asana/README.md b/rewrites/asana/README.md deleted file mode 100644 index e65cf93c..00000000 --- a/rewrites/asana/README.md +++ /dev/null @@ -1,453 +0,0 @@ -# asana.do - -> Work management. AI-coordinated. Open source. - -Asana pioneered modern work management. Tasks, projects, portfolios, goals - the whole hierarchy of getting things done. At $10.99-24.99/user/month, with premium features behind higher tiers and AI as another add-on, it's become expensive coordination tax. - -**asana.do** is work management reimagined. AI coordinates your work across teams. Goals cascade automatically. Tasks self-organize. The work manages itself. - -## The Problem - -Asana's tiered pricing: - -| Plan | Price | What's Missing | -|------|-------|----------------| -| Basic | Free | Views, timeline, portfolios, goals | -| Premium | $10.99/user/month | Portfolios, goals, resource mgmt | -| Business | $24.99/user/month | AI features, advanced reporting | -| Enterprise | Custom | SSO, advanced security | - -**200-person company on Business?** That's **$59,976/year** for work management. - -The hidden costs: -- **Goal tracking is Premium** - The whole point of work is locked behind paywall -- **Portfolio views are Premium** - Can't see across projects without upgrading -- **AI is Business-only** - The features that save time cost the most -- **Guest limits** - External collaborators count toward seats -- **Integration limits** - Good integrations require higher tiers - -## The Solution - -**asana.do** is work management that works for you: - -``` -Traditional Asana asana.do ------------------------------------------------------------------ -$10.99-24.99/user/month $0 - run your own -AI as Business add-on AI-native everywhere -Goals in Premium only Goals included, AI-cascaded -Portfolios locked Portfolios included -Their servers Your Cloudflare account -Limited integrations Unlimited integrations -Proprietary Open source -``` - -## One-Click Deploy - -```bash -npx create-dotdo asana -``` - -Your own Asana. Running on Cloudflare. AI that actually coordinates work. - -```bash -# Or add to existing workers.do project -npx dotdo add asana -``` - -## AI-Native API - -```typescript -import { asana } from 'asana.do' // Full SDK -import { asana } from 'asana.do/tiny' // Minimal client -import { asana } from 'asana.do/goals' // Goals-focused operations -``` - -Natural language for work management: - -```typescript -import { asana } from 'asana.do' - -// Talk to it like a colleague -const blocked = await asana`what's blocking the mobile app?` -const capacity = await asana`does engineering have bandwidth?` -const priorities = await asana`what should I focus on today?` - -// Chain like sentences -await asana`overdue tasks in Q1 Launch` - .map(task => asana`notify assignee of ${task}`) - -// Tasks that delegate themselves -await asana`new feature requests this week` - .analyze() // AI categorizes and estimates - .assign() // best-fit team members - .schedule() // realistic timelines -``` - -**Promise pipelining - chain work without Promise.all:** - -```typescript -// Goal tracking pipeline -await asana`Q1 goals` - .map(goal => asana`analyze progress on ${goal}`) - .map(goal => asana`update status for ${goal}`) - .notify(`Q1 goal status updated`) - -// Task delegation pipeline -await asana`new tasks in backlog` - .map(task => asana`estimate ${task}`) - .map(task => asana`assign ${task} to best fit`) - .map(task => asana`add ${task} to current sprint`) -``` - -One network round trip. Record-replay pipelining. Workers working for you. - -## Features - -### Tasks - -The atomic unit of work: - -```typescript -// Create tasks naturally -await asana`create "Implement user authentication" for Alice due Jan 20` -await asana`add OAuth2 flow task - Google SSO, GitHub SSO, email fallback` - -// Subtasks just work -await asana`add subtask "Set up OAuth providers" due Jan 15 to auth task` -await asana`add subtask "Implement session management" due Jan 18 to auth task` - -// Dependencies read like speech -await asana`session management depends on OAuth setup` -await asana`payment integration blocked by legal review` -``` - -### Projects - -Organize tasks into projects: - -```typescript -// Create projects naturally -await asana`create project "Backend Q1 2025" for engineering owned by Alice` -await asana`add sections Backlog, Ready, In Progress, In Review, Done to Backend Q1` - -// Set up custom fields -await asana`add priority field to Backend Q1 with Low, Medium, High, Critical` -await asana`add points field to Backend Q1` -await asana`add sprint field to Backend Q1 with Sprint 1, Sprint 2, Sprint 3` -``` - -### Views - -Multiple ways to see your work: - -```typescript -// Query views naturally -await asana`my incomplete tasks sorted by due date` -await asana`Backend Q1 board view` -await asana`Backend Q1 timeline with dependencies` -await asana`my calendar for this month` -await asana`engineering workload next 4 weeks` -``` - -### Portfolios - -See across all projects: - -```typescript -// Create portfolios naturally -await asana`create portfolio "Q1 2025 Initiatives" for CTO` -await asana`add Backend Q1, Frontend Q1, Mobile Q1, Infrastructure Q1 to Q1 portfolio` - -// Query portfolio views -await asana`Q1 portfolio by status` -await asana`Q1 portfolio timeline with milestones` -await asana`projects at risk in Q1 portfolio` -``` - -### Goals - -Align work to outcomes: - -```typescript -// Create goals naturally -await asana`create goal "Reach $10M ARR" for 2025 owned by CEO` -await asana`current ARR is $5.2M target is $10M` - -// Team goals cascade automatically -await asana`create goal "Launch enterprise features" for Q1 supporting $10M ARR` -await asana`link Backend Q1 to enterprise features goal` - -// Check goal progress -await asana`Q1 goals progress` -await asana`goals at risk` -await asana`how are we tracking to $10M ARR?` -``` - -## AI-Native Work Management - -AI doesn't just help - it coordinates. - -### AI Task Creation - -Describe work naturally: - -```typescript -// Just describe the work - AI creates the tasks -await asana` - We need to launch the new pricing page. - Alice should design the page by Friday. - Bob needs to implement it, depends on design. - Carol will write the copy, can start now. - Dave does QA before launch. -` - -// AI extracts assignees, maps dependencies, infers due dates -``` - -### AI Project Planning - -Let AI structure your project: - -```typescript -// Describe what you need - get a full plan -await asana`plan customer feedback portal for Alice Bob Carol by March 1` - .breakdown() // phases, milestones, dependencies - .assign() // best-fit team members - .schedule() // realistic timelines -``` - -### AI Task Assignment - -Smart work distribution: - -```typescript -// AI assigns based on expertise, workload, availability -await asana`assign new auth task to best fit on engineering` - -// Or let AI rebalance the team -await asana`rebalance engineering workload` - -// Check capacity before assigning -await asana`who can take the payment integration?` -``` - -### AI Goal Tracking - -AI monitors and reports on goals: - -```typescript -// Check goal health -await asana`how is $10M ARR goal tracking?` -await asana`goals at risk this quarter` -await asana`what's impacting enterprise features goal?` - -// Get recommendations -await asana`recommendations for Q1 goals` -``` - -### AI Status Updates - -Never write another status update: - -```typescript -// Generate updates from work activity -await asana`Backend Q1 status update for stakeholders` -await asana`weekly summary for engineering` -await asana`Q1 portfolio executive summary` - -// AI writes the narrative from actual work data -``` - -### Natural Language Queries - -Query your work naturally: - -```typescript -await asana`what's blocking the mobile app?` -await asana`does engineering have bandwidth?` -await asana`will we hit the Q1 launch date?` -await asana`what should I focus on today?` -``` - -## Inbox & My Tasks - -Personal work management: - -```typescript -// Your inbox -await asana`my unread notifications` -await asana`tasks assigned to me this week` -await asana`mentions and comments` - -// My Tasks sections -await asana`my tasks for today` -await asana`upcoming tasks` -await asana`move auth task to Today` -await asana`snooze payment task until Monday` -``` - -## Rules (Automations) - -Automate repetitive work: - -```typescript -// Set up rules naturally -await asana`when task completed move to Done section` -await asana`when due date is tomorrow notify assignee` -await asana`when task added to Urgent set priority Critical and notify #support-urgent` - -// Chain automations with pipelining -await asana`overdue tasks in support-queue` - .map(task => asana`escalate ${task} to manager`) - .map(task => asana`notify customer about ${task}`) -``` - -## API Compatible - -Full Asana API compatibility: - -```typescript -// REST API endpoints -GET /api/1.0/tasks -POST /api/1.0/tasks -GET /api/1.0/tasks/{task_gid} -PUT /api/1.0/tasks/{task_gid} -DELETE /api/1.0/tasks/{task_gid} - -GET /api/1.0/projects -GET /api/1.0/portfolios -GET /api/1.0/goals -GET /api/1.0/teams -GET /api/1.0/users -GET /api/1.0/workspaces - -POST /api/1.0/webhooks -``` - -Existing Asana SDK code works: - -```typescript -import Asana from 'asana' - -const client = Asana.Client.create({ - defaultHeaders: { 'Asana-Enable': 'new_user_task_lists' }, -}).useAccessToken(process.env.ASANA_TOKEN) - -// Just override the base URL -client.dispatcher.url = 'https://your-org.asana.do' - -const tasks = await client.tasks.findByProject(projectId) -``` - -## Architecture - -### Durable Object per Workspace - -``` -WorkspaceDO (workspace config, teams, users) - | - +-- ProjectDO:backend-q1 (tasks, sections, views) - | +-- SQLite: tasks, subtasks, comments - | +-- WebSocket: real-time updates - | - +-- PortfolioDO:q1-initiatives (project aggregation) - +-- GoalDO:company-2025 (goal hierarchy) - +-- InboxDO:user-123 (notifications) - +-- MyTasksDO:user-123 (personal organization) -``` - -### Real-Time Sync - -All changes sync instantly: - -```typescript -// Subscribe to project changes -await asana`watch Backend Q1 for changes` - .on('task:created', task => console.log('New task:', task)) - .on('task:completed', task => console.log('Done:', task)) -``` - -### Storage - -```typescript -// Hot: SQLite in Durable Object -// Active tasks, recent activity - -// Warm: R2 object storage -// Completed tasks, attachments - -// Cold: R2 archive -// Old projects, compliance retention -``` - -## Migration from Asana - -Import your existing Asana workspace: - -```bash -npx asana-do migrate \ - --token=your_asana_pat \ - --workspace=your_workspace_gid -``` - -Imports: -- All projects and tasks -- Sections and custom fields -- Subtasks and dependencies -- Portfolios and goals -- Rules/automations -- Teams and users -- Task history and comments - -## Roadmap - -- [x] Tasks, subtasks, dependencies -- [x] Projects with sections -- [x] All view types (list, board, timeline, calendar) -- [x] Custom fields -- [x] Portfolios -- [x] Goals with cascading -- [x] Rules (automations) -- [x] REST API compatibility -- [x] AI task creation and assignment -- [ ] Forms -- [ ] Proofing -- [ ] Approvals -- [ ] Resource management -- [ ] Reporting -- [ ] Asana import (full fidelity) - -## Why Open Source? - -Work coordination is too important for rent-seeking: - -1. **Your processes** - How work flows through your org -2. **Your goals** - Strategic alignment data -3. **Your history** - Decisions and outcomes over time -4. **Your AI** - Intelligence on your work should be yours - -Asana showed the world what work management could be. **asana.do** makes it open, intelligent, and self-coordinating. - -## Contributing - -We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - -Key areas: -- Views and visualization -- Goal tracking and OKRs -- AI coordination features -- API compatibility -- Integrations - -## License - -MIT License - Use it however you want. Build your business on it. Fork it. Make it your own. - ---- - -

- asana.do is part of the dotdo platform. -
- Website | Docs | Discord -

diff --git a/rewrites/athena/.beads/.gitignore b/rewrites/athena/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/athena/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/athena/.beads/README.md b/rewrites/athena/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/athena/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/athena/.beads/config.yaml b/rewrites/athena/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/athena/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/athena/.beads/interactions.jsonl b/rewrites/athena/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/athena/.beads/issues.jsonl b/rewrites/athena/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/athena/.beads/metadata.json b/rewrites/athena/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/athena/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/athena/.gitattributes b/rewrites/athena/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/athena/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/athena/AGENTS.md b/rewrites/athena/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/athena/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/athena/README.md b/rewrites/athena/README.md deleted file mode 100644 index d3f2f3f2..00000000 --- a/rewrites/athena/README.md +++ /dev/null @@ -1,270 +0,0 @@ -# athena.do - -> Practice Management. Revenue Cycle. Patient-First. AI-Native. - -Athenahealth built a $17B empire on the backs of independent physicians. Cloud-based practice management that promised to simplify healthcare, but instead delivered $250/patient/year in administrative burden, opaque billing, and interoperability that exists only in marketing materials. - -**athena.do** is the open-source alternative. Deploy your own practice management system. AI-native revenue cycle. FHIR-first interoperability. Zero per-claim fees. - -## The Problem - -Independent practices are being squeezed out of existence: - -- **$250/patient/year** in administrative costs - Documentation, billing, prior auth, phone tag -- **6-8% of revenue** to Athenahealth - Per-claim percentages on top of monthly fees -- **30% of healthcare spending** is administrative - $1.2 trillion annually in the US -- **45% claim denial rate** for first submissions - Rework costs $25-35 per claim -- **Prior auth purgatory** - 34 hours/week per practice on prior authorizations alone -- **Interoperability theater** - "FHIR-enabled" but data still trapped in silos - -Meanwhile, hospital systems with armies of billing staff acquire struggling practices for pennies on the dollar. The independent physician - the backbone of American healthcare - is disappearing. - -## The Solution - -**athena.do** levels the playing field: - -``` -Athenahealth athena.do ------------------------------------------------------------------ -$250/patient/year admin AI handles the paperwork -6-8% per-claim fee $0 - run your own -45% first-pass denial AI-optimized clean claims -34 hrs/week prior auth Automated PA submissions -Months to implement Deploy in hours -Proprietary everything Open source, MIT licensed -``` - -## AI-Native API - -Talk to your practice. Get things done. - -```typescript -import { athena } from 'athena.do' - -// Ask like you'd ask your office manager -await athena`who needs diabetes follow-up?` -await athena`appeal the winnable denials from this month` -await athena`bill today's visits` - -// Chain like conversation -await athena`denied claims over $1000`.appeal() -await athena`Dr. Chen's schedule tomorrow`.remind() -await athena`patients with balance over $500`.collect() -``` - -That's it. The AI figures out the rest. - -### Bring What You Need - -```typescript -import { athena } from 'athena.do' // Everything -import { athena } from 'athena.do/tiny' // Edge-optimized -``` - -## Deploy - -```bash -npx create-dotdo athena -``` - -Done. Your practice runs on your infrastructure. - -## Features - -### Patient Management - -```typescript -// Natural language patient lookup -const maria = await athena`find Maria Rodriguez` -const overdue = await athena`patients overdue for wellness visit` - -// Or scan an insurance card -const patient = await athena`register patient`.scan(insuranceCardPhoto) -``` - -### Scheduling - -```typescript -// Book like you're talking to the front desk -await athena`book Maria for a wellness visit with Dr. Chen next week` -await athena`find me a 30-minute slot for a new patient this Friday` - -// Manage the day -await athena`who's on the schedule today?` -await athena`move the 2pm to 3pm` -await athena`cancel Mrs. Johnson and notify her` -``` - -### Check-In - -```typescript -// Patient texts "here" or clicks a link -await athena`check in ${patient}` -// AI verifies insurance, collects copay, updates demographics -// Staff sees: "Maria Rodriguez - Checked in, $25 copay collected" -``` - -### Documentation - -```typescript -// Ambient: AI listens, documents, codes -await athena`document this visit`.listen() - -// Or dictate after -await athena`46yo diabetic here for wellness, doing great on metformin` - -// AI generates SOAP note, suggests codes, queues charges -``` - -### Orders - -```typescript -// Natural language orders -await athena`order A1c, lipid panel, and CMP for Maria` -await athena`refer to ophthalmology for diabetic eye exam` -await athena`refill her metformin, 90 day supply` - -// AI handles the details: correct codes, fasting instructions, e-prescribe routing -``` - -## Revenue Cycle - -AI fights for every dollar. - -```typescript -// End of day: bill everything -await athena`bill today's visits` - -// AI documents -> codes -> scrubs -> submits -// You review exceptions. That's it. -``` - -### Denials - -```typescript -// The billing specialist's dream -await athena`appeal the winnable denials` - -// AI analyzes each denial, drafts appeal letters, attaches documentation -// You approve. AI submits. - -// Or get specific -await athena`why was claim 12345 denied?` -await athena`appeal Mrs. Chen's MRI denial` -``` - -### Prior Auth - -```typescript -// 34 hours/week → 34 seconds -await athena`submit prior auth for knee replacement` - -// AI pulls clinical history, failed treatments, imaging -// Submits to payer. Tracks until resolution. -// Appeals automatically if denied. -``` - -### Payments - -```typescript -// ERA comes in, AI handles it -await athena`post today's payments` - -// Denials go to appeal queue. Patient balances get statements. -// You see the summary. -``` - -## REST API - -Full Athenahealth-compatible REST API for integrations that need it. - -``` -/patients, /appointments, /claims, /documents, /encounters -``` - -But you probably won't need it. Just talk to athena. - -## AI That Works - -### Documentation That Writes Itself - -```typescript -// AI listens to the visit, writes the note -await athena`document this visit`.listen() - -// Or voice-dictate afterward -await athena`stable diabetic, refill metformin, see in 3 months` -``` - -### Patient Outreach - -```typescript -// Find gaps, write messages, send them -await athena`reach out to patients overdue for mammograms` - -// AI writes personalized messages, sends via patient preference -``` - -## Architecture - -Each practice is isolated. Your data stays yours. - -``` -your-practice.athena.do → Durable Object (SQLite + R2) - ↓ - Encrypted, HIPAA-compliant - ↓ - You control it -``` - -## Who It's For - -- **Solo practices** - Run your own PM system for free -- **Small groups** - Central billing, shared scheduling -- **Telehealth** - Built-in video, ambient documentation -- **Billing services** - RCM without the PM (connect any EHR) - -## Deploy Anywhere - -```bash -npx create-dotdo athena # Cloudflare (recommended) -docker run dotdo/athena # Self-hosted -kubectl apply -f athena-hipaa.yaml # Kubernetes -``` - -## vs Athenahealth - -| | Athenahealth | athena.do | -|-|--------------|-----------| -| Cost | $250/patient/year + 6-8% | $0 | -| Setup | 3-6 months | Hours | -| AI | Premium add-on | Built-in | -| Data | Theirs | Yours | - -## Interoperability - -FHIR R4 native. Carequality. CommonWell. HL7. e-Prescribe. - -```typescript -// Pull records from other providers -await athena`get Maria's records from the hospital` - -// Send referrals -await athena`send referral to Austin Eye Associates` -``` - -## Open Source - -MIT License. Built for independent practice. - -```bash -git clone https://github.com/dotdo/athena.do -``` - -Contributions welcome from practice administrators, billers, coders, and physicians who know what the software should actually do. - ---- - -AI assists. Humans decide. Your responsibility. MIT License. - -**[athena.do](https://athena.do)** | Part of [workers.do](https://workers.do) diff --git a/rewrites/bamboohr/.beads/.gitignore b/rewrites/bamboohr/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/bamboohr/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/bamboohr/.beads/README.md b/rewrites/bamboohr/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/bamboohr/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/bamboohr/.beads/config.yaml b/rewrites/bamboohr/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/bamboohr/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/bamboohr/.beads/interactions.jsonl b/rewrites/bamboohr/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/bamboohr/.beads/issues.jsonl b/rewrites/bamboohr/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/bamboohr/.beads/metadata.json b/rewrites/bamboohr/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/bamboohr/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/bamboohr/.gitattributes b/rewrites/bamboohr/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/bamboohr/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/bamboohr/AGENTS.md b/rewrites/bamboohr/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/bamboohr/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/bamboohr/README.md b/rewrites/bamboohr/README.md deleted file mode 100644 index 0246c143..00000000 --- a/rewrites/bamboohr/README.md +++ /dev/null @@ -1,390 +0,0 @@ -# bamboohr.do - -> Simple HR for growing teams. AI-powered. Zero per-employee fees. - -You're a startup founder. Your team just hit 50 people. HR software wants $6/employee/month - that's $3,600/year just to track PTO and store employee records. For what is essentially a spreadsheet with a nice UI. Your growing team shouldn't be penalized for growing. - -**bamboohr.do** is the open-source alternative. Run it yourself or let us host it. No per-employee fees. Ever. - -## AI-Native API - -```typescript -import { bamboohr } from 'bamboohr.do' // Full SDK -import { bamboohr } from 'bamboohr.do/tiny' // Minimal client -import { olive } from 'bamboohr.do/agents' // AI HR assistant -``` - -Natural language for HR workflows: - -```typescript -import { bamboohr } from 'bamboohr.do' - -// Talk to it like a colleague -await bamboohr`hire Sarah Chen as Engineer starting Jan 15` -await bamboohr`who's on PTO next week?` -await bamboohr`employees in Engineering with anniversaries this month` - -// Chain like sentences -await bamboohr`new hires this month` - .map(emp => bamboohr`assign ${emp} onboarding workflow`) - .map(emp => bamboohr`schedule ${emp} orientation`) - -// Onboarding that flows naturally -await bamboohr`onboard Sarah Chen` - .provision(`laptop, Slack, GitHub`) - .assign(`Engineering`) - .buddy(`Alex Kim`) -``` - -## The Problem - -BambooHR built a great product for SMBs. But then they adopted enterprise pricing. - -**$6-8 per employee per month** sounds reasonable until: - -- Your 50-person startup pays $4,800/year for employee records -- Your 200-person company pays $19,200/year for spreadsheet replacement -- The "affordable" HR system costs more than your CRM - -What you actually need: -- Store employee info (spreadsheet could do this) -- Track time off (spreadsheet could do this) -- Onboard new hires (checklist could do this) -- Run performance reviews (Google Forms could do this) - -What you're paying for: -- Per-employee pricing that scales against you -- "Integrations" to connect to your other tools -- A nice UI around basic CRUD operations - -**bamboohr.do** gives you everything BambooHR does, running on your infrastructure, with AI that actually helps. - -## One-Click Deploy - -```bash -npx create-dotdo bamboohr -``` - -Your SMB-grade HR system is live. No per-employee fees. Ever. - -```typescript -import { BambooHR } from 'bamboohr.do' - -export default BambooHR({ - name: 'acme-startup', - domain: 'hr.acme.com', -}) -``` - -## Features - -### Employee Directory - -The heart of any HR system. Everyone in one place. - -```typescript -// Find anyone -const sarah = await bamboohr`Sarah Chen` -const engineering = await bamboohr`everyone in Engineering` -const managers = await bamboohr`all managers in San Francisco` - -// AI infers what you need -await bamboohr`Sarah Chen` // returns employee -await bamboohr`Sarah's manager` // returns Alex Kim -await bamboohr`who reports to Alex?` // returns team list -await bamboohr`org chart Engineering` // returns hierarchy -``` - -### Time Off - -Request, approve, track. No spreadsheets. - -```typescript -// Natural as asking a coworker -await bamboohr`how much PTO does Sarah have?` -await bamboohr`who's out this week?` -await bamboohr`vacation calendar for March` - -// Request time off in one line -await bamboohr`Sarah taking March 17-21 off for spring break` -// Auto-routes to manager for approval - -// Manager approves naturally -await bamboohr`approve Sarah's PTO request` - -// Bulk queries just work -await bamboohr`employees with more than 80 hours unused PTO` -await bamboohr`team utilization this quarter` -``` - -### Onboarding - -New hire checklists that actually get completed. - -```typescript -// Onboard in one line -await bamboohr`onboard Sarah Chen as Engineer on Alex's team` - .provision(`laptop, Slack, GitHub, Figma`) - .buddy(`Jamie Wong`) - -// Or step by step -await bamboohr`hire Sarah Chen as Engineer starting Jan 15` -await bamboohr`assign Sarah to Engineering onboarding` -await bamboohr`order laptop for Sarah` -await bamboohr`set up Sarah's accounts` - -// Track progress naturally -await bamboohr`Sarah's onboarding status` -await bamboohr`incomplete onboarding tasks this week` -await bamboohr`new hires without assigned buddies` -``` - -### Performance Management - -Goal setting, reviews, feedback. Lightweight but effective. - -```typescript -// Set goals naturally -await bamboohr`Sarah's Q1 goal: ship auth service with OAuth2 and SAML` -await bamboohr`Sarah's Q1 goal: reduce API errors by 50%` -await bamboohr`Sarah's Q1 goal: mentor Jamie on the codebase` - -// Request feedback -await bamboohr`request feedback on Sarah from her teammates` -await bamboohr`360 review for Sarah due March 15` - -// Reviews flow naturally -await bamboohr`start Sarah's Q1 review` -await bamboohr`Sarah exceeded expectations this quarter` - -// Bulk operations -await bamboohr`employees without Q1 goals` -await bamboohr`pending reviews this week` -await bamboohr`team feedback completion rates` -``` - -### Document Storage - -Employee documents in one place. - -```typescript -// Store documents naturally -await bamboohr`upload Sarah's offer letter` -await bamboohr`add I-9 for Sarah` - -// Find documents -await bamboohr`Sarah's documents` -await bamboohr`unsigned policy acknowledgments` -await bamboohr`expiring certifications this month` - -// E-signatures -await bamboohr`send handbook acknowledgment to Sarah for signature` -await bamboohr`policy update needs signature from all employees by Feb 1` -``` - -### Reporting - -See your workforce data clearly. - -```typescript -// Ask for what you need -await bamboohr`headcount by department this year` -await bamboohr`turnover rate by department 2024` -await bamboohr`PTO utilization across Engineering` -await bamboohr`tenure distribution company-wide` - -// Complex queries, natural language -await bamboohr`employees by department and location` -await bamboohr`hiring trend last 12 months` -await bamboohr`average tenure in Sales vs Engineering` -``` - -## AI Assistant - -**Olive** is your AI HR assistant. Named after the olive branch - extending help to everyone. - -```typescript -import { olive } from 'bamboohr.do/agents' - -// Employees talk to Olive naturally -await olive`How much PTO do I have?` -await olive`I need next Friday off` -await olive`what's our remote work policy?` -await olive`who's out this week?` - -// Olive handles the complexity -// - Checks balances -// - Creates requests -// - Routes for approval -// - Answers policy questions -// - All from natural language -``` - -### AI-Powered Onboarding - -Olive guides new hires through their first days: - -```typescript -// New hires talk to Olive -await olive`I'm new here, what should I do first?` -await olive`help me with my I-9` -await olive`where do I find the employee handbook?` -await olive`who's my buddy?` - -// Olive walks them through everything -// - Forms and paperwork -// - Account setup -// - Team introductions -// - First week tasks -``` - -### AI-Assisted Performance Reviews - -```typescript -// Managers get help writing reviews -await olive`help me write Sarah's performance review` -await olive`summarize Sarah's Q1 accomplishments` -await olive`what feedback did Sarah receive this quarter?` - -// Olive drafts, you approve -// - Pulls goals and achievements -// - Incorporates peer feedback -// - Drafts review summary -// - You review and finalize -``` - -### AI for HR Operations - -```typescript -// HR gets insights -await olive`onboarding completion rates by department` -await olive`which teams have low buddy assignment rates?` -await olive`draft announcement for new PTO policy` - -// Olive surfaces actionable insights -// - Identifies bottlenecks -// - Drafts communications -// - Suggests improvements -``` - -## Self-Service - -Employees handle their own HR tasks. No tickets required. - -```typescript -// Employees update their own info naturally -await bamboohr`update my address to 456 Oak Ave, San Francisco` -await bamboohr`my emergency contact is John Chen, spouse, 415-555-1234` -await bamboohr`update my direct deposit` -await bamboohr`show my pay stubs` -await bamboohr`update my profile photo` - -// All self-service, no HR tickets needed -``` - -## Architecture - -bamboohr.do runs on Cloudflare Workers + Durable Objects. - -``` -EmployeeDO - Individual employee record - | Profile, history, documents - | -OrganizationDO - Company structure - | Departments, teams, hierarchy - | -TimeOffDO - Leave management - | Balances, requests, policies - | -OnboardingDO - New hire workflows - | Checklists, progress, tasks - | -PerformanceDO - Goals and reviews - | OKRs, feedback, ratings - | -DocumentDO - Employee documents - Storage in R2, metadata in SQLite -``` - -**Storage:** -- **SQLite (in DO)** - Active employees, current data -- **R2** - Documents, photos, attachments -- **R2 Archive** - Terminated employees, historical data - -## Integrations - -Connect naturally: - -```typescript -// Payroll -await bamboohr`connect to Gusto` -await bamboohr`sync payroll with gusto.do` - -// SSO -await bamboohr`enable Okta SSO` - -// Slack -await bamboohr`connect Slack` -// Then employees ask Olive in Slack: -// @Olive how much PTO do I have? - -// Calendar -await bamboohr`sync time-off to Google Calendar` -``` - -## Pricing - -| Plan | Price | What You Get | -|------|-------|--------------| -| **Self-Hosted** | $0 | Run it yourself, unlimited employees | -| **Managed** | $99/mo | Hosted, automatic updates, support | -| **Enterprise** | Custom | SLA, dedicated support, customization | - -**No per-employee fees. Ever.** - -A 200-person company on BambooHR: ~$19,200/year -Same company on bamboohr.do Managed: $1,188/year - -## Migration - -Import from BambooHR: - -```typescript -// One line migration -await bamboohr`import from BambooHR` - -// Imports everything: -// - All employee records -// - Time off balances and history -// - Documents -// - Org structure -``` - -## Why This Exists - -BambooHR was founded to be simpler than enterprise HR. They succeeded. Then they adopted per-employee pricing and became what they replaced - software that costs more as you grow. - -HR shouldn't be a tax on headcount. Your employee records aren't more expensive because you have more employees. The marginal cost of row 51 vs row 50 in a database is zero. - -**bamboohr.do** returns to the original promise: simple HR for growing teams. Now with AI that actually helps. - -## Contributing - -bamboohr.do is open source under MIT license. - -```bash -git clone https://github.com/dotdo/bamboohr.do -cd bamboohr.do -npm install -npm run dev -``` - -See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - -## License - -MIT - Build on it, sell it, make it yours. - ---- - -**Simple HR. Smart AI. Zero per-employee fees.** diff --git a/rewrites/cerner/.beads/.gitignore b/rewrites/cerner/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/cerner/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/cerner/.beads/README.md b/rewrites/cerner/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/cerner/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/cerner/.beads/config.yaml b/rewrites/cerner/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/cerner/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/cerner/.beads/interactions.jsonl b/rewrites/cerner/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/cerner/.beads/issues.jsonl b/rewrites/cerner/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/cerner/.beads/metadata.json b/rewrites/cerner/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/cerner/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/cerner/.gitattributes b/rewrites/cerner/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/cerner/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/cerner/AGENTS.md b/rewrites/cerner/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/cerner/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/cerner/README.md b/rewrites/cerner/README.md deleted file mode 100644 index b11226cc..00000000 --- a/rewrites/cerner/README.md +++ /dev/null @@ -1,616 +0,0 @@ -# cerner.do - -> Electronic Health Records. Edge-Native. Open by Default. AI-First. - -Oracle paid $28.3 billion for Cerner. Now Oracle Health charges hospitals millions for implementations, locks them into proprietary systems, and treats interoperability as an afterthought. FHIR adoption moves at a glacier's pace. Clinicians drown in documentation. Patients can't access their own data. - -**cerner.do** is the open-source alternative. HIPAA-compliant. FHIR R4 native. Deploys in minutes, not months. AI that reduces clinician burden instead of adding to it. - -## AI-Native API - -```typescript -import { cerner } from 'cerner.do' // Full SDK -import { cerner } from 'cerner.do/tiny' // Minimal client -import { cerner } from 'cerner.do/fhir' // FHIR-only operations -``` - -Natural language for clinical workflows: - -```typescript -import { cerner } from 'cerner.do' - -// Talk to it like a colleague -const gaps = await cerner`care gaps for Sarah Chen` -const overdue = await cerner`colonoscopy overdue` -const critical = await cerner`A1C > 9 not seen in 6 months` - -// Chain like sentences -await cerner`diabetics needing A1C` - .notify(`Your A1C test is due`) - -// Visits that document themselves -await cerner`start visit Maria Rodriguez` - .listen() // ambient recording - .document() // AI generates SOAP note - .sign() // clinician approval -``` - -## The Problem - -Oracle Health (Cerner) dominates healthcare IT alongside Epic: - -| What Oracle Charges | The Reality | -|---------------------|-------------| -| **Implementation** | $10M-100M+ (multi-year projects) | -| **Annual Maintenance** | $1-5M/year per health system | -| **Per-Bed Licensing** | $10,000-25,000 per bed | -| **Interoperability** | FHIR "supported" but proprietary first | -| **Customization** | $500/hour consultants | -| **Vendor Lock-in** | Decades of data trapped | - -### The Oracle Tax - -Since the acquisition: - -- Aggressive cloud migration push (Oracle Cloud Infrastructure) -- Reduced on-premise support -- Increased licensing costs -- Slowed innovation -- Staff departures - -Healthcare systems are hostage to a database company that sees EHR as a cloud consumption vehicle. - -### The Interoperability Illusion - -Cerner talks FHIR. But: - -- Proprietary data formats underneath -- Custom APIs for critical workflows -- "Information blocking" disguised as configuration -- FHIR endpoints often read-only or incomplete -- Patient access portals are data traps - -The 21st Century Cures Act demands interoperability. Vendors deliver the minimum required. - -### The Clinician Burnout Crisis - -- **2 hours of EHR work** for every 1 hour of patient care -- Physicians spend more time clicking than healing -- AI "assistants" add complexity, not simplicity -- Mobile experience is an afterthought -- Alert fatigue kills patients - -## The Solution - -**cerner.do** reimagines EHR for clinicians and patients: - -``` -Oracle Health (Cerner) cerner.do ------------------------------------------------------------------ -$10M-100M implementation Deploy in minutes -$1-5M/year maintenance $0 - run your own -Proprietary formats FHIR R4 native -FHIR as afterthought FHIR as foundation -Oracle Cloud lock-in Your Cloudflare account -Months to customize Code-first, instant deploy -2:1 doc-to-patient ratio AI handles documentation -Patient portal trap Patient owns their data -Vendor lock-in Open source, MIT licensed -``` - -## One-Click Deploy - -```bash -npx create-dotdo cerner -``` - -A HIPAA-compliant EHR. Running on infrastructure you control. FHIR R4 native from day one. - -```typescript -import { Cerner } from 'cerner.do' - -export default Cerner({ - name: 'valley-health', - domain: 'ehr.valley-health.org', - fhir: { - version: 'R4', - smartOnFhir: true, - }, -}) -``` - -**Note:** Healthcare is heavily regulated. This is serious software for serious use cases. See compliance section below. - -## Features - -### Patient Demographics - -```typescript -// Find anyone -const maria = await cerner`Maria Rodriguez` -const diabetics = await cerner`diabetics in Austin` -const highrisk = await cerner`uncontrolled diabetes age > 65` - -// AI infers what you need -await cerner`Maria Rodriguez` // returns patient -await cerner`labs for Maria Rodriguez` // returns lab results -await cerner`Maria Rodriguez history` // returns full chart -``` - -### Encounters - -```typescript -// Visits are one line -await cerner`wellness visit Maria Rodriguez with Dr. Smith` - .document() // AI handles the note - .discharge() // done - -// Inpatient rounds -await cerner`round on 4 West` - .each(patient => patient.update().sign()) -``` - -### Clinical Documentation - -```typescript -// AI writes the note from the visit -await cerner`start visit Maria` - .document() // SOAP note generated from ambient audio - .sign() // you review and approve - -// Or dictate directly -await cerner`note for Maria: controlled HTN, continue lisinopril, f/u 3 months` -``` - -### Orders - -```typescript -// Just say it -await cerner`lisinopril 10mg daily for Maria Rodriguez` -await cerner`A1c and lipid panel for Maria, fasting` -await cerner`refer Maria to endocrinology` - -// AI suggests, you approve -await cerner`what does Maria need?` - .order() // submits pending your approval - -// Batch orders read like a prescription pad -await cerner` - Maria Rodriguez: - - metformin 500mg bid - - A1c in 3 months - - nutrition consult -` -``` - -### Results - -```typescript -// View results naturally -await cerner`Maria labs today` -await cerner`Maria A1c trend` -await cerner`abnormal results this week` - -// Critical values alert automatically -// AI flags what needs attention -``` - -### Problem List - -```typescript -// Add problems naturally -await cerner`add diabetes to Maria's problems` -await cerner`Maria has new onset hypertension` - -// Query the problem list -await cerner`Maria active problems` -await cerner`Maria chronic conditions` -``` - -### Allergies - -```typescript -// Document allergies naturally -await cerner`Maria allergic to sulfa - rash and swelling` -await cerner`Maria penicillin allergy severe` - -// Check before prescribing (AI does this automatically) -await cerner`Maria allergies` -``` - -### Immunizations - -```typescript -// Record vaccines -await cerner`gave Maria flu shot left arm lot ABC123` - -// Check what's due -await cerner`Maria vaccines due` -await cerner`pediatric patients needing MMR` -``` - -### Scheduling - -```typescript -// Natural as talking to a scheduler -await cerner`schedule Maria diabetes follow-up next week` -await cerner`when can Maria see endocrine?` -await cerner`Dr. Smith openings in April` - -// Bulk scheduling just works -await cerner`diabetics needing follow-up` - .schedule(`diabetes management visit`) -``` - -## FHIR R4 Native - -```typescript -// Same natural syntax, FHIR underneath -await cerner`Maria problems` // returns Condition resources -await cerner`Maria medications` // returns MedicationRequest resources -await cerner`Maria everything 2024` // returns Bundle - -// Bulk export for population health -await cerner`export diabetics since January` -``` - -### FHIR R4 Resources Supported - -| Category | Resources | -|----------|-----------| -| **Foundation** | Patient, Practitioner, Organization, Location, HealthcareService | -| **Clinical** | Condition, Procedure, Observation, DiagnosticReport, CarePlan | -| **Medications** | MedicationRequest, MedicationDispense, MedicationAdministration, MedicationStatement | -| **Diagnostics** | ServiceRequest, DiagnosticReport, Observation, ImagingStudy | -| **Documents** | DocumentReference, Composition, Bundle | -| **Workflow** | Encounter, EpisodeOfCare, Appointment, Schedule, Slot | -| **Financial** | Coverage, Claim, ExplanationOfBenefit, Account | - -### SMART on FHIR - -Third-party apps just work. Patient apps, clinician apps, standalone - all supported. - -### CDS Hooks - -Clinical decision support fires automatically. Drug interactions, allergy warnings, care gaps - surfaced at the point of care. - -## AI-Native Healthcare - -### Ambient Documentation - -```typescript -// Visits document themselves -await cerner`start visit Maria` - .listen() // AI listens to the conversation - .document() // generates SOAP note - .sign() // you review and sign - -// No typing during patient care -``` - -### Clinical Decision Support - -```typescript -// AI catches things automatically -// - Drug-allergy interactions -// - Dangerous drug combinations -// - Missing preventive care -// - Diagnostic suggestions - -// Or ask directly -await cerner`differential for fatigue weight loss night sweats` -``` - -### Prior Authorization Automation - -```typescript -// Prior auth in one line -await cerner`prior auth Ozempic for Maria to UHC` - -// AI builds the case automatically -// - Extracts clinical justification from chart -// - Generates submission with supporting docs -// - Submits electronically -// - Tracks status and appeals if needed - -// Check any prior auth -await cerner`Maria Ozempic auth status` -``` - -### Patient Communication - -```typescript -// Results with context, in their language -await cerner`send Maria her A1c results in Spanish` - -// AI explains at the right level, you approve before send -// No more copy-paste lab values - -// Bulk outreach -await cerner`diabetics needing A1c` - .notify(`Time for your A1c check`) -``` - -### Population Health - -```typescript -// Query your population like a database -await cerner`A1c > 9 no visit in 6 months` -await cerner`colonoscopy overdue` -await cerner`BP > 140/90 last 3 visits` - -// Close care gaps at scale -await cerner`diabetic care gaps` - .outreach() // personalized messages - .schedule() // book appointments - .track() // HEDIS/MIPS reporting - -// One line: find, notify, schedule, measure -await cerner`close Q1 diabetes gaps` -``` - -## Architecture - -### HIPAA Architecture - -Security is not optional: - -``` -Patient Data Flow: - -Internet --> Cloudflare WAF --> Edge Auth --> Durable Object --> SQLite - | | | | - DDoS Protection Zero Trust Encryption Encrypted - (mTLS, RBAC) Key at Rest - (per-tenant) -``` - -### Durable Object per Health System - -``` -HealthSystemDO (config, users, roles, facilities) - | - +-- PatientsDO (demographics, identifiers) - | |-- SQLite: Patient records (encrypted) - | +-- R2: Documents, consent forms (encrypted) - | - +-- ClinicalDO (notes, orders, results) - | |-- SQLite: Clinical data (encrypted) - | +-- R2: Imaging, attachments (encrypted) - | - +-- FHIRDO (FHIR resources, subscriptions) - | |-- SQLite: Resource store - | +-- Search indexes - | - +-- SchedulingDO (appointments, resources) - | |-- SQLite: Schedule data - | - +-- BillingDO (claims, payments, ERA) - |-- SQLite: Financial data (encrypted) - +-- R2: EOBs, statements -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Active patients, recent visits | <10ms | -| **Warm** | R2 + SQLite Index | Historical records (2-7 years) | <100ms | -| **Cold** | R2 Archive | Compliance retention (7+ years) | <1s | - -### Encryption - -Per-patient encryption with HSM-backed keys. AES-256-GCM. 90-day rotation. 7-year immutable audit logs. All automatic. - -## vs Oracle Health (Cerner) - -| Feature | Oracle Health (Cerner) | cerner.do | -|---------|----------------------|-----------| -| **Implementation** | $10M-100M+ | Deploy in minutes | -| **Annual Cost** | $1-5M+ | ~$100/month | -| **Architecture** | Monolithic, on-prem/Oracle Cloud | Edge-native, global | -| **FHIR** | Bolted on | Native foundation | -| **AI** | PowerChart Touch (limited) | AI-first design | -| **Data Location** | Oracle's data centers | Your Cloudflare account | -| **Customization** | $500/hour consultants | Code it yourself | -| **Patient Access** | Portal lock-in | Patients own their data | -| **Interoperability** | Minimum compliance | Open by default | -| **Updates** | Quarterly releases | Continuous deployment | -| **Lock-in** | Decades of migration | MIT licensed | - -## Use Cases - -### Patient Portals - -Patients get full access to their data. Records, appointments, messages, medications, bills - all in one place. FHIR Patient Access API and SMART on FHIR apps included. - -### Clinical Integrations - -Quest, LabCorp, Surescripts e-prescribe, PACS imaging, CommonWell HIE - all pre-configured. Just connect. - -### Analytics and Research - -```typescript -// Research exports -await cerner`export diabetics for research deidentified` - -// Quality measures -await cerner`HEDIS scores this quarter` -``` - -### Multi-Facility - -One deployment serves hospitals, urgent cares, and clinics. Master patient index, enterprise scheduling, unified billing - all automatic. - -## Compliance - -### HIPAA - -HIPAA compliance built in. Role-based access, break-glass procedures, AES-256 encryption, 7-year audit logs, TLS 1.3. Cloudflare provides BAA for Enterprise customers. - -### 21st Century Cures Act - -No information blocking. Patient Access API, USCDI v3, FHIR/C-CDA/PDF exports. All automatic. - -### ONC Certification - -cerner.do provides the technical foundation. Certification (2015 Edition Cures Update) must be obtained separately. - -## Deployment Options - -### Cloudflare Workers (HIPAA BAA Available) - -```bash -npx create-dotdo cerner -# Requires Cloudflare Enterprise with signed BAA -``` - -### Private Cloud - -```bash -# Deploy to your HIPAA-compliant infrastructure -docker run -p 8787:8787 dotdo/cerner - -# Or Kubernetes with encryption -kubectl apply -f cerner-do-hipaa.yaml -``` - -### On-Premises - -For organizations requiring complete control: - -```bash -./cerner-do-install.sh --on-premises --hipaa-mode --facility-count=5 -``` - -## Why Open Source for Healthcare? - -### 1. True Interoperability - -Oracle talks interoperability while maintaining proprietary lock-in. Open source means: -- FHIR R4 native, not FHIR-bolted-on -- No information blocking incentives -- Community-driven standards adoption -- CMS and ONC compliance by design - -### 2. Innovation Velocity - -Healthcare IT moves slowly because vendors profit from the status quo. Open source enables: -- Clinicians to influence development directly -- Researchers to build on clinical data -- Startups to integrate without vendor approval -- Health systems to customize for their needs - -### 3. Cost Liberation - -$10M-100M implementations are healthcare dollars diverted from patient care. Open source means: -- Minutes to deploy, not months -- No implementation consultants -- No per-bed licensing -- No vendor lock-in - -### 4. AI Enablement - -Closed EHRs control what AI you can use. Open source means: -- Integrate any LLM -- Build custom clinical decision support -- Reduce documentation burden -- Train models on your data (with governance) - -### 5. Patient Empowerment - -Patients should own their health data. Open source enables: -- True data portability -- Patient-controlled sharing -- No portal lock-in -- Health data as a patient right - -## Risk Acknowledgment - -Healthcare software is safety-critical. cerner.do is: -- **Not a substitute for clinical judgment** - AI assists, humans decide -- **Not FDA-cleared** - Not a medical device -- **Not ONC-certified** - Certification requires separate process -- **Your responsibility** - Deployment, configuration, compliance are on you - -Use with appropriate clinical governance, testing, and oversight. - -## Roadmap - -### Core EHR -- [x] Patient Demographics (FHIR Patient) -- [x] Encounters -- [x] Clinical Documentation -- [x] Orders (Medications, Labs, Imaging, Referrals) -- [x] Results -- [x] Problem List -- [x] Medications -- [x] Allergies -- [x] Immunizations -- [x] Scheduling -- [ ] Vital Signs Flowsheets -- [ ] Care Plans -- [ ] Surgical Scheduling - -### FHIR -- [x] FHIR R4 Resources -- [x] FHIR Search -- [x] SMART on FHIR -- [x] Bulk FHIR Export -- [x] Patient Access API -- [ ] CDS Hooks -- [ ] Subscriptions -- [ ] TEFCA Integration - -### AI -- [x] Ambient Documentation -- [x] Clinical Decision Support -- [x] Prior Authorization Automation -- [x] Patient Communication -- [ ] Predictive Analytics -- [ ] Risk Stratification -- [ ] Sepsis Early Warning - -### Compliance -- [x] HIPAA Technical Safeguards -- [x] Audit Logging -- [x] Per-Patient Encryption -- [x] 21st Century Cures Act -- [ ] ONC Certification Support -- [ ] HITRUST CSF -- [ ] SOC 2 Type II - -## Contributing - -cerner.do is open source under the MIT license. - -We especially welcome contributions from: -- Clinicians and nurses -- Health informaticists -- Security and compliance experts -- Patient advocates -- Healthcare AI researchers - -```bash -git clone https://github.com/dotdo/cerner.do -cd cerner.do -pnpm install -pnpm test -``` - -## License - -MIT License - For the health of everyone. - ---- - -

- The $28B acquisition ends here. -
- FHIR-native. AI-first. Patient-owned. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/clio/.beads/.gitignore b/rewrites/clio/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/clio/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/clio/.beads/README.md b/rewrites/clio/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/clio/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/clio/.beads/config.yaml b/rewrites/clio/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/clio/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/clio/.beads/interactions.jsonl b/rewrites/clio/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/clio/.beads/issues.jsonl b/rewrites/clio/.beads/issues.jsonl deleted file mode 100644 index f3b59466..00000000 --- a/rewrites/clio/.beads/issues.jsonl +++ /dev/null @@ -1,25 +0,0 @@ -{"id":"clio-035","title":"[RED] Activities API endpoint tests","description":"Write failing tests for Activities (Time/Expense Entries) API:\n- GET /api/v4/activities (list with pagination, filtering)\n- GET /api/v4/activities/{id} (single activity with fields)\n- POST /api/v4/activities (create time/expense entry)\n- PATCH /api/v4/activities/{id} (update activity)\n- DELETE /api/v4/activities/{id} (delete activity)\n\nActivity types: TimeEntry (hourly/flat-rate), ExpenseEntry\nFields: id, etag, type, date, quantity, price, total, note, billed, matter, user, activity_description\n\nPermissions: Activity Hour Visibility setting","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:52.517038-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:52.517038-06:00","labels":["activities","billing","phase-3","tdd-red"],"dependencies":[{"issue_id":"clio-035","depends_on_id":"clio-abo","type":"blocks","created_at":"2026-01-07T13:58:02.786949-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-035","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.023039-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-0zg","title":"[GREEN] Users API implementation","description":"Implement Users API to pass tests:\n- User model with SQLite schema\n- Read-only user listing\n- who_am_i endpoint for current user\n- Role-based permissions\n- Standard roles (Administrator, Billing)\n- Custom role support\n- Subscription type tracking\n- Default calendar association\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:44.156919-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:44.156919-06:00","labels":["phase-2","tdd-green","users"],"dependencies":[{"issue_id":"clio-0zg","depends_on_id":"clio-zf4","type":"blocks","created_at":"2026-01-07T13:59:50.124786-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-0zg","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.832196-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-17q","title":"[GREEN] Activities API implementation","description":"Implement Activities API to pass tests:\n- Activity model (TimeEntry, ExpenseEntry) with SQLite schema\n- CRUD operations with Hono routes\n- Time calculation (hourly rate * quantity)\n- Flat-rate time entries\n- Expense entries with receipts\n- Activity Hour Visibility permissions\n- Association with matters and users\n- Billed/unbilled status tracking\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:54.242428-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:54.242428-06:00","labels":["activities","billing","phase-3","tdd-green"],"dependencies":[{"issue_id":"clio-17q","depends_on_id":"clio-035","type":"blocks","created_at":"2026-01-07T13:58:02.723839-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-17q","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.084536-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-1ga","title":"[GREEN] Notes API implementation","description":"Implement Notes API to pass tests:\n- Note model with SQLite schema\n- Matter notes vs Contact notes\n- Required type parameter validation\n- Rich text support in detail field\n- Date tracking\n- User attribution\n- Association with matters and contacts\n- Timeline integration\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:29.277754-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:29.277754-06:00","labels":["notes","phase-4","tdd-green"],"dependencies":[{"issue_id":"clio-1ga","depends_on_id":"clio-fgg","type":"blocks","created_at":"2026-01-07T13:59:33.872774-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-1ga","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.709169-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-1wg","title":"[GREEN] Documents API implementation","description":"Implement Documents API to pass tests:\n- Document model with SQLite metadata\n- R2 storage for document content\n- Upload with multipart support\n- Download with streaming\n- Folder hierarchy support\n- Document versioning\n- Archive/bulk download\n- Matter association\n- Custom actions integration\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:36.346574-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:36.346574-06:00","labels":["documents","phase-4","tdd-green"],"dependencies":[{"issue_id":"clio-1wg","depends_on_id":"clio-jx6","type":"blocks","created_at":"2026-01-07T13:58:41.580882-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-1wg","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.336396-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-2os","title":"[RED] Matters API endpoint tests","description":"Write failing tests for Matters (cases) API:\n- GET /api/v4/matters (list with pagination, filtering)\n- GET /api/v4/matters/{id} (single matter with fields)\n- POST /api/v4/matters (create matter)\n- PATCH /api/v4/matters/{id} (update matter)\n- DELETE /api/v4/matters/{id} (delete matter)\n\nFields: id, etag, display_number, description, status, client, practice_area, billable, open_date, close_date, pending_date\n\nNested resources: contacts, activities, documents, calendar_entries, tasks","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:18.661322-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:18.661322-06:00","labels":["matters","phase-2","tdd-red"],"dependencies":[{"issue_id":"clio-2os","depends_on_id":"clio-9ec","type":"blocks","created_at":"2026-01-07T13:57:28.163739-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-2os","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:50.769977-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-44v","title":"[REFACTOR] Extract permission system","description":"Extract permission patterns:\n- Role-based access control (RBAC)\n- Resource-level permissions\n- Field-level permissions (Activity Hour Visibility)\n- Contacts Visibility permission\n- Bills permission\n- Custom role support\n- Permission middleware\n\nGoal: Centralized permission system matching Clio's model","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:00:09.020797-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:00:09.020797-06:00","labels":["permissions","phase-5","refactor"],"dependencies":[{"issue_id":"clio-44v","depends_on_id":"clio-fnk","type":"blocks","created_at":"2026-01-07T14:00:25.068903-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-44v","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:52.020259-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-7fw","title":"[RED] Tasks API endpoint tests","description":"Write failing tests for Tasks API:\n- GET /api/v4/tasks (list with pagination, filtering)\n- GET /api/v4/tasks/{id} (single task)\n- POST /api/v4/tasks (create task)\n- PATCH /api/v4/tasks/{id} (update task)\n- DELETE /api/v4/tasks/{id} (delete task)\n\nFields: id, etag, name, description, due_at, priority, status, assignee, matter, completed\n\nTask statuses: pending, in_progress, completed\nPriority levels support","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:09.887259-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:09.887259-06:00","labels":["phase-4","tasks","tdd-red"],"dependencies":[{"issue_id":"clio-7fw","depends_on_id":"clio-abo","type":"blocks","created_at":"2026-01-07T13:59:19.680206-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-7fw","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.522439-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-7i2","title":"[GREEN] Tasks API implementation","description":"Implement Tasks API to pass tests:\n- Task model with SQLite schema\n- CRUD operations with Hono routes\n- Assignment to users\n- Matter association\n- Due date tracking\n- Priority levels\n- Status transitions\n- Task completion tracking\n- Reminders/notifications\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:13.518362-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:13.518362-06:00","labels":["phase-4","tasks","tdd-green"],"dependencies":[{"issue_id":"clio-7i2","depends_on_id":"clio-7fw","type":"blocks","created_at":"2026-01-07T13:59:19.621729-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-7i2","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.587801-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-87a","title":"[RED] Calendar Events API endpoint tests","description":"Write failing tests for Calendar Events API:\n- GET /api/v4/calendar_entries (list with date range, filtering)\n- GET /api/v4/calendar_entries/{id} (single event)\n- POST /api/v4/calendar_entries (create event)\n- PATCH /api/v4/calendar_entries/{id} (update event)\n- DELETE /api/v4/calendar_entries/{id} (delete event)\n\nFields: id, etag, summary, description, start_at, end_at, all_day, location, matter, attendees, recurrence\n\nPrivacy: Private events return null fields for unauthorized users","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:53.019084-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:53.019084-06:00","labels":["calendar","phase-4","tdd-red"],"dependencies":[{"issue_id":"clio-87a","depends_on_id":"clio-abo","type":"blocks","created_at":"2026-01-07T13:59:01.436766-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-87a","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.398351-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-9ec","title":"[GREEN] OAuth 2.0 authentication implementation","description":"Implement OAuth 2.0 authentication to pass tests:\n- ClioAuth class with authorization URL generation\n- Token exchange endpoint handler\n- Token refresh mechanism\n- Token storage and validation\n- Multi-region endpoint routing\n- Rate limit retry logic\n\nReference: https://docs.developers.clio.com/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:56:57.582209-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:56:57.582209-06:00","labels":["auth","phase-1","tdd-green"],"dependencies":[{"issue_id":"clio-9ec","depends_on_id":"clio-dh4","type":"blocks","created_at":"2026-01-07T13:57:09.302287-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-9ec","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:50.702646-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-a3p","title":"[GREEN] Contacts API implementation","description":"Implement Contacts API to pass tests:\n- Contact model (Person, Company types) with SQLite schema\n- CRUD operations with Hono routes\n- Contact visibility permissions\n- Phone numbers and addresses as nested resources\n- Custom fields support\n- Company-person relationships\n- ETag support for optimistic concurrency\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:38.205521-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:38.205521-06:00","labels":["contacts","phase-2","tdd-green"],"dependencies":[{"issue_id":"clio-a3p","depends_on_id":"clio-z0l","type":"blocks","created_at":"2026-01-07T13:57:42.810618-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-a3p","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:50.960945-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-abo","title":"[GREEN] Matters API implementation","description":"Implement Matters API to pass tests:\n- Matter model with SQLite schema\n- CRUD operations with Hono routes\n- Field selection via query params\n- Pagination with cursor support\n- Filtering by status, client, practice_area\n- Nested resource expansion\n- ETag support for optimistic concurrency\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:20.16692-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:20.16692-06:00","labels":["matters","phase-2","tdd-green"],"dependencies":[{"issue_id":"clio-abo","depends_on_id":"clio-2os","type":"blocks","created_at":"2026-01-07T13:57:28.105074-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-abo","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:50.833642-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-c6t","title":"[RED] Bills API endpoint tests","description":"Write failing tests for Bills (Invoices) API:\n- GET /api/v4/bills (list with pagination, filtering)\n- GET /api/v4/bills/{id} (single bill with fields)\n- POST /api/v4/bills (generate bill)\n- PATCH /api/v4/bills/{id} (update bill)\n- DELETE /api/v4/bills/{id} (delete bill)\n\nFields: id, etag, number, issued_at, due_at, balance, total, state, matter, client, line_items\n\nBill states: draft, awaiting_approval, approved, awaiting_payment, paid\nRole permissions: Billing role required","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:14.86011-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:14.86011-06:00","labels":["billing","bills","phase-3","tdd-red"],"dependencies":[{"issue_id":"clio-c6t","depends_on_id":"clio-17q","type":"blocks","created_at":"2026-01-07T13:58:23.752148-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-c6t","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.150456-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-dh4","title":"[RED] OAuth 2.0 authentication flow tests","description":"Write failing tests for OAuth 2.0 authentication flow:\n- Authorization URL generation\n- Token exchange (code -\u003e access_token)\n- Token refresh\n- Token validation\n- Multi-region support (US, CA, EU, AU)\n- Rate limit handling (429 responses)\n\nTest against: app.clio.com, ca.app.clio.com, eu.app.clio.com, au.app.clio.com","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:56:56.458127-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:56:56.458127-06:00","labels":["auth","phase-1","tdd-red"],"dependencies":[{"issue_id":"clio-dh4","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:42.559519-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-dtt","title":"Clio API v4 Compatibility","description":"Implement Clio API v4 compatibility layer on Cloudflare Durable Objects. Clio is a leading legal practice management platform ($3B+) with REST API for matters, contacts, activities, billing, documents, and calendar. Target: Full API compatibility with OAuth 2.0 auth.","status":"open","priority":2,"issue_type":"epic","created_at":"2026-01-07T13:56:46.292963-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:56:46.292963-06:00"} -{"id":"clio-e2d","title":"[REFACTOR] Extract field selection and nested resource expansion","description":"Extract field handling patterns:\n- Field selection parser (?fields=id,name,matter{id,description})\n- Nested resource expansion\n- Default fields per resource type\n- Field validation against schema\n- Curly bracket syntax for nested fields\n\nExample: /api/v4/activities?fields=id,etag,type,matter{id,description}","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:00:06.961581-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:00:06.961581-06:00","labels":["architecture","phase-5","refactor"],"dependencies":[{"issue_id":"clio-e2d","depends_on_id":"clio-fnk","type":"blocks","created_at":"2026-01-07T14:00:22.635073-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-e2d","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.95758-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-fgg","title":"[RED] Notes API endpoint tests","description":"Write failing tests for Notes API:\n- GET /api/v4/notes?type=Matter (list matter notes)\n- GET /api/v4/notes?type=Contact (list contact notes)\n- GET /api/v4/notes/{id} (single note)\n- POST /api/v4/notes (create note)\n- PATCH /api/v4/notes/{id} (update note)\n- DELETE /api/v4/notes/{id} (delete note)\n\nFields: id, etag, subject, detail, date, type, matter, contact, user\nRequired 'type' parameter for list queries","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:27.748876-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:27.748876-06:00","labels":["notes","phase-4","tdd-red"],"dependencies":[{"issue_id":"clio-fgg","depends_on_id":"clio-a3p","type":"blocks","created_at":"2026-01-07T13:59:33.9305-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-fgg","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.649071-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-fnk","title":"[REFACTOR] Extract common CRUD patterns to base class","description":"Extract common patterns after GREEN implementations:\n- BaseResource class with CRUD operations\n- Field selection middleware\n- Pagination middleware (cursor-based)\n- ETag handling middleware\n- Error response formatting (4xx, 5xx)\n- Rate limiting middleware\n- JSON response serialization\n\nGoal: DRY code, consistent API behavior across all endpoints","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:00:05.890455-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:00:05.890455-06:00","labels":["architecture","phase-5","refactor"],"dependencies":[{"issue_id":"clio-fnk","depends_on_id":"clio-abo","type":"blocks","created_at":"2026-01-07T14:00:19.736271-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-fnk","depends_on_id":"clio-a3p","type":"blocks","created_at":"2026-01-07T14:00:19.795504-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-fnk","depends_on_id":"clio-17q","type":"blocks","created_at":"2026-01-07T14:00:19.854454-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-fnk","depends_on_id":"clio-qva","type":"blocks","created_at":"2026-01-07T14:00:19.912679-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-fnk","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.892642-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-jx6","title":"[RED] Documents API endpoint tests","description":"Write failing tests for Documents API:\n- GET /api/v4/documents (list with pagination, filtering)\n- GET /api/v4/documents/{id} (single document metadata)\n- GET /api/v4/documents/{id}/download (download content)\n- POST /api/v4/documents (upload document)\n- PATCH /api/v4/documents/{id} (update metadata)\n- DELETE /api/v4/documents/{id} (delete document)\n- POST /api/v4/document_archives (bulk download)\n\nFields: id, etag, name, content_type, size, created_at, updated_at, matter, parent_folder\nCustom actions support for documents","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:32.72108-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:32.72108-06:00","labels":["documents","phase-4","tdd-red"],"dependencies":[{"issue_id":"clio-jx6","depends_on_id":"clio-abo","type":"blocks","created_at":"2026-01-07T13:58:41.639507-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-jx6","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.275449-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-kt0","title":"[REFACTOR] Extract webhook system for custom actions","description":"Extract webhook patterns:\n- Webhook registration API\n- Event types (create, update, delete)\n- Field-level change detection\n- Custom actions for Activities, Contacts, Documents, Matters\n- Payload formatting\n- Retry logic with exponential backoff\n- Webhook signature verification\n\nReference: Clio Custom Actions documentation","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:00:10.334226-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:00:10.334226-06:00","labels":["phase-5","refactor","webhooks"],"dependencies":[{"issue_id":"clio-kt0","depends_on_id":"clio-fnk","type":"blocks","created_at":"2026-01-07T14:00:26.732518-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-kt0","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:52.080309-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-llt","title":"[GREEN] Calendar Events API implementation","description":"Implement Calendar Events API to pass tests:\n- CalendarEntry model with SQLite schema\n- Event creation with attendees\n- Recurrence rules (RRULE support)\n- All-day events\n- Private event permissions\n- Matter association\n- Time zone handling\n- Conflict detection\n- iCal export capability\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:54.234476-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:54.234476-06:00","labels":["calendar","phase-4","tdd-green"],"dependencies":[{"issue_id":"clio-llt","depends_on_id":"clio-87a","type":"blocks","created_at":"2026-01-07T13:59:01.375028-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-llt","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.461553-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-qva","title":"[GREEN] Bills API implementation","description":"Implement Bills API to pass tests:\n- Bill model with SQLite schema\n- Bill generation from unbilled activities\n- Line items (time entries, expenses, flat fees)\n- State machine (draft -\u003e approved -\u003e paid)\n- PDF generation capability\n- Due date calculation\n- Balance tracking\n- Role-based access (Billing permission)\n\nReference: https://docs.developers.clio.com/api-reference/","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:58:16.80969-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:58:16.80969-06:00","labels":["billing","bills","phase-3","tdd-green"],"dependencies":[{"issue_id":"clio-qva","depends_on_id":"clio-c6t","type":"blocks","created_at":"2026-01-07T13:58:23.689256-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-qva","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.213645-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-z0l","title":"[RED] Contacts API endpoint tests","description":"Write failing tests for Contacts (clients/companies) API:\n- GET /api/v4/contacts (list with pagination, filtering)\n- GET /api/v4/contacts/{id} (single contact with fields)\n- POST /api/v4/contacts (create Person or Company)\n- PATCH /api/v4/contacts/{id} (update contact)\n- DELETE /api/v4/contacts/{id} (delete contact)\n\nContact types: Person, Company\nFields: id, etag, type, name, first_name, last_name, email, phone_numbers, addresses, company, custom_fields\n\nVisibility permissions: honor Contacts Visibility user permission","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:57:36.313271-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:57:36.313271-06:00","labels":["contacts","phase-2","tdd-red"],"dependencies":[{"issue_id":"clio-z0l","depends_on_id":"clio-9ec","type":"blocks","created_at":"2026-01-07T13:57:42.870593-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-z0l","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:50.895783-06:00","created_by":"nathanclevenger"}]} -{"id":"clio-zf4","title":"[RED] Users API endpoint tests","description":"Write failing tests for Users API:\n- GET /api/v4/users (list firm users)\n- GET /api/v4/users/{id} (single user)\n- GET /api/v4/users/who_am_i (current user)\n\nFields: id, etag, name, first_name, last_name, email, role, enabled, subscription_type, default_calendar_id\n\nRoles: Administrator, Billing, custom roles\nPermissions enforcement based on role","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T13:59:42.488319-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T13:59:42.488319-06:00","labels":["phase-2","tdd-red","users"],"dependencies":[{"issue_id":"clio-zf4","depends_on_id":"clio-9ec","type":"blocks","created_at":"2026-01-07T13:59:50.183624-06:00","created_by":"nathanclevenger"},{"issue_id":"clio-zf4","depends_on_id":"clio-dtt","type":"parent-child","created_at":"2026-01-07T14:00:51.77071-06:00","created_by":"nathanclevenger"}]} diff --git a/rewrites/clio/.beads/metadata.json b/rewrites/clio/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/clio/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/clio/.gitattributes b/rewrites/clio/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/clio/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/clio/AGENTS.md b/rewrites/clio/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/clio/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/clio/README.md b/rewrites/clio/README.md deleted file mode 100644 index dde130b1..00000000 --- a/rewrites/clio/README.md +++ /dev/null @@ -1,359 +0,0 @@ -# clio.do - -> Legal Practice Management. AI-powered. Open source. - -Clio built a $3B+ company charging $39-149 per user per month for legal practice management. 150,000+ law firms pay for the privilege of tracking their own time, managing their own documents, and billing their own clients. - -**clio.do** is the open-source alternative. Your own legal practice management system. AI that captures billable time automatically. Deploy in one click. - -## The Problem - -Legal practice management software has a dirty secret: it creates more administrative burden than it solves. - -- **$39-149/user/month** - A 10-attorney firm pays $4,680-$17,880/year just for software -- **Billable hour leakage** - Attorneys forget to track 10-30% of billable time -- **Document chaos** - Files scattered across email, drives, portals, desks -- **Time tracking friction** - Context-switching to log time interrupts legal work -- **Per-seat scaling** - Growing your firm means growing your software costs linearly -- **Data hostage** - Client matters, billing history, documents - all locked in their system - -Meanwhile, solo practitioners and small firms are priced out of professional tools. Legal aid organizations run on spreadsheets. Access to justice suffers because practice management is a profit center. - -## The Solution - -**clio.do** reimagines legal practice management for the AI era: - -``` -Clio clio.do ------------------------------------------------------------------ -$39-149/user/month $0 - run your own -Per-seat licensing Unlimited attorneys -Manual time tracking AI captures time automatically -Document silos Unified matter workspace -Their servers, their rules Your infrastructure, your data -Billable hour leakage AI reconstructs missed time -API access costs extra Full API included -``` - -## Talk to Your Practice - -```typescript -import clio from 'clio.do' - -// That's it. No config. No auth setup. -// Context flows naturally - like talking to your paralegal. - -await clio`bill Smith for January` -// → Finds matters, pulls unbilled time, reviews for block billing, -// generates invoice, sends to client. One sentence. - -await clio`what's overdue?` -// → Shows past-due invoices with aging. - -await clio`reconstruct my time today` -// → AI reviews emails, docs, calls, calendar. -// Creates draft entries. You approve. - -// Chain like conversation: -await clio`open matters` - .bill() - .send() - -await clio`Smith deposition` - .prep() // Gathers exhibits, drafts outline - .schedule() // Finds available times, books court reporter - -// Research happens naturally: -await clio`research negligence for Johnson v. Acme` -// → Finds the matter, researches the issue, attaches memo. - -// Intake is one line: -await clio`intake Sarah Connor, dog bite case` -// → Conflicts check, engagement letter, fee agreement, e-sign. Done. -``` - -**The test:** Can you dictate this walking to court? If not, we simplified it. - -### When You Need More - -```typescript -// Default: Just works -import clio from 'clio.do' - -// Need the AI team explicitly? -import { clio, tom, priya } from 'clio.do' -await tom`review ${brief}` - -// Cloudflare Workers service binding? -import clio from 'clio.do/rpc' -``` - -## Before & After - -### Matters - -```typescript -// Old way (other software) -await clio.matters.create({ - client: 'CL-001', - name: 'Smith v. Johnson - Personal Injury', - practiceArea: 'Personal Injury', - responsibleAttorney: 'atty-001', - billingMethod: 'contingency', - // ... 15 more fields -}) - -// clio.do -await clio`new PI matter for John Smith, rear-end collision` -// → Creates matter, sets practice area, assigns you, detects billing type. - -await clio`add opposing counsel Big Defense LLP` -// → Context: knows you mean the current matter. -``` - -### Time - -```typescript -// Old way -await clio.time.create({ - matter: 'MAT-001', - user: 'atty-001', - duration: 1.5, - date: new Date(), - description: 'Draft motion for summary judgment; review case law', - activityType: 'Drafting', - billable: true, - rate: 350, -}) - -// clio.do -await clio`1.5 hours drafting MSJ for Smith` -// → Finds matter, knows your rate, marks billable. Done. - -// Or just let AI handle it: -await clio`what did I miss today?` -// → Reviews your emails, docs, calendar. Suggests entries. -// → You approve with one tap. -``` - -### Billing - -```typescript -// Old way -const invoice = await clio.invoices.create({ - matter: 'MAT-001', client: 'CL-001', billTo: 'contact-001', - dateFrom: new Date('2025-01-01'), dateTo: new Date('2025-01-31'), - includeUnbilled: true -}) -await invoice.finalize() -await invoice.send({ method: 'email', includeStatement: true }) - -// clio.do -await clio`bill Smith for January` -// → Everything above, plus block billing review. One sentence. -``` - -### Trust - -```typescript -// Old way: 30 lines of trust deposit, transfer, reconciliation code - -// clio.do -await clio`deposit $10k retainer from Smith` -// → Creates trust deposit, assigns to matter, compliant ledger entry. - -await clio`transfer earned fees for Smith invoice 1234` -// → Validates against invoice, transfers to operating, updates ledger. - -await clio`reconcile trust` -// → Three-way reconciliation. Alerts on discrepancies. -``` - -### Documents - -```typescript -// Old way: 20 lines with buffers, categories, metadata - -// clio.do -await clio`file the MSJ draft` -// → Knows the current matter, categorizes as pleading, versions it. - -await clio`engagement letter for new client Sarah Connor` -// → Generates from template, fills variables, sends for e-sign. -``` - -### Calendar - -```typescript -// Old way -await clio.calendar.create({ - matter: 'MAT-001', - title: 'Deposition - Jane Doe', - start: new Date('2025-02-15T09:00:00'), - end: new Date('2025-02-15T12:00:00'), - location: 'Court Reporter Services, 456 Legal Ave', - attendees: ['atty-001', 'paralegal-001'], - reminders: [{ type: 'email', before: '1 day' }] -}) - -// clio.do -await clio`schedule Jane Doe depo next Tuesday morning` -// → Finds available time, books court reporter, adds to matter, sets reminders. - -await clio`opposition due March 1` -// → Adds deadline, auto-calculates warning dates per California rules. -``` - -## Workflows That Flow - -Complex pipelines become simple chains. Say what you want to happen. - -```typescript -// Discovery → Review → Meet-and-confer → File -// OLD: 50 lines of .map() chains with explicit agent calls - -// NEW: -await clio`review discovery responses for Smith` - .meetAndConfer() - .send() -// → AI identifies evasive answers, drafts letter, reviews tone, files, sends. - -// Research → Verify → Attach -// OLD: 20 lines with tom, quinn, explicit matter IDs - -// NEW: -await clio`research rear-end presumption for Smith` -// → Researches, verifies citations, attaches memo to matter. One line. - -// Depo prep -await clio`prep for Jane Doe deposition` -// → Gathers docs, identifies exhibits, creates outline, builds binder. -``` - -### Time Capture - -```typescript -// End of day - one sentence -await clio`capture my time today` -// → Reviews emails, docs, calls, calendar. -// → Creates draft entries. You approve what's right. - -// Weekly audit -await clio`audit this week's time` -// → Flags block billing, vague descriptions, duplicates. -// → You fix what needs fixing. -``` - -### Billing Pipeline - -```typescript -// The entire billing cycle -await clio`bill all open matters` - .review() // Block billing, write-downs, ethical check - .send() // Emails with narrative - .collect() // Tracks until paid - -// Or matter by matter -await clio`bill Smith`.send() -``` - -### Trust Accounting - -```typescript -// Transfer earned fees -await clio`transfer Smith earned fees` -// → Checks against invoice, moves funds, updates ledger, sends statement. - -// Reconciliation -await clio`reconcile trust` -// → Three-way reconciliation. Alerts on issues. -``` - -### Client Communication - -```typescript -// Status update -await clio`update Smith on case progress` -// → Drafts email with recent work, next steps. Sends after you approve. - -// Full intake -await clio`intake new client, dog bite` -// → Conflicts, engagement letter, questionnaire, fee agreement, e-sign. -// All triggered by seven words. -``` - -## Drop-In Migration - -Already on Clio? One command to switch: - -```bash -npx clio-do migrate -# → Exports from Clio, imports to your instance. -# → Matters, contacts, time, invoices, documents, trust. Everything. -``` - -Existing integrations keep working. Same API, better address: - -```typescript -// Change one line -baseUrl: 'https://your-firm.clio.do' -``` - -## Under the Hood - -Each firm gets a Durable Object - a dedicated SQLite database at the edge: - -- **Your matters stay together** - Transactional consistency for trust accounting -- **Your data stays separate** - Complete isolation from other firms -- **Your practice stays fast** - Edge locations near every courthouse -- **Documents on R2** - Unlimited storage, 7-year retention -- **Search via Vectorize** - Find anything instantly - -Security is automatic: encryption at rest, audit logs, MFA. Configure SSO if you want it. - -## For Every Practice - -**Solo:** Stop paying $50/month. Voice-first time tracking. ~$5/month on Cloudflare. - -**Small Firm:** Unlimited attorneys. No per-seat scaling. Full trust accounting. - -**Legal Aid:** Professional tools at zero cost. Grant tracking. Pro bono metrics. - -**Virtual:** Multi-jurisdiction. Work from anywhere. E-signature built in. - -## Get Started - -```bash -npx create-dotdo clio -``` - -That's it. AI features are on by default. Add your firm name when prompted. - -First time you talk to it: - -```typescript -await clio`I'm Jane, $350/hour, California bar` -// → Creates your profile. Ready to practice. -``` - -## Why - -Your billable hour shouldn't be spent fighting with software. - -Every solo deserves Big Law tools. Every legal aid org deserves professional practice management. Every small firm should compete on service, not software budgets. - -**Your data. Your workflow. Your economics. Your AI.** - -## Open Source - -MIT license. Your practice, your data, your terms. - -```bash -git clone https://github.com/dotdo/clio.do && cd clio.do && npm install && npm test -``` - ---- - -

- clio.do | Docs | Discord -

diff --git a/rewrites/cloudera/.beads/.gitignore b/rewrites/cloudera/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/cloudera/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/cloudera/.beads/README.md b/rewrites/cloudera/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/cloudera/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/cloudera/.beads/config.yaml b/rewrites/cloudera/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/cloudera/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/cloudera/.beads/interactions.jsonl b/rewrites/cloudera/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/cloudera/.beads/issues.jsonl b/rewrites/cloudera/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/cloudera/.beads/metadata.json b/rewrites/cloudera/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/cloudera/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/cloudera/.gitattributes b/rewrites/cloudera/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/cloudera/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/cloudera/AGENTS.md b/rewrites/cloudera/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/cloudera/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/composio/.beads/.gitignore b/rewrites/composio/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/composio/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/composio/.beads/README.md b/rewrites/composio/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/composio/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/composio/.beads/config.yaml b/rewrites/composio/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/composio/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/composio/.beads/interactions.jsonl b/rewrites/composio/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/composio/.beads/issues.jsonl b/rewrites/composio/.beads/issues.jsonl deleted file mode 100644 index 1f1008ec..00000000 --- a/rewrites/composio/.beads/issues.jsonl +++ /dev/null @@ -1,14 +0,0 @@ -{"id":"composio-1gq","title":"[RED] Test TriggerDO webhook subscriptions","description":"Write failing tests for subscribing to app events and receiving webhooks.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:41:07.869736-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:07.869736-06:00"} -{"id":"composio-5hh","title":"[GREEN] Implement agent framework adapters","description":"Implement adapters for LangChain, CrewAI, Autogen, and LlamaIndex with common interface.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:41:02.10782-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:02.10782-06:00","dependencies":[{"issue_id":"composio-5hh","depends_on_id":"composio-rfe","type":"blocks","created_at":"2026-01-07T14:41:20.898854-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-5ix","title":"[RED] Test MCP server tool exposure","description":"Write failing tests for creating MCP server that exposes Composio tools with proper JSON schemas.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:41:01.839622-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:01.839622-06:00"} -{"id":"composio-66g","title":"[GREEN] Implement ExecutionDO with retry and rate limiting","description":"Implement ExecutionDO for sandboxed action execution with automatic retries, timeouts, and per-entity rate limiting.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:54.403667-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:54.403667-06:00","dependencies":[{"issue_id":"composio-66g","depends_on_id":"composio-uk1","type":"blocks","created_at":"2026-01-07T14:41:20.767932-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-6rl","title":"[RED] Test Composio client initialization and config","description":"Write failing tests for Composio client construction with apiKey, baseUrl, and rateLimit config options.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:37.619847-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:37.619847-06:00"} -{"id":"composio-co0","title":"[GREEN] Implement ToolsDO with schema caching","description":"Implement ToolsDO for tool registry with KV-backed schema caching for fast lookups.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:54.2301-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:54.2301-06:00","dependencies":[{"issue_id":"composio-co0","depends_on_id":"composio-hcj","type":"blocks","created_at":"2026-01-07T14:41:20.705348-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-cov","title":"[GREEN] Implement ConnectionDO with secure token storage","description":"Implement ConnectionDO Durable Object for per-entity credential isolation with SQLite storage and automatic token refresh.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:47.338513-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:47.338513-06:00","dependencies":[{"issue_id":"composio-cov","depends_on_id":"composio-i3y","type":"blocks","created_at":"2026-01-07T14:41:20.641917-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-dkx","title":"[GREEN] Implement Composio client with config validation","description":"Implement the Composio class to pass the initialization tests with proper config handling.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:47.160001-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:47.160001-06:00","dependencies":[{"issue_id":"composio-dkx","depends_on_id":"composio-6rl","type":"blocks","created_at":"2026-01-07T14:41:20.576318-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-hcj","title":"[RED] Test ToolsDO app and action registry","description":"Write failing tests for listing apps, getting actions, and searching the tool registry.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:54.142207-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:54.142207-06:00"} -{"id":"composio-i3y","title":"[RED] Test ConnectionDO OAuth flow","description":"Write failing tests for OAuth connect flow including redirect URL generation and token exchange.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:47.250377-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:47.250377-06:00"} -{"id":"composio-rfe","title":"[RED] Test LangChain adapter","description":"Write failing tests for LangChain tool adapter that converts Composio tools to LangChain format.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:41:02.018769-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:02.018769-06:00"} -{"id":"composio-tvp","title":"[GREEN] Implement MCP server with dynamic tool generation","description":"Implement MCP server that dynamically generates tool definitions from the registry with proper input/output schemas.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:41:01.928585-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:01.928585-06:00","dependencies":[{"issue_id":"composio-tvp","depends_on_id":"composio-5ix","type":"blocks","created_at":"2026-01-07T14:41:20.83129-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-tzi","title":"[GREEN] Implement TriggerDO with Queue-based delivery","description":"Implement TriggerDO for webhook subscriptions with Cloudflare Queues for reliable delivery.","status":"open","priority":2,"issue_type":"task","created_at":"2026-01-07T14:41:07.959458-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:41:07.959458-06:00","dependencies":[{"issue_id":"composio-tzi","depends_on_id":"composio-1gq","type":"blocks","created_at":"2026-01-07T14:41:20.966165-06:00","created_by":"nathanclevenger"}]} -{"id":"composio-uk1","title":"[RED] Test ExecutionDO action sandbox","description":"Write failing tests for executing actions in isolated sandbox with proper credential injection.","status":"open","priority":1,"issue_type":"task","created_at":"2026-01-07T14:40:54.319581-06:00","created_by":"nathanclevenger","updated_at":"2026-01-07T14:40:54.319581-06:00"} diff --git a/rewrites/composio/.beads/metadata.json b/rewrites/composio/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/composio/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/composio/.gitattributes b/rewrites/composio/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/composio/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/composio/AGENTS.md b/rewrites/composio/AGENTS.md deleted file mode 100644 index c726de3d..00000000 --- a/rewrites/composio/AGENTS.md +++ /dev/null @@ -1,212 +0,0 @@ -# Composio Rewrite - AI Assistant Guidance - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Project Overview - -**Goal:** Build a Cloudflare-native Composio alternative - AI agent tool integration platform with managed auth and MCP-native tool definitions. - -**Package:** `@dotdo/composio` with domains `tools.do` / `composio.do` - -**Core Primitives:** -- Durable Objects - Per-entity credential isolation -- Workers KV - Tool schema caching -- R2 - Large response storage -- Cloudflare Queues - Webhook delivery - -## Reference Documents - -1. **../CLAUDE.md** - General rewrites guidance and TDD patterns -2. **../inngest/README.md** - Reference rewrite pattern for DO architecture -3. **../fsx/README.md** - Reference for MCP integration - -## Key Composio Concepts - -### Connections -```typescript -// OAuth flow -const { redirectUrl } = await composio.connect({ - userId: 'user-123', - app: 'github', - redirectUrl: 'https://myapp.com/callback' -}) - -// API key auth -await composio.connect({ - userId: 'user-123', - app: 'openai', - credentials: { apiKey: 'sk-...' } -}) -``` - -### Tool Execution -```typescript -const result = await composio.execute({ - action: 'github_create_issue', - params: { repo: 'owner/repo', title: 'Bug fix' }, - entityId: 'user-123' -}) -``` - -### Agent Framework Integration -```typescript -// LangChain -import { composioTools } from '@dotdo/composio/langchain' -const tools = await composioTools.getTools({ apps: ['github'], entityId: 'user-123' }) - -// MCP Native -import { createMCPServer } from '@dotdo/composio/mcp' -const mcpServer = createMCPServer({ apps: ['github'], entityId: 'user-123' }) -``` - -### Triggers -```typescript -await composio.triggers.subscribe({ - app: 'github', - event: 'push', - entityId: 'user-123', - webhookUrl: 'https://myapp.com/webhooks' -}) -``` - -## Architecture - -``` -composio/ - src/ - core/ # Business logic - app-registry.ts # App definitions and schemas - action-executor.ts # Action execution with retries - auth-manager.ts # OAuth/API key handling - durable-object/ # DO implementations - ToolsDO.ts # Tool registry and discovery - ConnectionDO.ts # Per-entity credential storage - ExecutionDO.ts # Sandboxed action execution - TriggerDO.ts # Webhook subscriptions - EntityDO.ts # User-to-connection mapping - RateDO.ts # Rate limiting per entity - adapters/ # Framework adapters - langchain.ts # LangChain tools - crewai.ts # CrewAI tools - autogen.ts # Autogen tools - llamaindex.ts # LlamaIndex tools - mcp/ # MCP integration - server.ts # MCP server implementation - tools.ts # MCP tool definitions - sdk/ # Client SDK - composio.ts # Composio class - types.ts # TypeScript types - .beads/ # Issue tracking - AGENTS.md # This file - README.md # User documentation -``` - -## TDD Workflow - -Follow strict TDD for all implementation: - -```bash -# Find ready work (RED tests first) -bd ready - -# Claim work -bd update composio-xxx --status in_progress - -# After tests pass -bd close composio-xxx - -# Check what's unblocked -bd ready -``` - -### TDD Cycle Pattern -1. **[RED]** Write failing tests first -2. **[GREEN]** Implement minimum code to pass -3. **[REFACTOR]** Clean up without changing behavior - -## Implementation Priorities - -### Phase 1: Core SDK (composio-001 to composio-003) -1. [RED] Test Composio client initialization -2. [GREEN] Implement Composio class with config -3. [REFACTOR] Add TypeScript types - -### Phase 2: Connection Management (composio-004 to composio-006) -1. [RED] Test OAuth connect flow -2. [GREEN] Implement ConnectionDO with token storage -3. [REFACTOR] Add token refresh logic - -### Phase 3: Tool Registry (composio-007 to composio-009) -1. [RED] Test app and action listing -2. [GREEN] Implement ToolsDO with schemas -3. [REFACTOR] Add KV caching layer - -### Phase 4: Action Execution (composio-010 to composio-012) -1. [RED] Test action execution -2. [GREEN] Implement ExecutionDO sandbox -3. [REFACTOR] Add retry and rate limiting - -### Phase 5: MCP Integration (composio-013 to composio-015) -1. [RED] Test MCP server creation -2. [GREEN] Implement MCP tool exposure -3. [REFACTOR] Dynamic schema generation - -### Phase 6: Agent Adapters (composio-016 to composio-018) -1. [RED] Test LangChain adapter -2. [GREEN] Implement framework adapters -3. [REFACTOR] Common adapter interface - -### Phase 7: Triggers (composio-019 to composio-021) -1. [RED] Test webhook subscription -2. [GREEN] Implement TriggerDO -3. [REFACTOR] Queue-based delivery - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -bd dep tree # View dependency tree -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - -## Testing Commands - -```bash -# Run tests -npm test - -# Run with coverage -npm run test:coverage - -# Watch mode -npm run test:watch -``` diff --git a/rewrites/composio/README.md b/rewrites/composio/README.md deleted file mode 100644 index 02384526..00000000 --- a/rewrites/composio/README.md +++ /dev/null @@ -1,364 +0,0 @@ -# composio.do - -> Hands for AI Agents. 150+ Tools. Zero Auth Headaches. - -Your agent can think. It can reason. It can plan. But it needs **hands** to act. - -"Fix the bug and open a PR" means 5 APIs: GitHub branches, commits, PRs, Slack webhooks, Linear tickets. 40+ hours of OAuth nightmares. 3am token refresh bugs. Rate limit hell. - -**composio.do** gives your agents hands. 150+ tools. One line per action. - -## AI-Native API - -```typescript -import { composio } from 'composio.do' // Full SDK -import { composio } from 'composio.do/tiny' // Minimal client -import { composio } from 'composio.do/mcp' // MCP tools only -``` - -Natural language for tool integrations: - -```typescript -import { composio } from 'composio.do' - -// Talk to it like a colleague -await composio`connect GitHub for user-123` -await composio`available actions for Slack` -await composio`execute send-message on Slack #general "Hello team"` - -// Chain like sentences -await composio`create GitHub issue ${bug}` - .then(issue => composio`notify Slack #bugs about ${issue}`) - -// Agents get tools automatically -await ralph.with(composio)`fix the login bug and open a PR` -``` - -## The Problem - -Composio (the original) charges for every action: - -| What They Charge | The Reality | -|------------------|-------------| -| **Per-Action Pricing** | $0.001-0.01 per tool call adds up fast | -| **Vendor Lock-in** | Your agent's hands belong to them | -| **Latency** | Round trip to their servers per action | -| **Tool Limits** | Quota overages kill your agent mid-task | -| **No Self-Hosting** | Your credentials, their servers | - -### The Real Cost - -At scale, tool calls explode: -- Simple bug fix: 10-20 tool calls -- Feature implementation: 50-100 tool calls -- Daily agent operations: 1,000+ tool calls - -**$100-1,000/month** just for your agent to have hands. - -## The Solution - -**composio.do** is the open-source alternative: - -``` -Composio (Original) composio.do ------------------------------------------------------------------ -Per-action pricing $0 - run your own -Their servers Your Cloudflare account -Latency per action Edge-native, global -Quota limits Unlimited -Vendor lock-in Open source, MIT licensed -``` - -## One-Click Deploy - -```bash -npx create-dotdo composio -``` - -150+ tool integrations. Running on infrastructure you control. Zero per-action costs. - -```typescript -import { Composio } from 'composio.do' - -export default Composio({ - name: 'my-agent-tools', - domain: 'tools.myagent.ai', - apps: ['github', 'slack', 'notion', 'linear'], -}) -``` - -## Features - -### Connections - -```typescript -// Connect anyone to anything -await composio`connect GitHub for user-123` -await composio`connect Slack for team-acme` -await composio`connect Salesforce for org-enterprise` - -// AI infers what you need -await composio`user-123 connections` // returns all connections -await composio`GitHub status for user-123` // returns connection health -await composio`refresh tokens for user-123` // refreshes expired tokens -``` - -### Tool Discovery - -```typescript -// Find tools naturally -await composio`what can I do with GitHub?` -await composio`Slack actions for messaging` -await composio`search tools for file management` - -// Get tool schemas -await composio`schema for github_create_issue` -await composio`required params for slack_send_message` -``` - -### Tool Execution - -```typescript -// Just say it -await composio`create GitHub issue: Login bug on mobile in acme/webapp` -await composio`send Slack message to #engineering: Deployment complete` -await composio`create Notion page: Sprint 23 Retro in Engineering space` - -// AI parses intent, executes the right action -await composio`add label "urgent" to issue #42 on acme/webapp` -await composio`assign @tom to PR #123` -await composio`close Linear ticket ENG-456 as done` -``` - -### Agent Integration - -```typescript -import { ralph } from 'agents.do' - -// Ralph gets 150+ tools automatically -await ralph.with(composio)`fix the login bug and open a PR` -// Uses: github_create_branch, github_commit, github_create_pr, slack_send_message - -// Ralph doesn't know about OAuth, tokens, or API schemas -// He just acts -``` - -### Promise Pipelining - -```typescript -import { priya, ralph, tom, quinn } from 'agents.do' - -// Plan to deploy - one network round trip -const deployment = await priya`plan deployment for v2.0` - .map(plan => ralph.with(composio)`implement ${plan}`) - .map(code => tom.with(composio)`review and approve ${code}`) - .map(approved => composio`notify Slack #releases: ${approved}`) - -// Cross-platform workflows -await composio`merge GitHub PR #${prNumber}` - .map(merge => composio`deploy Vercel production`) - .map(deploy => composio`create Datadog deployment marker`) - .map(marker => composio`announce in Slack #releases: ${marker}`) -``` - -## 150+ Tools, Zero Config - -| Category | Apps | Example Commands | -|----------|------|------------------| -| **Developer** | GitHub, GitLab, Linear, Jira | `create issue`, `open PR`, `merge branch` | -| **Communication** | Slack, Discord, Teams, Email | `send message`, `create channel`, `notify team` | -| **Productivity** | Notion, Asana, Monday, Trello | `create task`, `update page`, `move card` | -| **CRM** | Salesforce, HubSpot, Pipedrive | `create contact`, `log activity`, `update deal` | -| **Storage** | Google Drive, Dropbox, Box | `upload file`, `share folder`, `create doc` | -| **Analytics** | Datadog, Mixpanel, Amplitude | `track event`, `create dashboard`, `query metric` | -| **AI/ML** | OpenAI, Anthropic, Cohere | `generate text`, `create embedding`, `analyze` | - -```typescript -// All tools, one interface -await composio`create GitHub issue: ${bug}` -await composio`send Slack message: ${notification}` -await composio`create Notion page: ${doc}` -await composio`log Salesforce activity: ${call}` -await composio`upload to Drive: ${file}` -await composio`track Mixpanel event: ${action}` -``` - -## MCP Native - -Every tool is an MCP tool. Claude uses them directly: - -```typescript -// Expose tools to Claude -await composio`serve MCP for user-123 with github slack notion` - -// Claude sees: -// - github_create_issue -// - github_create_pr -// - slack_send_message -// - notion_create_page -// ... 150+ tools -``` - -## Framework Adapters - -```typescript -// Works with every framework -// But really, just use agents.do - -import { ralph } from 'agents.do' -await ralph.with(composio)`do the thing` // This is the way -``` - -## Auth Made Invisible - -The hardest part of integrations is auth. Just say it: - -```typescript -// OAuth - one line -await composio`connect Salesforce for user-123` - -// API Keys - stored securely -await composio`connect OpenAI for user-123 with ${apiKey}` - -// Check connection status -await composio`status for user-123 connections` - -// Disconnect -await composio`disconnect GitHub for user-123` - -// Token refresh - automatic -// Rate limits - managed -// Error retry - built in -``` - -Each user gets isolated credentials. Per-entity Durable Objects. No cross-contamination. - -## Architecture - -### Durable Object per Entity - -``` -ComposioDO (config, apps, schemas) - | - +-- ConnectionDO (per user - OAuth tokens, API keys) - | |-- SQLite: Credentials (encrypted) - | +-- R2: Connection metadata - | - +-- ExecutionDO (sandboxed tool execution) - | |-- Rate limiting per entity - | +-- Audit logging - | - +-- WebhookDO (inbound events from apps) - |-- GitHub webhooks - +-- Slack events -``` - -**Key insight**: Each user entity gets its own ConnectionDO for credential isolation. Tool execution happens in sandboxed ExecutionDO instances with rate limiting per entity. - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Active connections, recent executions | <10ms | -| **Warm** | R2 + Index | Historical logs, audit trails | <100ms | -| **Cold** | R2 Archive | Compliance retention | <1s | - -### Encryption - -Per-entity encryption for credentials. AES-256-GCM. Automatic key rotation. Immutable audit logs. - -## vs Composio (Original) - -| Feature | Composio (Original) | composio.do | -|---------|---------------------|-------------| -| **Pricing** | Per-action fees | $0 - run your own | -| **Data Location** | Their servers | Your Cloudflare account | -| **Latency** | Centralized | Edge-native, global | -| **Quotas** | Monthly limits | Unlimited | -| **Customization** | Limited | Full source access | -| **Lock-in** | Vendor dependency | MIT licensed | -| **MCP** | Partial support | Native | - -## Use Cases - -### AI Coding Agents - -```typescript -// Agent that ships code -await ralph.with(composio)` - fix the login bug in auth.ts, - create a branch, - commit the fix, - open a PR, - notify #engineering on Slack -` -``` - -### Workflow Automation - -```typescript -// When issue created, assign and notify -await composio`watch GitHub issues on acme/webapp` - .on('created', issue => composio` - assign @tom to ${issue}, - add label "triage", - notify #support: New issue ${issue.title} - `) -``` - -### Multi-App Orchestration - -```typescript -// Sales flow across apps -await composio`new contact ${lead} in HubSpot` - .map(contact => composio`create task in Asana: Follow up with ${contact}`) - .map(task => composio`schedule calendar meeting for ${task.due}`) - .map(meeting => composio`send Slack reminder: ${meeting}`) -``` - -## Deployment Options - -### Cloudflare Workers - -```bash -npx create-dotdo composio -# Your account, your data -``` - -### Private Cloud - -```bash -# Docker deployment -docker run -p 8787:8787 dotdo/composio - -# Kubernetes -kubectl apply -f composio-do.yaml -``` - -## Contributing - -composio.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/composio.do -cd composio.do -pnpm install -pnpm test -``` - -## License - -MIT License - Give your agents hands. - ---- - -

- Per-action pricing ends here. -
- 150+ tools. Zero cost. Your infrastructure. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/composio/package.json b/rewrites/composio/package.json deleted file mode 100644 index 67d4441a..00000000 --- a/rewrites/composio/package.json +++ /dev/null @@ -1,102 +0,0 @@ -{ - "name": "@dotdo/composio", - "version": "0.0.1", - "description": "Composio on Cloudflare - AI agent tool integration platform with managed auth", - "type": "module", - "main": "dist/index.js", - "types": "dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js" - }, - "./core": { - "types": "./dist/core/index.d.ts", - "import": "./dist/core/index.js" - }, - "./do": { - "types": "./dist/durable-object/index.d.ts", - "import": "./dist/durable-object/index.js" - }, - "./mcp": { - "types": "./dist/mcp/index.d.ts", - "import": "./dist/mcp/index.js" - }, - "./langchain": { - "types": "./dist/adapters/langchain.d.ts", - "import": "./dist/adapters/langchain.js" - }, - "./crewai": { - "types": "./dist/adapters/crewai.d.ts", - "import": "./dist/adapters/crewai.js" - }, - "./autogen": { - "types": "./dist/adapters/autogen.d.ts", - "import": "./dist/adapters/autogen.js" - }, - "./llamaindex": { - "types": "./dist/adapters/llamaindex.d.ts", - "import": "./dist/adapters/llamaindex.js" - } - }, - "files": [ - "dist", - "LICENSE", - "README.md" - ], - "scripts": { - "build": "tsc", - "test": "vitest run", - "test:watch": "vitest", - "test:coverage": "vitest run --coverage", - "dev": "wrangler dev", - "deploy": "wrangler deploy", - "lint": "eslint src test --ext .ts", - "typecheck": "tsc --noEmit" - }, - "keywords": [ - "composio", - "ai-agents", - "tool-integration", - "oauth", - "cloudflare", - "durable-objects", - "mcp", - "langchain", - "crewai", - "autogen", - "llamaindex" - ], - "author": "dot-do", - "license": "MIT", - "repository": { - "type": "git", - "url": "https://github.com/dot-do/composio" - }, - "homepage": "https://github.com/dot-do/composio#readme", - "bugs": { - "url": "https://github.com/dot-do/composio/issues" - }, - "engines": { - "node": ">=18.0.0" - }, - "devDependencies": { - "@cloudflare/workers-types": "^4.20241218.0", - "@types/node": "^22.10.2", - "typescript": "^5.7.2", - "vitest": "^2.1.8", - "wrangler": "^3.99.0" - }, - "peerDependencies": { - "@langchain/core": ">=0.2.0", - "hono": ">=4.0.0" - }, - "peerDependenciesMeta": { - "@langchain/core": { - "optional": true - }, - "hono": { - "optional": true - } - } -} diff --git a/rewrites/composio/src/index.ts b/rewrites/composio/src/index.ts deleted file mode 100644 index 39693e16..00000000 --- a/rewrites/composio/src/index.ts +++ /dev/null @@ -1,19 +0,0 @@ -/** - * @dotdo/composio - AI agent tool integration platform with managed auth - * - * Composio on Cloudflare - 150+ tool integrations for AI agents with managed OAuth/API key auth, - * tool execution sandbox, and first-class MCP support. - */ - -export { Composio, type ComposioConfig, type ComposioClient } from './sdk/composio' -export type { - App, - Action, - Connection, - Entity, - Trigger, - ExecutionResult, - ConnectOptions, - ExecuteOptions, - GetToolsOptions, -} from './sdk/types' diff --git a/rewrites/composio/src/sdk/composio.ts b/rewrites/composio/src/sdk/composio.ts deleted file mode 100644 index acc429f2..00000000 --- a/rewrites/composio/src/sdk/composio.ts +++ /dev/null @@ -1,198 +0,0 @@ -/** - * Composio SDK client - * - * Main entry point for interacting with Composio on Cloudflare. - */ - -import type { - App, - Action, - Connection, - Entity, - Trigger, - ExecutionResult, - ConnectOptions, - ExecuteOptions, - GetToolsOptions, - MCPTool, - RateLimitConfig, -} from './types' - -export interface ComposioConfig { - apiKey?: string - baseUrl?: string - rateLimit?: RateLimitConfig -} - -export interface ComposioClient { - // App management - apps: { - list(): Promise - get(appId: string): Promise - } - - // Action management - actions: { - list(options?: { app?: string }): Promise - get(actionId: string): Promise - search(options: { query: string; apps?: string[] }): Promise - } - - // Connection management - connect(options: ConnectOptions): Promise<{ redirectUrl?: string; connection?: Connection }> - completeAuth(options: { userId: string; app: string; code: string }): Promise - disconnect(options: { userId: string; app: string }): Promise - getConnections(userId: string): Promise - - // Action execution - execute(options: ExecuteOptions): Promise> - executeBatch(options: ExecuteOptions[]): Promise - - // Tool generation - getTools(options: GetToolsOptions): Promise - - // Triggers - triggers: { - subscribe(options: { - app: string - event: string - entityId: string - webhookUrl: string - }): Promise - unsubscribe(triggerId: string): Promise - list(entityId: string): Promise - } - - // Entity management - entities: { - get(entityId: string): Promise - create(externalId: string): Promise - delete(entityId: string): Promise - } - - // Rate limiting - setAppRateLimit(options: { app: string } & RateLimitConfig): Promise -} - -/** - * Create a Composio client - */ -export class Composio implements ComposioClient { - private config: ComposioConfig - - constructor(config: ComposioConfig = {}) { - this.config = { - baseUrl: 'https://composio.do', - ...config, - } - } - - // App management - apps = { - list: async (): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - get: async (_appId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - } - - // Action management - actions = { - list: async (_options?: { app?: string }): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - get: async (_actionId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - search: async (_options: { query: string; apps?: string[] }): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - } - - // Connection management - async connect(_options: ConnectOptions): Promise<{ redirectUrl?: string; connection?: Connection }> { - // TODO: Implement - throw new Error('Not implemented') - } - - async completeAuth(_options: { userId: string; app: string; code: string }): Promise { - // TODO: Implement - throw new Error('Not implemented') - } - - async disconnect(_options: { userId: string; app: string }): Promise { - // TODO: Implement - throw new Error('Not implemented') - } - - async getConnections(_userId: string): Promise { - // TODO: Implement - throw new Error('Not implemented') - } - - // Action execution - async execute(_options: ExecuteOptions): Promise> { - // TODO: Implement - throw new Error('Not implemented') - } - - async executeBatch(_options: ExecuteOptions[]): Promise { - // TODO: Implement - throw new Error('Not implemented') - } - - // Tool generation - async getTools(_options: GetToolsOptions): Promise { - // TODO: Implement - throw new Error('Not implemented') - } - - // Triggers - triggers = { - subscribe: async (_options: { - app: string - event: string - entityId: string - webhookUrl: string - }): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - unsubscribe: async (_triggerId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - list: async (_entityId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - } - - // Entity management - entities = { - get: async (_entityId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - create: async (_externalId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - delete: async (_entityId: string): Promise => { - // TODO: Implement - throw new Error('Not implemented') - }, - } - - // Rate limiting - async setAppRateLimit(_options: { app: string } & RateLimitConfig): Promise { - // TODO: Implement - throw new Error('Not implemented') - } -} diff --git a/rewrites/composio/src/sdk/types.ts b/rewrites/composio/src/sdk/types.ts deleted file mode 100644 index ef568e55..00000000 --- a/rewrites/composio/src/sdk/types.ts +++ /dev/null @@ -1,163 +0,0 @@ -/** - * TypeScript types for Composio SDK - */ - -/** - * Supported app definition - */ -export interface App { - id: string - name: string - description: string - category: AppCategory - authMethod: AuthMethod - logo?: string - docsUrl?: string -} - -export type AppCategory = - | 'developer' - | 'communication' - | 'productivity' - | 'crm' - | 'storage' - | 'calendar' - | 'ai' - | 'finance' - | 'marketing' - | 'other' - -export type AuthMethod = 'oauth2' | 'api_key' | 'bearer_token' | 'basic_auth' - -/** - * Action definition - */ -export interface Action { - id: string - name: string - description: string - app: string - inputSchema: JSONSchema - outputSchema: JSONSchema - scopes?: string[] -} - -export interface JSONSchema { - type: 'object' | 'array' | 'string' | 'number' | 'boolean' | 'null' - properties?: Record - required?: string[] - items?: JSONSchema - description?: string - enum?: string[] - default?: unknown -} - -/** - * Connection to an app for an entity - */ -export interface Connection { - id: string - entityId: string - app: string - status: ConnectionStatus - createdAt: Date - updatedAt: Date - expiresAt?: Date - scopes?: string[] -} - -export type ConnectionStatus = 'active' | 'expired' | 'revoked' | 'pending' - -/** - * Entity represents a user/account in the system - */ -export interface Entity { - id: string - externalId: string - connections: Connection[] - createdAt: Date - updatedAt: Date -} - -/** - * Trigger definition for webhooks - */ -export interface Trigger { - id: string - entityId: string - app: string - event: string - webhookUrl: string - status: TriggerStatus - createdAt: Date -} - -export type TriggerStatus = 'active' | 'paused' | 'failed' - -/** - * Result of executing an action - */ -export interface ExecutionResult { - success: boolean - data?: T - error?: { - code: string - message: string - details?: unknown - } - executionTime: number - requestId: string -} - -/** - * Options for connecting to an app - */ -export interface ConnectOptions { - userId: string - app: string - redirectUrl?: string - credentials?: Credentials - scopes?: string[] -} - -export type Credentials = - | { apiKey: string } - | { bearerToken: string } - | { email: string; apiToken: string } - | { clientId: string; clientSecret: string } - -/** - * Options for executing an action - */ -export interface ExecuteOptions { - action: string - params: Record - entityId: string - timeout?: number -} - -/** - * Options for getting tools - */ -export interface GetToolsOptions { - apps?: string[] - actions?: string[] - entityId: string -} - -/** - * MCP Tool definition - */ -export interface MCPTool { - name: string - description: string - inputSchema: JSONSchema -} - -/** - * Rate limit configuration - */ -export interface RateLimitConfig { - maxRequests: number - windowMs: number -} diff --git a/rewrites/confluence/.beads/.gitignore b/rewrites/confluence/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/confluence/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/confluence/.beads/README.md b/rewrites/confluence/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/confluence/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/confluence/.beads/config.yaml b/rewrites/confluence/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/confluence/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/confluence/.beads/interactions.jsonl b/rewrites/confluence/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/confluence/.beads/issues.jsonl b/rewrites/confluence/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/confluence/.beads/metadata.json b/rewrites/confluence/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/confluence/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/confluence/.gitattributes b/rewrites/confluence/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/confluence/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/confluence/AGENTS.md b/rewrites/confluence/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/confluence/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/confluence/README.md b/rewrites/confluence/README.md deleted file mode 100644 index 4824d86b..00000000 --- a/rewrites/confluence/README.md +++ /dev/null @@ -1,475 +0,0 @@ -# confluence.do - -> Enterprise wiki. AI-native. Open source. - -Confluence became the default for team documentation. It also became the place where knowledge goes to die. Outdated pages, broken links, impossible search, and the constant question: "Is this doc still accurate?" - -**confluence.do** reimagines team knowledge for the AI era. AI writes docs from your code. AI keeps docs in sync. AI answers questions. Your wiki finally works. - -## AI-Native API - -```typescript -import { confluence } from 'confluence.do' // Full SDK -import { confluence } from 'confluence.do/tiny' // Minimal client -import { confluence } from 'confluence.do/search' // Search-only operations -``` - -Natural language for knowledge workflows: - -```typescript -import { confluence } from 'confluence.do' - -// Talk to it like a colleague -const answer = await confluence`how do we handle authentication?` -const stale = await confluence`outdated docs in Engineering` -const api = await confluence`pages mentioning API redesign` - -// Chain like sentences -await confluence`docs about payments` - .notify(`Please review for accuracy`) - -// Documentation that writes itself -await confluence`document the auth system from code` - .review() // AI checks accuracy - .publish() // your approval -``` - -### Promise Pipelining with Agents - -Chain work without Promise.all: - -```typescript -import { confluence } from 'confluence.do' -import { mark, tom, priya } from 'agents.do' - -// Keep docs in sync with code -await confluence`docs about authentication` - .map(doc => tom`verify ${doc} against codebase`) - .map(doc => mark`update ${doc} if stale`) - .map(doc => priya`review ${doc}`) - -// Generate docs from code changes -await git`recent commits to ${repo}` - .map(commit => mark`document ${commit}`) - .map(doc => confluence`publish ${doc} to Engineering`) - -// Close documentation gaps at scale -await confluence`stale docs in Engineering` - .map(doc => mark`update ${doc} from current code`) - .each(doc => doc.publish()) -``` - -One network round trip. Record-replay pipelining. Workers working for you. - -## The Problem - -Atlassian bundled Confluence with Jira, and companies got stuck: - -| Confluence Plan | Price | Reality | -|-----------------|-------|---------| -| Free | $0/10 users | 2GB storage, limited features | -| Standard | $6.05/user/month | Basic wiki | -| Premium | $11.55/user/month | Analytics, archiving | -| Enterprise | Custom | Unlimited everything, unlimited cost | - -**500-person company on Premium?** That's **$69,300/year** for a wiki. - -The hidden costs are worse: -- **Search that doesn't work** - Can't find what you're looking for -- **Stale documentation** - Nobody knows if docs are current -- **Content sprawl** - 10,000 pages, 10 useful ones -- **No code integration** - Docs and code drift apart -- **Template hell** - 50 templates, none quite right - -## The Solution - -**confluence.do** is what a wiki should be: - -``` -Traditional Confluence confluence.do ------------------------------------------------------------------ -$6-12/user/month $0 - run your own -Terrible search AI-powered semantic search -Stale documentation AI-verified freshness -Manual everything AI writes, updates, maintains -Their servers Your Cloudflare account -Content lock-in Markdown + structured data -Complex macros Simple, powerful components -``` - -## One-Click Deploy - -```bash -npx create-dotdo confluence -``` - -Your team wiki. Running on Cloudflare. AI-native. - -```typescript -import { Confluence } from 'confluence.do' - -export default Confluence({ - name: 'acme-wiki', - domain: 'wiki.acme.com', - spaces: ['Engineering', 'Product', 'Design'], -}) -``` - -## Features - -### Spaces & Pages - -```typescript -// Create naturally -await confluence`create space "Engineering" for technical docs` -await confluence`create page "Auth Architecture" in Engineering under Architecture` - -// Or just write content -await confluence` - page "API Guidelines" in Engineering: - - # API Guidelines - - All APIs must be RESTful with JSON responses... -` -``` - -### Search & Discovery - -```typescript -// Find anything -const authDocs = await confluence`pages about authentication` -const stale = await confluence`docs not updated in 6 months` -const byAlice = await confluence`pages Alice wrote last month` - -// AI infers what you need -await confluence`authentication` // returns relevant pages -await confluence`how do we deploy?` // returns answer with sources -await confluence`what did we decide about caching?` // finds ADRs -``` - -### Real-Time Collaboration - -Multiple editors, zero conflicts. CRDT-based editing syncs instantly. - -```typescript -// Presence just works -// See who's editing, where their cursor is -// Changes merge automatically -``` - -### Page Trees & Navigation - -```typescript -// Navigate naturally -await confluence`Engineering page tree` -await confluence`pages under Architecture` -await confluence`related to Auth Architecture` -``` - -### Templates - -```typescript -// Use templates by name -await confluence`new ADR "Use PostgreSQL for User Data"` -await confluence`new runbook for payment-service` -await confluence`new meeting notes from standup` - -// AI fills in context automatically -``` - -## AI-Native Documentation - -The real magic. AI that keeps your documentation alive. - -### AI Search - -Search that actually understands: - -```typescript -// Semantic search - not just keywords -await confluence`how do we handle authentication?` -// Finds pages about auth, JWT, login, sessions - even without those exact words - -// Question answering with sources -await confluence`what database do we use for user data?` -// Returns: "PostgreSQL, as decided in ADR-042. The users table schema is defined in..." - -// Complex queries -await confluence`how do I deploy to production? show sources` -``` - -### AI Writes Documentation - -Stop staring at blank pages: - -```typescript -// Generate docs from code -await confluence`document the API routes in src/routes/` -await confluence`write technical docs for payment-service` - -// Generate from context -await confluence`write runbook for payment-service with @backend-team oncall` - -// Generate meeting notes -await confluence`meeting notes from standup recording` - .review() // you approve - .publish() // done -``` - -### AI Freshness Verification - -Know if your docs are current: - -```typescript -// AI checks documentation freshness -await confluence`check freshness of API docs against code` -await confluence`find broken links in Engineering` -await confluence`docs that contradict each other` - -// Or just ask -await confluence`is the deployment guide current?` -``` - -### AI Doc Sync - -Keep docs and code in sync automatically: - -```typescript -// Sync API docs with OpenAPI spec -await confluence`sync API Reference from openapi.yaml on every push` - -// Sync README with wiki -await confluence`sync payment-service README bidirectionally` - -// Auto-generate changelog -await confluence`generate changelog from commits weekly` -``` - -### AI Q&A Bot - -Answer questions from your wiki: - -```typescript -// Deploy a Q&A bot - one line -await confluence`deploy bot to #engineering and #help` - -// In Slack: "@wiki-bot how do we deploy to production?" -// Bot responds with synthesized answer + source links -``` - -## Integration with jira.do - -Seamless connection with your issue tracker: - -```typescript -// Auto-link issues mentioned in docs -// Writing "BACKEND-123" in a page automatically links to jira.do - -// Create design doc from issue -await confluence`design doc for BACKEND-500` - -// Or chain from jira -await jira`BACKEND-500` - .map(issue => confluence`write design doc for ${issue}`) - .map(doc => confluence`publish to Design Documents`) - -// Embed issue status in docs -// {% jira BACKEND-500 %} shows live status -``` - -## Content Components - -Rich components for modern documentation: - -### Diagrams - -```typescript -// Generate diagrams naturally -await confluence`diagram the auth flow` -await confluence`architecture diagram for payment-service` - -// Or embed Mermaid/Excalidraw directly in content -``` - -### Code Blocks - -```typescript -// Live code from GitHub - always current -await confluence`embed src/models/user.ts lines 10-25` - -// Code stays in sync automatically -``` - -### Tables & Databases - -```typescript -// Dynamic tables from queries -await confluence`table of open bugs in BACKEND` - -// Embedded databases (Notion-style) -await confluence`database of features with status, owner, priority` -``` - -### Callouts & Panels - -```markdown -{% info %} -This is informational content. -{% /info %} - -{% warning %} -Be careful when modifying production data. -{% /warning %} - -{% danger %} -This action cannot be undone. -{% /danger %} -``` - -## API Compatible - -Drop-in compatibility with Confluence REST API: - -```typescript -// Standard Confluence REST API -GET /wiki/rest/api/space -GET /wiki/rest/api/content -POST /wiki/rest/api/content -PUT /wiki/rest/api/content/{id} -DELETE /wiki/rest/api/content/{id} - -GET /wiki/rest/api/content/{id}/child/page -GET /wiki/rest/api/content/search?cql={query} -``` - -Existing Confluence integrations work. Just change the host. - -## Architecture - -### Durable Object per Space - -Each space runs in its own Durable Object: - -``` -WikiDO (global config, search index) - | - +-- SpaceDO:ENG (Engineering space) - | +-- SQLite: pages, comments, attachments metadata - | +-- R2: attachments, images - | +-- WebSocket: real-time collaboration - | - +-- SpaceDO:PRODUCT (Product space) - +-- SpaceDO:DESIGN (Design space) - +-- SearchDO (vector embeddings for semantic search) -``` - -### Real-Time Editing - -CRDT-based collaborative editing: - -``` -User A types "Hello" at position 0 -User B types "World" at position 100 - -Both operations merge automatically: -- No conflicts -- No lost changes -- Instant sync -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Current pages, recent edits | <10ms | -| **Warm** | R2 + Index | Attachments, page history | <100ms | -| **Cold** | R2 Archive | Old versions, deleted content | <1s | - -## vs Atlassian Confluence - -| Feature | Atlassian Confluence | confluence.do | -|---------|---------------------|---------------| -| **Annual Cost** | $69,300 (500 users) | ~$50/month | -| **Search** | Keyword matching | AI semantic search | -| **Freshness** | Manual verification | AI-verified | -| **Documentation** | Write everything | AI generates from code | -| **Architecture** | Centralized cloud | Edge-native, global | -| **Data Location** | Atlassian's servers | Your Cloudflare account | -| **Lock-in** | Years of migration | MIT licensed | - -## Migration from Confluence - -One-time import of your existing wiki: - -```bash -npx confluence-do migrate \ - --url=https://your-company.atlassian.net/wiki \ - --email=admin@company.com \ - --token=your_api_token -``` - -Imports: -- All spaces and pages -- Page hierarchy and relationships -- Attachments and images -- Comments and inline comments -- Labels and categories -- User permissions -- Page history - -## Roadmap - -- [x] Spaces and pages -- [x] Page hierarchy and navigation -- [x] Real-time collaborative editing -- [x] Templates -- [x] Search (full-text + semantic) -- [x] REST API compatibility -- [x] AI search and Q&A -- [x] AI freshness verification -- [ ] Advanced permissions (page-level) -- [ ] Blueprints marketplace -- [ ] Confluence macro compatibility -- [ ] PDF export -- [ ] Jira.do deep integration -- [ ] Git-based version control - -## Why Open Source? - -Your knowledge is too valuable to be locked away: - -1. **Your documentation** - Institutional knowledge is irreplaceable -2. **Your search** - Finding information shouldn't require luck -3. **Your freshness** - Stale docs are worse than no docs -4. **Your AI** - Intelligence on your knowledge should be yours - -Confluence showed the world what team wikis could be. **confluence.do** makes them intelligent, accurate, and always up-to-date. - -## Contributing - -We welcome contributions! See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines. - -Key areas: -- Editor and collaboration features -- Search and AI capabilities -- Confluence API compatibility -- Component and macro development -- Migration tooling - -## License - -MIT License - For teams who deserve better documentation. - ---- - -

- The $70k wiki ends here. -
- AI-native. Always fresh. Knowledge that works. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/coupa/.beads/.gitignore b/rewrites/coupa/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/coupa/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/coupa/.beads/README.md b/rewrites/coupa/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/coupa/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/coupa/.beads/config.yaml b/rewrites/coupa/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/coupa/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/coupa/.beads/interactions.jsonl b/rewrites/coupa/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/coupa/.beads/issues.jsonl b/rewrites/coupa/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/coupa/.beads/metadata.json b/rewrites/coupa/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/coupa/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/coupa/.gitattributes b/rewrites/coupa/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/coupa/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/coupa/AGENTS.md b/rewrites/coupa/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/coupa/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/coupa/README.md b/rewrites/coupa/README.md deleted file mode 100644 index 6c7434df..00000000 --- a/rewrites/coupa/README.md +++ /dev/null @@ -1,520 +0,0 @@ -# coupa.do - -> Procurement + Spend Management. AI-native. Control your spend. - -Coupa built a $8B company (acquired by Thoma Bravo) managing enterprise procurement - purchase orders, invoices, supplier management. At $150-300 per user per month, with implementations running $500K-2M, the "Business Spend Management" platform has become its own budget line item. - -**coupa.do** is the open-source alternative. One-click deploy your own procurement platform. AI that actually finds savings. No per-seat ransomware. - -## AI-Native API - -```typescript -import { coupa } from 'coupa.do' // Full SDK -import { coupa } from 'coupa.do/tiny' // Minimal client -import { coupa } from 'coupa.do/p2p' // Procure-to-pay only -``` - -Natural language for procurement workflows: - -```typescript -import { coupa } from 'coupa.do' - -// Talk to it like a procurement manager -const pending = await coupa`POs pending approval over $10k` -const maverick = await coupa`off-contract spend by department this quarter` -const savings = await coupa`find me $500k in savings this year` - -// Chain like sentences -await coupa`contracts expiring Q2` - .map(contract => coupa`prepare renewal brief for ${contract}`) - -// Procure-to-pay in one flow -await coupa`requisition 5 laptops for Engineering` - .approve() // routes based on amount and policy - .order() // generates PO to preferred vendor - .receive() // confirms delivery - .pay() // three-way match and payment -``` - -## The Problem - -Procurement software that costs as much as the procurement team: - -- **$150-300/user/month** - A 200-person procurement org? $360K-720K annually -- **$500K-2M implementation** - Before you process a single PO -- **Per-transaction fees** - Additional charges for supplier network, e-invoicing -- **AI as premium upsell** - Spend intelligence costs extra -- **Supplier network lock-in** - Your suppliers locked into their network -- **Consultant dependency** - Change a workflow? Call Deloitte - -Companies implement procurement systems to control spend. Then spend a fortune on the procurement system. - -## The Solution - -**coupa.do** returns procurement to procurement: - -``` -Coupa coupa.do ------------------------------------------------------------------ -$150-300/user/month $0 - run your own -$500K-2M implementation Deploy in hours -Supplier network fees Your suppliers, direct -AI as premium tier AI-native from day one -Consultant dependency Configure yourself -Proprietary workflows Open, customizable -``` - -## One-Click Deploy - -```bash -npx create-dotdo coupa -``` - -A full procurement platform. Running on infrastructure you control. Every user, every supplier, no per-seat pricing. - -```typescript -import { Coupa } from 'coupa.do' - -export default Coupa({ - name: 'acme-procurement', - domain: 'procurement.acme.com', -}) -``` - -## Features - -### Requisitions - -Request what you need: - -```typescript -import { coupa } from 'coupa.do' - -// Just ask for it -await coupa`5 MacBook Pro M3 Max for Q1 hires, deliver to HQ IT closet by Feb 15` - -// AI figures out supplier, pricing, coding, routing -await coupa`office supplies for marketing, about $500` - -// Bulk requests read like a shopping list -await coupa` - Engineering needs: - - 10 monitors for new hires - - standing desks for the Austin office - - ergonomic keyboards, same as last order -` -``` - -### Approvals - -Approvals flow naturally: - -```typescript -// Check what needs your attention -await coupa`my pending approvals` -await coupa`urgent approvals over $50k` - -// Approve with context -await coupa`approve REQ-001, consolidate with Q2 order if possible` -await coupa`reject REQ-002, use preferred vendor instead` -await coupa`return REQ-003 to Sarah for more detail` - -// Delegation when you're out -await coupa`delegate my approvals to Mike for two weeks` -``` - -### Purchase Orders - -POs in plain English: - -```typescript -// Create PO from approved requisition -await coupa`create PO for REQ-001 to Apple, Net 30, ship to SF office` - -// Or generate directly -await coupa`PO to Dell for 20 laptops from the IT equipment contract` - -// Check status -await coupa`open POs for Engineering this month` -await coupa`POs pending supplier acknowledgment` - -// Receiving is one line -await coupa`received all items on PO-001` -await coupa`received 5 of 10 monitors on PO-002, rest backordered` -``` - -### Invoices - -Three-way matching in natural language: - -```typescript -// Process incoming invoices -await coupa`match invoice from Apple INV-2025-001234` - -// Query invoice status -await coupa`invoices not matched to POs` -await coupa`invoices pending approval over $10k` -await coupa`overdue invoices by supplier` - -// Handle exceptions naturally -await coupa`why won't INV-001 match?` -await coupa`approve INV-001 despite price variance, vendor raised prices` - -// Payment scheduling -await coupa`schedule INV-001 for early pay discount` -await coupa`hold payment on INV-002 until quality issue resolved` -``` - -### Supplier Management - -Know your suppliers: - -```typescript -// Find and query suppliers -await coupa`our top 10 suppliers by spend` -await coupa`suppliers in IT Equipment category` -await coupa`which suppliers have declining scores?` - -// Onboard naturally -await coupa`add Apple as IT Equipment vendor, enterprise@apple.com` -await coupa`request W-9 and insurance certificate from Apple` - -// Performance tracking -await coupa`Apple scorecard` -await coupa`suppliers with late delivery issues this quarter` -await coupa`suppliers with expiring certifications` -``` - -### Contracts - -Contract management without the forms: - -```typescript -// Query contracts -await coupa`contracts expiring in 90 days` -await coupa`our agreement with Apple` -await coupa`contracts with volume discounts we're not hitting` - -// Contract intelligence -await coupa`which contracts have renewal leverage?` -await coupa`off-contract spend by category this quarter` -await coupa`are we getting the contracted price on PO-001?` - -// Renewal prep -await coupa`prepare negotiation brief for Apple renewal` -``` - -### Catalogs - -Shopping from approved sources: - -```typescript -// Browse what's available -await coupa`show me approved laptops` -await coupa`office supplies catalog` -await coupa`what can I buy from Amazon Business?` - -// Smart recommendations -await coupa`I need a laptop for a new developer` -// AI suggests: "MacBook Pro 16" M3 Max from Apple contract at $3,499 -// - 15% discount vs list -// - 2-day delivery available -// - 47 others ordered this quarter" -``` - -## Spend Analytics - -Ask questions about your spend: - -```typescript -// Understand where money goes -await coupa`spend by category this year` -await coupa`top 20 suppliers by spend` -await coupa`IT spend trend last 12 months` - -// Find savings opportunities -await coupa`maverick spend by department this quarter` -await coupa`contracts we're underutilizing` -await coupa`duplicate purchases across departments` - -// Executive dashboards -await coupa`spend summary for the board` -await coupa`year-over-year savings from procurement` -``` - -## AI-Native Procurement - -### Procurement Intelligence - -```typescript -// One question unlocks insights -await coupa`find me $500K in savings this year` -// AI analyzes spend patterns, contract leverage, pricing variance, -// maverick spend, and early pay discounts - returns actionable plan - -// Continuous monitoring -await coupa`supplier risk alerts` -await coupa`contracts expiring without coverage` -await coupa`price increases above inflation` -``` - -### Invoice Automation - -```typescript -// Drop in an invoice, AI handles the rest -await coupa`process invoice from Apple` - .match() // three-way match to PO and receipt - .code() // GL coding from patterns - .approve() // auto-approve if clean - -// Exception handling -await coupa`invoices with matching exceptions` - .map(inv => coupa`explain why ${inv} won't match`) -``` - -### Negotiation Support - -```typescript -import { sally } from 'agents.do' - -// Prep for renewals in one line -await sally`prepare Salesforce renewal brief, current spend $450K` -// Returns: market benchmarks, leverage points, target pricing, BATNA - -// Bulk renewal prep -await coupa`contracts expiring Q2` - .map(contract => sally`negotiation brief for ${contract}`) -``` - -### Supplier Risk Monitoring - -```typescript -import { tom } from 'agents.do' - -// Continuous risk assessment -await tom`assess risk across our top 50 suppliers` - -// Alert on changes -await coupa`suppliers with credit downgrades` -await coupa`single-source categories` -await coupa`suppliers with expiring SOC 2 certs` -``` - -### Smart Recommendations - -```typescript -import { priya } from 'agents.do' - -// AI suggests based on context -await priya`recommend laptops for 10 new hires` -// Returns options ranked by value, performance, compliance -// with pricing, availability, and approval requirements -``` - -## Architecture - -### Durable Object per Company - -``` -CompanyDO (config, users, approval rules) - | - +-- RequisitionsDO (purchase requests) - | |-- SQLite: Requisition data - | +-- R2: Attachments - | - +-- PurchaseOrdersDO (POs) - | |-- SQLite: PO data, status - | +-- R2: PO documents - | - +-- InvoicesDO (invoices, matching) - | |-- SQLite: Invoice data - | +-- R2: Invoice images/PDFs - | - +-- SuppliersDO (supplier master) - | |-- SQLite: Supplier data - | +-- R2: Supplier documents - | - +-- AnalyticsDO (spend analytics) - |-- SQLite: Aggregated spend - +-- R2: Data warehouse -``` - -### Integrations - -Connect to your systems naturally: - -```typescript -// ERP sync just works -await coupa`sync vendors and GL accounts from NetSuite` -await coupa`push approved POs to SAP` - -// Supplier portal included -await coupa`enable supplier portal for our vendors` -// Suppliers can view POs, submit invoices, check payment status -``` - -## vs Coupa - -| Feature | Coupa | coupa.do | -|---------|-------|----------| -| **Per-User Cost** | $150-300/month | $0 - run your own | -| **Implementation** | $500K-2M | Deploy in hours | -| **Supplier Network** | Locked to their network | Your suppliers, direct | -| **AI/Analytics** | Premium upsell | AI-native from day one | -| **Customization** | Consultant required | Configure yourself | -| **Data Location** | Their cloud | Your Cloudflare account | -| **Workflow Changes** | Change request process | Code it yourself | -| **API Access** | Limited, extra cost | Full access, open | -| **Lock-in** | Years of migration | MIT licensed | - -## Why Open Source for Procurement? - -**1. Procurement Is Universal** - -Every company buys things. The process is well-understood: -- Request -> Approve -> Order -> Receive -> Pay -- No competitive advantage in the software itself -- Value is in the process and data - -**2. Data Is Strategic** - -Your spend data reveals: -- Business strategy -- Supplier relationships -- Cost structure -- Negotiating positions - -This shouldn't live on a third-party platform. - -**3. Integration Hell** - -Procurement touches everything: -- ERP for financials -- HR for approvals -- IT for assets -- Legal for contracts - -Open source = integrate without permission. - -**4. AI Needs Access** - -To find savings, optimize payments, assess risk - AI needs full access to: -- Spend data -- Contracts -- Supplier information -- Historical patterns - -Closed platforms gatekeep this data. - -**5. Per-Seat Pricing Is Extractive** - -Procurement should have MORE users, not fewer: -- Requestors across the company -- Approvers at all levels -- Receivers in warehouses -- AP clerks processing invoices - -Per-seat pricing creates friction against adoption. - -## Deployment - -### Cloudflare Workers - -```bash -npx create-dotdo coupa -# Global deployment -# Fast for every user -``` - -### Self-Hosted - -```bash -# Docker -docker run -p 8787:8787 dotdo/coupa - -# Kubernetes -kubectl apply -f coupa-do-deployment.yaml -``` - -### Hybrid - -```typescript -// Edge for fast user experience, origin for heavy analytics -await coupa`run requisitions and approvals at the edge` -await coupa`run analytics on origin for heavy queries` -``` - -## Roadmap - -### Procurement -- [x] Requisitions -- [x] Approvals -- [x] Purchase Orders -- [x] Receiving -- [x] Three-Way Match -- [ ] Blanket Orders -- [ ] RFx (RFQ/RFP/RFI) - -### Suppliers -- [x] Supplier Master -- [x] Supplier Portal -- [x] Supplier Qualification -- [x] Performance Scorecards -- [ ] Supplier Diversity -- [ ] Risk Monitoring - -### Invoices -- [x] Invoice Processing -- [x] Three-Way Match -- [x] Exception Handling -- [x] Payment Scheduling -- [ ] Virtual Cards -- [ ] Dynamic Discounting - -### Analytics -- [x] Spend Analytics -- [x] Contract Utilization -- [x] Maverick Spend -- [ ] Predictive Analytics -- [ ] Benchmarking - -### AI -- [x] Invoice OCR -- [x] Spend Analysis -- [x] Savings Identification -- [ ] Autonomous Procurement -- [ ] Negotiation Assistance - -## Contributing - -coupa.do is open source under the MIT license. - -We welcome contributions from: -- Procurement professionals -- Supply chain experts -- Finance/AP specialists -- ERP integration developers - -```bash -git clone https://github.com/dotdo/coupa.do -cd coupa.do -npm install -npm test -``` - -## License - -MIT License - Procure freely. - ---- - -

- The $8B acquisition ends here. -
- AI-native. Every user. No per-seat pricing. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/customerio/.beads/.gitignore b/rewrites/customerio/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/customerio/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/customerio/.beads/README.md b/rewrites/customerio/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/customerio/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/customerio/.beads/config.yaml b/rewrites/customerio/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/customerio/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/customerio/.beads/interactions.jsonl b/rewrites/customerio/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/customerio/.beads/issues.jsonl b/rewrites/customerio/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/customerio/.beads/metadata.json b/rewrites/customerio/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/customerio/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/customerio/.gitattributes b/rewrites/customerio/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/customerio/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/customerio/AGENTS.md b/rewrites/customerio/AGENTS.md deleted file mode 100644 index f84088db..00000000 --- a/rewrites/customerio/AGENTS.md +++ /dev/null @@ -1,112 +0,0 @@ -# Agent Instructions - customerio.do - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Project Overview - -**customerio.do** is a Customer.io-compatible marketing automation platform built on Cloudflare Durable Objects. It provides event tracking, workflow orchestration, dynamic segmentation, multi-channel message delivery, and Liquid template rendering. - -Package: `@dotdo/customerio` - -## Architecture - -``` -customerio/ - src/ - core/ # Pure business logic - durable-objects/ # DO classes - CustomerDO # User profiles, events, preferences - JourneyDO # Workflow execution via CF Workflows - SegmentDO # Dynamic audience computation - DeliveryDO # Channel orchestration - TemplateDO # Template storage and rendering - channels/ # Channel adapters - email/ # Resend, SendGrid, SES - push/ # APNs, FCM - sms/ # Twilio, Vonage - in-app/ # WebSocket - mcp/ # AI tool definitions - .beads/ # Issue tracking -``` - -## TDD Workflow - -All work follows strict Red-Green-Refactor: - -1. **[RED]** - Write failing tests first -2. **[GREEN]** - Minimal implementation to pass tests -3. **[REFACTOR]** - Clean up without changing behavior - -```bash -# Find next RED task -bd ready | grep RED - -# Claim and work -bd update customerio-xxx --status in_progress - -# After tests pass, close RED -bd close customerio-xxx - -# Move to GREEN (now unblocked) -bd ready | grep GREEN -``` - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -bd dep tree # View dependency tree -``` - -## Key APIs (Customer.io Compatible) - -### Track API -```typescript -// POST /v1/identify -await customerio.identify(userId, { email, name, plan }) - -// POST /v1/track -await customerio.track(userId, 'purchase', { amount: 99 }) - -// POST /v1/batch -await customerio.batch([...events]) -``` - -### Workflow API -```typescript -// POST /v1/workflows/:id/trigger -await customerio.workflows.trigger('onboarding', { - recipients: ['user_123'], - data: { plan: 'pro' } -}) -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds diff --git a/rewrites/customerio/README.md b/rewrites/customerio/README.md deleted file mode 100644 index 642e74f8..00000000 --- a/rewrites/customerio/README.md +++ /dev/null @@ -1,432 +0,0 @@ -# customerio.do - -> Marketing Automation. Edge-Native. Natural Language. AI-First. - -Customer.io charges $150/month for 12,000 profiles. Their API requires configuration objects, event schemas, and workflow builders. Every integration is a developer task. - -**customerio.do** is the open-source alternative. Natural language. Deploy in seconds. Talk to it like a marketer. - -## AI-Native API - -```typescript -import { customerio } from 'customerio.do' // Full SDK -import { customerio } from 'customerio.do/tiny' // Minimal client -import { customerio } from 'customerio.do/segments' // Segments-only -``` - -Natural language for marketing automation: - -```typescript -import { customerio } from 'customerio.do' - -// Talk to it like a colleague -await customerio`user-123 signed up from Google Ads` -await customerio`alice@example.com upgraded to Pro` -await customerio`send welcome sequence to users signed up today` - -// Chain like sentences -await customerio`users who abandoned cart` - .notify('Complete your purchase - 10% off!') - -// Campaigns that write themselves -await customerio`users inactive for 30 days` - .map(user => customerio`win back ${user}`) -``` - -## The Problem - -Customer.io dominates marketing automation alongside Braze and Iterable: - -| What Customer.io Charges | The Reality | -|--------------------------|-------------| -| **Essentials** | $100/month for 5,000 profiles | -| **Premium** | $1,000+/month | -| **Per-Message** | Overage charges on email/SMS | -| **Enterprise** | Custom pricing (read: expensive) | -| **Integration** | Developers write event tracking | -| **Vendor Lock-in** | Data trapped in their platform | - -### The Integration Tax - -Every marketing tool requires: - -- Schema definitions for events -- Code changes for tracking -- Developer time for campaigns -- Platform-specific syntax - -Marketers can't move without engineering help. - -### The Configuration Hell - -```typescript -// OLD: Customer.io SDK (verbose, developer-only) -await cio.identify('user_123', { - email: 'alice@example.com', - name: 'Alice', - plan: 'pro', - created_at: Date.now() -}) - -await cio.track('user_123', 'order_placed', { - order_id: 'order_456', - amount: 99.99, - items: [{ sku: 'widget-1', qty: 2 }] -}) -``` - -Nobody talks like that. A marketer would say: "Alice placed a $100 order." - -## The Solution - -**customerio.do** reimagines marketing automation: - -``` -Customer.io customerio.do ------------------------------------------------------------------ -$100-1000+/month $0 - run your own -Configuration objects Natural language -Developer-required Marketer-friendly -Proprietary workflows Cloudflare Workflows -Data trapped Your Cloudflare account -Weeks to integrate Deploy in seconds -``` - -## One-Click Deploy - -```bash -npx create-dotdo customerio -``` - -A complete marketing automation platform. Running on infrastructure you control. - -```typescript -import { CustomerIO } from 'customerio.do' - -export default CustomerIO({ - name: 'my-startup', - domain: 'marketing.my-startup.com', -}) -``` - -## Features - -### User Tracking - -```typescript -// Just describe what happened -await customerio`user-123 signed up from Google Ads` -await customerio`alice@example.com is on the Pro plan` -await customerio`bob viewed pricing page 3 times today` - -// AI infers what you need -await customerio`user-123` // returns user profile -await customerio`user-123 activity` // returns event history -await customerio`users from Twitter` // returns segment -``` - -### Events - -```typescript -// Natural event tracking -await customerio`alice placed a $99 order` -await customerio`bob abandoned cart with 3 items` -await customerio`user-123 used the export feature` - -// Batch events read like a log -await customerio` - alice upgraded to Pro - bob canceled subscription - charlie started trial -` -``` - -### Segments - -```typescript -// Query your audience like a database -await customerio`users who signed up this week` -await customerio`Pro users who haven't logged in for 30 days` -await customerio`users from Google Ads with no purchase` - -// Segments that update themselves -await customerio`active Pro users` - .count() // real-time membership - -await customerio`cart abandoners from yesterday` - .each(user => customerio`remind ${user} about cart`) -``` - -### Campaigns - -```typescript -// Trigger campaigns naturally -await customerio`send welcome email to alice@example.com` -await customerio`start onboarding sequence for new signups` -await customerio`notify Pro users about the new feature` - -// Multi-step journeys -await customerio`users who signed up today` - .notify('Welcome! Here is how to get started...') - .wait('3 days') - .notify('Have you tried our export feature?') - .wait('7 days') - .notify('Ready to upgrade to Pro?') -``` - -### Multi-Channel Delivery - -```typescript -// Just say what you want -await customerio`email alice about the sale` -await customerio`push notification to mobile users` -await customerio`SMS bob about order shipped` - -// Automatic fallback -await customerio`notify alice about new feature` - // Tries push, falls back to email, then SMS -``` - -### Templates - -```typescript -// AI writes personalized content -await customerio`welcome email for Pro users` - // Generates context-aware template - -// Or preview with data -await customerio`preview welcome email for alice` -``` - -## Promise Pipelining - -Chain operations without waiting: - -```typescript -// One network round trip -const campaigns = await customerio`users from Google Ads this week` - .map(user => customerio`send welcome sequence to ${user}`) - .map(result => customerio`track ${result} delivery status`) - -// Parallel outreach -await customerio`inactive Pro users` - .map(user => [ - customerio`email ${user} win-back offer`, - customerio`push ${user} reminder`, - ]) -``` - -## Architecture - -``` -Internet --> Cloudflare Worker --> Durable Object --> SQLite - | | | - Edge Routing Customer Data User Profiles - Event History Segments - Campaign State Preferences -``` - -### Durable Object per Workspace - -``` -CustomerIODO (config, channels, templates) - | - +-- UsersDO (profiles, attributes) - | |-- SQLite: User records - | +-- R2: Profile data - | - +-- EventsDO (tracking, history) - | |-- SQLite: Event stream - | +-- R2: Event archive - | - +-- SegmentsDO (audiences, rules) - | |-- SQLite: Segment definitions - | - +-- CampaignsDO (journeys, workflows) - |-- SQLite: Campaign state - +-- CF Workflows: Journey engine -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Active users, recent events | <10ms | -| **Warm** | R2 + SQLite Index | Historical events (30-90 days) | <100ms | -| **Cold** | R2 Archive | Compliance retention (1+ years) | <1s | - -## vs Customer.io - -| Feature | Customer.io | customerio.do | -|---------|-------------|---------------| -| **Pricing** | $100-1000+/month | ~$10/month | -| **API Style** | Configuration objects | Natural language | -| **Setup** | Developer required | Marketer-friendly | -| **Workflows** | Proprietary builder | Cloudflare Workflows | -| **Data Location** | Their cloud | Your Cloudflare account | -| **Customization** | Limited | Code it yourself | -| **Lock-in** | Data export fees | MIT licensed | - -## Use Cases - -### Product-Led Growth - -```typescript -// Trial conversion automation -await customerio`users on day 7 of trial who haven't activated` - .notify('Need help getting started?') - .wait('3 days') - .notify('Your trial ends soon - upgrade now for 20% off') - -// Feature adoption -await customerio`Pro users who never used export` - .notify('Did you know you can export your data?') -``` - -### E-Commerce - -```typescript -// Cart recovery -await customerio`cart abandoners in the last hour` - .wait('1 hour') - .notify('You left something behind!') - .wait('24 hours') - .notify('Still thinking about it? Here is 10% off') - -// Post-purchase -await customerio`customers who ordered today` - .notify('Your order is confirmed!') - .wait('3 days') - .notify('How was your experience? Leave a review') -``` - -### SaaS Onboarding - -```typescript -// Welcome sequence -await customerio`new signups` - .notify('Welcome! Here is how to get started') - .wait('1 day') - .notify('Have you completed your first project?') - .wait('3 days') - .notify('Meet the features that power users love') - -// Re-engagement -await customerio`users inactive for 30 days` - .notify('We miss you! Here is what is new') -``` - -## AI-Native Marketing - -### Audience Discovery - -```typescript -// AI finds your best audiences -await customerio`users most likely to convert` -await customerio`customers at risk of churning` -await customerio`users who would benefit from Pro` -``` - -### Content Generation - -```typescript -// AI writes the copy -await customerio`write win-back email for churned Pro users` -await customerio`generate subject lines for cart abandonment` -await customerio`personalize welcome email for alice` -``` - -### Campaign Optimization - -```typescript -// AI optimizes timing and content -await customerio`best time to email alice` -await customerio`which subject line performs better for Pro users` -await customerio`optimize cart abandonment sequence` -``` - -## Multi-Agent Marketing - -Every AI agent gets their own marketing instance: - -```typescript -import { mark, sally } from 'agents.do' -import { customerio } from 'customerio.do' - -// Mark handles marketing campaigns -await mark`announce the new feature to Pro users` - // Uses customerio under the hood - -// Sally handles sales outreach -await sally`reach out to trial users about to expire` - // Different campaigns, same platform -``` - -## The workers.do Platform - -customerio.do is a core service of [workers.do](https://workers.do) - the platform for building Autonomous Startups. - -```typescript -import { priya, ralph, tom, mark } from 'agents.do' - -// AI agents with marketing built in -await mark`launch announcement for AI Assistant` -await sally`follow up with leads from the webinar` -await priya`notify beta users about the new feature` -``` - -Both kinds of workers. Working for you. - -## Roadmap - -### Core Features -- [x] User Profiles -- [x] Event Tracking -- [x] Dynamic Segments -- [x] Multi-Channel Delivery -- [x] Campaign Workflows -- [ ] A/B Testing -- [ ] Predictive Analytics - -### Channels -- [x] Email (via Resend/Postmark) -- [x] Push Notifications -- [x] SMS (via Twilio) -- [x] In-App Messages -- [ ] WhatsApp -- [ ] Slack - -### AI -- [x] Natural Language Queries -- [x] Content Generation -- [ ] Send Time Optimization -- [ ] Churn Prediction -- [ ] Audience Discovery - -## Contributing - -customerio.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/customerio.do -cd customerio.do -pnpm install -pnpm test -``` - -## License - -MIT License - Marketing automation for everyone. - ---- - -

- Talk to your marketing platform. -
- Natural language. Edge-native. AI-first. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/databricks/.beads/.gitignore b/rewrites/databricks/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/databricks/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/databricks/.beads/README.md b/rewrites/databricks/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/databricks/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/databricks/.beads/config.yaml b/rewrites/databricks/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/databricks/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/databricks/.beads/interactions.jsonl b/rewrites/databricks/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/databricks/.beads/metadata.json b/rewrites/databricks/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/databricks/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/databricks/.gitattributes b/rewrites/databricks/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/databricks/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/databricks/AGENTS.md b/rewrites/databricks/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/databricks/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/databricks/README.md b/rewrites/databricks/README.md deleted file mode 100644 index 6410e15d..00000000 --- a/rewrites/databricks/README.md +++ /dev/null @@ -1,391 +0,0 @@ -# databricks.do - -> The $62B Data Lakehouse. Now open source. AI-native. Zero complexity. - -Databricks built a $62B empire on Apache Spark. Unity Catalog costs $6/DBU. Serverless SQL warehouses start at $0.22/DBU. MLflow "Enterprise" requires premium tiers. A 50-person data team easily spends $500K-$1M/year. - -**databricks.do** is the open-source alternative. Lakehouse architecture on Cloudflare. Delta tables backed by R2. SQL warehouses at the edge. ML pipelines without the bill. - -## AI-Native API - -```typescript -import { databricks } from 'databricks.do' // Full SDK -import { databricks } from 'databricks.do/tiny' // Minimal client -import { databricks } from 'databricks.do/sql' // SQL-only operations -``` - -Natural language for data workflows: - -```typescript -import { databricks } from 'databricks.do' - -// Talk to it like a colleague -const revenue = await databricks`sales by region last quarter` -const top = await databricks`top 10 customers by revenue` -const trend = await databricks`monthly revenue trend this year` - -// Chain like sentences -await databricks`customers who churned last month` - .map(c => databricks`analyze churn reasons for ${c}`) - -// Pipelines that build themselves -await databricks`run sales ETL daily at 6am` - .monitor() // AI watches for anomalies - .alert() // notify on failures -``` - -## The Problem - -Databricks has become synonymous with "enterprise data platform": - -| The Databricks Tax | Reality | -|--------------------|---------| -| **Unity Catalog** | $6/DBU premium + compute costs | -| **Serverless SQL** | $0.22-$0.70/DBU depending on tier | -| **MLflow "Enterprise"** | Requires premium workspace | -| **Jobs compute** | $0.15-$0.40/DBU depending on tier | -| **Model serving** | $0.07/1000 requests | -| **Total platform** | $500K-$1M+/year for serious data teams | - -### The DBU Tax - -The real cost of Databricks: - -- Costs explode when workloads scale -- Proprietary Unity Catalog lock-in -- Databricks-specific skills (not transferable) -- Cross-cloud data sharing is expensive -- Simple SQL hidden behind Spark complexity - -Meanwhile, Databricks' core value - **unified analytics on a lakehouse** - is a storage and compute coordination problem. Edge compute with columnar storage solves this better. - -## The Solution - -**databricks.do** reimagines the lakehouse for data engineers: - -``` -Databricks databricks.do ------------------------------------------------------------------ -$500K-$1M+/year Deploy in minutes -$6/DBU Unity Catalog $0 - open source -Spark required SQL-first -Cloud-specific Edge-native -AI as premium feature AI at the core -DBU-based billing Pay for actual compute -``` - -## One-Click Deploy - -```bash -npx create-dotdo databricks -``` - -Your own Lakehouse. Running on infrastructure you control. AI-native from day one. - -```typescript -import { Databricks } from 'databricks.do' - -export default Databricks({ - name: 'company-lakehouse', - domain: 'data.company.com', - storage: 'r2', -}) -``` - -## Features - -### Data Catalog - -```typescript -// Just say what you need -await databricks`create production catalog` -await databricks`create sales schema in production` -await databricks`give analysts read access to sales` - -// AI infers what you need -await databricks`production.sales` // returns schema info -await databricks`lineage for orders` // returns data lineage -await databricks`who can access sales?` // returns permissions -``` - -### Delta Tables - -```typescript -// Create tables naturally -await databricks`create orders table with order_id customer_id amount status` -await databricks`partition orders by date` -await databricks`add order_id 1 customer 100 amount 99.99 to orders` - -// Time travel is one word -await databricks`orders as of yesterday` -await databricks`orders version 42` -await databricks`orders changes since last week` - -// Merges read like English -await databricks`upsert staging_orders into orders on order_id` -``` - -### SQL Analytics - -```typescript -// Just ask questions -await databricks`monthly revenue this year` -await databricks`customers ranked by spend` -await databricks`cohort analysis for Q1 signups` - -// Complex analysis, simple words -await databricks`customer lifetime value by acquisition channel` -await databricks`revenue attribution by marketing touchpoint` -await databricks`funnel conversion rates by segment` -``` - -### MLflow - -```typescript -// Experiments without the ceremony -await databricks`track churn prediction experiment` -await databricks`log accuracy 0.89 precision 0.85 recall 0.92` -await databricks`register churn model v1` - -// Deploy with one line -await databricks`deploy churn model to production` -await databricks`predict churn for customer 12345` - -// AI manages the lifecycle -await databricks`what's the best performing churn model?` -await databricks`compare model v1 vs v2` -``` - -### Notebooks - -```typescript -// Run analysis like you'd describe it -await databricks`run sales analysis notebook` -await databricks`schedule daily report at 8am Pacific` - -// AI writes the code -await databricks`analyze sales trends and visualize as line chart` -await databricks`create dashboard for executive summary` -``` - -### SQL Warehouses - -```typescript -// Warehouses that manage themselves -await databricks`create analytics warehouse auto-stop 15 min` -await databricks`scale warehouse to handle Black Friday traffic` - -// Stream large results naturally -await databricks`stream all events from last month` - .each(batch => process(batch)) -``` - -### ETL Pipelines - -```typescript -// Pipelines in plain English -await databricks`create sales ETL bronze silver gold` -await databricks`run sales pipeline` -await databricks`backfill orders from January` - -// Quality expectations built-in -await databricks`fail if order_id is null` -await databricks`drop rows where amount <= 0` -``` - -## Promise Pipelining - -Chain operations without callback hell: - -```typescript -// Build an ML pipeline with one chain -const deployed = await databricks`load customer transactions` - .map(data => databricks`clean and validate ${data}`) - .map(clean => databricks`engineer churn features from ${clean}`) - .map(features => databricks`train XGBoost on ${features}`) - .map(model => databricks`evaluate ${model}`) - .map(evaluated => databricks`deploy if metrics pass`) - -// One network round trip. Record-replay pipelining. -``` - -### AI Agents as Data Engineers - -```typescript -import { priya, ralph, tom, quinn } from 'agents.do' - -// The whole team, naturally -const spec = await priya`design customer 360 data model` -const pipeline = await ralph`implement ${spec} with DLT` -const review = await tom`review ${pipeline} architecture` -const tests = await quinn`create quality tests for ${pipeline}` - -// Chain the team -await priya`plan Q1 data roadmap` - .map(plan => ralph`implement ${plan}`) - .map(code => [priya, tom, quinn].map(r => r`review ${code}`)) -``` - -### AI-Powered Data Quality - -```typescript -// Monitor automatically -await databricks`monitor orders for anomalies` -await databricks`alert if row count drops 20%` -await databricks`explain why orders dropped yesterday` -// "Row count dropped 35% - source API returned errors from 2-4 AM UTC. -// 12,453 orders failed to ingest. Re-run ingestion for affected window." -``` - -## Architecture - -### Durable Object per Workspace - -``` -LakehouseDO (config, users, catalogs) - | - +-- CatalogDO (schemas, tables, permissions) - | |-- SQLite: Metadata (encrypted) - | +-- R2: Delta tables (Parquet) - | - +-- WarehouseDO (queries, cache) - | |-- SQLite: Query state - | +-- Query engine - | - +-- MLflowDO (experiments, models) - | |-- SQLite: Tracking data - | +-- R2: Artifacts - | - +-- PipelineDO (ETL, schedules) - |-- SQLite: Pipeline state -``` - -### Storage Tiers - -| Tier | Storage | Use Case | Query Speed | -|------|---------|----------|-------------| -| **Hot** | SQLite | Metadata, recent queries | <10ms | -| **Warm** | R2 Parquet | Delta tables, features | <100ms | -| **Cold** | R2 Archive | Historical versions, audit | <1s | - -## vs Databricks - -| Feature | Databricks | databricks.do | -|---------|------------|---------------| -| **Implementation** | DBU billing complexity | Deploy in minutes | -| **Annual Cost** | $500K-$1M+ | ~$50/month | -| **Architecture** | Spark clusters | Edge-native, global | -| **Data Catalog** | $6/DBU premium | Included | -| **AI** | Premium add-ons | AI-first design | -| **Data Location** | Databricks managed | Your Cloudflare account | -| **Lock-in** | Proprietary | MIT licensed | - -## Use Cases - -### Data Engineering - -```typescript -// Streaming pipelines in plain English -await databricks`stream events from Kafka to bronze` -await databricks`clean bronze events to silver` -await databricks`aggregate silver to gold metrics` - -// Backfills are one line -await databricks`backfill orders from last month` -``` - -### Data Science - -```typescript -// ML without the ceremony -await databricks`train churn model on customer features` -await databricks`tune hyperparameters for best accuracy` -await databricks`deploy if accuracy > 0.9` - -// Feature engineering -await databricks`create customer lifetime value features` -await databricks`store features in feature store` -``` - -### Business Intelligence - -```typescript -// Dashboards from questions -await databricks`create executive dashboard with revenue and customers` -await databricks`add filter for date range and region` -await databricks`schedule refresh daily at 6am` -``` - -### Migration from Databricks - -```typescript -// One-liner migration -await databricks`import from cloud.databricks.com` - -// Or step by step -await databricks`import catalog production from databricks` -await databricks`import notebooks from /workspace` -await databricks`import mlflow experiments` -``` - -## Deployment Options - -### Cloudflare Workers - -```bash -npx create-dotdo databricks -# Deploys to your Cloudflare account -``` - -### Private Cloud - -```bash -docker run -p 8787:8787 dotdo/databricks -``` - -## Roadmap - -### Core Lakehouse -- [x] Unity Catalog (catalogs, schemas, tables) -- [x] Delta Lake tables with ACID -- [x] SQL Warehouse queries -- [x] MLflow tracking and registry -- [x] Natural language queries -- [ ] DLT Pipelines (full) -- [ ] Real-time streaming -- [ ] Vector search - -### AI -- [x] Natural language to SQL -- [x] AI data quality monitoring -- [ ] AutoML integration -- [ ] Feature store -- [ ] Model serving - -## Contributing - -databricks.do is open source under the MIT license. - -```bash -git clone https://github.com/dotdo/databricks.do -cd databricks.do -pnpm install -pnpm test -``` - -## License - -MIT License - For the data democratizers. - ---- - -

- The $62B lakehouse ends here. -
- SQL-first. AI-native. No DBU pricing. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/datadog/.beads/.gitignore b/rewrites/datadog/.beads/.gitignore deleted file mode 100644 index 4a7a77df..00000000 --- a/rewrites/datadog/.beads/.gitignore +++ /dev/null @@ -1,39 +0,0 @@ -# SQLite databases -*.db -*.db?* -*.db-journal -*.db-wal -*.db-shm - -# Daemon runtime files -daemon.lock -daemon.log -daemon.pid -bd.sock -sync-state.json -last-touched - -# Local version tracking (prevents upgrade notification spam after git ops) -.local_version - -# Legacy database files -db.sqlite -bd.db - -# Worktree redirect file (contains relative path to main repo's .beads/) -# Must not be committed as paths would be wrong in other clones -redirect - -# Merge artifacts (temporary files from 3-way merge) -beads.base.jsonl -beads.base.meta.json -beads.left.jsonl -beads.left.meta.json -beads.right.jsonl -beads.right.meta.json - -# NOTE: Do NOT add negation patterns (e.g., !issues.jsonl) here. -# They would override fork protection in .git/info/exclude, allowing -# contributors to accidentally commit upstream issue databases. -# The JSONL files (issues.jsonl, interactions.jsonl) and config files -# are tracked by git by default since no pattern above ignores them. diff --git a/rewrites/datadog/.beads/README.md b/rewrites/datadog/.beads/README.md deleted file mode 100644 index 50f281f0..00000000 --- a/rewrites/datadog/.beads/README.md +++ /dev/null @@ -1,81 +0,0 @@ -# Beads - AI-Native Issue Tracking - -Welcome to Beads! This repository uses **Beads** for issue tracking - a modern, AI-native tool designed to live directly in your codebase alongside your code. - -## What is Beads? - -Beads is issue tracking that lives in your repo, making it perfect for AI coding agents and developers who want their issues close to their code. No web UI required - everything works through the CLI and integrates seamlessly with git. - -**Learn more:** [github.com/steveyegge/beads](https://github.com/steveyegge/beads) - -## Quick Start - -### Essential Commands - -```bash -# Create new issues -bd create "Add user authentication" - -# View all issues -bd list - -# View issue details -bd show - -# Update issue status -bd update --status in_progress -bd update --status done - -# Sync with git remote -bd sync -``` - -### Working with Issues - -Issues in Beads are: -- **Git-native**: Stored in `.beads/issues.jsonl` and synced like code -- **AI-friendly**: CLI-first design works perfectly with AI coding agents -- **Branch-aware**: Issues can follow your branch workflow -- **Always in sync**: Auto-syncs with your commits - -## Why Beads? - -✨ **AI-Native Design** -- Built specifically for AI-assisted development workflows -- CLI-first interface works seamlessly with AI coding agents -- No context switching to web UIs - -🚀 **Developer Focused** -- Issues live in your repo, right next to your code -- Works offline, syncs when you push -- Fast, lightweight, and stays out of your way - -🔧 **Git Integration** -- Automatic sync with git commits -- Branch-aware issue tracking -- Intelligent JSONL merge resolution - -## Get Started with Beads - -Try Beads in your own projects: - -```bash -# Install Beads -curl -sSL https://raw.githubusercontent.com/steveyegge/beads/main/scripts/install.sh | bash - -# Initialize in your repo -bd init - -# Create your first issue -bd create "Try out Beads" -``` - -## Learn More - -- **Documentation**: [github.com/steveyegge/beads/docs](https://github.com/steveyegge/beads/tree/main/docs) -- **Quick Start Guide**: Run `bd quickstart` -- **Examples**: [github.com/steveyegge/beads/examples](https://github.com/steveyegge/beads/tree/main/examples) - ---- - -*Beads: Issue tracking that moves at the speed of thought* ⚡ diff --git a/rewrites/datadog/.beads/config.yaml b/rewrites/datadog/.beads/config.yaml deleted file mode 100644 index f2427856..00000000 --- a/rewrites/datadog/.beads/config.yaml +++ /dev/null @@ -1,62 +0,0 @@ -# Beads Configuration File -# This file configures default behavior for all bd commands in this repository -# All settings can also be set via environment variables (BD_* prefix) -# or overridden with command-line flags - -# Issue prefix for this repository (used by bd init) -# If not set, bd init will auto-detect from directory name -# Example: issue-prefix: "myproject" creates issues like "myproject-1", "myproject-2", etc. -# issue-prefix: "" - -# Use no-db mode: load from JSONL, no SQLite, write back after each command -# When true, bd will use .beads/issues.jsonl as the source of truth -# instead of SQLite database -# no-db: false - -# Disable daemon for RPC communication (forces direct database access) -# no-daemon: false - -# Disable auto-flush of database to JSONL after mutations -# no-auto-flush: false - -# Disable auto-import from JSONL when it's newer than database -# no-auto-import: false - -# Enable JSON output by default -# json: false - -# Default actor for audit trails (overridden by BD_ACTOR or --actor) -# actor: "" - -# Path to database (overridden by BEADS_DB or --db) -# db: "" - -# Auto-start daemon if not running (can also use BEADS_AUTO_START_DAEMON) -# auto-start-daemon: true - -# Debounce interval for auto-flush (can also use BEADS_FLUSH_DEBOUNCE) -# flush-debounce: "5s" - -# Git branch for beads commits (bd sync will commit to this branch) -# IMPORTANT: Set this for team projects so all clones use the same sync branch. -# This setting persists across clones (unlike database config which is gitignored). -# Can also use BEADS_SYNC_BRANCH env var for local override. -# If not set, bd sync will require you to run 'bd config set sync.branch '. -# sync-branch: "beads-sync" - -# Multi-repo configuration (experimental - bd-307) -# Allows hydrating from multiple repositories and routing writes to the correct JSONL -# repos: -# primary: "." # Primary repo (where this database lives) -# additional: # Additional repos to hydrate from (read-only) -# - ~/beads-planning # Personal planning repo -# - ~/work-planning # Work planning repo - -# Integration settings (access with 'bd config get/set') -# These are stored in the database, not in this file: -# - jira.url -# - jira.project -# - linear.url -# - linear.api-key -# - github.org -# - github.repo diff --git a/rewrites/datadog/.beads/interactions.jsonl b/rewrites/datadog/.beads/interactions.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/datadog/.beads/issues.jsonl b/rewrites/datadog/.beads/issues.jsonl deleted file mode 100644 index e69de29b..00000000 diff --git a/rewrites/datadog/.beads/metadata.json b/rewrites/datadog/.beads/metadata.json deleted file mode 100644 index c787975e..00000000 --- a/rewrites/datadog/.beads/metadata.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "database": "beads.db", - "jsonl_export": "issues.jsonl" -} \ No newline at end of file diff --git a/rewrites/datadog/.gitattributes b/rewrites/datadog/.gitattributes deleted file mode 100644 index 807d5983..00000000 --- a/rewrites/datadog/.gitattributes +++ /dev/null @@ -1,3 +0,0 @@ - -# Use bd merge for beads JSONL files -.beads/issues.jsonl merge=beads diff --git a/rewrites/datadog/AGENTS.md b/rewrites/datadog/AGENTS.md deleted file mode 100644 index df7a4af9..00000000 --- a/rewrites/datadog/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Agent Instructions - -This project uses **bd** (beads) for issue tracking. Run `bd onboard` to get started. - -## Quick Reference - -```bash -bd ready # Find available work -bd show # View issue details -bd update --status in_progress # Claim work -bd close # Complete work -bd sync # Sync with git -``` - -## Landing the Plane (Session Completion) - -**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds. - -**MANDATORY WORKFLOW:** - -1. **File issues for remaining work** - Create issues for anything that needs follow-up -2. **Run quality gates** (if code changed) - Tests, linters, builds -3. **Update issue status** - Close finished work, update in-progress items -4. **PUSH TO REMOTE** - This is MANDATORY: - ```bash - git pull --rebase - bd sync - git push - git status # MUST show "up to date with origin" - ``` -5. **Clean up** - Clear stashes, prune remote branches -6. **Verify** - All changes committed AND pushed -7. **Hand off** - Provide context for next session - -**CRITICAL RULES:** -- Work is NOT complete until `git push` succeeds -- NEVER stop before pushing - that leaves work stranded locally -- NEVER say "ready to push when you are" - YOU must push -- If push fails, resolve and retry until it succeeds - diff --git a/rewrites/datadog/README.md b/rewrites/datadog/README.md deleted file mode 100644 index 606e06e1..00000000 --- a/rewrites/datadog/README.md +++ /dev/null @@ -1,519 +0,0 @@ -# datadog.do - -> The $18B observability platform. Now open source. AI-native. - -Datadog dominates modern observability. But at $15-34/host/month for infrastructure, $1.27/GB for logs, and custom metrics that can cost $100k+/year, bills that regularly hit 7 figures, it's time for a new approach. - -**datadog.do** reimagines observability for the AI era. Full APM. Unlimited logs. Zero per-host pricing. - -## AI-Native API - -```typescript -import { datadog } from 'datadog.do' // Full SDK -import { datadog } from 'datadog.do/tiny' // Minimal client -import { datadog } from 'datadog.do/metrics' // Metrics-only operations -``` - -Natural language for observability: - -```typescript -import { datadog } from 'datadog.do' - -// Talk to it like an SRE -const health = await datadog`production health right now` -const errors = await datadog`errors in production last hour` -const slow = await datadog`slowest endpoints this week` - -// Chain like sentences -await datadog`services with high error rate` - .map(service => datadog`root cause for ${service}`) - .map(cause => datadog`recommended fix for ${cause}`) - -// Incidents that investigate themselves -await datadog`investigate the 3am outage` - .correlate() // cross-reference logs, traces, metrics - .timeline() // build incident timeline - .report() // generate post-mortem -``` - -## The Problem - -Datadog built an observability empire on: - -| What Datadog Charges | The Reality | -|----------------------|-------------| -| **Per-host pricing** | $15/host (Infra), $31/host (APM) | -| **Per-GB logs** | $0.10/GB ingested, $1.27/GB indexed | -| **Custom metrics** | $0.05/metric/month, scales to $100k+ | -| **Retention** | Historical data requires expensive plans | -| **Fragmentation** | APM, Logs, Metrics, RUM all priced separately | -| **Unpredictable** | Usage spikes = surprise invoices | - -### The Observability Tax - -A 500-host infrastructure with logs and APM? **$300k+/year**. With custom metrics and long retention? **$500k+**. - -### The Dashboard Maze - -When production is down: -- Navigate five dashboards -- Write three queries -- Correlate manually -- Every minute costs money - -### The Alert Fatigue - -- Too many alerts = ignored alerts -- Too few alerts = missed incidents -- Threshold tuning is a full-time job -- AI "assistants" add complexity, not clarity - -## The Solution - -**datadog.do** reimagines observability: - -``` -Traditional Datadog datadog.do ------------------------------------------------------------------ -$15-34/host/month $0 - run your own -$1.27/GB logs Unlimited (R2 storage costs) -$0.05/custom metric Unlimited metrics -15-day log retention Unlimited retention -Unpredictable bills Predictable costs -Vendor lock-in Open source, your data -``` - -## One-Click Deploy - -```bash -npx create-dotdo datadog -``` - -Your own Datadog. Running on Cloudflare. No per-host fees. - -```typescript -import { Datadog } from 'datadog.do' - -export default Datadog({ - name: 'acme-observability', - domain: 'monitor.acme.io', - retention: { - metrics: '90d', - logs: '365d', - traces: '30d', - }, -}) -``` - -## Features - -### Metrics - -```typescript -// Query metrics naturally -await datadog`CPU usage for web servers last 24 hours` -await datadog`memory trend for api-gateway this week` -await datadog`disk usage above 80%` - -// AI infers what you need -await datadog`web servers` // returns host list -await datadog`web servers CPU` // returns metrics -await datadog`web servers CPU trending up` // returns analysis -``` - -### Logs - -```typescript -// Search logs naturally -await datadog`logs containing "timeout" in production` -await datadog`errors from api-gateway last hour` -await datadog`payment failures today` - -// Chain for investigation -await datadog`errors spiking in checkout` - .each(error => error.trace()) // get related traces -``` - -### Traces - -```typescript -// Distributed tracing in plain English -await datadog`slow requests to /api/users` -await datadog`traces with errors in payment-service` -await datadog`p99 latency for checkout flow` - -// Follow the request path -await datadog`trace the slow request from user-123` - .visualize() // flame graph -``` - -### Alerting - -```typescript -// Alerts as sentences -await datadog`alert when CPU > 90% for 5 minutes` -await datadog`alert when error rate > 5% for 5 minutes` -await datadog`alert when p99 latency > 2 seconds` -await datadog`alert when disk usage > 80%` - -// Smart alerts -await datadog`alert when traffic is anomalous` -await datadog`alert when checkout errors spike` - -// Composite alerts -await datadog`alert when high CPU and high error rate together` -``` - -### Dashboards - -```typescript -// Create dashboards naturally -await datadog`dashboard for production infrastructure` -await datadog`dashboard for api-gateway performance` -await datadog`dashboard for checkout flow` - -// Query dashboards -await datadog`show me the infrastructure dashboard` -``` - -### Infrastructure - -```typescript -// Host and container monitoring -await datadog`hosts with high CPU` -await datadog`containers in production namespace` -await datadog`pods restarting in kubernetes` - -// Cloud resources -await datadog`cloudflare workers by region` -await datadog`r2 buckets by size` -await datadog`d1 databases by query count` -``` - -### APM - -```typescript -// Application performance in plain English -await datadog`services with highest error rate` -await datadog`slowest database queries` -await datadog`api-gateway dependencies` - -// Service health -await datadog`is checkout service healthy?` -await datadog`what's blocking payment-service?` -``` - -### RUM - -```typescript -// Real user monitoring -await datadog`page load times this week` -await datadog`javascript errors on checkout page` -await datadog`users affected by slow performance` - -// User journeys -await datadog`users dropping off at checkout` -await datadog`conversion rate by browser` -``` - -### Synthetic Monitoring - -```typescript -// Proactive testing as sentences -await datadog`check api.acme.com/health every minute` -await datadog`test login flow from all regions` -await datadog`verify checkout completes under 3 seconds` -``` - -## AI-Native Observability - -### Incident Response - -```typescript -// Ask questions about your infrastructure -await datadog`what's wrong with production right now?` -await datadog`why is the API slow?` -await datadog`what changed before the errors started?` - -// Diagnostic questions -await datadog`root cause of the 3am outage` -await datadog`how does this week compare to last week?` -await datadog`what's different about the slow requests?` -``` - -### Watchdog (AI Detection) - -```typescript -// AI finds anomalies automatically -await datadog`enable watchdog for production services` -await datadog`watchdog alerts last 24 hours` -await datadog`explain the latency anomaly in api-gateway` - -// AI correlates across signals -await datadog`correlate error spike with recent deployments` -await datadog`what else changed when checkout broke?` -``` - -### AI Agents as SREs - -```typescript -import { tom, quinn } from 'agents.do' -import { datadog } from 'datadog.do' - -// Tech lead investigates incident -await tom`investigate the elevated error rate in production` - .using(datadog) - .report() - -// QA validates fix -await quinn`verify the deployment fixed the issue` - .compare('error rate before and after') - -// Chain investigation into resolution -await datadog`services with errors right now` - .map(service => tom`diagnose ${service} and suggest fix`) - .map(fix => quinn`test that ${fix} resolves the issue`) -``` - -## Architecture - -### Data Collection - -``` -Applications/Hosts - | - v -+---------------+ -| dd-agent | (Open-source agent) -| Worker | -+---------------+ - | - v -+---------------+ -| Edge Worker | (Ingest + Route) -+---------------+ - | - +---+---+---+ - | | | | - v v v v -Metrics Logs Traces Events -``` - -### Durable Objects - -``` - +------------------------+ - | datadog.do Worker | - | (API + Ingest) | - +------------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | MetricsDO | | LogsDO | | TracesDO | - | (Time Series) | | (Log Store) | | (Span Store) | - +------------------+ +------------------+ +------------------+ - | | | - +---------------+---------------+ - | - +--------------------+--------------------+ - | | | - +----------+ +-----------+ +------------+ - | Analytics| | R2 | | D1 | - | Engine | | (Archive) | | (Metadata) | - +----------+ +-----------+ +------------+ -``` - -### Storage Tiers - -- **Hot (Analytics Engine)** - Real-time metrics, last 24h of logs -- **Warm (SQLite/D1)** - Indexed logs, trace metadata, dashboards -- **Cold (R2)** - Log archives, trace archives, long-term metrics - -### Query Engine - -```typescript -// Natural language queries compile to DQL -await datadog`average CPU across web servers` -// -> avg:system.cpu.user{role:web} by {host} - -await datadog`requests per second in production` -// -> sum:app.requests{env:prod}.as_count().rollup(sum, 60) - -await datadog`slowest 10 endpoints` -// -> top(avg:app.latency{*} by {endpoint}, 10, mean) -``` - -## Agent Installation - -### Docker - -```bash -docker run -d \ - -v /var/run/docker.sock:/var/run/docker.sock:ro \ - -v /proc/:/host/proc/:ro \ - -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \ - -e DD_API_KEY=your-api-key \ - -e DD_SITE=your-org.datadog.do \ - datadog.do/agent:latest -``` - -### Kubernetes - -```yaml -apiVersion: apps/v1 -kind: DaemonSet -metadata: - name: datadog-agent -spec: - template: - spec: - containers: - - name: agent - image: datadog.do/agent:latest - env: - - name: DD_API_KEY - valueFrom: - secretKeyRef: - name: datadog - key: api-key - - name: DD_SITE - value: your-org.datadog.do -``` - -### Cloudflare Workers - -```typescript -import { datadog } from 'datadog.do' - -// Auto-instrumentation - just wrap your worker -export default datadog.instrument({ - async fetch(request, env, ctx) { - // Your worker code - // Traces, logs, metrics collected automatically - }, -}) -``` - -## Migration from Datadog - -### Agent Compatibility - -Point your existing agent to datadog.do: - -```bash -DD_SITE=your-org.datadog.do DD_API_KEY=your-key datadog-agent run -``` - -### Dashboard Migration - -```typescript -// Migrate dashboards with one command -await datadog`import dashboards from datadog` -await datadog`import monitors from datadog` -await datadog`import synthetic tests from datadog` -``` - -Or use the CLI: - -```bash -npx datadog-migrate import --from-datadog --all -``` - -## Integrations - -```typescript -// Enable integrations naturally -await datadog`connect to AWS` -await datadog`connect to Cloudflare` -await datadog`connect to PostgreSQL` - -// Or specify details -await datadog`monitor kubernetes cluster production` -await datadog`track GitHub Actions deployments` -``` - -Pre-built integrations for common services: - -| Category | Integrations | -|----------|-----------| -| **Cloud** | AWS, GCP, Azure, Cloudflare | -| **Containers** | Docker, Kubernetes, ECS | -| **Databases** | PostgreSQL, MySQL, Redis, MongoDB | -| **Web** | Nginx, Apache, HAProxy | -| **Languages** | Node.js, Python, Go, Java, Ruby | -| **CI/CD** | GitHub Actions, GitLab CI, Jenkins | - -## vs Datadog - -| Feature | Datadog | datadog.do | -|---------|---------|------------| -| **Infrastructure** | $15/host/month | $0 - run your own | -| **APM** | $31/host/month | $0 - run your own | -| **Logs** | $1.27/GB indexed | Unlimited (R2 storage) | -| **Custom metrics** | $0.05/metric/month | Unlimited | -| **Retention** | 15 days (logs) | Unlimited | -| **Data location** | Datadog's cloud | Your Cloudflare account | -| **Query language** | DQL | Natural language + DQL | -| **AI features** | Watchdog | AI-native from day one | -| **Lock-in** | Proprietary | MIT licensed | - -## Roadmap - -- [x] Metrics collection (agent) -- [x] Log management -- [x] APM (distributed tracing) -- [x] Dashboards -- [x] Alerting (monitors) -- [x] AI anomaly detection (Watchdog) -- [ ] RUM (browser SDK) -- [ ] Synthetic monitoring -- [ ] Security monitoring -- [ ] Network monitoring -- [ ] Database monitoring -- [ ] CI visibility - -## Why Open Source? - -Observability shouldn't cost millions: - -1. **Your metrics** - Infrastructure data is critical for operations -2. **Your logs** - Log data retention shouldn't cost per-GB -3. **Your traces** - APM shouldn't require per-host licensing -4. **Your alerts** - Monitoring is too important to be vendor-locked - -Datadog showed the world what modern observability could be. **datadog.do** makes it accessible to everyone. - -## Contributing - -datadog.do is open source under the MIT license. - -We especially welcome contributions from: -- SREs and DevOps engineers -- Observability experts -- AI/ML engineers -- Infrastructure engineers - -```bash -git clone https://github.com/dotdo/datadog.do -cd datadog.do -pnpm install -pnpm test -``` - -## License - -MIT License - Monitor everything. Alert on anything. Pay for storage, not seats. - ---- - -

- The $18B valuation ends here. -
- AI-native. Open source. Your data. -

- Website | - Docs | - Discord | - GitHub -

diff --git a/rewrites/docs/research/analytics-cdp-scope.md b/rewrites/docs/research/analytics-cdp-scope.md deleted file mode 100644 index 417621f9..00000000 --- a/rewrites/docs/research/analytics-cdp-scope.md +++ /dev/null @@ -1,755 +0,0 @@ -# Analytics & CDP Rewrites - Scoping Document - -**Date**: 2026-01-07 -**Author**: Research Phase -**Status**: Draft Scope - ---- - -## Executive Summary - -This document evaluates five analytics/CDP platforms for Cloudflare Workers rewrites: - -| Platform | Verdict | Effort | Edge Synergy | -|----------|---------|--------|--------------| -| **Segment** | **RECOMMENDED** | Medium | High | -| **RudderStack** | **RECOMMENDED** | Medium | Very High | -| **Mixpanel** | Possible | High | Medium | -| **Amplitude** | Possible | High | Medium | -| **Heap** | Not Recommended | Very High | Low | - -**Top Recommendations**: -1. `segment.do` - Event routing CDP with identity resolution -2. `analytics.do` - Unified analytics ingestion (RudderStack-inspired) - ---- - -## Platform Analysis - -### 1. Segment (Twilio) - -**Core Value Proposition**: Customer Data Platform that collects events from any source and routes them to 300+ destinations. Single API, many outputs. - -**Key APIs/Features**: -- Track API: Record user actions with properties -- Identify API: Link user traits to profiles -- Page/Screen API: Track page views -- Group API: Associate users with organizations -- Destinations: 300+ integrations (analytics, marketing, data warehouses) -- Protocols: Schema enforcement and data governance - -**Event Schema** (Segment Spec): -```typescript -interface SegmentEvent { - type: 'track' | 'identify' | 'page' | 'screen' | 'group' | 'alias' - anonymousId?: string - userId?: string - timestamp: string - context: { - ip?: string - userAgent?: string - locale?: string - campaign?: { source, medium, term, content, name } - device?: { type, manufacturer, model } - os?: { name, version } - app?: { name, version, build } - } - integrations?: Record - // Type-specific fields - properties?: Record // track, page, screen - traits?: Record // identify, group - event?: string // track only - groupId?: string // group only -} -``` - -**Pricing Pain Point**: $25K-$200K/year enterprise, MTU-based billing at ~$0.01-0.03 per user. - -**Cloudflare Workers Rewrite Potential**: - -| Component | Edge Capability | Storage | -|-----------|-----------------|---------| -| Event Ingestion | Workers (sub-ms) | - | -| Schema Validation | Workers | D1 (schemas) | -| Identity Resolution | Durable Objects | SQLite | -| Destination Routing | Workers + Queues | KV (configs) | -| Real-time Streaming | WebSockets/DO | - | -| Warehouse Sync | R2 + Pipelines | Parquet/Iceberg | - -**Killer Edge Feature**: First-party data collection that bypasses ad blockers, sub-millisecond ingestion, GDPR-compliant regional processing. - ---- - -### 2. RudderStack - -**Core Value Proposition**: Open-source, warehouse-native CDP. Data stays in your infrastructure, Segment API-compatible. - -**Key APIs/Features**: -- 100% Segment API compatible -- Transformations API: Custom JavaScript transforms -- Warehouse-native: Identity resolution in your warehouse -- Self-hostable: Control plane + data plane architecture - -**Architecture**: -``` -Control Plane (React UI) Data Plane (Go backend) - │ │ - └──────── Config ───────────▶│ - │ -Sources ─────▶ Ingest ─────▶ Transform ─────▶ Destinations - │ │ - └──── Queue ───┘ -``` - -**Cloudflare Workers Rewrite Potential**: - -| Component | Implementation | Storage | -|-----------|---------------|---------| -| Control Plane | Workers + React | D1 | -| Data Plane | Workers | - | -| Transformations | Workers (isolates) | - | -| Event Queue | Queues | - | -| Identity Graph | Durable Objects | SQLite | -| Warehouse Sync | Pipelines + R2 | Iceberg | - -**Killer Edge Feature**: Warehouse-native approach means R2/D1 IS the warehouse. No external dependencies, complete data ownership. - ---- - -### 3. Mixpanel - -**Core Value Proposition**: Product analytics focused on user behavior funnels, retention, and cohort analysis. - -**Key APIs/Features**: -- Track API: Event ingestion (2GB/min rate limit) -- Import API: Historical data (>5 days old) -- Engage API: User profiles -- Query APIs: JQL (JavaScript Query Language), Insights, Funnels, Retention - -**Data Model**: -```typescript -interface MixpanelEvent { - event: string - properties: { - distinct_id: string // User identifier - time: number // Unix timestamp (ms) - $insert_id?: string // Deduplication - // Standard properties - $browser?: string - $city?: string - $os?: string - $device?: string - // Custom properties - [key: string]: unknown - } -} -``` - -**Cloudflare Workers Rewrite Potential**: - -| Component | Feasibility | Notes | -|-----------|-------------|-------| -| Event Ingestion | High | Workers | -| User Profiles | High | Durable Objects | -| Funnels | Medium | Analytics Engine + D1 | -| Retention Charts | Medium | Pre-aggregated in DO | -| JQL Queries | Low | Complex runtime needed | -| Cohort Analysis | Medium | Window functions needed | - -**Challenge**: Mixpanel's query power (funnels, retention, cohorts) requires complex analytics infrastructure. Would need ClickHouse or significant Analytics Engine work. - ---- - -### 4. Amplitude - -**Core Value Proposition**: Enterprise product analytics with behavioral cohorts, A/B testing, and data governance. - -**Key APIs/Features**: -- HTTP V2 API: Event ingestion (1000 events/sec limit) -- Identify API: User property updates -- Export API: Bulk data export -- Dashboard REST API: Query analytics data -- Warehouse sync: Snowflake, BigQuery native - -**Data Model**: -```typescript -interface AmplitudeEvent { - user_id?: string - device_id?: string - event_type: string - time: number - event_properties?: Record - user_properties?: Record - groups?: Record - // Device info - platform?: string - os_name?: string - os_version?: string - device_brand?: string - device_model?: string - // Location - country?: string - region?: string - city?: string - // Revenue - price?: number - quantity?: number - revenue?: number - productId?: string -} -``` - -**Cloudflare Workers Rewrite Potential**: - -| Component | Feasibility | Notes | -|-----------|-------------|-------| -| Event Ingestion | High | Workers | -| User Properties | High | Durable Objects | -| Behavioral Cohorts | Low | Complex ML needed | -| A/B Testing | Medium | Feature flags are simpler | -| Dashboard Queries | Low | Full OLAP engine needed | - -**Challenge**: Amplitude's value is in its analysis engine, not just collection. A rewrite would need significant analytics infrastructure. - ---- - -### 5. Heap - -**Core Value Proposition**: Auto-capture analytics - no instrumentation needed. Records all user interactions automatically. - -**Key APIs/Features**: -- Auto-capture: Clicks, pageviews, form submissions, change events -- Virtual Events: Define events retroactively from captured data -- Session Replay: Full user session recordings -- Retroactive Analysis: Query historical data with new event definitions - -**Architecture Challenge**: -``` -Heap captures EVERYTHING: -- Every click (with full DOM context) -- Every form field change -- Every scroll position -- Every session recording frame - -This generates: -- 1 PB stored data -- 1B events/day ingested -- 250K analyses/week -``` - -**Cloudflare Workers Rewrite Potential**: - -| Component | Feasibility | Notes | -|-----------|-------------|-------| -| Auto-capture SDK | Medium | JavaScript snippet | -| Event Ingestion | Medium | High volume challenge | -| Session Replay | Low | Massive storage, complex | -| Retroactive Queries | Very Low | Requires full data scan | -| Virtual Events | Low | Real-time pattern matching | - -**Not Recommended**: Heap's value proposition (auto-capture + retroactive analysis) requires storing and querying massive amounts of raw data. This doesn't align well with edge computing constraints. - ---- - -## Recommended Rewrites - -### Primary: `segment.do` (CDP + Event Router) - -**Package**: `segment.do` -**Domain**: `segment.do`, `cdp.do`, `events.do` - -**Architecture**: -``` -┌─────────────────────────────────────────────────────────────────────────────┐ -│ segment.do │ -├─────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ │ -│ │ Sources │ │ Ingestion │ │ Identity │ │Destinations│ │ -│ │ │───▶│ Worker │───▶│Resolution DO│───▶│ Router │ │ -│ │ • Web SDK │ │ │ │ │ │ │ │ -│ │ • Server │ │ • Validate │ │ • Merge │ │ • Queues │ │ -│ │ • Mobile │ │ • Enrich │ │ • Graph │ │ • Webhooks │ │ -│ │ • HTTP API │ │ • Sample │ │ • Persist │ │ • Streams │ │ -│ └─────────────┘ └─────────────┘ └─────────────┘ └────────────┘ │ -│ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Storage Layer │ │ -│ ├─────────────────┬─────────────────┬─────────────────┬───────────────┤ │ -│ │ KV │ D1 │ R2 │ Queues │ │ -│ │ • Configs │ • Schemas │ • Archives │ • Events │ │ -│ │ • Caches │ • Sources │ • Warehouse │ • Webhooks │ │ -│ │ • Rate limits │ • Dests │ • Exports │ • Retries │ │ -│ └─────────────────┴─────────────────┴─────────────────┴───────────────┘ │ -│ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Durable Objects │ │ -│ ├─────────────────────────────────────────────────────────────────────┤ │ -│ │ IdentityDO (per user) │ WorkspaceDO (per tenant) │ │ -│ │ • SQLite identity graph │ • SQLite config store │ │ -│ │ • Trait history │ • Schema registry │ │ -│ │ • Merge operations │ • Destination configs │ │ -│ └─────────────────────────────────────────────────────────────────────┘ │ -│ │ -└─────────────────────────────────────────────────────────────────────────────┘ -``` - -**Core Features**: - -1. **Event Ingestion** (Workers) - - Segment-compatible API (`/v1/track`, `/v1/identify`, `/v1/page`, etc.) - - Schema validation against Tracking Plan - - Context enrichment (geo, device, campaign) - - Sub-millisecond response times - -2. **Identity Resolution** (Durable Objects) - - Per-user DO with SQLite identity graph - - Deterministic matching (email, phone, user_id) - - Probabilistic matching (device fingerprint, IP patterns) - - Merge operations with conflict resolution - -3. **Destination Routing** (Workers + Queues) - - 50+ common destinations (GA4, Amplitude, Mixpanel, etc.) - - Webhook destinations for custom integrations - - Batching and retry logic via Queues - - Per-destination transformation rules - -4. **Data Warehouse Sync** (R2 + Pipelines) - - Real-time CDC to R2 in Parquet/Iceberg format - - Compatible with ClickHouse, Snowflake, BigQuery - - Schema evolution support - - Time travel queries - -**API Design**: -```typescript -// Track event -POST /v1/track -{ - "userId": "user_123", - "event": "Order Completed", - "properties": { - "orderId": "order_456", - "revenue": 99.99, - "products": [...] - }, - "context": { - "ip": "auto", - "userAgent": "auto" - } -} - -// Identify user -POST /v1/identify -{ - "userId": "user_123", - "traits": { - "email": "user@example.com", - "name": "John Doe", - "plan": "premium" - } -} - -// Batch events -POST /v1/batch -{ - "batch": [ - { "type": "track", ... }, - { "type": "identify", ... } - ] -} -``` - -**SDK**: -```typescript -import { Analytics } from 'segment.do' - -const analytics = Analytics({ - writeKey: 'your-write-key', - // Edge-specific options - flushAt: 20, - flushInterval: 10000, - // First-party tracking (bypass ad blockers) - apiHost: 'analytics.yoursite.com' -}) - -analytics.track('Order Completed', { - orderId: 'order_123', - revenue: 99.99 -}) - -analytics.identify('user_123', { - email: 'user@example.com', - plan: 'premium' -}) -``` - -**Complexity Assessment**: MEDIUM -- Event ingestion: Low complexity -- Identity resolution: Medium complexity (graph algorithms) -- Destination routing: Medium complexity (many integrations) -- Warehouse sync: Low complexity (existing R2/Iceberg patterns) - -**Dependencies**: -- `kafka.do` (optional, for high-volume streaming) -- `mongo.do` OLAP layer patterns (for analytics queries) - ---- - -### Secondary: `analytics.do` (Unified Analytics Ingestion) - -**Package**: `analytics.do` -**Domain**: `analytics.do`, `track.do`, `metrics.do` - -**Positioning**: While `segment.do` is a full CDP, `analytics.do` is focused purely on analytics ingestion and basic querying - think "Analytics Engine on steroids". - -**Architecture**: -``` -┌─────────────────────────────────────────────────────────────────────────────┐ -│ analytics.do │ -├─────────────────────────────────────────────────────────────────────────────┤ -│ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Ingestion Layer (Workers) │ │ -│ ├─────────────────┬─────────────────┬─────────────────────────────────┤ │ -│ │ Track API │ Page API │ Batch API │ │ -│ │ /v1/track │ /v1/page │ /v1/batch │ │ -│ └────────┬────────┴────────┬────────┴─────────────┬───────────────────┘ │ -│ │ │ │ │ -│ ▼ ▼ ▼ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Workers Analytics Engine (Native) │ │ -│ │ │ │ -│ │ • 25 data points per invocation │ │ -│ │ • Automatic sampling (ABR) │ │ -│ │ • 90-day retention (free) │ │ -│ │ • SQL API for queries │ │ -│ └─────────────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Extended Storage (Long-term) │ │ -│ ├─────────────────────────────────────────────────────────────────────┤ │ -│ │ R2 (Parquet/Iceberg) │ ClickHouse (Query Engine) │ │ -│ │ • Unlimited retention │ • Complex aggregations │ │ -│ │ • Time travel │ • Funnels, cohorts │ │ -│ │ • Schema evolution │ • Window functions │ │ -│ └─────────────────────────────────────────────────────────────────────┘ │ -│ │ -│ ┌─────────────────────────────────────────────────────────────────────┐ │ -│ │ Query Layer │ │ -│ ├─────────────────────────────────────────────────────────────────────┤ │ -│ │ • SQL API (Analytics Engine + ClickHouse) │ │ -│ │ • Grafana integration │ │ -│ │ • Dashboard API │ │ -│ │ • Export API │ │ -│ └─────────────────────────────────────────────────────────────────────┘ │ -│ │ -└─────────────────────────────────────────────────────────────────────────────┘ -``` - -**Key Differentiator**: Leverages Cloudflare's native Analytics Engine for hot data, with overflow to R2/ClickHouse for long-term storage and complex queries. - -**SDK**: -```typescript -import { track, page, identify } from 'analytics.do' - -// Simple tracking -track('Button Clicked', { buttonId: 'cta-signup' }) - -// Page view -page('Home', { referrer: document.referrer }) - -// User identification -identify('user_123', { plan: 'pro' }) - -// Query API -import { query } from 'analytics.do' - -const results = await query.sql(` - SELECT - blob1 as event, - count() as count, - avg(double1) as avgValue - FROM events - WHERE timestamp > now() - INTERVAL 7 DAY - GROUP BY event - ORDER BY count DESC - LIMIT 10 -`) -``` - -**Complexity Assessment**: LOW-MEDIUM -- Event ingestion: Low (Workers + Analytics Engine) -- Storage: Low (native Analytics Engine, R2 for overflow) -- Querying: Medium (SQL API wrapper) - ---- - -## Implementation Roadmap - -### Phase 1: Core Ingestion (2 weeks) -- [ ] Event ingestion API (track, page, identify) -- [ ] Schema validation -- [ ] Context enrichment (geo, device) -- [ ] Analytics Engine integration -- [ ] Basic SDK (browser, node) - -### Phase 2: Identity Resolution (2 weeks) -- [ ] IdentityDO with SQLite graph -- [ ] Deterministic matching -- [ ] Anonymous → identified merging -- [ ] Identity API - -### Phase 3: Destinations (3 weeks) -- [ ] Destination framework -- [ ] 10 core destinations (GA4, Mixpanel, etc.) -- [ ] Webhook destinations -- [ ] Queue-based delivery with retries - -### Phase 4: Warehouse & Analytics (2 weeks) -- [ ] R2 export (Parquet/Iceberg) -- [ ] ClickHouse integration -- [ ] Query API -- [ ] Dashboard widgets - -### Phase 5: Advanced Features (2 weeks) -- [ ] Tracking Plans (schema enforcement) -- [ ] Data governance (PII detection) -- [ ] Replay/debugging tools -- [ ] Multi-tenant workspace isolation - ---- - -## Edge Computing Advantages - -### 1. First-Party Data Collection -``` -Traditional: Edge (segment.do): - -Browser ─── 3rd Party ──▶ CDN Browser ─── Same Domain ──▶ Edge - │ │ │ │ - │ (Blocked by │ (First-party, - │ ad blockers) │ not blocked) - ▼ ▼ - Partial Data Complete Data -``` - -### 2. GDPR/Privacy Compliance -``` -Traditional: Edge: - -EU User ──▶ US Server EU User ──▶ EU Edge ──▶ Process Locally - │ │ │ │ - │ (Data transfer │ (No cross-border - │ compliance) │ transfer needed) -``` - -### 3. Real-Time Processing -``` -Traditional: Edge: - -Event ──▶ Queue ──▶ Process Event ──▶ Process at Edge - │ │ │ │ │ - │ (100ms+) │ (<10ms) -``` - -### 4. Cost Optimization -``` -Segment Pricing: segment.do Pricing: -• $0.01-0.03 per MTU • Workers: $0.30/million requests -• $25K-$200K/year • Queues: $0.40/million operations -• Scales with users • R2: $0.015/GB stored - • Estimated: 80-90% cost reduction -``` - ---- - -## Technical Considerations - -### Identity Resolution Algorithm - -```typescript -interface IdentityGraph { - // Core identity - canonicalId: string - - // Known identifiers (deterministic) - identifiers: { - userId?: string[] - email?: string[] - phone?: string[] - deviceId?: string[] - } - - // Anonymous identifiers - anonymous: { - anonymousId: string - firstSeen: Date - merged?: Date - }[] - - // Probabilistic signals - signals: { - ipAddresses: string[] - userAgents: string[] - fingerprints: string[] - } - - // Merged identities - mergedFrom: string[] -} - -// Matching strategy -async function resolveIdentity(event: SegmentEvent): Promise { - // 1. Deterministic match (exact identifiers) - if (event.userId) { - return getOrCreateByUserId(event.userId) - } - - // 2. Trait-based match (email, phone) - if (event.traits?.email) { - const match = await findByEmail(event.traits.email) - if (match) return match.canonicalId - } - - // 3. Anonymous tracking - if (event.anonymousId) { - const existing = await findByAnonymousId(event.anonymousId) - if (existing) return existing.canonicalId - } - - // 4. Probabilistic (optional, lower confidence) - const probabilistic = await probabilisticMatch(event.context) - if (probabilistic?.confidence > 0.9) { - return probabilistic.canonicalId - } - - // 5. Create new identity - return createNewIdentity(event) -} -``` - -### Destination Routing - -```typescript -interface Destination { - id: string - type: 'webhook' | 'api' | 'warehouse' - config: { - url?: string - apiKey?: string - mapping?: Record - filters?: EventFilter[] - batching?: { - maxSize: number - maxWait: number - } - } -} - -// Fan-out to destinations -async function routeEvent(event: ResolvedEvent) { - const workspace = await getWorkspace(event.workspaceId) - const destinations = workspace.destinations.filter(d => - matchesFilters(event, d.config.filters) - ) - - // Queue for delivery - for (const dest of destinations) { - const transformed = applyMapping(event, dest.config.mapping) - await env.DESTINATION_QUEUE.send({ - destinationId: dest.id, - event: transformed, - attempt: 0 - }) - } -} -``` - -### Storage Strategy - -| Data Type | Hot (0-90 days) | Warm (90 days - 1 year) | Cold (1+ years) | -|-----------|-----------------|-------------------------|-----------------| -| Events | Analytics Engine | R2 (Parquet) | R2 (Iceberg archive) | -| Identities | Durable Objects | D1 (backup) | R2 (export) | -| Configs | KV | D1 | - | -| Schemas | D1 | - | - | - ---- - -## Competitive Analysis - -| Feature | Segment | RudderStack | segment.do | -|---------|---------|-------------|------------| -| Pricing | $25K-$200K/yr | $0 (OSS) + infra | $500-$5K/yr | -| Edge Native | No | No | Yes | -| Identity Resolution | Yes | Warehouse | Edge DO | -| Destinations | 300+ | 200+ | 50+ (initial) | -| Real-time | Near (seconds) | Near (seconds) | Sub-ms | -| First-party | Requires proxy | Requires proxy | Native | -| GDPR Edge Processing | No | No | Yes | - ---- - -## Risk Assessment - -| Risk | Impact | Mitigation | -|------|--------|------------| -| Analytics Engine limits | Medium | Overflow to R2/ClickHouse | -| DO per-user cost at scale | Medium | Tiered identity resolution | -| Destination API changes | Low | Abstraction layer | -| Complex analytics queries | High | Partner with ClickHouse | - ---- - -## Conclusion - -**Recommended Path**: - -1. **Start with `segment.do`** - Full CDP with identity resolution -2. **Offer `analytics.do`** as simpler alternative - Just analytics, no CDP features -3. **Share infrastructure** - Both use same ingestion, storage, query layers - -**Unique Value Proposition**: -- **First-party tracking** that actually works (no ad blockers) -- **Edge-native** identity resolution (sub-ms, GDPR compliant) -- **90% cost reduction** vs cloud CDPs -- **Segment API compatible** (drop-in replacement) - ---- - -## Sources - -### Segment -- [Segment Track Spec](https://segment.com/docs/connections/spec/track/) -- [Segment Public API Documentation](https://docs.segmentapis.com/) -- [Segment Destinations Overview](https://segment.com/docs/connections/destinations/) -- [Segment Pricing Guide](https://www.spendflo.com/blog/segment-pricing-guide) - -### RudderStack -- [RudderStack Open Source](https://www.rudderstack.com/docs/get-started/rudderstack-open-source/) -- [RudderStack GitHub](https://github.com/rudderlabs/rudder-server) - -### Mixpanel -- [Mixpanel Ingestion API](https://developer.mixpanel.com/reference/ingestion-api) -- [Mixpanel Track Events](https://docs.mixpanel.com/docs/quickstart/capture-events/track-events) - -### Amplitude -- [Amplitude HTTP V2 API](https://amplitude.com/docs/apis/analytics/http-v2) -- [Amplitude Snowflake Integration](https://amplitude.com/docs/data/destination-catalog/snowflake) - -### Heap -- [Heap AutoCapture](https://www.heap.io/platform/autocapture) -- [Heap Architecture](https://www.heap.io/blog/heaps-next-generation-data-platform) - -### Cloudflare -- [Workers Analytics Engine](https://developers.cloudflare.com/analytics/analytics-engine/) -- [Analytics Engine SQL API](https://developers.cloudflare.com/analytics/analytics-engine/sql-api/) -- [Durable Objects Overview](https://developers.cloudflare.com/durable-objects/) - -### Identity Resolution -- [Identity Resolution Guide](https://www.twilio.com/en-us/blog/insights/identity-resolution) -- [Identity Resolution Algorithms for CDP](https://blog.ahmadwkhan.com/guide-to-identity-resolution-algorithms-for-cdp) - -### Privacy & Edge -- [First-Party Data Compliance 2025](https://secureprivacy.ai/blog/first-party-data-collection-compliance-gdpr-ccpa-2025) -- [GDPR Compliance in Edge Computing](https://www.gdpr-advisor.com/gdpr-compliance-in-edge-computing-managing-decentralized-data-storage/) diff --git a/rewrites/docs/research/auth-scope.md b/rewrites/docs/research/auth-scope.md deleted file mode 100644 index 42b429a8..00000000 --- a/rewrites/docs/research/auth-scope.md +++ /dev/null @@ -1,834 +0,0 @@ -# auth.do / id.do - Authentication & Identity Rewrite Scope - -**Cloudflare Workers Edge Authentication Platform** - -A comprehensive authentication and identity platform running entirely on Cloudflare Workers with Durable Objects, providing enterprise-grade security at the edge with zero cold starts. - ---- - -## Executive Summary - -This document scopes the development of `auth.do` (or `id.do`), a Cloudflare Workers rewrite of authentication platforms like Auth0, Clerk, Supabase Auth, WorkOS, Stytch, and FusionAuth. The platform will provide: - -- **Zero-latency JWT validation** at the edge with cached JWKS -- **Stateful session management** using Durable Objects -- **Complete OAuth 2.1/OIDC provider** capabilities -- **Enterprise SSO** (SAML, OIDC) with directory sync (SCIM) -- **Modern passwordless auth** (magic links, passkeys/WebAuthn) -- **Multi-tenant B2B** support with per-organization settings - ---- - -## Platform Research Summary - -### 1. Auth0 (Okta) - -**Core Value Proposition**: Enterprise-grade identity-as-a-service with comprehensive protocol support. - -**Key Features**: -- Universal Login with customizable UX -- Token Vault for external API access (Google Calendar, GitHub, etc.) -- Multi-Resource Refresh Tokens (MRRT) - single refresh token for multiple APIs -- Auth for GenAI - identity for AI agents -- Organizations - up to 2M business customers per tenant -- Fine-Grained Authorization (FGA) -- FAPI 2 certification (Q2 2025) - -**Edge Rewrite Opportunities**: -- JWT validation happens server-side - move to edge -- JWKS caching strategy (15-second max-age with stale-while-revalidate) -- Token introspection latency - eliminate with edge validation - -**Sources**: -- [Auth0 Platform](https://auth0.com/) -- [Auth0 APIs Documentation](https://auth0.com/docs/api) -- [Auth0 July 2025 Updates](https://auth0.com/blog/july-2025-product-updates-new-security-features-global-regions-and-developer-previews/) - ---- - -### 2. Clerk - -**Core Value Proposition**: Developer-first authentication with React-native components. - -**Key Features**: -- Pre-built UI components (``, ``) -- Short-lived JWTs (60 seconds) with automatic background refresh -- API Keys for machine authentication (2025) -- Organizations with RBAC -- Session tokens stored in `__session` cookie -- Native Android components (2025) - -**Architecture Insights**: -- JWT verification via `clerkMiddleware()` at request start -- Public key available at `/.well-known/jwks.json` -- Cookie size limit: 4KB (browser limitation) -- Claims: `exp`, `nbf`, `azp` (authorized parties) - -**Edge Rewrite Opportunities**: -- Middleware already edge-optimized - match or exceed -- Session sync across regions needs Durable Objects -- Cookie-based sessions ideal for edge - -**Sources**: -- [Clerk Documentation](https://clerk.com/docs) -- [Clerk Session Tokens](https://clerk.com/docs/guides/sessions/session-tokens) -- [Clerk JWT Verification](https://clerk.com/docs/guides/sessions/manual-jwt-verification) - ---- - -### 3. Supabase Auth - -**Core Value Proposition**: Open-source auth tightly integrated with PostgreSQL and Row Level Security. - -**Key Features**: -- OAuth 2.1 and OpenID Connect provider (new) -- Asymmetric keys (RS256 default, ECC/Ed25519 optional) -- Row Level Security (RLS) integration -- "Sign in with [Your App]" capability -- MCP authentication support -- Self-hostable (GoTrue-based) - -**Architecture Insights**: -- JWTs include `user_id`, `role`, `client_id` claims -- RLS policies automatically apply to OAuth tokens -- Authorization code flow with PKCE - -**Edge Rewrite Opportunities**: -- RLS-like authorization at edge with D1 -- OAuth 2.1 server running on Workers -- JWT signing/verification with WebCrypto - -**Sources**: -- [Supabase Auth Documentation](https://supabase.com/docs/guides/auth) -- [Supabase OAuth 2.1 Server](https://supabase.com/docs/guides/auth/oauth-server) -- [Supabase Auth GitHub](https://github.com/supabase/auth) - ---- - -### 4. WorkOS - -**Core Value Proposition**: Enterprise-ready features (SSO, SCIM) for B2B SaaS. - -**Key Features**: -- Single Sign-On supporting SAML + OIDC with one API -- Directory Sync (SCIM) with real-time webhooks -- Admin Portal for self-service configuration -- Audit Logs with SIEM streaming -- Radar for fraud detection -- Vault for encryption key management -- Fine-grained authorization (Warrant acquisition) - -**Pricing Model** (2025): -- SSO: $125/connection/month -- Directory Sync: $125/connection/month -- User Management: Free up to 1M MAU - -**Edge Rewrite Opportunities**: -- SAML assertion validation at edge -- Directory sync webhooks via Workers -- Audit log streaming to R2 - -**Sources**: -- [WorkOS Documentation](https://workos.com/docs) -- [WorkOS SSO](https://workos.com/single-sign-on) -- [SCIM vs SSO Guide](https://workos.com/guide/scim-vs-sso) - ---- - -### 5. Stytch - -**Core Value Proposition**: Passwordless-first authentication with fraud detection. - -**Key Features**: -- Magic links, OTPs, passkeys (FIDO2/WebAuthn) -- Native mobile biometrics -- 99.99% bot detection accuracy -- Device fingerprinting -- Multi-tenant RBAC -- Per-organization auth settings - -**Edge Rewrite Opportunities**: -- Passwordless flows ideal for edge (stateless) -- Device fingerprinting at edge -- Rate limiting at edge - -**Sources**: -- [Stytch Platform](https://stytch.com/) -- [Stytch Passwordless](https://stytch.com/solutions/passwordless) -- [Passwordless Authentication Guide](https://stytch.com/blog/what-is-passwordless-authentication/) - ---- - -### 6. FusionAuth - -**Core Value Proposition**: Self-hosted, downloadable CIAM with full API access. - -**Key Features**: -- Completely self-hostable (Docker, K8s, bare metal) -- RESTful API for everything (UI built on same APIs) -- Community edition free for unlimited users -- No rate limits when self-hosted -- Machine-to-machine authentication -- SCIM provisioning - -**Edge Rewrite Opportunities**: -- Full API surface to replicate -- Self-hosted model maps to DO isolation -- No rate limits = edge-native scaling - -**Sources**: -- [FusionAuth Platform](https://fusionauth.io/) -- [FusionAuth Self-Hosting](https://fusionauth.io/platform/self-hosting) -- [FusionAuth API Overview](https://fusionauth.io/docs/apis/) - ---- - -## Architecture Vision - -``` -auth.do / id.do -| -+-- Edge Layer (Cloudflare Workers) -| | -| +-- JWT Validation (cached JWKS, zero-latency) -| +-- Session Cookie Management -| +-- Rate Limiting (per IP, per user) -| +-- Device Fingerprinting -| +-- Geographic Restrictions -| -+-- Auth Durable Objects -| | -| +-- UserDO (per-user state) -| | +-- Sessions -| | +-- MFA enrollment -| | +-- Passkey credentials -| | +-- Login history -| | -| +-- OrganizationDO (per-org state) -| | +-- SSO connections -| | +-- Directory sync -| | +-- RBAC policies -| | +-- Audit logs -| | -| +-- SessionDO (per-session state) -| +-- Refresh token rotation -| +-- Device binding -| +-- Activity tracking -| -+-- Storage Layer -| | -| +-- D1 (SQLite) -| | +-- Users table -| | +-- Sessions table -| | +-- Organizations table -| | +-- SAML/OIDC connections -| | -| +-- KV (Caching) -| | +-- JWKS cache -| | +-- Session cache -| | +-- Rate limit counters -| | -| +-- R2 (Object Storage) -| +-- SAML certificates -| +-- Audit log archives -| +-- SCIM sync snapshots -| -+-- OAuth/OIDC Provider -| | -| +-- Authorization Server -| +-- Token Endpoint -| +-- Introspection Endpoint -| +-- Revocation Endpoint -| +-- JWKS Endpoint -| +-- Discovery (.well-known/openid-configuration) -| -+-- Enterprise SSO -| | -| +-- SAML SP (Service Provider) -| +-- OIDC Client -| +-- IdP Metadata Parser -| +-- Assertion Validation -| -+-- Directory Sync -| | -| +-- SCIM 2.0 Server -| +-- Webhook Delivery -| +-- Delta Sync -| -+-- Admin API - | - +-- User Management - +-- Organization Management - +-- Connection Management - +-- Audit Log Queries -``` - ---- - -## Core Features Specification - -### 1. JWT Validation at Edge (Zero Latency) - -**Implementation**: -```typescript -// Using jose library with WebCrypto -import { jwtVerify, createRemoteJWKSet } from 'jose' - -const JWKS = createRemoteJWKSet(new URL('https://auth.do/.well-known/jwks.json')) - -// Cache JWKS in KV with stale-while-revalidate pattern -async function validateJWT(token: string, env: Env): Promise { - // Try cached JWKS first - const cachedJWKS = await env.KV.get('jwks', { type: 'json', cacheTtl: 60 }) - - const { payload } = await jwtVerify(token, JWKS, { - issuer: 'https://auth.do', - audience: env.CLIENT_ID, - }) - - return payload -} -``` - -**JWKS Caching Strategy**: -- Primary cache: KV with 60-second TTL -- Background refresh: Stale-while-revalidate pattern -- Fallback: Direct fetch if cache miss -- Key rotation: Support multiple active keys - -**Performance Target**: <5ms JWT validation at edge - ---- - -### 2. Session Management with Durable Objects - -**SessionDO Architecture**: -```typescript -export class SessionDO extends DurableObject { - private session: SessionState | null = null - - async create(userId: string, deviceInfo: DeviceInfo): Promise { - const sessionId = crypto.randomUUID() - const refreshToken = generateSecureToken() - - this.session = { - id: sessionId, - userId, - deviceInfo, - refreshToken: await hashToken(refreshToken), - createdAt: Date.now(), - lastActivityAt: Date.now(), - expiresAt: Date.now() + SESSION_TTL, - } - - await this.ctx.storage.put('session', this.session) - - return { - sessionId, - accessToken: await this.generateAccessToken(), - refreshToken, - expiresIn: ACCESS_TOKEN_TTL, - } - } - - async refresh(refreshToken: string): Promise { - // Validate refresh token - // Rotate refresh token (single-use) - // Generate new access token - // Update last activity - } - - async revoke(): Promise { - await this.ctx.storage.deleteAll() - this.session = null - } -} -``` - -**Session Consistency Across Regions**: -- Durable Objects provide single-writer guarantee -- Session state lives in one location (user's primary region) -- Access tokens validated at edge (stateless JWT) -- Refresh requires round-trip to session DO - ---- - -### 3. OAuth 2.1 / OIDC Provider - -**Endpoints**: -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/.well-known/openid-configuration` | GET | Discovery document | -| `/.well-known/jwks.json` | GET | JSON Web Key Set | -| `/authorize` | GET | Authorization endpoint | -| `/token` | POST | Token endpoint | -| `/userinfo` | GET | User info endpoint | -| `/introspect` | POST | Token introspection | -| `/revoke` | POST | Token revocation | - -**Supported Flows**: -- Authorization Code with PKCE (recommended) -- Client Credentials (machine-to-machine) -- Refresh Token with rotation - -**Token Types**: -- Access Token: Short-lived JWT (15 min - 1 hour) -- Refresh Token: Long-lived, single-use with rotation -- ID Token: OpenID Connect identity assertion - ---- - -### 4. Enterprise SSO (SAML + OIDC) - -**SAML Service Provider**: -```typescript -interface SAMLConnection { - id: string - organizationId: string - idpMetadataUrl?: string - idpMetadata?: string - idpEntityId: string - idpSsoUrl: string - idpCertificate: string - spEntityId: string - spAcsUrl: string - attributeMapping: Record -} -``` - -**SAML Flow at Edge**: -1. User initiates login with organization domain -2. Worker generates AuthnRequest, redirects to IdP -3. IdP authenticates user, posts SAMLResponse to ACS -4. Worker validates signature, extracts assertions -5. Creates session, issues tokens - -**OIDC Federation**: -- Support for Google, Microsoft, Okta as IdPs -- Dynamic client registration -- Standard claims mapping - ---- - -### 5. Multi-Factor Authentication - -**Supported Methods**: -| Method | Implementation | Storage | -|--------|----------------|---------| -| TOTP | RFC 6238 | Secret in UserDO | -| WebAuthn/Passkeys | FIDO2 | Credential in D1 | -| SMS OTP | External provider | Temporary in KV | -| Email OTP | SendGrid/Resend | Temporary in KV | -| Recovery Codes | One-time use | Hashed in UserDO | - -**WebAuthn/Passkeys at Edge**: -```typescript -// Registration -async function registerPasskey(challenge: string, attestation: AttestationObject) { - // Verify attestation using WebCrypto - // Extract public key - // Store credential in D1 -} - -// Authentication -async function verifyPasskey(challenge: string, assertion: AssertionObject) { - // Retrieve credential from D1 - // Verify signature using stored public key - // Update sign count -} -``` - ---- - -### 6. Directory Sync (SCIM 2.0) - -**SCIM Endpoints**: -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/scim/v2/Users` | GET, POST | List/create users | -| `/scim/v2/Users/{id}` | GET, PUT, PATCH, DELETE | User operations | -| `/scim/v2/Groups` | GET, POST | List/create groups | -| `/scim/v2/Groups/{id}` | GET, PUT, PATCH, DELETE | Group operations | -| `/scim/v2/Schemas` | GET | Schema discovery | - -**Webhook Events**: -- `user.created`, `user.updated`, `user.deleted` -- `group.created`, `group.updated`, `group.deleted` -- `membership.added`, `membership.removed` - ---- - -## Security Considerations - -### 1. Key Rotation - -**JWKS Rotation Strategy**: -```typescript -interface JWKSRotation { - currentKey: JWK // Primary signing key - nextKey?: JWK // Pre-published for rotation - previousKey?: JWK // Grace period for existing tokens - rotationSchedule: string // Cron expression -} -``` - -- New key published 24 hours before activation -- Old key retained for token lifetime -- Automatic rotation with zero downtime - -### 2. Brute Force Protection - -**Rate Limiting Layers**: -```typescript -// Layer 1: Global rate limit (Cloudflare WAF) -// Layer 2: Per-IP rate limit (KV counters) -// Layer 3: Per-account rate limit (UserDO) - -async function checkRateLimit(ip: string, userId?: string): Promise { - const ipKey = `ratelimit:ip:${ip}` - const ipCount = await env.KV.get(ipKey, { type: 'json' }) || 0 - - if (ipCount > IP_RATE_LIMIT) { - return false // Block - } - - await env.KV.put(ipKey, ipCount + 1, { expirationTtl: 60 }) - - if (userId) { - // Check per-user rate limit in UserDO - } - - return true -} -``` - -**Account Lockout**: -- Progressive delays after failed attempts -- Lockout after N failures (configurable) -- CAPTCHA challenge on suspicious activity -- Breach detection integration (HaveIBeenPwned) - -### 3. CSRF/XSS Prevention - -**CSRF Protection**: -- State parameter in OAuth flows (stored in DO) -- SameSite=Strict cookies for sessions -- Origin header validation - -**XSS Prevention**: -- HTTPOnly cookies for tokens -- Content-Security-Policy headers -- Input sanitization in UI - -### 4. Audit Logging - -**Logged Events**: -| Event | Data | Retention | -|-------|------|-----------| -| `auth.login.success` | userId, ip, device, method | 90 days | -| `auth.login.failure` | email, ip, reason | 30 days | -| `auth.logout` | userId, sessionId | 90 days | -| `auth.mfa.enrolled` | userId, method | Permanent | -| `auth.password.changed` | userId | Permanent | -| `auth.token.revoked` | userId, tokenId | 90 days | - -**Storage**: -- Hot: D1 for recent events (30 days) -- Warm: R2 for archives (1 year) -- SIEM streaming: Webhooks to external systems - ---- - -## Compliance Requirements - -### SOC 2 Type II - -**Requirements**: -- Access controls and authentication -- Encryption at rest and in transit -- Audit logging and monitoring -- Incident response procedures -- Vendor management - -**Implementation**: -- Cloudflare provides infrastructure compliance -- auth.do provides application-level controls -- Audit logs exportable for auditors - -### GDPR - -**Requirements**: -- Right to access (data export) -- Right to erasure (account deletion) -- Data portability -- Consent management - -**Implementation**: -- User data export API -- Hard delete capability -- Consent tracking in UserDO - -### HIPAA (Optional) - -**BAA Required**: -- Cloudflare Enterprise with BAA -- PHI never stored in auth system -- Audit logs for access tracking - ---- - -## API Surface - -### Authentication API - -```typescript -interface AuthAPI { - // User Authentication - signup(email: string, password: string): Promise - login(email: string, password: string): Promise - loginWithMagicLink(email: string): Promise - loginWithPasskey(credential: Credential): Promise - logout(): Promise - - // Session Management - refreshToken(refreshToken: string): Promise - revokeSession(sessionId: string): Promise - listSessions(): Promise - - // MFA - enrollTOTP(): Promise - verifyTOTP(code: string): Promise - enrollPasskey(): Promise - - // Password - resetPassword(email: string): Promise - changePassword(current: string, newPassword: string): Promise -} -``` - -### Management API - -```typescript -interface ManagementAPI { - // Users - createUser(user: CreateUserInput): Promise - getUser(userId: string): Promise - updateUser(userId: string, updates: UpdateUserInput): Promise - deleteUser(userId: string): Promise - listUsers(filters?: UserFilters): Promise - - // Organizations - createOrganization(org: CreateOrgInput): Promise - getOrganization(orgId: string): Promise - addMember(orgId: string, userId: string, role: string): Promise - removeMember(orgId: string, userId: string): Promise - - // SSO Connections - createSSOConnection(connection: SSOConnectionInput): Promise - updateSSOConnection(id: string, updates: SSOConnectionInput): Promise - deleteSSOConnection(id: string): Promise - - // Directory Sync - createDirectory(directory: DirectoryInput): Promise - syncDirectory(directoryId: string): Promise -} -``` - ---- - -## SDK Design - -### JavaScript/TypeScript SDK - -```typescript -import { Auth } from 'auth.do' - -// Initialize -const auth = Auth({ - domain: 'myapp.auth.do', - clientId: 'xxx', -}) - -// Login -const session = await auth.login({ - email: 'user@example.com', - password: 'secret', -}) - -// Access protected resources -const response = await fetch('/api/data', { - headers: { - Authorization: `Bearer ${session.accessToken}`, - }, -}) - -// Refresh token -const newSession = await auth.refreshToken() - -// Logout -await auth.logout() -``` - -### Middleware Integration - -```typescript -// Hono middleware -import { authMiddleware } from 'auth.do/hono' - -app.use('/*', authMiddleware({ - publicPaths: ['/health', '/docs'], - requireAuth: true, -})) - -// Next.js middleware -import { withAuth } from 'auth.do/nextjs' - -export default withAuth({ - publicPaths: ['/', '/login'], -}) -``` - ---- - -## Implementation Phases - -### Phase 1: Core Authentication (4-6 weeks) - -- [ ] JWT signing/verification with WebCrypto -- [ ] JWKS endpoint with rotation -- [ ] Email/password authentication -- [ ] Session management (DO) -- [ ] Refresh token rotation -- [ ] Basic user management API - -### Phase 2: OAuth/OIDC Provider (4-6 weeks) - -- [ ] Authorization endpoint -- [ ] Token endpoint -- [ ] PKCE support -- [ ] OpenID Connect -- [ ] Discovery document -- [ ] Token introspection/revocation - -### Phase 3: Passwordless (2-4 weeks) - -- [ ] Magic link authentication -- [ ] Email OTP -- [ ] WebAuthn/Passkeys registration -- [ ] WebAuthn/Passkeys authentication - -### Phase 4: Social Logins (2-3 weeks) - -- [ ] Google OAuth -- [ ] GitHub OAuth -- [ ] Microsoft OAuth -- [ ] Apple Sign In -- [ ] Generic OIDC provider - -### Phase 5: Enterprise SSO (4-6 weeks) - -- [ ] SAML Service Provider -- [ ] OIDC Federation -- [ ] IdP metadata parsing -- [ ] Attribute mapping -- [ ] Just-in-time provisioning - -### Phase 6: Directory Sync (3-4 weeks) - -- [ ] SCIM 2.0 server -- [ ] Webhook delivery -- [ ] Delta sync -- [ ] Group membership - -### Phase 7: Multi-Factor Authentication (2-3 weeks) - -- [ ] TOTP enrollment/verification -- [ ] Recovery codes -- [ ] MFA enforcement policies -- [ ] Step-up authentication - -### Phase 8: Admin & Compliance (3-4 weeks) - -- [ ] Admin dashboard -- [ ] Audit logging -- [ ] Data export/deletion -- [ ] SIEM integration - ---- - -## Competitive Positioning - -| Feature | auth.do | Auth0 | Clerk | Supabase | WorkOS | -|---------|---------|-------|-------|----------|--------| -| Edge JWT Validation | Native | Proxy | Native | Proxy | Proxy | -| Session Storage | DO | Cloud | Cloud | Postgres | Cloud | -| Cold Start | None | ~100ms | ~50ms | ~200ms | ~100ms | -| Self-Host Option | Yes | No | No | Yes | No | -| Open Source | Yes | No | No | Yes | No | -| AI Agent Auth | Native | New | No | New | No | -| Pricing | Usage | Seat | MAU | Usage | Connection | - ---- - -## Estimated Effort - -| Phase | Effort | Priority | -|-------|--------|----------| -| Phase 1: Core Auth | 4-6 weeks | P0 | -| Phase 2: OAuth/OIDC | 4-6 weeks | P0 | -| Phase 3: Passwordless | 2-4 weeks | P1 | -| Phase 4: Social Logins | 2-3 weeks | P1 | -| Phase 5: Enterprise SSO | 4-6 weeks | P1 | -| Phase 6: Directory Sync | 3-4 weeks | P2 | -| Phase 7: MFA | 2-3 weeks | P1 | -| Phase 8: Admin | 3-4 weeks | P2 | - -**Total: 24-36 weeks** for full feature parity - -**MVP (Phase 1-2): 8-12 weeks** for core authentication platform - ---- - -## Open Questions - -1. **Domain Strategy**: Use `auth.do` or `id.do`? (Current codebase has `id.org.ai` for WorkOS integration) - -2. **Key Management**: Should we integrate with Cloudflare's native key management or implement our own with KV/R2? - -3. **Social Providers**: Which providers are priority? (Google, GitHub, Microsoft seem essential) - -4. **SAML Complexity**: Build full SAML SP or recommend WorkOS/Auth0 for complex enterprise needs? - -5. **Pricing Model**: Per-MAU (Clerk), per-connection (WorkOS), or pure usage-based? - -6. **AI Agent Authentication**: How deep should Agent Token support go? (Currently implemented in WorkOSDO) - ---- - -## References - -### Primary Sources -- [Auth0 Platform](https://auth0.com/) -- [Clerk Documentation](https://clerk.com/docs) -- [Supabase Auth](https://supabase.com/docs/guides/auth) -- [WorkOS Documentation](https://workos.com/docs) -- [Stytch Platform](https://stytch.com/) -- [FusionAuth Documentation](https://fusionauth.io/docs/) - -### Edge Authentication -- [JWT Validation at Edge](https://securityboulevard.com/2025/11/how-to-validate-jwts-efficiently-at-the-edge-with-cloudflare-workers-and-vercel/) -- [Cloudflare Workers JWT](https://github.com/tsndr/cloudflare-worker-jwt) -- [jose library](https://www.npmjs.com/package/jose) - -### Durable Objects Session Management -- [UserDO Authentication Pattern](https://github.com/acoyfellow/UserDO) -- [Durable Objects Sessions Demo](https://github.com/saibotsivad/cloudflare-durable-object-sessions) -- [MCP Agent Auth with DO](https://blog.cloudflare.com/building-ai-agents-with-mcp-authn-authz-and-durable-objects/) - -### Standards -- [OAuth 2.1 Draft](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-12) -- [OpenID Connect](https://openid.net/connect/) -- [SAML 2.0](http://docs.oasis-open.org/security/saml/v2.0/) -- [SCIM 2.0](https://www.rfc-editor.org/rfc/rfc7644) -- [WebAuthn](https://www.w3.org/TR/webauthn/) -- [FIDO2/Passkeys](https://fidoalliance.org/passkeys/) diff --git a/rewrites/docs/research/billing-scope.md b/rewrites/docs/research/billing-scope.md deleted file mode 100644 index 8f8e2c81..00000000 --- a/rewrites/docs/research/billing-scope.md +++ /dev/null @@ -1,1284 +0,0 @@ -# Billing Rewrite Scope: billing.do / meter.do - -## Executive Summary - -This document scopes a Cloudflare Workers rewrite of billing and metering functionality, combining the best patterns from leading platforms: Stripe Billing, Paddle, LemonSqueezy, Orb, Lago, and Stigg. The goal is to create a unified billing infrastructure that leverages edge computing for real-time usage metering, fast entitlement checks, and seamless integration with the workers.do ecosystem. - ---- - -## Platform Research Summary - -### 1. Stripe Billing - -**Core Value Proposition**: Industry-leading subscription management with flexible pricing models, automatic proration, and the new Entitlements API for feature gating. - -**Key APIs/Features**: -- **Subscription lifecycle**: Create, update, pause, cancel, resume with automatic proration -- **Usage metering**: New Meter API (replaces legacy usage records) - 1,000 events/sec standard, 10,000/sec with streams -- **Invoice generation**: Automatic with line items, prorations, and PDF rendering -- **Payment processing**: Smart Retries recovered $6.5B in 2024 -- **Pricing models**: Flat, per-seat, usage-based, tiered, graduated, volume -- **Entitlements API**: Feature-to-product mapping, webhook-based provisioning - -**Architecture Notes**: -- Billing Meters don't require subscriptions before reporting usage -- Idempotency keys prevent duplicate event reporting -- 35-day timestamp window for usage events -- Entitlements recommend caching in your database for performance - -Sources: [Stripe Billing](https://stripe.com/billing), [Entitlements API](https://docs.stripe.com/billing/entitlements), [Meters API](https://docs.stripe.com/api/billing/meter) - ---- - -### 2. Paddle - -**Core Value Proposition**: Merchant of Record (MoR) handling all payments, tax compliance, fraud protection, and global payouts. - -**Key APIs/Features**: -- **Subscription lifecycle**: Create, pause, resume, cancel with automatic renewals -- **Tax compliance**: Automatic VAT/GST calculation and remittance globally -- **Fraud protection**: Built-in chargeback handling -- **Checkout**: Hosted or embeddable with localized payment methods -- **Pricing**: 5% + $0.50 per transaction (no monthly fees) - -**Architecture Notes**: -- Paddle Classic vs Paddle Billing (v2) - rebuilt from scratch -- Custom data storage on entities for integration -- Webhook-driven subscription management - -Sources: [Paddle](https://www.paddle.com), [Paddle Developer Docs](https://developer.paddle.com) - ---- - -### 3. LemonSqueezy - -**Core Value Proposition**: Simple merchant of record for digital products with license key management and file hosting. - -**Key APIs/Features**: -- **Products**: Variants, pay-what-you-want, subscription and one-time -- **License keys**: Automatic issuance, deactivation, re-issuance -- **File hosting**: Unlimited files per product, secure delivery -- **Webhooks**: Full webhook management API -- **API**: JSON:API spec with Bearer authentication - -**Architecture Notes**: -- REST API at `api.lemonsqueezy.com/v1/` -- Product status: draft or published -- Good for indie developers and digital products - -Sources: [LemonSqueezy](https://www.lemonsqueezy.com), [API Docs](https://docs.lemonsqueezy.com/api) - ---- - -### 4. Orb - -**Core Value Proposition**: Purpose-built for high-volume usage-based billing with raw event architecture and real-time pricing flexibility. - -**Key APIs/Features**: -- **Event ingestion**: 1,000 events/sec standard, scales to billions/day for enterprise -- **Raw event storage**: Pricing calculated dynamically, not locked at ingestion -- **Billable metrics**: Flexible queries over raw events, auto-materialized -- **Real-time invoicing**: Amounts refresh as events stream in -- **Pricing models**: Tiered, graduated, prepaid consumption, hybrid - -**Architecture Notes**: -- Events require: idempotency_key, customer identifier, timestamp -- Schema-less properties (key/value primitives) -- RevGraph stores all usage data in raw form -- Retrospective pricing changes without rewriting pipelines - -Sources: [Orb](https://www.withorb.com), [Event Ingestion](https://docs.withorb.com/events-and-metrics/event-ingestion) - ---- - -### 5. Lago - -**Core Value Proposition**: Open-source billing API for metering and usage-based pricing, self-hostable with 15,000 events/sec capacity. - -**Key APIs/Features**: -- **Billable metrics**: COUNT, UNIQUE_COUNT, SUM, MAX, LATEST, WEIGHTED_SUM, CUSTOM -- **Event processing**: Real-time aggregation with idempotency -- **Pricing models**: Tiers, packages, seat-based, prepaid credits, minimums -- **Payment agnostic**: Integrates with Stripe, GoCardless, or any processor -- **Invoicing**: Automatic calculation and PDF generation - -**Architecture Notes**: -- AGPLv3 license, self-hosted free forever -- Expression-based unit calculations (ceil, concat, round, +, -, /, *) -- Recurring vs metered metrics (reset to 0 or persist) - -Sources: [Lago](https://www.getlago.com), [GitHub](https://github.com/getlago/lago) - ---- - -### 6. Stigg - -**Core Value Proposition**: Entitlements-first platform for pricing and packaging with instant feature gating and experiment support. - -**Key APIs/Features**: -- **Entitlements**: Metered and unmetered, hard and soft limits -- **Feature gating**: useBooleanEntitlement, useNumericEntitlement, useMeteredEntitlement hooks -- **Caching**: In-memory (30s polling), WebSocket real-time, persistent Redis -- **Sync**: Automatic sync to billing, CRM, CPQ, data warehouse -- **Migration**: 20M+ subscriptions/hour pipeline - -**Architecture Notes**: -- Edge API with 300+ global PoPs, <100ms entitlement checks -- Sidecar service for low-latency with minimal host footprint -- Offline mode with buffered usage metering - -Sources: [Stigg](https://www.stigg.io), [Local Caching](https://docs.stigg.io/docs/local-caching-and-fallback-strategy) - ---- - -## Cloudflare Workers Rewrite Potential - -### Why Edge Billing? - -1. **Usage Metering**: Ingest events at the edge, aggregate in Durable Objects -2. **Entitlement Checks**: Sub-millisecond feature gating from KV cache -3. **Webhook Processing**: Instant webhook handling at global edge -4. **Invoice Generation**: Generate PDFs in Workers with R2 storage - -### Edge-Native Advantages - -| Capability | Traditional | Edge (Workers) | -|------------|-------------|----------------| -| Usage ingestion | Centralized, batched | Real-time, global | -| Entitlement check | API call (~100ms) | KV read (~10ms) | -| Webhook processing | Queue-based | Instant at edge | -| Event deduplication | Database lookup | DO-based state | - ---- - -## Architecture Vision - -``` -billing.do / meter.do -├── metering/ # Edge usage ingestion -│ ├── ingest.ts # High-throughput event endpoint -│ ├── aggregate.ts # DO-based aggregation -│ └── dedup.ts # Idempotency handling -│ -├── entitlements/ # Feature gating -│ ├── cache.ts # KV-based entitlement cache -│ ├── check.ts # Fast boolean/numeric checks -│ └── sync.ts # Stripe/Stigg sync -│ -├── subscriptions/ # Subscription state -│ ├── durable-object/ # SubscriptionDO class -│ ├── lifecycle.ts # Create, update, cancel -│ └── proration.ts # Mid-cycle changes -│ -├── invoicing/ # Invoice engine -│ ├── generator.ts # Line item calculation -│ ├── pdf.ts # PDF generation -│ └── storage.ts # R2 invoice archive -│ -├── pricing/ # Pricing models -│ ├── flat.ts # Flat-rate pricing -│ ├── usage.ts # Usage-based pricing -│ ├── tiered.ts # Tiered/graduated -│ └── hybrid.ts # Combined models -│ -├── webhooks/ # Event processing -│ ├── stripe.ts # Stripe webhook handler -│ ├── paddle.ts # Paddle webhook handler -│ └── internal.ts # Internal event bus -│ -├── adapters/ # Payment processor adapters -│ ├── stripe.ts # Stripe Connect -│ ├── paddle.ts # Paddle Billing -│ └── interface.ts # Common adapter interface -│ -└── analytics/ # Revenue metrics - ├── mrr.ts # MRR/ARR calculations - ├── churn.ts # Churn analysis - └── cohorts.ts # Cohort analysis -``` - ---- - -## Core Components - -### 1. MeterDO (Durable Object) - -```typescript -// rewrites/billing/src/metering/durable-object/meter.ts - -export class MeterDO extends DurableObject { - private sql: SqlStorage - - constructor(ctx: DurableObjectState, env: Env) { - super(ctx, env) - this.sql = ctx.storage.sql - this.initSchema() - } - - private initSchema() { - this.sql.exec(` - CREATE TABLE IF NOT EXISTS events ( - idempotency_key TEXT PRIMARY KEY, - customer_id TEXT NOT NULL, - event_type TEXT NOT NULL, - timestamp INTEGER NOT NULL, - properties TEXT, -- JSON - created_at INTEGER DEFAULT (unixepoch()) - ); - - CREATE TABLE IF NOT EXISTS aggregates ( - customer_id TEXT NOT NULL, - metric_id TEXT NOT NULL, - period_start INTEGER NOT NULL, - period_end INTEGER NOT NULL, - value REAL NOT NULL, - PRIMARY KEY (customer_id, metric_id, period_start) - ); - - CREATE INDEX IF NOT EXISTS idx_events_customer - ON events(customer_id, timestamp); - `) - } - - async ingest(event: MeterEvent): Promise { - // Idempotency check - const existing = this.sql.exec( - `SELECT 1 FROM events WHERE idempotency_key = ?`, - event.idempotencyKey - ).one() - - if (existing) { - return { status: 'duplicate', idempotencyKey: event.idempotencyKey } - } - - // Insert event - this.sql.exec(` - INSERT INTO events (idempotency_key, customer_id, event_type, timestamp, properties) - VALUES (?, ?, ?, ?, ?) - `, event.idempotencyKey, event.customerId, event.eventType, - event.timestamp, JSON.stringify(event.properties)) - - // Update aggregate - await this.updateAggregate(event) - - return { status: 'ingested', idempotencyKey: event.idempotencyKey } - } - - async getUsage(customerId: string, metricId: string, period: Period): Promise { - const result = this.sql.exec(` - SELECT value FROM aggregates - WHERE customer_id = ? AND metric_id = ? - AND period_start = ? AND period_end = ? - `, customerId, metricId, period.start, period.end).one() - - return result?.value ?? 0 - } -} -``` - -### 2. EntitlementCache (KV-based) - -```typescript -// rewrites/billing/src/entitlements/cache.ts - -export interface EntitlementCache { - // Check if customer has boolean feature - hasFeature(customerId: string, featureKey: string): Promise - - // Get numeric limit for feature - getLimit(customerId: string, featureKey: string): Promise - - // Check metered usage against limit - checkUsage(customerId: string, featureKey: string): Promise<{ - current: number - limit: number - remaining: number - exceeded: boolean - }> - - // Sync entitlements from Stripe/source of truth - sync(customerId: string): Promise -} - -export class KVEntitlementCache implements EntitlementCache { - constructor(private kv: KVNamespace, private meterDO: DurableObjectStub) {} - - async hasFeature(customerId: string, featureKey: string): Promise { - const key = `entitlement:${customerId}:${featureKey}` - const cached = await this.kv.get(key) - - if (cached !== null) { - return cached === 'true' - } - - // Cache miss - sync from source - await this.sync(customerId) - const refreshed = await this.kv.get(key) - return refreshed === 'true' - } - - async getLimit(customerId: string, featureKey: string): Promise { - const key = `limit:${customerId}:${featureKey}` - const cached = await this.kv.get(key) - - if (cached !== null) { - return parseInt(cached, 10) - } - - return null - } - - async checkUsage(customerId: string, featureKey: string) { - const [limit, current] = await Promise.all([ - this.getLimit(customerId, featureKey), - this.getCurrentUsage(customerId, featureKey) - ]) - - const numLimit = limit ?? Infinity - return { - current, - limit: numLimit, - remaining: Math.max(0, numLimit - current), - exceeded: current >= numLimit - } - } - - private async getCurrentUsage(customerId: string, featureKey: string): Promise { - // Query MeterDO for current period usage - const period = this.getCurrentBillingPeriod(customerId) - return await this.meterDO.getUsage(customerId, featureKey, period) - } -} -``` - -### 3. SubscriptionDO (Durable Object) - -```typescript -// rewrites/billing/src/subscriptions/durable-object/subscription.ts - -export class SubscriptionDO extends DurableObject { - private sql: SqlStorage - - constructor(ctx: DurableObjectState, env: Env) { - super(ctx, env) - this.sql = ctx.storage.sql - this.initSchema() - } - - private initSchema() { - this.sql.exec(` - CREATE TABLE IF NOT EXISTS subscriptions ( - id TEXT PRIMARY KEY, - customer_id TEXT NOT NULL, - plan_id TEXT NOT NULL, - status TEXT NOT NULL, -- active, canceled, past_due, paused - current_period_start INTEGER NOT NULL, - current_period_end INTEGER NOT NULL, - cancel_at_period_end INTEGER DEFAULT 0, - metadata TEXT, - created_at INTEGER DEFAULT (unixepoch()), - updated_at INTEGER DEFAULT (unixepoch()) - ); - - CREATE TABLE IF NOT EXISTS subscription_items ( - id TEXT PRIMARY KEY, - subscription_id TEXT NOT NULL, - price_id TEXT NOT NULL, - quantity INTEGER DEFAULT 1, - FOREIGN KEY (subscription_id) REFERENCES subscriptions(id) - ); - - CREATE TABLE IF NOT EXISTS prorations ( - id TEXT PRIMARY KEY, - subscription_id TEXT NOT NULL, - type TEXT NOT NULL, -- upgrade, downgrade, quantity_change - amount INTEGER NOT NULL, -- cents - description TEXT, - applied_at INTEGER, - created_at INTEGER DEFAULT (unixepoch()), - FOREIGN KEY (subscription_id) REFERENCES subscriptions(id) - ); - `) - } - - async create(params: CreateSubscriptionParams): Promise { - const id = crypto.randomUUID() - const now = Math.floor(Date.now() / 1000) - const periodEnd = this.calculatePeriodEnd(now, params.billingInterval) - - this.sql.exec(` - INSERT INTO subscriptions - (id, customer_id, plan_id, status, current_period_start, current_period_end) - VALUES (?, ?, ?, 'active', ?, ?) - `, id, params.customerId, params.planId, now, periodEnd) - - // Create subscription items - for (const item of params.items) { - this.sql.exec(` - INSERT INTO subscription_items (id, subscription_id, price_id, quantity) - VALUES (?, ?, ?, ?) - `, crypto.randomUUID(), id, item.priceId, item.quantity ?? 1) - } - - // Sync entitlements to KV - await this.syncEntitlements(id) - - return this.get(id)! - } - - async update(id: string, params: UpdateSubscriptionParams): Promise { - const current = this.get(id) - if (!current) throw new Error('Subscription not found') - - // Calculate proration if changing plan - if (params.planId && params.planId !== current.planId) { - const proration = this.calculateProration(current, params.planId) - this.sql.exec(` - INSERT INTO prorations (id, subscription_id, type, amount, description) - VALUES (?, ?, ?, ?, ?) - `, crypto.randomUUID(), id, proration.type, proration.amount, proration.description) - } - - // Update subscription - this.sql.exec(` - UPDATE subscriptions - SET plan_id = COALESCE(?, plan_id), - status = COALESCE(?, status), - cancel_at_period_end = COALESCE(?, cancel_at_period_end), - updated_at = unixepoch() - WHERE id = ? - `, params.planId, params.status, params.cancelAtPeriodEnd ? 1 : 0, id) - - // Re-sync entitlements - await this.syncEntitlements(id) - - return this.get(id)! - } - - private calculateProration(current: Subscription, newPlanId: string): Proration { - const now = Math.floor(Date.now() / 1000) - const periodTotal = current.currentPeriodEnd - current.currentPeriodStart - const periodRemaining = current.currentPeriodEnd - now - const ratio = periodRemaining / periodTotal - - const currentPlan = this.getPlan(current.planId) - const newPlan = this.getPlan(newPlanId) - - const currentProrated = Math.round(currentPlan.amount * ratio) - const newProrated = Math.round(newPlan.amount * ratio) - const amount = newProrated - currentProrated - - return { - type: amount > 0 ? 'upgrade' : 'downgrade', - amount, - description: `Prorated ${amount > 0 ? 'charge' : 'credit'} for plan change` - } - } -} -``` - -### 4. InvoiceEngine - -```typescript -// rewrites/billing/src/invoicing/generator.ts - -export interface InvoiceLineItem { - description: string - quantity: number - unitAmount: number // cents - amount: number // cents - period?: { start: number; end: number } -} - -export interface Invoice { - id: string - customerId: string - subscriptionId?: string - status: 'draft' | 'open' | 'paid' | 'void' - currency: string - lineItems: InvoiceLineItem[] - subtotal: number - tax: number - total: number - dueDate: number - pdfUrl?: string - createdAt: number -} - -export class InvoiceEngine { - constructor( - private subscriptionDO: DurableObjectStub, - private meterDO: DurableObjectStub, - private pricingEngine: PricingEngine, - private r2: R2Bucket - ) {} - - async generateInvoice(customerId: string, subscriptionId: string): Promise { - const subscription = await this.subscriptionDO.get(subscriptionId) - if (!subscription) throw new Error('Subscription not found') - - const lineItems: InvoiceLineItem[] = [] - - // Add subscription line items - for (const item of subscription.items) { - const price = await this.pricingEngine.getPrice(item.priceId) - - if (price.type === 'recurring') { - lineItems.push({ - description: price.nickname || price.productName, - quantity: item.quantity, - unitAmount: price.unitAmount, - amount: price.unitAmount * item.quantity, - period: { - start: subscription.currentPeriodStart, - end: subscription.currentPeriodEnd - } - }) - } else if (price.type === 'metered') { - const usage = await this.meterDO.getUsage( - customerId, - price.meterId, - { - start: subscription.currentPeriodStart, - end: subscription.currentPeriodEnd - } - ) - - const amount = await this.pricingEngine.calculateUsageAmount( - price.id, - usage - ) - - lineItems.push({ - description: `${price.nickname || price.productName} (${usage} units)`, - quantity: usage, - unitAmount: amount / usage, - amount, - period: { - start: subscription.currentPeriodStart, - end: subscription.currentPeriodEnd - } - }) - } - } - - // Add prorations - const prorations = await this.subscriptionDO.getProrations(subscriptionId) - for (const proration of prorations) { - if (!proration.appliedAt) { - lineItems.push({ - description: proration.description, - quantity: 1, - unitAmount: proration.amount, - amount: proration.amount - }) - } - } - - const subtotal = lineItems.reduce((sum, item) => sum + item.amount, 0) - const tax = await this.calculateTax(customerId, subtotal) - - const invoice: Invoice = { - id: crypto.randomUUID(), - customerId, - subscriptionId, - status: 'draft', - currency: 'usd', - lineItems, - subtotal, - tax, - total: subtotal + tax, - dueDate: subscription.currentPeriodEnd, - createdAt: Math.floor(Date.now() / 1000) - } - - return invoice - } - - async generatePDF(invoice: Invoice): Promise { - const html = await this.renderInvoiceHTML(invoice) - const pdf = await this.htmlToPDF(html) - - const key = `invoices/${invoice.customerId}/${invoice.id}.pdf` - await this.r2.put(key, pdf, { - httpMetadata: { contentType: 'application/pdf' } - }) - - return key - } -} -``` - -### 5. PricingEngine - -```typescript -// rewrites/billing/src/pricing/engine.ts - -export interface Price { - id: string - productId: string - type: 'one_time' | 'recurring' | 'metered' - billingScheme: 'per_unit' | 'tiered' - unitAmount?: number - tiers?: PriceTier[] - meterId?: string - transformQuantity?: { - divideBy: number - round: 'up' | 'down' - } -} - -export interface PriceTier { - upTo: number | null // null = infinity - unitAmount?: number - flatAmount?: number -} - -export class PricingEngine { - calculateUsageAmount(price: Price, usage: number): number { - if (price.billingScheme === 'per_unit') { - return this.calculatePerUnit(price, usage) - } else { - return this.calculateTiered(price, usage) - } - } - - private calculatePerUnit(price: Price, usage: number): number { - let quantity = usage - - if (price.transformQuantity) { - quantity = price.transformQuantity.round === 'up' - ? Math.ceil(usage / price.transformQuantity.divideBy) - : Math.floor(usage / price.transformQuantity.divideBy) - } - - return quantity * (price.unitAmount ?? 0) - } - - private calculateTiered(price: Price, usage: number): number { - if (!price.tiers) return 0 - - let total = 0 - let remaining = usage - - for (const tier of price.tiers) { - if (remaining <= 0) break - - const tierLimit = tier.upTo ?? Infinity - const inTier = Math.min(remaining, tierLimit) - - if (tier.flatAmount) { - total += tier.flatAmount - } - - if (tier.unitAmount) { - total += inTier * tier.unitAmount - } - - remaining -= inTier - } - - return total - } - - // Graduated pricing (each tier only applies to units in that tier) - calculateGraduated(price: Price, usage: number): number { - if (!price.tiers) return 0 - - let total = 0 - let previousLimit = 0 - - for (const tier of price.tiers) { - const tierLimit = tier.upTo ?? Infinity - - if (usage <= previousLimit) break - - const inTier = Math.min(usage, tierLimit) - previousLimit - - if (tier.flatAmount && usage > previousLimit) { - total += tier.flatAmount - } - - if (tier.unitAmount) { - total += inTier * tier.unitAmount - } - - previousLimit = tierLimit - } - - return total - } - - // Volume pricing (tier applies to ALL units) - calculateVolume(price: Price, usage: number): number { - if (!price.tiers) return 0 - - for (const tier of price.tiers) { - const tierLimit = tier.upTo ?? Infinity - - if (usage <= tierLimit) { - let total = tier.flatAmount ?? 0 - if (tier.unitAmount) { - total += usage * tier.unitAmount - } - return total - } - } - - return 0 - } -} -``` - ---- - -## Integration Points - -### Stripe Connect (Actual Payments) - -```typescript -// rewrites/billing/src/adapters/stripe.ts - -export class StripeAdapter implements PaymentAdapter { - private stripe: Stripe - - constructor(secretKey: string) { - this.stripe = new Stripe(secretKey) - } - - async createCustomer(params: CreateCustomerParams): Promise { - const customer = await this.stripe.customers.create({ - email: params.email, - name: params.name, - metadata: params.metadata - }) - return customer.id - } - - async createSubscription(params: CreateSubscriptionParams): Promise { - const subscription = await this.stripe.subscriptions.create({ - customer: params.customerId, - items: params.items.map(item => ({ - price: item.priceId, - quantity: item.quantity - })), - payment_behavior: 'default_incomplete', - expand: ['latest_invoice.payment_intent'] - }) - return subscription.id - } - - async reportUsage(subscriptionItemId: string, quantity: number): Promise { - // Use new Meter API - await this.stripe.billing.meterEvents.create({ - event_name: 'usage', - payload: { - stripe_customer_id: customerId, - value: quantity.toString() - } - }) - } - - async chargeInvoice(invoiceId: string): Promise { - await this.stripe.invoices.pay(invoiceId) - } -} -``` - -### Feature Flags (Entitlements) - -```typescript -// rewrites/billing/src/entitlements/middleware.ts - -export function requireFeature(featureKey: string) { - return async (c: Context, next: Next) => { - const customerId = c.get('customerId') - const cache = c.get('entitlementCache') as EntitlementCache - - const hasFeature = await cache.hasFeature(customerId, featureKey) - - if (!hasFeature) { - return c.json({ - error: 'feature_not_available', - message: `Your plan does not include access to ${featureKey}`, - upgradeUrl: `/billing/upgrade?feature=${featureKey}` - }, 403) - } - - await next() - } -} - -export function checkUsageLimit(featureKey: string) { - return async (c: Context, next: Next) => { - const customerId = c.get('customerId') - const cache = c.get('entitlementCache') as EntitlementCache - - const usage = await cache.checkUsage(customerId, featureKey) - - if (usage.exceeded) { - return c.json({ - error: 'usage_limit_exceeded', - message: `You have exceeded your ${featureKey} limit`, - current: usage.current, - limit: usage.limit, - upgradeUrl: `/billing/upgrade?feature=${featureKey}` - }, 429) - } - - // Add usage info to context for metering - c.set('usageInfo', usage) - - await next() - } -} -``` - -### Analytics (Revenue Metrics) - -```typescript -// rewrites/billing/src/analytics/mrr.ts - -export class RevenueAnalytics { - constructor(private sql: SqlStorage) {} - - async calculateMRR(): Promise { - const result = this.sql.exec(` - SELECT - SUM(CASE WHEN status = 'active' THEN monthly_amount ELSE 0 END) as mrr, - SUM(CASE - WHEN created_at >= date('now', '-1 month') - THEN monthly_amount ELSE 0 - END) as new_mrr, - SUM(CASE - WHEN previous_amount < monthly_amount - THEN monthly_amount - previous_amount ELSE 0 - END) as expansion_mrr, - SUM(CASE - WHEN previous_amount > monthly_amount - THEN previous_amount - monthly_amount ELSE 0 - END) as contraction_mrr, - SUM(CASE - WHEN status = 'canceled' AND canceled_at >= date('now', '-1 month') - THEN monthly_amount ELSE 0 - END) as churned_mrr - FROM subscriptions - WHERE status IN ('active', 'canceled') - `).one() - - return { - mrr: result.mrr, - newMRR: result.new_mrr, - expansionMRR: result.expansion_mrr, - contractionMRR: result.contraction_mrr, - churnedMRR: result.churned_mrr, - netNewMRR: result.new_mrr + result.expansion_mrr - result.contraction_mrr - result.churned_mrr - } - } - - async calculateChurnRate(period: 'monthly' | 'annual'): Promise { - const periodDays = period === 'monthly' ? 30 : 365 - - const result = this.sql.exec(` - SELECT - COUNT(CASE WHEN status = 'canceled' AND canceled_at >= date('now', '-${periodDays} days') THEN 1 END) as churned, - COUNT(*) as total - FROM subscriptions - WHERE created_at < date('now', '-${periodDays} days') - `).one() - - return result.total > 0 ? (result.churned / result.total) * 100 : 0 - } -} -``` - ---- - -## Deep Dive: Key Technical Challenges - -### 1. Usage Aggregation Strategies - -**Edge Ingestion Pattern**: -```typescript -// High-throughput edge ingestion with DO fan-out -export default { - async fetch(request: Request, env: Env) { - const events = await request.json() - - // Fan out to customer-specific DOs for aggregation - const promises = events.map(async (event) => { - const doId = env.METER.idFromName(event.customerId) - const stub = env.METER.get(doId) - return stub.ingest(event) - }) - - const results = await Promise.allSettled(promises) - return Response.json({ - ingested: results.filter(r => r.status === 'fulfilled').length, - failed: results.filter(r => r.status === 'rejected').length - }) - } -} -``` - -**Aggregation Windows**: -- **Real-time**: Per-event updates in DO SQLite -- **Hourly**: Background job aggregates to hourly buckets -- **Daily**: Roll up hourly to daily for reporting -- **Billing period**: Sum daily for invoice generation - -### 2. Billing Period Handling - -```typescript -interface BillingPeriod { - start: number // Unix timestamp - end: number - interval: 'day' | 'week' | 'month' | 'year' - intervalCount: number - anchorDate?: number // For anniversary billing -} - -function calculateNextPeriod(current: BillingPeriod): BillingPeriod { - const startDate = new Date(current.end * 1000) - let endDate: Date - - switch (current.interval) { - case 'day': - endDate = addDays(startDate, current.intervalCount) - break - case 'week': - endDate = addWeeks(startDate, current.intervalCount) - break - case 'month': - endDate = addMonths(startDate, current.intervalCount) - // Handle month-end anchoring - if (current.anchorDate) { - const day = Math.min(current.anchorDate, getDaysInMonth(endDate)) - endDate.setDate(day) - } - break - case 'year': - endDate = addYears(startDate, current.intervalCount) - break - } - - return { - start: Math.floor(startDate.getTime() / 1000), - end: Math.floor(endDate.getTime() / 1000), - interval: current.interval, - intervalCount: current.intervalCount, - anchorDate: current.anchorDate - } -} -``` - -### 3. Proration Calculations - -```typescript -interface ProrationContext { - subscription: Subscription - oldPrice: Price - newPrice: Price - effectiveDate: number -} - -function calculateProration(ctx: ProrationContext): number { - const { subscription, oldPrice, newPrice, effectiveDate } = ctx - - // Time-based proration - const periodTotal = subscription.currentPeriodEnd - subscription.currentPeriodStart - const periodElapsed = effectiveDate - subscription.currentPeriodStart - const periodRemaining = subscription.currentPeriodEnd - effectiveDate - - // Calculate unused portion of old plan - const oldUnused = (oldPrice.unitAmount * periodRemaining) / periodTotal - - // Calculate cost of new plan for remaining period - const newCost = (newPrice.unitAmount * periodRemaining) / periodTotal - - // Proration amount (positive = charge, negative = credit) - return Math.round(newCost - oldUnused) -} - -// Different proration behaviors -type ProrationBehavior = - | 'create_prorations' // Default - charge/credit immediately - | 'none' // No proration, full charge at next renewal - | 'always_invoice' // Invoice proration immediately - | 'pending_if_incomplete' // Queue if payment incomplete -``` - -### 4. Multi-Currency Support - -```typescript -interface CurrencyConfig { - code: string // ISO 4217 - symbol: string - decimalPlaces: number - smallestUnit: number // Cents, pence, etc. -} - -const CURRENCIES: Record = { - usd: { code: 'USD', symbol: '$', decimalPlaces: 2, smallestUnit: 1 }, - eur: { code: 'EUR', symbol: '€', decimalPlaces: 2, smallestUnit: 1 }, - gbp: { code: 'GBP', symbol: '£', decimalPlaces: 2, smallestUnit: 1 }, - jpy: { code: 'JPY', symbol: '¥', decimalPlaces: 0, smallestUnit: 1 }, - // ... -} - -interface MultiCurrencyPrice { - default: { currency: string; amount: number } - localized: Record // currency -> amount -} - -function getPriceForCustomer( - price: MultiCurrencyPrice, - customer: Customer -): { currency: string; amount: number } { - // Check for localized price - if (customer.currency && price.localized[customer.currency]) { - return { - currency: customer.currency, - amount: price.localized[customer.currency] - } - } - - // Fall back to default - return price.default -} - -// Exchange rate handling for cosmetic localization -async function convertCurrency( - amount: number, - from: string, - to: string, - rates: ExchangeRates -): Promise { - if (from === to) return amount - - const fromRate = rates[from] - const toRate = rates[to] - - // Convert to USD (base) then to target - const inUSD = amount / fromRate - return Math.round(inUSD * toRate) -} -``` - ---- - -## SDK Interface - -```typescript -// sdks/billing.do/index.ts - -import { createClient, type ClientOptions } from 'rpc.do' - -export interface BillingClient { - // Metering - meter: { - ingest(event: MeterEvent): Promise - ingestBatch(events: MeterEvent[]): Promise - getUsage(customerId: string, metricId: string, period: Period): Promise - } - - // Entitlements - entitlements: { - check(customerId: string, featureKey: string): Promise - list(customerId: string): Promise - sync(customerId: string): Promise - } - - // Subscriptions - subscriptions: { - create(params: CreateSubscriptionParams): Promise - get(id: string): Promise - update(id: string, params: UpdateSubscriptionParams): Promise - cancel(id: string, options?: CancelOptions): Promise - list(customerId?: string): Promise - } - - // Invoices - invoices: { - create(params: CreateInvoiceParams): Promise - get(id: string): Promise - finalize(id: string): Promise - pay(id: string): Promise - getPDF(id: string): Promise - list(customerId?: string): Promise - } - - // Pricing - prices: { - create(params: CreatePriceParams): Promise - get(id: string): Promise - list(productId?: string): Promise - calculate(priceId: string, quantity: number): Promise - } - - // Webhooks - webhooks: { - constructEvent(payload: string, signature: string, secret: string): WebhookEvent - } - - // Analytics - analytics: { - mrr(): Promise - churn(period: 'monthly' | 'annual'): Promise - cohorts(params: CohortParams): Promise - } -} - -export function Billing(options?: ClientOptions): BillingClient { - return createClient('https://billing.do', options) -} - -export const billing: BillingClient = Billing() - -export default billing -``` - ---- - -## Implementation Phases - -### Phase 1: Core Metering (Week 1-2) -- [ ] MeterDO with event ingestion and aggregation -- [ ] Idempotency handling -- [ ] Basic aggregation types (COUNT, SUM) -- [ ] Usage query API - -### Phase 2: Entitlements (Week 3) -- [ ] KV-based entitlement cache -- [ ] Boolean and numeric feature checks -- [ ] Metered entitlement checks -- [ ] Stripe entitlements sync - -### Phase 3: Subscriptions (Week 4-5) -- [ ] SubscriptionDO with lifecycle management -- [ ] Proration calculations -- [ ] Plan change handling -- [ ] Stripe subscription sync - -### Phase 4: Invoicing (Week 6) -- [ ] Invoice generation from subscriptions -- [ ] Usage-based line items -- [ ] PDF generation -- [ ] R2 storage - -### Phase 5: Pricing Engine (Week 7) -- [ ] Per-unit pricing -- [ ] Tiered pricing (graduated and volume) -- [ ] Transform quantity -- [ ] Multi-currency support - -### Phase 6: Analytics & Polish (Week 8) -- [ ] MRR/ARR calculations -- [ ] Churn analysis -- [ ] SDK finalization -- [ ] Documentation - ---- - -## File Structure - -``` -rewrites/billing/ -├── .beads/ -│ └── issues.jsonl -├── src/ -│ ├── metering/ -│ │ ├── durable-object/ -│ │ │ └── meter.ts -│ │ ├── ingest.ts -│ │ ├── aggregate.ts -│ │ ├── aggregate.test.ts -│ │ └── index.ts -│ ├── entitlements/ -│ │ ├── cache.ts -│ │ ├── cache.test.ts -│ │ ├── check.ts -│ │ ├── middleware.ts -│ │ ├── sync.ts -│ │ └── index.ts -│ ├── subscriptions/ -│ │ ├── durable-object/ -│ │ │ └── subscription.ts -│ │ ├── lifecycle.ts -│ │ ├── lifecycle.test.ts -│ │ ├── proration.ts -│ │ ├── proration.test.ts -│ │ └── index.ts -│ ├── invoicing/ -│ │ ├── generator.ts -│ │ ├── generator.test.ts -│ │ ├── pdf.ts -│ │ ├── storage.ts -│ │ └── index.ts -│ ├── pricing/ -│ │ ├── engine.ts -│ │ ├── engine.test.ts -│ │ ├── flat.ts -│ │ ├── usage.ts -│ │ ├── tiered.ts -│ │ ├── currency.ts -│ │ └── index.ts -│ ├── webhooks/ -│ │ ├── stripe.ts -│ │ ├── paddle.ts -│ │ ├── internal.ts -│ │ └── index.ts -│ ├── adapters/ -│ │ ├── stripe.ts -│ │ ├── paddle.ts -│ │ ├── interface.ts -│ │ └── index.ts -│ ├── analytics/ -│ │ ├── mrr.ts -│ │ ├── churn.ts -│ │ ├── cohorts.ts -│ │ └── index.ts -│ └── index.ts -├── package.json -├── tsconfig.json -├── vitest.config.ts -├── wrangler.toml -└── SCOPE.md -``` - ---- - -## Sources - -### Stripe Billing -- [Stripe Billing Overview](https://stripe.com/billing) -- [Subscriptions API](https://docs.stripe.com/api/subscriptions) -- [Billing Meters](https://docs.stripe.com/api/billing/meter) -- [Entitlements API](https://docs.stripe.com/billing/entitlements) -- [Record Usage API](https://docs.stripe.com/billing/subscriptions/usage-based/recording-usage-api) - -### Paddle -- [Paddle Platform](https://www.paddle.com) -- [Paddle Developer Docs](https://developer.paddle.com) -- [Webhooks Overview](https://developer.paddle.com/webhooks/overview) - -### LemonSqueezy -- [LemonSqueezy](https://www.lemonsqueezy.com) -- [API Reference](https://docs.lemonsqueezy.com/api) -- [Digital Products](https://www.lemonsqueezy.com/ecommerce/digital-products) - -### Orb -- [Orb Platform](https://www.withorb.com) -- [Event Ingestion](https://docs.withorb.com/events-and-metrics/event-ingestion) -- [Metering](https://www.withorb.com/products/metering) - -### Lago -- [Lago](https://www.getlago.com) -- [GitHub Repository](https://github.com/getlago/lago) -- [Billable Metrics](https://doc.getlago.com/api-reference/billable-metrics/object) - -### Stigg -- [Stigg Platform](https://www.stigg.io) -- [Local Caching](https://docs.stigg.io/docs/local-caching-and-fallback-strategy) -- [Persistent Caching](https://docs.stigg.io/docs/persistent-caching) -- [Entitlements Blog](https://www.stigg.io/blog-posts/entitlements-untangled-the-modern-way-to-software-monetization) diff --git a/rewrites/docs/research/errors-scope.md b/rewrites/docs/research/errors-scope.md deleted file mode 100644 index a99d3421..00000000 --- a/rewrites/docs/research/errors-scope.md +++ /dev/null @@ -1,767 +0,0 @@ -# errors.do - Error Monitoring Rewrite Scoping Document - -## Executive Summary - -This document scopes a Cloudflare Workers-native error monitoring platform that provides Sentry SDK compatibility with edge-native processing. The goal is to deliver low-latency error ingestion, intelligent issue grouping, and seamless integration with the workers.do ecosystem. - -**Domain Options**: `errors.do`, `sentry.do`, `bugs.do`, `exceptions.do` - ---- - -## 1. Platform Analysis - -### 1.1 Sentry - Market Leader - -**Core Value Proposition**: End-to-end error tracking with source map support, issue grouping, and release management. - -**Key Technical Details**: -- **Envelope Protocol**: Binary format for batching errors, attachments, sessions - - Endpoint: `POST /api/{PROJECT_ID}/envelope/` - - Headers + Items with individual payloads - - Max 40MB compressed, 200MB decompressed - - Supports: events, transactions, attachments, sessions, logs -- **Issue Grouping**: Multi-stage fingerprinting - - Stack trace normalization - - Server-side fingerprint rules - - AI-powered semantic grouping (new) - - Hierarchical hashes for group subdivision -- **Source Maps**: Rust-based `symbolic` crate for symbolication - - Source map upload via release artifacts - - JS-specific: module name + filename + context-line - - Native: function name cleaning (generics, params removed) -- **Architecture**: Relay (Rust edge proxy) -> Kafka -> Workers/Symbolication -> ClickHouse - -**Cloudflare Integration**: Official `@sentry/cloudflare` SDK with `withSentry()` wrapper. - -**Rewrite Opportunity**: Sentry Relay is already a Rust edge proxy - we can replace the entire backend. - -### 1.2 Bugsnag - -**Core Value Proposition**: Error monitoring with stability scores and breadcrumb tracking. - -**Key Technical Details**: -- REST API at `notify.bugsnag.com` -- JSON payload with apiKey, releaseStage, user, context, metaData -- Breadcrumb system: circular buffer of 25 events -- GroupingHash override for custom deduplication -- Session tracking for stability scores - -**Differentiator**: Automatic device/app state capture, feature flag correlation. - -### 1.3 Rollbar - -**Core Value Proposition**: Real-time error aggregation with custom fingerprinting. - -**Key Technical Details**: -- REST API at `api.rollbar.com/api/1/item/` -- JSON payload with environment, level, body (trace/message/crash) -- Max 1024KB payload -- Telemetry timeline ("breadcrumbs") -- Custom fingerprinting via payload handlers - -**Differentiator**: Simple API, strong Python ecosystem. - -### 1.4 Datadog Error Tracking - -**Core Value Proposition**: Unified APM with errors, traces, and logs correlation. - -**Key Technical Details**: -- Error tracking built on APM spans -- Required attributes: `error.stack`, `error.message`, `error.type` -- Fingerprinting: error type + message + stack frames -- Correlation via `DD_ENV`, `DD_SERVICE`, `DD_VERSION` tags -- OpenTelemetry compatible - -**Differentiator**: Full-stack observability in one platform. - -### 1.5 LogRocket - -**Core Value Proposition**: Session replay with error context. - -**Key Technical Details**: -- DOM recording via MutationObserver (rrweb-based) -- Network request capture -- Console log capture -- `captureException(error)` for server-side -- User identification via `identify()` method - -**Differentiator**: Visual debugging - see exactly what user was doing. - -### 1.6 Highlight.io - -**Core Value Proposition**: Open-source full-stack monitoring. - -**Key Technical Details**: -- Session replay + error monitoring + logging + tracing -- OpenTelemetry integration -- rrweb for DOM recording -- Self-hostable (Docker, 8GB RAM minimum) -- Recently acquired by LaunchDarkly (March 2025) - -**Differentiator**: Open source, self-hosted option. - ---- - -## 2. Cloudflare Workers Rewrite Architecture - -### 2.1 Architecture Overview - -``` -errors.do -├── Edge Ingestion Layer (Cloudflare Workers) -│ ├── /api/{project}/envelope/ - Sentry protocol -│ ├── /api/{project}/store/ - Legacy Sentry -│ ├── /notify - Bugsnag protocol -│ └── /api/1/item/ - Rollbar protocol -│ -├── Processing Layer (Durable Objects) -│ ├── ErrorIngestionDO - Rate limiting, sampling -│ ├── IssueGroupingDO - Fingerprinting, dedup -│ ├── SymbolicationDO - Source map processing -│ └── AlertingDO - Real-time notifications -│ -├── Storage Layer -│ ├── D1 - Issues, fingerprints, metadata -│ ├── R2 - Source maps, attachments -│ ├── KV - Hot cache, rate limits -│ └── Analytics Engine - Time-series error data -│ -└── Query Layer - ├── Dashboard API - REST/GraphQL - ├── MCP Tools - AI agent integration - └── WebSocket - Real-time updates -``` - -### 2.2 Durable Object Design - -#### ErrorIngestionDO - -Per-project Durable Object for ingestion control: - -```typescript -export class ErrorIngestionDO extends DurableObject { - // SQLite tables - // - rate_limits: token bucket per client - // - sampling_config: rules for which errors to keep - // - project_settings: DSN, auth, quotas - - async ingest(envelope: SentryEnvelope): Promise { - // 1. Authenticate (DSN validation) - // 2. Rate limit check - // 3. Sampling decision - // 4. Parse envelope items - // 5. Route to appropriate processor - } -} -``` - -#### IssueGroupingDO - -Per-project Durable Object for issue management: - -```typescript -export class IssueGroupingDO extends DurableObject { - // SQLite tables - // - issues: id, fingerprint, title, first_seen, last_seen - // - events: id, issue_id, timestamp, payload_hash - // - fingerprint_rules: custom grouping rules - - async processEvent(event: ErrorEvent): Promise { - // 1. Normalize stack trace - // 2. Apply fingerprint rules - // 3. Generate fingerprint hash - // 4. Find or create issue - // 5. Update issue statistics - // 6. Trigger alerts if new issue - } -} -``` - -#### SymbolicationDO - -Singleton DO for source map processing: - -```typescript -export class SymbolicationDO extends DurableObject { - // SQLite tables - // - source_maps: release, filename, r2_key - // - symbolication_cache: frame_hash -> symbolicated - - async symbolicate( - stacktrace: Stacktrace, - release: string - ): Promise { - // 1. Check cache for each frame - // 2. Load source maps from R2 - // 3. Parse with source-map-js (WASM too heavy for DO) - // 4. Apply source map to each frame - // 5. Cache results - // 6. Return enhanced stack trace - } -} -``` - -### 2.3 Protocol Support Matrix - -| Protocol | Endpoint | Priority | Compatibility | -|----------|----------|----------|---------------| -| Sentry Envelope | `/api/{project}/envelope/` | P0 | Drop-in SDK | -| Sentry Store (legacy) | `/api/{project}/store/` | P1 | Legacy support | -| Bugsnag | `/notify` | P2 | SDK compatible | -| Rollbar | `/api/1/item/` | P2 | SDK compatible | -| OpenTelemetry | `/v1/traces`, `/v1/logs` | P1 | OTLP export | -| Custom | `/api/errors` | P0 | Native SDK | - -### 2.4 Storage Strategy - -**Hot Storage (D1/SQLite in DO)**: -- Active issues (last 7 days) -- Recent events (last 24 hours) -- Fingerprint mappings -- Rate limit counters -- Sampling decisions - -**Warm Storage (R2)**: -- Source maps (per release) -- Event payloads (compressed JSON) -- Attachments (screenshots, logs) -- Session replay data - -**Cold Storage (R2 Archive)**: -- Historical events (>30 days) -- Audit logs -- Compliance data - -**Analytics (Analytics Engine)**: -- Error counts over time -- Error rates by release -- Geographic distribution -- Browser/device breakdowns - ---- - -## 3. Key Features Specification - -### 3.1 Error Ingestion - -**Sentry Envelope Parsing**: -```typescript -interface EnvelopeHeader { - event_id?: string - dsn?: string - sdk?: { name: string; version: string } - sent_at?: string // RFC 3339 -} - -interface EnvelopeItem { - type: 'event' | 'transaction' | 'attachment' | 'session' - length?: number - content_type?: string - filename?: string -} -``` - -**Rate Limiting**: -- Token bucket per project (configurable burst/rate) -- Dynamic sampling based on quota usage -- Burst protection for sudden spikes -- Graceful degradation (accept envelope, defer processing) - -### 3.2 Issue Grouping Algorithm - -**Phase 1: Stack Trace Normalization** -- Remove PII from paths -- Normalize file paths (remove hash suffixes) -- Mark frames as in-app vs system -- Clean function names (remove anonymous wrappers) - -**Phase 2: Fingerprint Generation** -```typescript -function generateFingerprint(event: ErrorEvent): string[] { - const components: string[] = [] - - // 1. Exception type and message (cleaned) - if (event.exception) { - components.push(event.exception.type) - components.push(cleanMessage(event.exception.value)) - } - - // 2. Stack trace (in-app frames only) - const frames = event.stacktrace?.frames ?? [] - for (const frame of frames.filter(f => f.in_app)) { - components.push(`${frame.module}:${frame.function}:${frame.lineno}`) - } - - return [hash(components.join('\n'))] -} -``` - -**Phase 3: Custom Rules** -```yaml -# Example fingerprint rules -rules: - - match: - type: "NetworkError" - message: "*timeout*" - fingerprint: ["network-timeout"] - - - match: - function: "handleApiError" - fingerprint: ["{{ default }}", "{{ tags.endpoint }}"] -``` - -### 3.3 Source Map Processing - -**Challenges**: -- Source maps can be large (>10MB for enterprise apps) -- Parsing is CPU-intensive -- Multiple source maps per release -- Mapping accuracy depends on build tooling - -**Strategy**: -1. **Upload**: Source maps uploaded via CLI/CI during release -2. **Storage**: Compressed in R2 with release/filename index -3. **Lazy Loading**: Load source maps on-demand, not preloaded -4. **Caching**: Cache symbolicated frames (hash of frame -> result) -5. **Library**: Use `source-map-js` (pure JS, no WASM overhead) - -```typescript -// Source map storage schema -interface SourceMapEntry { - release: string - filename: string // Minified file path - r2_key: string // R2 object key - uploaded_at: number - size_bytes: number - map_hash: string // For cache invalidation -} -``` - -### 3.4 PII Scrubbing - -**Default Scrubbing Rules**: -- Email addresses -- IP addresses -- Credit card numbers -- Phone numbers -- Social security numbers -- API keys/tokens (pattern matching) - -**Implementation**: -```typescript -const scrubbers = [ - { pattern: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, replacement: '[email]' }, - { pattern: /\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b/g, replacement: '[ip]' }, - { pattern: /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g, replacement: '[card]' }, - { pattern: /\b(sk_live_|pk_live_|sk_test_|pk_test_)[a-zA-Z0-9]+\b/g, replacement: '[api_key]' }, -] -``` - -### 3.5 Real-Time Alerting - -**Alert Triggers**: -- New issue detected -- Issue regression (reoccurrence after resolution) -- Error rate spike (>N% increase in window) -- Quota threshold reached - -**Alert Channels**: -- Webhook (generic) -- Slack -- Discord -- Email -- PagerDuty - -**Implementation via Durable Object Alarms**: -```typescript -class AlertingDO extends DurableObject { - async alarm() { - // Check pending alerts - const alerts = await this.sql` - SELECT * FROM pending_alerts WHERE processed = 0 - ` - for (const alert of alerts) { - await this.dispatch(alert) - await this.sql`UPDATE pending_alerts SET processed = 1 WHERE id = ${alert.id}` - } - // Schedule next check - this.ctx.storage.setAlarm(Date.now() + 10_000) // 10s - } -} -``` - ---- - -## 4. SDK Strategy - -### 4.1 Sentry SDK Compatibility - -**Goal**: Zero-code migration from Sentry by pointing DSN to errors.do - -```typescript -import * as Sentry from '@sentry/browser' - -Sentry.init({ - dsn: 'https://key@errors.do/123', // Just change the host! - // All existing config works -}) -``` - -**Implementation**: -- Accept Sentry envelope format exactly -- Support Sentry auth header format -- Return Sentry-compatible responses -- Handle Sentry SDK version quirks - -### 4.2 Native SDK (errors.do) - -For new projects, provide a lightweight native SDK: - -```typescript -import { Errors } from 'errors.do' - -const errors = Errors({ - dsn: 'https://key@errors.do/123', - release: '1.0.0', - environment: 'production', -}) - -// Auto-capture -errors.install() - -// Manual capture -errors.captureException(new Error('Something went wrong')) -errors.captureMessage('User clicked deprecated button', 'warning') - -// Add context -errors.setUser({ id: '123', email: 'user@example.com' }) -errors.setTag('feature', 'checkout') -errors.addBreadcrumb({ category: 'ui', message: 'Button clicked' }) -``` - -### 4.3 Cloudflare Workers SDK - -Deep integration for Workers: - -```typescript -import { withErrors } from 'errors.do/cloudflare' - -export default withErrors({ - dsn: 'https://key@errors.do/123', - handler: { - async fetch(request, env, ctx) { - // Errors automatically captured - throw new Error('Oops!') - } - } -}) -``` - ---- - -## 5. API Design - -### 5.1 Ingestion Endpoints - -``` -POST /api/{project_id}/envelope/ - - Sentry envelope format - - Auth via X-Sentry-Auth header or DSN in envelope - -POST /api/{project_id}/store/ - - Legacy Sentry JSON event - - Deprecated but supported - -POST /notify - - Bugsnag format - - Auth via Bugsnag-Api-Key header - -POST /api/1/item/ - - Rollbar format - - Auth via access_token in payload -``` - -### 5.2 Management API - -``` -GET /api/projects # List projects -POST /api/projects # Create project -GET /api/projects/{id} # Get project -DELETE /api/projects/{id} # Delete project - -GET /api/projects/{id}/issues # List issues -GET /api/projects/{id}/issues/{id} # Get issue details -POST /api/projects/{id}/issues/{id}/resolve # Resolve issue -POST /api/projects/{id}/issues/{id}/ignore # Ignore issue - -GET /api/projects/{id}/events # List events -GET /api/projects/{id}/events/{id} # Get event details - -POST /api/projects/{id}/releases # Create release -POST /api/projects/{id}/sourcemaps # Upload source maps - -GET /api/projects/{id}/stats # Error statistics -``` - -### 5.3 MCP Tools - -```typescript -const tools = { - errors_list_issues: { - description: 'List error issues for a project', - parameters: { - project_id: { type: 'string', required: true }, - status: { type: 'string', enum: ['unresolved', 'resolved', 'ignored'] }, - limit: { type: 'number', default: 20 }, - }, - }, - - errors_get_issue: { - description: 'Get detailed information about an error issue', - parameters: { - project_id: { type: 'string', required: true }, - issue_id: { type: 'string', required: true }, - }, - }, - - errors_resolve_issue: { - description: 'Mark an error issue as resolved', - parameters: { - project_id: { type: 'string', required: true }, - issue_id: { type: 'string', required: true }, - reason: { type: 'string' }, - }, - }, - - errors_search_events: { - description: 'Search for error events', - parameters: { - project_id: { type: 'string', required: true }, - query: { type: 'string', required: true }, - timeframe: { type: 'string', default: '24h' }, - }, - }, -} -``` - ---- - -## 6. Complexity Analysis - -### 6.1 High Complexity Components - -| Component | Complexity | Reason | Mitigation | -|-----------|------------|--------|------------| -| Source Map Parsing | High | CPU intensive, large files | Cache aggressively, lazy load | -| Issue Grouping | High | ML/heuristics for semantic matching | Start with deterministic, add AI later | -| Multi-Protocol Support | Medium-High | Different formats, auth schemes | Abstract into common internal format | -| Rate Limiting | Medium | Distributed state | DO per-project handles local state | -| PII Scrubbing | Medium | Many patterns, performance | Precompiled regex, streaming | - -### 6.2 Cloudflare Workers Constraints - -| Constraint | Limit | Impact | Solution | -|------------|-------|--------|----------| -| CPU Time | 30s (unbound) | Source map parsing | Chunk processing, caching | -| Memory | 128MB | Large payloads | Streaming, compression | -| Subrequest Limit | 1000/request | Fanout operations | Batch, queue | -| D1 Row Size | 1MB | Event payloads | Store in R2, reference | -| R2 PUT Size | 5GB | Source maps | No issue | - -### 6.3 Performance Targets - -| Metric | Target | Rationale | -|--------|--------|-----------| -| Ingestion Latency (p50) | <50ms | Edge processing benefit | -| Ingestion Latency (p99) | <200ms | Acceptable for async | -| Issue Query Latency | <100ms | Dashboard responsiveness | -| Source Map Lookup | <500ms | Acceptable for async symbolication | -| Alert Delivery | <30s | Real-time notification | - ---- - -## 7. Implementation Phases - -### Phase 1: Core Ingestion (MVP) - -**Duration**: 4-6 weeks - -**Deliverables**: -- [ ] Sentry envelope parser -- [ ] Basic event storage (D1) -- [ ] Simple fingerprinting (type + message + stack hash) -- [ ] Issue creation/grouping -- [ ] Basic dashboard API -- [ ] Native SDK (browser + Node) - -**Architecture**: -``` -Worker -> ErrorIngestionDO -> IssueGroupingDO -> D1 -``` - -### Phase 2: Source Maps & Symbolication - -**Duration**: 3-4 weeks - -**Deliverables**: -- [ ] Source map upload API -- [ ] R2 storage integration -- [ ] Symbolication service (source-map-js) -- [ ] Release management -- [ ] CLI for uploads - -**Architecture**: -``` -SymbolicationDO <- R2 (source maps) - <- Cache (SQLite) -``` - -### Phase 3: Advanced Grouping & Alerting - -**Duration**: 3-4 weeks - -**Deliverables**: -- [ ] Custom fingerprint rules -- [ ] Issue merging/splitting -- [ ] Alert configuration -- [ ] Webhook integrations -- [ ] Slack/Discord notifications - -### Phase 4: Multi-Protocol & Ecosystem - -**Duration**: 4-6 weeks - -**Deliverables**: -- [ ] Bugsnag protocol support -- [ ] Rollbar protocol support -- [ ] OpenTelemetry export -- [ ] Session replay (basic) -- [ ] MCP tools - -### Phase 5: Polish & Scale - -**Duration**: 2-4 weeks - -**Deliverables**: -- [ ] Analytics Engine integration -- [ ] Performance optimization -- [ ] PII scrubbing rules UI -- [ ] Team/organization management -- [ ] Usage quotas and billing hooks - ---- - -## 8. Integration with workers.do Ecosystem - -### 8.1 Service Bindings - -```typescript -// From other workers.do services -interface Env { - ERRORS: Service -} - -// Usage -await env.ERRORS.captureException(error, { - tags: { service: 'payments.do' }, - user: { id: userId }, -}) -``` - -### 8.2 Agent Integration - -```typescript -import { quinn } from 'agents.do' - -// Quinn (QA agent) can query errors -quinn`what are the top unresolved errors this week?` - -// Ralph (Dev agent) can investigate -ralph`investigate error ERR-123 and suggest a fix` -``` - -### 8.3 Workflow Integration - -```typescript -import { Workflow } from 'workflows.do' - -export const errorTriage = Workflow({ - trigger: { type: 'error', severity: 'critical' }, - phases: { - investigate: { assignee: quinn, then: 'fix' }, - fix: { assignee: ralph, then: 'review' }, - review: { assignee: tom, then: 'deploy' }, - deploy: { assignee: ralph, checkpoint: true }, - }, -}) -``` - ---- - -## 9. Competitive Positioning - -| Feature | Sentry | errors.do | Advantage | -|---------|--------|-----------|-----------| -| Edge Ingestion | Relay (self-host) | Native | Zero latency, no setup | -| Pricing | Per-event | Per-project | Predictable costs | -| Source Maps | Upload required | Upload required | Parity | -| SDK Compat | N/A | Full Sentry | Migration ease | -| AI Integration | Seer | MCP native | Deeper agents.do integration | -| Self-Host | Complex | N/A | Managed simplicity | -| Cloudflare Native | SDK only | First-class | Service bindings, DO integration | - ---- - -## 10. Open Questions - -1. **Semantic Grouping**: Should we implement AI-powered grouping in Phase 1, or start deterministic? - - Recommendation: Start deterministic, add AI via workers AI in Phase 3 - -2. **Session Replay**: Full replay or just breadcrumbs? - - Recommendation: Breadcrumbs first, full replay as separate product (replay.do?) - -3. **Pricing Model**: Per-event (Sentry), per-project, or usage-based? - - Recommendation: Tiered per-project with event quotas - -4. **Trace Integration**: Build tracing into errors.do or separate service? - - Recommendation: Separate (traces.do), with correlation IDs - -5. **Dashboard**: Build custom or integrate with existing (Grafana)? - - Recommendation: Build minimal custom, focus on MCP/API - ---- - -## 11. References - -### Documentation -- [Sentry Developer Documentation](https://develop.sentry.dev/) -- [Sentry Envelope Format](https://develop.sentry.dev/sdk/data-model/envelopes/) -- [Sentry Event Payloads](https://develop.sentry.dev/sdk/event-payloads/) -- [Sentry Grouping](https://develop.sentry.dev/backend/application-domains/grouping/) -- [Sentry Relay](https://github.com/getsentry/relay) -- [Bugsnag API](https://docs.bugsnag.com/api/error-reporting/) -- [Rollbar API](https://docs.rollbar.com/reference/create-item) -- [Datadog Error Tracking](https://docs.datadoghq.com/tracing/error_tracking/) -- [LogRocket Session Replay](https://docs.logrocket.com/docs/session-replay) -- [Highlight.io](https://github.com/highlight/highlight) - -### Libraries -- [source-map-js](https://www.npmjs.com/package/source-map-js) - Pure JS source map parsing -- [@sentry/cloudflare](https://docs.sentry.io/platforms/javascript/guides/cloudflare/) - Official SDK -- [rrweb](https://github.com/rrweb-io/rrweb) - Session replay (used by Highlight, LogRocket) - ---- - -## 12. Conclusion - -errors.do presents a compelling opportunity to build a Cloudflare-native error monitoring solution that: - -1. **Provides instant migration** from Sentry via protocol compatibility -2. **Delivers lower latency** via edge-native ingestion -3. **Integrates deeply** with the workers.do agent ecosystem -4. **Offers predictable pricing** via per-project model -5. **Simplifies operations** with fully managed infrastructure - -The technical complexity is manageable with a phased approach, starting with core ingestion and expanding to advanced features. The key risks are source map processing performance and achieving high-quality issue grouping - both addressable with caching and iterative algorithm improvement. - -**Recommended next step**: Create TDD issues in `rewrites/errors/.beads/` following the established pattern from fsx and redis rewrites. diff --git a/rewrites/docs/research/flags-scope.md b/rewrites/docs/research/flags-scope.md deleted file mode 100644 index 06be6441..00000000 --- a/rewrites/docs/research/flags-scope.md +++ /dev/null @@ -1,890 +0,0 @@ -# Feature Flags & Experimentation Rewrite Scope - -## flags.do / experiments.do - -**Date**: 2026-01-07 -**Status**: Research Complete - Ready for Implementation Planning - ---- - -## Executive Summary - -This document scopes a Cloudflare Workers rewrite for feature flag evaluation and A/B testing experimentation. The goal is to provide sub-millisecond flag evaluation at the edge, consistent user bucketing via Durable Objects, and integrated statistical analysis for experiments. - ---- - -## 1. Competitive Landscape Analysis - -### 1.1 LaunchDarkly - -**Core Value Proposition**: Runtime control plane for features, AI, and releases. - -**Key Metrics**: -- 99.99% uptime -- 45T+ flag evaluations daily -- <200ms global flag change propagation -- 100+ points of presence - -**Key Features**: -- Progressive rollouts with blast radius control -- Real-time feature-level performance tracking -- Automatic rollback on custom thresholds -- Bayesian and Frequentist statistical analysis -- Multi-armed bandits for dynamic traffic allocation - -**SDK Architecture**: -- Client-side, Server-side, AI, and Edge SDKs -- Cloudflare Workers SDK uses KV as persistent store -- Pushes config directly to KV (no evaluation-time API calls) -- Event sending optional via `{ sendEvents: true }` - -**Edge Implementation Pattern**: -```typescript -const client = init({ - clientSideID: 'client-side-id', - options: { kvNamespace: env.LD_KV } -}) -await client.waitForInitialization() -const value = await client.variation('flag-key', context, defaultValue) -``` - -### 1.2 Split.io (now Harness) - -**Core Value Proposition**: Release faster with feature flags connected to impact data. - -**Key Features**: -- Flexible targeting rules for gradual rollouts -- Automatic performance metric capture during releases -- Impact detection across concurrent rollouts -- Sub-minute issue identification via alerts -- AI-powered insights explaining metric impacts - -**SDK Architecture**: -- 14+ SDKs (client and server) -- Local flag evaluation (no sensitive data transmission) -- Cloudflare Workers: "Partial consumer mode" - - Uses external cache for flag definitions - - Cron trigger updates cache periodically - - Events sent directly to Harness (not cached) - -**Integration Ecosystem**: -- Datadog, New Relic, Sentry monitoring -- Segment, mParticle CDP connectors -- Google Analytics, Jira integrations - -### 1.3 Optimizely - -**Core Value Proposition**: Ship features with confidence via code-level experiments. - -**Key Features**: -- Proprietary Stats Engine for result validation -- Opal AI assistant for test ideation -- Omni-channel experimentation -- CDP integration for advanced audience targeting -- Multiple features per single flag - -**SDK Architecture**: -- 9,000+ developers using platform -- "Universal" JS SDK (excludes datafile manager for performance) -- Cloudflare Cache API for datafile caching -- Custom event dispatching via platform helpers - -**Cloudflare Workers Template**: -```typescript -// Uses Cache API for datafile -// Custom getOptimizelyClient() helper -// Event dispatch through Workers -``` - -### 1.4 Statsig - -**Core Value Proposition**: Same tools as world's largest tech companies - unified A/B testing, feature management, and analytics. - -**Key Features**: -- Feature gates (boolean flags) -- Experiments with variant configurations -- Layers for grouped experiment management -- Dynamic configs targeted to users -- Auto-exposure logging on every check - -**SDK Architecture**: -- Extensive SDK coverage (JS, React, Node, Python, Go, Rust, C++, etc.) -- Initialize: Fetches all rule sets from servers -- Local evaluation: No network calls after init -- Auto-flush: Events sent every 60 seconds -- Config polling: Every 10 seconds (configurable) - -**Statistical Capabilities**: -- P-value analysis (0.05 threshold) -- Confidence intervals -- CUPED variance reduction -- Winsorization for outliers -- Sample ratio mismatch detection - -### 1.5 Flagsmith (Open Source) - -**Core Value Proposition**: Ship faster with open-source feature flag management. - -**Key Features**: -- Toggle features across web, mobile, server -- User segmentation based on stored traits -- Staged rollouts to percentage cohorts -- A/B testing with analytics integration -- Self-hosted via Kubernetes/Helm/OpenShift - -**SDK Architecture**: -- 13+ language SDKs -- Two evaluation modes: - 1. **Remote**: Blocking network request per evaluation - 2. **Local**: Async fetch on init, poll every 60s -- Identity-based evaluation: `getIdentityFlags(identifier, traits)` -- Default fallback handlers for failures - -**Open Source Advantage**: -- Full self-hosting capability -- On-premises/private cloud deployment -- GitHub-available codebase - -### 1.6 GrowthBook (Open Source) - -**Core Value Proposition**: #1 open-source feature flags and experimentation platform. - -**Key Metrics**: -- 100B+ feature flag lookups daily -- 2,700+ companies -- 99.9999% infrastructure uptime -- SOC II certified, GDPR compliant - -**Key Features**: -- Visual editor for no-code A/B tests -- Data warehouse native integration -- Enterprise-class statistics engine -- Smallest SDK footprint (9KB JS) - -**Statistical Engine**: -- **Bayesian** (default): Probability distributions, intuitive results -- **Frequentist**: Two-sample t-tests, CUPED, sequential testing -- Configurable priors (Normal distribution, mean 0, SD 0.3) -- Multiple testing corrections (Benjamini-Hochberg, Bonferroni) -- SRM (Sample Ratio Mismatch) detection - -**Hashing Algorithm (FNV32a v2)**: -```javascript -// Deterministic bucketing -n = fnv32a(fnv32a(seed + userId) + "") -bucket = (n % 10000) / 10000 // 0.0 to 1.0 -``` - -**Cloudflare Workers SDK**: -```typescript -import { GrowthBook } from '@growthbook/edge-cloudflare' -// Webhook-based or JIT payload caching via KV -// Sticky bucketing for consistent UX -// Automatic attribute collection (device, browser, UTM) -``` - ---- - -## 2. Architecture Vision for flags.do - -### 2.1 Domain Structure - -``` -flags.do # Primary domain -experiments.do # Alias for experimentation focus -ab.do # A/B testing shorthand (if available) -``` - -### 2.2 High-Level Architecture - -``` -flags.do -├── Flag Evaluation (edge, KV cached) -│ ├── Boolean flags (gates) -│ ├── Multivariate flags (strings, numbers, JSON) -│ └── Percentage rollouts -├── User Bucketing (deterministic hash) -│ ├── FNV32a hashing (GrowthBook compatible) -│ ├── Sticky bucketing via Durable Objects -│ └── Cross-device identity resolution -├── Targeting Rules Engine -│ ├── User attributes (traits) -│ ├── Segment membership -│ ├── Geographic targeting -│ └── Time-based activation -├── Event Tracking -│ ├── Exposure logging (auto + manual) -│ ├── Conversion events -│ └── Analytics pipeline integration -├── Admin API -│ ├── Flag CRUD operations -│ ├── Experiment management -│ ├── Segment definitions -│ └── Audit logging -└── Stats Engine - ├── Bayesian analysis (default) - ├── Frequentist analysis - ├── CUPED variance reduction - └── SRM detection -``` - -### 2.3 Durable Object Design - -```typescript -// FlagConfigDO - Single source of truth for flag configuration -// One per project/environment -class FlagConfigDO extends DurableObject { - // SQLite: flag definitions, targeting rules, segments - // WebSocket: Real-time config updates to edge - // Methods: getFlags, updateFlag, createExperiment -} - -// UserBucketDO - Consistent bucketing per user -// One per user (or user segment for efficiency) -class UserBucketDO extends DurableObject { - // SQLite: user assignments, sticky bucket history - // Methods: getBucket, assignToExperiment, getAssignments -} - -// ExperimentDO - Experiment state and results -// One per experiment -class ExperimentDO extends DurableObject { - // SQLite: exposures, conversions, computed stats - // Methods: recordExposure, recordConversion, getResults -} - -// AnalyticsDO - Aggregated analytics -// Sharded by time period for write scalability -class AnalyticsDO extends DurableObject { - // SQLite: aggregated counts, metrics - // Methods: ingest, query, export -} -``` - -### 2.4 Storage Strategy - -| Data Type | Hot Storage | Warm Storage | Archive | -|-----------|-------------|--------------|---------| -| Flag configs | KV (edge cached) | SQLite in DO | - | -| User buckets | SQLite in DO | - | R2 (old users) | -| Exposures | SQLite in DO | R2 (batch export) | Analytics warehouse | -| Experiment results | SQLite in DO | R2 (historical) | - | - -### 2.5 Edge Evaluation Flow - -``` -Request arrives at edge - │ - ▼ -┌─────────────────────────┐ -│ Check KV for flags │ ◄── Sub-ms read -│ (cached flag config) │ -└────────────┬────────────┘ - │ - ▼ -┌─────────────────────────┐ -│ Hash user ID + flag │ ◄── FNV32a (no network) -│ Deterministic bucket │ -└────────────┬────────────┘ - │ - ▼ -┌─────────────────────────┐ -│ Evaluate targeting │ ◄── In-memory rules -│ rules against context │ -└────────────┬────────────┘ - │ - ▼ -┌─────────────────────────┐ -│ Return variation │ ◄── < 1ms total -│ Log exposure (async) │ -└─────────────────────────┘ -``` - ---- - -## 3. API Design - -### 3.1 SDK Interface - -```typescript -import { Flags } from 'flags.do' - -const flags = Flags({ - projectKey: 'my-project', - environment: 'production' -}) - -// Boolean flag -const showNewUI = await flags.isEnabled('new-ui', { - userId: 'user-123', - attributes: { plan: 'pro', country: 'US' } -}) - -// Multivariate flag -const buttonColor = await flags.getValue('button-color', { - userId: 'user-123', - default: 'blue' -}) - -// Experiment assignment -const experiment = await flags.getExperiment('checkout-flow', { - userId: 'user-123' -}) -// { variation: 'B', payload: { layout: 'single-page' } } - -// Track conversion -await flags.track('purchase', { - userId: 'user-123', - value: 99.99, - properties: { sku: 'WIDGET-001' } -}) -``` - -### 3.2 REST API - -``` -GET /flags/:projectKey/:flagKey - ?userId=xxx&attributes=base64json - Returns: { value, variation, reason } - -POST /flags/:projectKey/:flagKey/evaluate - Body: { userId, attributes } - Returns: { value, variation, reason } - -POST /events/:projectKey - Body: { events: [{ type, userId, ... }] } - Returns: { accepted: number } - -GET /experiments/:projectKey/:experimentKey/results - Returns: { variations: [...], winner, confidence } -``` - -### 3.3 Admin API - -``` -# Flags -GET /admin/projects/:projectKey/flags -POST /admin/projects/:projectKey/flags -GET /admin/projects/:projectKey/flags/:flagKey -PUT /admin/projects/:projectKey/flags/:flagKey -DELETE /admin/projects/:projectKey/flags/:flagKey - -# Experiments -GET /admin/projects/:projectKey/experiments -POST /admin/projects/:projectKey/experiments -GET /admin/projects/:projectKey/experiments/:experimentKey -PUT /admin/projects/:projectKey/experiments/:experimentKey -POST /admin/projects/:projectKey/experiments/:experimentKey/start -POST /admin/projects/:projectKey/experiments/:experimentKey/stop - -# Segments -GET /admin/projects/:projectKey/segments -POST /admin/projects/:projectKey/segments -PUT /admin/projects/:projectKey/segments/:segmentKey -DELETE /admin/projects/:projectKey/segments/:segmentKey -``` - ---- - -## 4. Targeting Rules Engine - -### 4.1 Rule Syntax (MongoDB-style, GrowthBook compatible) - -```typescript -interface TargetingRule { - id: string - condition: Condition // MongoDB-style query - coverage: number // 0.0 - 1.0 - variations: VariationWeight[] - hashAttribute?: string // Default: 'userId' -} - -// Condition examples -{ country: 'US' } -{ plan: { $in: ['pro', 'enterprise'] } } -{ $and: [ - { country: 'US' }, - { age: { $gte: 18 } } -]} -{ $or: [ - { betaTester: true }, - { employeeId: { $exists: true } } -]} -``` - -### 4.2 Operators Supported - -| Operator | Description | Example | -|----------|-------------|---------| -| `$eq` | Equals | `{ plan: { $eq: 'pro' } }` | -| `$ne` | Not equals | `{ status: { $ne: 'banned' } }` | -| `$in` | In array | `{ country: { $in: ['US', 'CA'] } }` | -| `$nin` | Not in array | `{ country: { $nin: ['CN', 'RU'] } }` | -| `$gt`, `$gte` | Greater than | `{ age: { $gte: 18 } }` | -| `$lt`, `$lte` | Less than | `{ usage: { $lt: 1000 } }` | -| `$exists` | Property exists | `{ premiumFeature: { $exists: true } }` | -| `$regex` | Regex match | `{ email: { $regex: '@company\\.com$' } }` | -| `$and`, `$or`, `$not` | Logical | `{ $and: [...] }` | - -### 4.3 Evaluation Order - -1. **Kill switch check** - If flag is disabled globally, return default -2. **Individual targeting** - Check if user has specific override -3. **Segment rules** - Evaluate rules in priority order -4. **Percentage rollout** - Hash user into bucket -5. **Default rule** - Fall back to default variation - ---- - -## 5. Statistical Engine - -### 5.1 Bayesian Analysis (Default) - -```typescript -interface BayesianResult { - variationId: string - users: number - conversions: number - conversionRate: number - chanceToWin: number // P(this > control) - expectedLoss: number // Risk if choosing this - credibleInterval: [number, number] // 95% CI - uplift: { - mean: number - distribution: number[] // For visualization - } -} -``` - -**Prior Configuration**: -- Default: Uninformative prior -- Optional: Normal(0, 0.3) to shrink extreme results - -### 5.2 Frequentist Analysis - -```typescript -interface FrequentistResult { - variationId: string - users: number - conversions: number - conversionRate: number - pValue: number - significanceLevel: 0.01 | 0.05 | 0.1 - isSignificant: boolean - confidenceInterval: [number, number] - relativeUplift: number - standardError: number -} -``` - -### 5.3 Variance Reduction (CUPED) - -```typescript -// CUPED: Controlled Using Pre-Experiment Data -// Reduces variance by adjusting for pre-experiment behavior - -interface CUPEDConfig { - enabled: boolean - covariate: string // e.g., 'pre_experiment_sessions' - lookbackDays: number // How far back to look -} - -// Adjusted metric = Y - theta * (X - mean(X)) -// Where X is the covariate, theta is regression coefficient -``` - -### 5.4 Data Quality Checks - -| Check | Description | Action | -|-------|-------------|--------| -| SRM | Sample Ratio Mismatch | Alert if allocation differs from expected | -| MDE | Minimum Detectable Effect | Warn if sample too small | -| Novelty | Early results instability | Suggest waiting period | -| Carryover | Previous experiment contamination | Flag affected users | - ---- - -## 6. Edge Advantages - -### 6.1 Performance Benefits - -| Metric | Traditional SDK | flags.do Edge | -|--------|----------------|---------------| -| Flag evaluation | 1-5ms | <0.5ms | -| Cold start | 50-200ms | ~0ms (isolates) | -| Network round trips | 1-2 per eval | 0 (cached) | -| Global latency | Variable | Consistent <50ms | - -### 6.2 Unique Edge Capabilities - -1. **Server-Side Rendering Support** - - Evaluate flags before HTML generation - - No flash of wrong content (FOWC) - - SEO-safe personalization - -2. **Bot Detection Integration** - - Exclude bots from experiments automatically - - Use Cloudflare Bot Management signals - - Prevent analytics pollution - -3. **Geographic Targeting** - - Native access to `cf-ipcountry`, `cf-ipcity` - - No client-side geo lookup needed - - Real-time geo-based rollouts - -4. **Request Header Targeting** - - Target by User-Agent, Accept-Language - - Device type detection - - A/B test by request characteristics - -5. **Response Transformation** - - Modify HTML at edge based on flags - - Inject experiment tracking scripts - - Serve different static assets - ---- - -## 7. Integration Points - -### 7.1 Internal Platform Services - -```typescript -// Bind to other workers.do services -interface Env { - FLAGS: DurableObjectNamespace // Flag config DO - USERS: DurableObjectNamespace // User bucket DO - EXPERIMENTS: DurableObjectNamespace - ANALYTICS: DurableObjectNamespace - - // Platform integrations - LLM: Service // AI-powered targeting - ANALYTICS_DO: Service // analytics.do integration -} -``` - -### 7.2 External Integrations - -| Integration | Purpose | Implementation | -|-------------|---------|----------------| -| Segment | Event routing | Webhook receiver | -| Amplitude | Analytics | Event export | -| Mixpanel | Analytics | Event export | -| BigQuery | Data warehouse | Scheduled export | -| Snowflake | Data warehouse | Scheduled export | -| Slack | Notifications | Webhook sender | -| PagerDuty | Alerts | Webhook sender | - -### 7.3 MCP Integration - -```typescript -// AI agents can control experiments -const mcpTools = { - 'flags_get': { /* Get flag value */ }, - 'flags_set': { /* Update flag */ }, - 'experiment_create': { /* Create A/B test */ }, - 'experiment_results': { /* Get stats */ }, - 'experiment_decide': { /* AI recommends winner */ } -} -``` - ---- - -## 8. Implementation Phases - -### Phase 1: Core Flag Evaluation (MVP) - -**Duration**: 2-3 weeks - -**Deliverables**: -- [ ] Boolean flag evaluation at edge -- [ ] KV-cached flag configuration -- [ ] FNV32a deterministic bucketing -- [ ] Basic targeting rules (equals, in) -- [ ] SDK: `isEnabled()`, `getValue()` -- [ ] Admin API: Flag CRUD - -**TDD Issues**: -``` -[EPIC] Core flag evaluation - [RED] Test flag evaluation returns correct value - [GREEN] Implement KV-cached flag lookup - [RED] Test deterministic bucketing - [GREEN] Implement FNV32a hashing - [RED] Test basic targeting rules - [GREEN] Implement condition evaluator - [REFACTOR] Extract targeting engine -``` - -### Phase 2: Experiments & Events - -**Duration**: 2-3 weeks - -**Deliverables**: -- [ ] Experiment configuration -- [ ] Exposure tracking (auto-logged) -- [ ] Conversion event ingestion -- [ ] Event batching and export -- [ ] SDK: `getExperiment()`, `track()` - -**TDD Issues**: -``` -[EPIC] Experimentation engine - [RED] Test experiment assignment - [GREEN] Implement variation selection - [RED] Test exposure logging - [GREEN] Implement auto-exposure tracking - [RED] Test event batching - [GREEN] Implement event pipeline -``` - -### Phase 3: Statistics Engine - -**Duration**: 3-4 weeks - -**Deliverables**: -- [ ] Bayesian analysis -- [ ] Frequentist analysis -- [ ] CUPED variance reduction -- [ ] SRM detection -- [ ] Results API - -**TDD Issues**: -``` -[EPIC] Statistics engine - [RED] Test Bayesian probability calculation - [GREEN] Implement Beta distribution - [RED] Test frequentist t-test - [GREEN] Implement significance testing - [RED] Test CUPED adjustment - [GREEN] Implement variance reduction - [RED] Test SRM detection - [GREEN] Implement sample ratio check -``` - -### Phase 4: Advanced Features - -**Duration**: 2-3 weeks - -**Deliverables**: -- [ ] Segments management -- [ ] Mutual exclusion groups -- [ ] Multi-armed bandits -- [ ] Sticky bucketing -- [ ] Audit logging - -### Phase 5: Dashboard & Integrations - -**Duration**: 2-3 weeks - -**Deliverables**: -- [ ] Web dashboard (React) -- [ ] Analytics integrations -- [ ] Webhook notifications -- [ ] MCP tools - ---- - -## 9. Technical Specifications - -### 9.1 Hashing Implementation - -```typescript -// FNV32a implementation (GrowthBook v2 compatible) -function fnv32a(str: string): number { - let hash = 2166136261 - for (let i = 0; i < str.length; i++) { - hash ^= str.charCodeAt(i) - hash = Math.imul(hash, 16777619) - } - return hash >>> 0 -} - -function getBucket(userId: string, seed: string): number { - const n = fnv32a(fnv32a(seed + userId).toString() + '') - return (n % 10000) / 10000 // 0.0000 to 0.9999 -} - -function inBucket(bucket: number, range: [number, number]): boolean { - return bucket >= range[0] && bucket < range[1] -} -``` - -### 9.2 Flag Configuration Schema - -```typescript -interface Flag { - key: string - name: string - description?: string - type: 'boolean' | 'string' | 'number' | 'json' - defaultValue: any - enabled: boolean - - // Targeting - rules: TargetingRule[] - - // Experiment settings (if A/B test) - experiment?: { - key: string - variations: Variation[] - trafficAllocation: number // 0.0 - 1.0 - status: 'draft' | 'running' | 'paused' | 'completed' - } - - // Metadata - tags: string[] - createdAt: string - updatedAt: string - createdBy: string -} - -interface Variation { - id: string - key: string - name: string - value: any - weight: number // 0.0 - 1.0, must sum to 1.0 -} -``` - -### 9.3 Event Schema - -```typescript -interface ExposureEvent { - type: 'exposure' - timestamp: string - userId: string - flagKey: string - variationKey: string - experimentKey?: string - attributes: Record - source: 'sdk' | 'edge' | 'api' -} - -interface TrackEvent { - type: 'track' - timestamp: string - userId: string - eventName: string - value?: number - properties: Record - source: 'sdk' | 'api' -} -``` - ---- - -## 10. Consistency Guarantees - -### 10.1 Flag Propagation - -| Change Type | Propagation Time | Mechanism | -|-------------|------------------|-----------| -| Flag enable/disable | <1s | WebSocket push | -| Targeting rule update | <5s | KV cache invalidation | -| New flag creation | <10s | KV write + cache | -| Experiment start/stop | <1s | WebSocket + DO state | - -### 10.2 Bucketing Consistency - -**Guarantee**: Same user + same flag + same seed = same bucket - -**Implementation**: -- Deterministic FNV32a hash (no randomness) -- Sticky bucketing optional (DO-persisted) -- Cross-device: Identity resolution layer - -### 10.3 Event Delivery - -**Guarantee**: At-least-once delivery to analytics - -**Implementation**: -- Events buffered in DO SQLite -- Batch export every 60s -- Retry with exponential backoff -- Dead letter queue for failures - ---- - -## 11. Pricing Model Considerations - -| Tier | Flags | MAU | Experiments | Price | -|------|-------|-----|-------------|-------| -| Free | 5 | 10K | 1 | $0 | -| Starter | 25 | 100K | 5 | $29/mo | -| Pro | Unlimited | 1M | Unlimited | $99/mo | -| Enterprise | Unlimited | Unlimited | Unlimited | Custom | - -**Usage-Based Additions**: -- $0.10 per 100K additional evaluations -- $0.50 per 100K additional events -- $10 per additional 100K MAU - ---- - -## 12. Competitive Differentiation - -### 12.1 vs LaunchDarkly - -| Aspect | LaunchDarkly | flags.do | -|--------|--------------|----------| -| Edge evaluation | Yes (via KV) | Native | -| Pricing | $$$$ | $ | -| Open source | No | Core open | -| Self-hosting | No | Yes | -| Stats engine | Basic | Advanced (Bayesian/Freq) | - -### 12.2 vs GrowthBook - -| Aspect | GrowthBook | flags.do | -|--------|------------|----------| -| Edge evaluation | Plugin | Native | -| Managed hosting | Limited | Full | -| Real-time updates | Polling | WebSocket | -| Infrastructure | BYO | Included | -| Analytics | Warehouse-native | Integrated + export | - -### 12.3 Unique Value Props - -1. **True Edge-Native**: Built for Cloudflare from ground up -2. **Unified Platform**: Part of workers.do ecosystem -3. **AI-Ready**: MCP integration for agent-controlled experiments -4. **Developer Experience**: Natural language API patterns -5. **Transparent Pricing**: Simple, predictable costs - ---- - -## 13. Open Questions - -1. **Domain choice**: `flags.do` vs `experiments.do` vs both? -2. **Stats engine**: Build custom or integrate existing (e.g., GrowthBook's)? -3. **Visual editor**: Priority for no-code A/B tests? -4. **SDK strategy**: Universal SDK or platform-specific? -5. **Warehouse integration**: Native connectors or export-only? - ---- - -## 14. References - -- [LaunchDarkly Cloudflare SDK](https://launchdarkly.com/docs/sdk/edge/cloudflare) -- [GrowthBook Edge SDK](https://docs.growthbook.io/lib/edge/cloudflare) -- [GrowthBook Statistics Overview](https://docs.growthbook.io/statistics/overview) -- [GrowthBook SDK Build Guide](https://docs.growthbook.io/lib/build-your-own) -- [Statsig Documentation](https://docs.statsig.com/) -- [Flagsmith Documentation](https://docs.flagsmith.com/) -- [Harness Feature Management](https://www.harness.io/products/feature-management-experimentation) - ---- - -## 15. Next Steps - -1. **Create beads workspace**: `bd init --prefix=flags` in `rewrites/flags/` -2. **Create TDD epics**: Core evaluation, experiments, statistics -3. **Scaffold project**: DO classes, SDK structure, API routes -4. **Implement Phase 1**: Core flag evaluation MVP -5. **Validate with users**: Get feedback on API design - ---- - -*Document version: 1.0* -*Last updated: 2026-01-07* diff --git a/rewrites/docs/research/notify-scope.md b/rewrites/docs/research/notify-scope.md deleted file mode 100644 index f8f92d21..00000000 --- a/rewrites/docs/research/notify-scope.md +++ /dev/null @@ -1,1120 +0,0 @@ -# notify.do - Customer Engagement Infrastructure - -**Cloudflare Workers Rewrite Scoping Document** - -A unified notification and customer engagement platform built on Cloudflare Workers, combining the best patterns from Customer.io, Intercom, Braze, OneSignal, Knock, and Novu. - ---- - -## Executive Summary - -notify.do provides notification infrastructure for developers building customer engagement into their products. It handles event ingestion, workflow orchestration, template rendering, and multi-channel delivery - all running at the edge with Cloudflare Workers. - -**Target domain**: `notify.do` (primary), `engage.do` (alias) - -**Core value**: Replace fragmented notification infrastructure with a single, edge-native platform that developers configure once and delivers across all channels. - ---- - -## 1. Platform Analysis - -### 1.1 Customer.io - -**Core Value Proposition**: Data-driven customer engagement with visual workflow builder. Powers 56+ billion messages annually for 8,000+ brands. - -**Key Capabilities**: -- Event-driven campaigns triggered by user actions -- Visual drag-and-drop journey builder -- Multi-channel: email, SMS, push, in-app -- Real-time data pipelines with reverse ETL (Snowflake, BigQuery) -- A/B testing and cohort experimentation -- Ad audience sync (Google, Facebook, Instagram) - -**Technical Architecture**: -- Track API: Event ingestion at 3000 req/3sec rate limit -- v2 Edge API with batch endpoints -- Anonymous ID to User ID resolution -- Object associations (accounts, courses, etc.) - -**Key Insight**: Customer.io excels at connecting behavioral data to messaging workflows. Their "objects" concept (grouping people into accounts) is valuable for B2B use cases. - -### 1.2 Intercom - -**Core Value Proposition**: AI-first customer service platform with "Fin AI Agent" for automated support resolution. - -**Key Capabilities**: -- Conversational messaging across channels -- AI agent (Fin) for automated query resolution -- Help desk integration -- Multi-region support (US, EU, Australia) -- Real-time chat widgets - -**Technical Architecture**: -- REST API with JSON encoding -- Regional endpoint deployment -- Webhook-based event delivery -- Conversation threading model - -**Key Insight**: Intercom's strength is conversational AI and support workflows. For notify.do, the relevant pattern is their real-time messaging infrastructure and conversation state management. - -### 1.3 Braze - -**Core Value Proposition**: Enterprise customer engagement with AI-powered journey orchestration. Powers cross-channel experiences at massive scale. - -**Key Capabilities**: -- Canvas Flow: No-code journey builder with "unlimited ingress/specific egress" -- BrazeAI: Predictive, generative, and agentic intelligence -- Cross-channel: email, SMS/RCS, push, in-app, WhatsApp, web, LINE -- Real-time user tracking via `/users/track` endpoint -- Liquid templating for personalization -- Segment and campaign management APIs - -**Technical Architecture**: -- Sub-second latency at any scale -- Component-based workflow: Action Paths, Audience Paths, Decision Splits -- User Update Component for in-journey data capture -- Experiment Paths for A/B testing -- Data Platform with direct warehouse connections - -**Key Insight**: Braze's Canvas Flow architecture - deterministic step-based journeys with unlimited inputs - is the gold standard for workflow orchestration. Their real-time triggering model is essential. - -### 1.4 OneSignal - -**Core Value Proposition**: Push notification infrastructure for developers, powering ~20% of all mobile apps with 12B+ daily messages. - -**Key Capabilities**: -- Push notifications (mobile + web) -- Email, SMS/RCS, in-app messaging -- Live Activities (iOS) -- Dynamic segmentation with no-code workflows -- 200+ integrations (CDPs, analytics, CRMs) -- Real-time analytics and optimization - -**Technical Architecture**: -- REST API + SDKs (Android, iOS, Flutter, React Native, Unity) -- 99.95% uptime SLA -- SOC 2, GDPR, CCPA, HIPAA, ISO 27001/27701 certified -- Behavior-based personalization engine - -**Key Insight**: OneSignal's specialization in push notifications and their integration ecosystem shows the value of channel-specific optimization and broad connectivity. - -### 1.5 Knock - -**Core Value Proposition**: Developer-first notification infrastructure with emphasis on workflow orchestration and preference management. - -**Key Capabilities**: -- Multi-channel workflows (email, SMS, push, in-app, Slack, MS Teams) -- Drag-and-drop workflow builder -- Functions: delay, batch, branch, fetch, throttle -- User preference management with opt-out support -- Link and open tracking -- Real-time in-app notifications via WebSocket -- Version control with rollback - -**Technical Architecture**: -- SDKs in 8 languages (Node, Python, Ruby, Go, Java, .NET, Elixir, PHP) -- Management API for programmatic workflow control -- CI/CD integration for pre-deployment validation -- SAML SSO, SCIM directory sync -- 99.99% uptime, HIPAA/SOC2/GDPR/CCPA compliant - -**Key Insight**: Knock's developer experience - SDKs, CLI tooling, version control for workflows - is best-in-class. Their preference management system handles commercial unsubscribe compliance well. - -### 1.6 Novu - -**Core Value Proposition**: Open-source notification infrastructure with unified API and self-hosting option. - -**Key Capabilities**: -- Unified API for all channels -- Visual workflow editor + code-based Novu Framework SDK -- In-app Inbox component (6 lines of code) -- Digest engine to consolidate notifications -- Multi-provider support per channel -- Internationalization built-in -- Subscriber preference management - -**Technical Architecture**: -- Open-source core (MIT), commercial enterprise features -- NestJS + MongoDB stack -- Docker self-hosting support -- REST + WebSocket architecture -- Event-driven trigger system -- Provider integrations: - - Email: SendGrid, Mailgun, AWS SES, Postmark, SMTP - - SMS: Twilio, Vonage, Plivo, SNS - - Push: FCM, Expo, APNs, Pushpad - - Chat: Slack, Discord, Microsoft Teams - -**Key Insight**: Novu's open-source model and provider abstraction layer is compelling. Their digest engine for notification consolidation reduces user fatigue. - ---- - -## 2. Architecture Vision - -``` -notify.do -├── Edge Layer (Workers) -│ ├── Event Ingestion API -│ ├── Webhook Receivers -│ └── Real-time WebSocket Gateway -│ -├── Core Services (Durable Objects) -│ ├── UserDO (profiles + preferences) -│ ├── WorkflowDO (journey state machine) -│ ├── TemplateDO (template storage + rendering) -│ └── DeliveryDO (channel orchestration) -│ -├── Orchestration (CF Workflows) -│ ├── Journey Engine -│ ├── Batch/Digest Processor -│ └── Retry/Escalation Handler -│ -├── Storage -│ ├── D1 (user profiles, event history, analytics) -│ ├── R2 (template assets, large payloads) -│ └── KV (rate limiting, feature flags) -│ -├── Channel Adapters -│ ├── Email (Resend, SendGrid, Mailgun, SES) -│ ├── SMS (Twilio, Vonage, MessageBird) -│ ├── Push (APNs, FCM) -│ ├── In-App (WebSocket) -│ ├── Chat (Slack, Discord, Teams) -│ └── Webhooks -│ -└── Analytics Pipeline - ├── Event Stream (R2 + batch processing) - └── Real-time Counters (DO) -``` - ---- - -## 3. Core Components - -### 3.1 Event Ingestion Layer - -**Problem**: High-volume event tracking with low latency at edge. - -**Design**: -```typescript -// Edge Worker - validates and routes events -export default { - async fetch(request: Request, env: Env): Promise { - const event = await request.json() as TrackEvent - - // Validate and normalize - const normalized = validateEvent(event) - - // Route to user's DO for state updates - const userId = normalized.userId || normalized.anonymousId - const userDO = env.USERS.get(env.USERS.idFromName(userId)) - - // Fire-and-forget ingestion (async durability) - ctx.waitUntil(userDO.track(normalized)) - - // Immediate response for low latency - return new Response('OK', { status: 202 }) - } -} - -// Event schema (Customer.io compatible) -interface TrackEvent { - userId?: string - anonymousId?: string - event: string - properties?: Record - timestamp?: string - context?: EventContext -} -``` - -**Rate Limiting**: Use KV for sliding window counters per API key. - -**Anonymous ID Resolution**: When `identify()` is called with both `anonymousId` and `userId`, merge profiles in UserDO. - -### 3.2 User Profile Management - -**Problem**: Store user attributes, track events, and manage preferences at scale. - -**Design (Durable Object)**: -```typescript -export class UserDO extends DurableObject { - private sql: SqlStorage - - constructor(state: DurableObjectState, env: Env) { - super(state, env) - this.sql = state.storage.sql - this.initSchema() - } - - private initSchema() { - this.sql.exec(` - CREATE TABLE IF NOT EXISTS attributes ( - key TEXT PRIMARY KEY, - value TEXT, - updated_at INTEGER - ); - - CREATE TABLE IF NOT EXISTS events ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - event TEXT NOT NULL, - properties TEXT, - timestamp INTEGER NOT NULL, - UNIQUE(event, timestamp) - ); - - CREATE TABLE IF NOT EXISTS preferences ( - workflow_id TEXT, - channel TEXT, - enabled INTEGER DEFAULT 1, - PRIMARY KEY (workflow_id, channel) - ); - - CREATE TABLE IF NOT EXISTS channel_tokens ( - channel TEXT PRIMARY KEY, - token TEXT, - metadata TEXT - ); - - CREATE INDEX IF NOT EXISTS idx_events_event ON events(event); - CREATE INDEX IF NOT EXISTS idx_events_time ON events(timestamp); - `) - } - - async identify(attributes: Record) { - const now = Date.now() - for (const [key, value] of Object.entries(attributes)) { - this.sql.exec(` - INSERT OR REPLACE INTO attributes (key, value, updated_at) - VALUES (?, ?, ?) - `, key, JSON.stringify(value), now) - } - } - - async track(event: TrackEvent) { - const timestamp = event.timestamp - ? new Date(event.timestamp).getTime() - : Date.now() - - this.sql.exec(` - INSERT INTO events (event, properties, timestamp) - VALUES (?, ?, ?) - `, event.event, JSON.stringify(event.properties || {}), timestamp) - - // Trigger workflow evaluation - await this.evaluateWorkflows(event) - } - - async getPreferences(): Promise { - const rows = this.sql.exec(`SELECT * FROM preferences`).toArray() - return rows.reduce((acc, row) => { - acc[row.workflow_id] = acc[row.workflow_id] || {} - acc[row.workflow_id][row.channel] = Boolean(row.enabled) - return acc - }, {} as UserPreferences) - } - - async setChannelToken(channel: string, token: string, metadata?: object) { - this.sql.exec(` - INSERT OR REPLACE INTO channel_tokens (channel, token, metadata) - VALUES (?, ?, ?) - `, channel, token, JSON.stringify(metadata || {})) - } -} -``` - -### 3.3 Workflow Engine - -**Problem**: Execute multi-step, multi-channel notification journeys with branching, delays, and batching. - -**Design (Cloudflare Workflows)**: -```typescript -// Workflow definition (Braze Canvas-inspired) -interface WorkflowDefinition { - id: string - name: string - trigger: WorkflowTrigger - steps: WorkflowStep[] -} - -interface WorkflowTrigger { - type: 'event' | 'segment' | 'api' | 'schedule' - event?: string - segment?: string - schedule?: string // cron expression -} - -type WorkflowStep = - | MessageStep - | DelayStep - | BranchStep - | BatchStep - | ThrottleStep - | FetchStep - | UpdateUserStep - -interface MessageStep { - type: 'message' - channel: Channel - template: string - fallback?: string // fallback channel if primary fails -} - -interface DelayStep { - type: 'delay' - duration: string // "5m", "1h", "1d" - until?: string // ISO timestamp -} - -interface BranchStep { - type: 'branch' - conditions: BranchCondition[] - default: string // step id -} - -interface BatchStep { - type: 'batch' - window: string // "1h" - collect events for 1 hour - maxSize: number // max events before forced delivery - digestTemplate: string -} - -// Cloudflare Workflow implementation -export class NotifyWorkflow extends WorkflowEntrypoint { - async run(event: WorkflowEvent, step: WorkflowStep) { - const { userId, workflowId, trigger } = event.payload - - // Load workflow definition - const definition = await step.do('load-workflow', async () => { - return await this.env.WORKFLOWS_KV.get(workflowId, 'json') - }) - - // Check user preferences - const userDO = this.env.USERS.get(this.env.USERS.idFromName(userId)) - const preferences = await step.do('check-preferences', async () => { - return await userDO.getPreferences() - }) - - // Execute steps - for (const stepDef of definition.steps) { - await this.executeStep(step, stepDef, userId, preferences) - } - } - - private async executeStep( - step: WorkflowStep, - stepDef: WorkflowStep, - userId: string, - preferences: UserPreferences - ) { - switch (stepDef.type) { - case 'delay': - await step.sleep(stepDef.id, parseDuration(stepDef.duration)) - break - - case 'message': - // Check preferences - if (!preferences[stepDef.channel]?.enabled) { - if (stepDef.fallback) { - return this.executeStep(step, { ...stepDef, channel: stepDef.fallback }, userId, preferences) - } - return // user opted out, no fallback - } - - await step.do(`send-${stepDef.channel}`, async () => { - const deliveryDO = this.env.DELIVERY.get( - this.env.DELIVERY.idFromName(`${userId}:${stepDef.channel}`) - ) - return await deliveryDO.send(userId, stepDef.template) - }) - break - - case 'branch': - const user = await step.do('load-user', async () => { - const userDO = this.env.USERS.get(this.env.USERS.idFromName(userId)) - return await userDO.getProfile() - }) - - for (const condition of stepDef.conditions) { - if (evaluateCondition(condition, user)) { - // Jump to condition's target step - return condition.goto - } - } - return stepDef.default - break - - case 'batch': - // Handled by BatchDO - collects events until window closes - const batchDO = this.env.BATCH.get( - this.env.BATCH.idFromName(`${userId}:${stepDef.id}`) - ) - await batchDO.addToBatch(trigger, stepDef.window, stepDef.maxSize) - break - } - } -} -``` - -### 3.4 Template Engine - -**Problem**: Dynamic template rendering with personalization at scale. - -**Design**: -```typescript -// Template storage (supports Liquid-like syntax) -interface Template { - id: string - channel: Channel - subject?: string // email subject - body: string - variables: string[] // extracted variable names for validation - version: number -} - -// Lightweight Liquid-compatible template engine -export class TemplateEngine { - private cache = new Map() - - compile(template: string): CompiledTemplate { - // Parse {{ variable }} and {% if %} / {% for %} blocks - // Return compiled function for efficient re-rendering - } - - async render( - templateId: string, - context: TemplateContext, - env: Env - ): Promise { - // Load template - const template = await this.loadTemplate(templateId, env) - - // Get or compile - let compiled = this.cache.get(templateId) - if (!compiled || compiled.version !== template.version) { - compiled = this.compile(template.body) - this.cache.set(templateId, compiled) - } - - // Render with user context - return { - subject: this.renderString(template.subject, context), - body: compiled.render(context) - } - } -} - -// Context includes user attributes, event data, computed fields -interface TemplateContext { - user: UserProfile - event?: TrackEvent - computed?: Record - digest?: DigestData // for batch templates -} -``` - -### 3.5 Channel Adapters - -**Problem**: Reliable delivery across email, SMS, push, in-app, and chat with provider abstraction. - -**Design**: -```typescript -// Abstract channel interface -interface ChannelAdapter { - readonly channel: Channel - send(recipient: Recipient, content: RenderedContent): Promise - validateRecipient(recipient: Recipient): boolean - getDeliveryStatus(messageId: string): Promise -} - -// Email adapter with provider selection -export class EmailAdapter implements ChannelAdapter { - readonly channel = 'email' - - constructor( - private providers: EmailProvider[], - private selector: ProviderSelector - ) {} - - async send(recipient: Recipient, content: RenderedContent): Promise { - const provider = this.selector.select(this.providers, recipient) - - try { - const result = await provider.send({ - to: recipient.email, - subject: content.subject, - html: content.body, - from: this.getFromAddress(recipient), - replyTo: content.replyTo, - headers: this.buildHeaders(recipient, content) - }) - - return { - success: true, - messageId: result.id, - provider: provider.name - } - } catch (error) { - // Try fallback provider - const fallback = this.selector.fallback(this.providers, provider) - if (fallback) { - return this.sendWithProvider(fallback, recipient, content) - } - throw error - } - } -} - -// Push adapter with APNs + FCM -export class PushAdapter implements ChannelAdapter { - readonly channel = 'push' - - async send(recipient: Recipient, content: RenderedContent): Promise { - const tokens = await this.getTokens(recipient.userId) - - const results = await Promise.allSettled( - tokens.map(token => this.sendToToken(token, content)) - ) - - // Handle token invalidation - const invalidTokens = results - .filter(r => r.status === 'rejected' && isInvalidToken(r.reason)) - .map((r, i) => tokens[i]) - - if (invalidTokens.length > 0) { - await this.removeInvalidTokens(recipient.userId, invalidTokens) - } - - return { - success: results.some(r => r.status === 'fulfilled'), - messageId: generateId(), - details: results - } - } -} - -// In-app adapter via WebSocket -export class InAppAdapter implements ChannelAdapter { - readonly channel = 'in_app' - - async send(recipient: Recipient, content: RenderedContent): Promise { - // Store in inbox - const inboxDO = this.env.INBOX.get( - this.env.INBOX.idFromName(recipient.userId) - ) - const messageId = await inboxDO.addMessage(content) - - // Push via WebSocket if connected - const ws = await this.getWebSocket(recipient.userId) - if (ws) { - ws.send(JSON.stringify({ - type: 'notification', - message: content - })) - } - - return { success: true, messageId } - } -} -``` - -### 3.6 Digest/Batch Processing - -**Problem**: Consolidate multiple notifications into single digests to reduce user fatigue. - -**Design**: -```typescript -export class BatchDO extends DurableObject { - private sql: SqlStorage - private alarm: DurableObjectAlarm - - async addToBatch(event: TrackEvent, window: string, maxSize: number) { - // Store event - this.sql.exec(` - INSERT INTO batch_events (event, properties, timestamp) - VALUES (?, ?, ?) - `, event.event, JSON.stringify(event.properties), Date.now()) - - // Check if we should deliver early (max size reached) - const count = this.sql.exec(`SELECT COUNT(*) as c FROM batch_events`).one().c - if (count >= maxSize) { - await this.deliverBatch() - return - } - - // Set alarm for window close if not set - const alarm = await this.state.storage.getAlarm() - if (!alarm) { - const windowMs = parseDuration(window) - await this.state.storage.setAlarm(Date.now() + windowMs) - } - } - - async alarm() { - await this.deliverBatch() - } - - private async deliverBatch() { - const events = this.sql.exec(`SELECT * FROM batch_events ORDER BY timestamp`).toArray() - - if (events.length === 0) return - - // Build digest context - const digestContext: DigestData = { - count: events.length, - events: events.map(e => ({ - event: e.event, - properties: JSON.parse(e.properties), - timestamp: new Date(e.timestamp) - })), - summary: this.generateSummary(events) - } - - // Trigger digest delivery - const userId = this.state.id.toString().split(':')[0] - const deliveryDO = this.env.DELIVERY.get( - this.env.DELIVERY.idFromName(`${userId}:digest`) - ) - await deliveryDO.sendDigest(userId, digestContext) - - // Clear batch - this.sql.exec(`DELETE FROM batch_events`) - } -} -``` - -### 3.7 Preference Management - -**Problem**: Allow users to control notification delivery while maintaining compliance. - -**Design**: -```typescript -// Preference levels (Knock-inspired) -interface PreferenceSchema { - // Global opt-out - global: { - enabled: boolean - } - - // Per-workflow preferences - workflows: { - [workflowId: string]: { - enabled: boolean - channels?: { - [channel: string]: boolean - } - } - } - - // Per-channel preferences - channels: { - [channel: string]: { - enabled: boolean - quietHours?: { - start: string // "22:00" - end: string // "08:00" - timezone: string - } - } - } -} - -// Preference evaluation -function shouldDeliver( - preferences: PreferenceSchema, - workflowId: string, - channel: string -): boolean { - // Check global opt-out - if (!preferences.global.enabled) return false - - // Check channel opt-out - if (preferences.channels[channel]?.enabled === false) return false - - // Check workflow opt-out - const workflow = preferences.workflows[workflowId] - if (workflow?.enabled === false) return false - if (workflow?.channels?.[channel] === false) return false - - // Check quiet hours - const channelPrefs = preferences.channels[channel] - if (channelPrefs?.quietHours) { - if (isInQuietHours(channelPrefs.quietHours)) return false - } - - return true -} - -// Preference API endpoints -app.get('/v1/users/:userId/preferences', async (c) => { - const userDO = c.env.USERS.get(c.env.USERS.idFromName(c.req.param('userId'))) - return c.json(await userDO.getPreferences()) -}) - -app.put('/v1/users/:userId/preferences', async (c) => { - const userDO = c.env.USERS.get(c.env.USERS.idFromName(c.req.param('userId'))) - const preferences = await c.req.json() - await userDO.setPreferences(preferences) - return c.json({ success: true }) -}) -``` - ---- - -## 4. API Design - -### 4.1 Track API (Customer.io Compatible) - -```typescript -// POST /v1/track -interface TrackRequest { - userId?: string - anonymousId?: string - event: string - properties?: Record - timestamp?: string - context?: { - ip?: string - userAgent?: string - locale?: string - timezone?: string - } -} - -// POST /v1/identify -interface IdentifyRequest { - userId: string - anonymousId?: string // for merging - traits: Record -} - -// POST /v1/batch -interface BatchRequest { - batch: (TrackRequest | IdentifyRequest)[] -} -``` - -### 4.2 Workflow API (Knock-inspired) - -```typescript -// POST /v1/workflows/:workflowId/trigger -interface WorkflowTriggerRequest { - recipients: string | string[] // user IDs - data?: Record // template variables - actor?: string // who triggered (for "X liked your post") - tenant?: string // for multi-tenant apps -} - -// Response -interface WorkflowTriggerResponse { - workflowRunId: string - status: 'accepted' -} - -// GET /v1/workflows/:workflowId/runs/:runId -interface WorkflowRunStatus { - id: string - status: 'pending' | 'running' | 'completed' | 'failed' - steps: StepStatus[] - startedAt: string - completedAt?: string -} -``` - -### 4.3 Message API - -```typescript -// GET /v1/users/:userId/messages -interface MessageListResponse { - messages: Message[] - cursor?: string -} - -interface Message { - id: string - workflowId: string - channel: string - status: 'pending' | 'sent' | 'delivered' | 'failed' | 'read' - content: { - subject?: string - body: string - } - sentAt?: string - deliveredAt?: string - readAt?: string -} - -// POST /v1/users/:userId/messages/:messageId/read -// Mark message as read - -// DELETE /v1/users/:userId/messages/:messageId -// Archive/delete message -``` - -### 4.4 In-App Feed API - -```typescript -// GET /v1/users/:userId/feed -interface FeedResponse { - entries: FeedEntry[] - meta: { - unreadCount: number - unseenCount: number - totalCount: number - } - cursor?: string -} - -// WebSocket connection for real-time updates -// ws://notify.do/v1/users/:userId/feed/realtime -interface FeedWebSocketMessage { - type: 'new' | 'update' | 'delete' - entry?: FeedEntry - entryId?: string -} -``` - ---- - -## 5. Integration Points - -### 5.1 Email Providers - -| Provider | Priority | Use Case | -|----------|----------|----------| -| Resend | Primary | Transactional, great DX | -| SendGrid | Secondary | High volume, enterprise | -| Mailgun | Tertiary | Europe, validation | -| AWS SES | Fallback | Cost optimization | -| Postmark | Specialty | Transactional focus | - -### 5.2 SMS Providers - -| Provider | Priority | Use Case | -|----------|----------|----------| -| Twilio | Primary | Global coverage | -| Vonage | Secondary | Enterprise | -| MessageBird | Regional | Europe focus | -| SNS | Fallback | AWS integration | - -### 5.3 Push Providers - -| Provider | Platform | Notes | -|----------|----------|-------| -| APNs | iOS | Direct integration | -| FCM | Android/Web | Firebase Cloud Messaging | -| Expo | React Native | Simplified push | - -### 5.4 Chat Integrations - -| Platform | Features | -|----------|----------| -| Slack | Channels, DMs, threads | -| Discord | Webhooks, bot messages | -| MS Teams | Adaptive cards | - ---- - -## 6. Key Design Decisions - -### 6.1 Workflow State Management - -**Decision**: Use Cloudflare Workflows for long-running orchestration, Durable Objects for real-time state. - -**Rationale**: -- CF Workflows handle delays, retries, and durability automatically -- DOs provide sub-millisecond access to user state and preferences -- Combination gives both reliability and performance - -### 6.2 Template Personalization - -**Decision**: Compile templates to JavaScript functions, cache in DO memory. - -**Rationale**: -- Avoids re-parsing on every render -- DO memory is fast and persistent within session -- Version tracking ensures cache invalidation - -### 6.3 Delivery Reliability - -**Decision**: At-least-once delivery with idempotency keys. - -**Rationale**: -- Each message gets unique ID -- Channel adapters check for duplicate delivery -- Retry with exponential backoff on transient failures - -### 6.4 Rate Limiting - -**Decision**: Per-channel rate limits using token bucket in KV. - -**Rationale**: -- Email: 100/hour per user (deliverability) -- SMS: 10/day per user (cost + spam regulations) -- Push: 50/hour per user (user experience) -- In-app: Unlimited (low cost) - ---- - -## 7. Implementation Phases - -### Phase 1: Core Infrastructure -- [ ] Event ingestion API (track, identify, batch) -- [ ] UserDO with profile and event storage -- [ ] Basic template engine -- [ ] Email channel adapter (Resend) - -### Phase 2: Workflow Engine -- [ ] WorkflowDO for definition storage -- [ ] CF Workflow integration -- [ ] Delay, branch, and message steps -- [ ] Push notification adapter (APNs, FCM) - -### Phase 3: Advanced Features -- [ ] Batch/digest processing -- [ ] Preference management API -- [ ] In-app notifications with WebSocket -- [ ] SMS channel adapter - -### Phase 4: Developer Experience -- [ ] SDK generation (TypeScript, Python) -- [ ] CLI for workflow management -- [ ] Version control for workflows -- [ ] Analytics dashboard - -### Phase 5: Enterprise -- [ ] Multi-tenant support -- [ ] Audit logging -- [ ] SSO/SCIM integration -- [ ] Custom domain support - ---- - -## 8. Competitive Positioning - -| Feature | notify.do | Customer.io | Knock | Novu | -|---------|-----------|-------------|-------|------| -| Edge-native | Yes | No | No | No | -| Open source | Yes | No | No | Yes | -| Self-hostable | Yes (CF) | No | No | Yes | -| Sub-ms latency | Yes | No | No | No | -| Workflow builder | Yes | Yes | Yes | Yes | -| Multi-channel | Yes | Yes | Yes | Yes | -| Digest engine | Yes | Yes | Yes | Yes | -| MCP integration | Yes | No | No | No | - -**Unique Value**: notify.do combines edge-native performance with the developer experience of Knock and the open infrastructure approach of Novu, all running on Cloudflare's global network. - ---- - -## 9. SDK Design - -```typescript -// TypeScript SDK (tree-shakable) -import { Notify } from 'notify.do' - -const notify = Notify({ - apiKey: process.env.NOTIFY_API_KEY -}) - -// Track event -await notify.track({ - userId: 'user_123', - event: 'order_placed', - properties: { - orderId: 'order_456', - amount: 99.99 - } -}) - -// Identify user -await notify.identify({ - userId: 'user_123', - traits: { - email: 'alice@example.com', - name: 'Alice', - plan: 'pro' - } -}) - -// Trigger workflow -await notify.workflows.trigger('order-confirmation', { - recipients: ['user_123'], - data: { - orderId: 'order_456', - items: [{ name: 'Widget', qty: 2 }] - } -}) - -// Get user preferences -const prefs = await notify.users.getPreferences('user_123') - -// Update preferences -await notify.users.setPreferences('user_123', { - workflows: { - 'marketing-weekly': { enabled: false } - } -}) -``` - ---- - -## 10. MCP Integration - -```typescript -// MCP tools for AI agents -const mcpTools = { - // Send notification - notify_send: { - description: 'Send a notification to a user', - parameters: { - userId: { type: 'string', required: true }, - channel: { type: 'string', enum: ['email', 'sms', 'push', 'in_app'] }, - message: { type: 'string', required: true } - } - }, - - // Trigger workflow - notify_workflow: { - description: 'Trigger a notification workflow', - parameters: { - workflowId: { type: 'string', required: true }, - recipients: { type: 'array', items: { type: 'string' } }, - data: { type: 'object' } - } - }, - - // Check delivery status - notify_status: { - description: 'Check notification delivery status', - parameters: { - messageId: { type: 'string', required: true } - } - }, - - // Get user preferences - notify_preferences: { - description: 'Get or update user notification preferences', - parameters: { - userId: { type: 'string', required: true }, - action: { type: 'string', enum: ['get', 'set'] }, - preferences: { type: 'object' } - } - } -} -``` - ---- - -## Sources - -- [Customer.io Documentation](https://docs.customer.io/integrations/api/) -- [Customer.io - Track Events](https://docs.customer.io/integrations/data-in/custom-events/) -- [Braze API Basics](https://www.braze.com/docs/api/basics/) -- [Braze Canvas Flow Architecture](https://www.braze.com/resources/articles/building-braze-how-braze-built-our-canvas-flow-customer-journey-tool) -- [OneSignal Documentation](https://documentation.onesignal.com/docs) -- [Knock Documentation](https://docs.knock.app/) -- [Knock Workflows](https://docs.knock.app/concepts/workflows) -- [Novu Documentation](https://docs.novu.co/) -- [Novu GitHub](https://github.com/novuhq/novu) -- [Novu Architecture Overview](https://dev.to/elie222/inside-the-open-source-novu-notification-engine-311g) diff --git a/rewrites/docs/research/search-scope.md b/rewrites/docs/research/search-scope.md deleted file mode 100644 index 1ea64c57..00000000 --- a/rewrites/docs/research/search-scope.md +++ /dev/null @@ -1,540 +0,0 @@ -# Search & Vector Database Rewrite Scope - -## Executive Summary - -This document outlines the architecture for a Cloudflare Workers-native search platform (`search.do` / `vectors.do`) that consolidates the capabilities of Algolia, Typesense, Meilisearch, Pinecone, Weaviate, and Qdrant into a unified edge-first search and vector database service. - -**Key Insight**: Cloudflare's native Vectorize + Workers AI + D1 stack provides all the building blocks needed to replace external search/vector services with a more cost-effective, lower-latency edge solution. - ---- - -## 1. Platform Competitive Analysis - -### 1.1 Traditional Search Platforms - -#### Algolia -- **Core Value**: Sub-20ms search with typo tolerance and synonyms -- **Key Features**: - - NeuralSearch (hybrid keyword + vector) - - AI Ranking, Personalization, AI Synonyms - - Faceting, filtering, InstantSearch UI -- **Pricing**: $0.50/1K searches (Grow), $1.75/1K (AI features) -- **Limitation**: No self-hosted option, expensive at scale - -#### Typesense -- **Core Value**: Open-source, typo-tolerant search -- **Key Features**: - - Vector search with ONNX models - - Hybrid search with configurable alpha (keyword vs semantic) - - Built-in embedding generation (OpenAI, PaLM, custom ONNX) - - Cosine similarity with distance threshold -- **Pricing**: ~$30/mo starting (cloud), free self-hosted -- **Limitation**: Requires dedicated cluster - -#### Meilisearch -- **Core Value**: Developer-friendly instant search -- **Key Features**: - - Multi-search (federated search across indexes) - - Hybrid search via embedders (OpenAI, HuggingFace, Cohere) - - Faceted filtering, geosearch - - Conversational search endpoint -- **Pricing**: $30/mo starting (cloud), free open-source -- **Limitation**: Limited vector dimensions - -### 1.2 Vector Databases - -#### Pinecone -- **Core Value**: Managed vector database for production AI -- **Key Features**: - - Serverless with on-demand pricing - - Namespaces for multi-tenancy - - Metadata filtering - - Inference API (built-in embeddings) -- **Pricing**: Free tier (2GB), $0.33/GB storage, $16/M reads -- **Limitation**: US-only free tier, expensive at scale - -#### Weaviate -- **Core Value**: Open-source AI-native vector database -- **Key Features**: - - Schema-based with modules - - Hybrid search (semantic + keyword) - - RAG integration, Query Agent - - Multi-tenant isolation -- **Pricing**: $45/mo Flex, $400/mo Premium -- **Limitation**: Complex deployment - -#### Qdrant -- **Core Value**: AI-native vector search with payload filtering -- **Key Features**: - - Universal Query API - - Multiple distance metrics (dot, cosine, euclidean, manhattan) - - ACORN algorithm for filtered search - - Multi-vector per point - - FastEmbed for embeddings -- **Pricing**: Free 1GB cluster, pay-as-you-go cloud -- **Limitation**: No serverless option - ---- - -## 2. Cloudflare Native Stack - -### 2.1 Vectorize (Vector Database) - -**Capabilities**: -- 5M vectors per index, 50K indexes per account -- 1536 dimensions max (float32) -- 10KB metadata per vector -- Namespaces (50K per index) -- Metadata indexing (10 per index) -- Cosine, euclidean, dot-product distance - -**Limits**: -| Resource | Free | Paid | -|----------|------|------| -| Indexes | 100 | 50,000 | -| Vectors/index | 5M | 5M | -| Namespaces | 1K | 50K | -| topK (with metadata) | 20 | 20 | -| topK (without) | 100 | 100 | - -**Pricing**: -- 30M queried dimensions/mo free, then $0.01/M -- 5M stored dimensions free, then $0.05/100M -- Example: 50K vectors @ 768 dims, 200K queries/mo = $1.94/mo - -### 2.2 Workers AI (Embeddings) - -**Models Available**: -| Model | Dimensions | Notes | -|-------|------------|-------| -| bge-small-en-v1.5 | 384 | Compact, fast | -| bge-base-en-v1.5 | 768 | Balanced | -| bge-large-en-v1.5 | 1024 | High quality | -| bge-m3 | Variable | Multilingual | -| EmbeddingGemma-300m | TBD | Google, 100+ languages | -| Qwen3-Embedding-0.6b | TBD | Text ranking | - -**Advantages**: -- Zero egress between Workers AI and Vectorize -- Pay per token, no idle costs -- Edge deployment (lower latency) - -### 2.3 D1 (Metadata + Full-Text) - -**Capabilities**: -- SQLite with FTS5 for full-text search -- 10GB per database (10K databases allowed) -- Read replicas for global edge reads -- Time Travel (30 days) - -**Use Cases in Search**: -- Document metadata storage -- Inverted index for keyword search -- Facet value storage -- Analytics and query logs - ---- - -## 3. Architecture Vision - -### 3.1 Domain Structure - -``` -search.do # Unified search API (keyword + semantic) - | - +-- vectors.do # Pure vector operations - +-- indexes.do # Index management - +-- facets.do # Faceted search - +-- suggest.do # Autocomplete/suggestions -``` - -### 3.2 Internal Architecture - -``` -search.do/ - src/ - core/ - embedding/ # Workers AI embedding generation - tokenizer/ # Text tokenization (for FTS) - scorer/ # Relevance scoring (TF-IDF, BM25) - ranker/ # Result ranking & fusion - storage/ - vectorize/ # Vectorize wrapper - d1/ # D1 metadata + FTS5 - cache/ # Edge caching layer - api/ - search/ # Unified search endpoint - index/ # Index CRUD - vector/ # Raw vector ops - facet/ # Facet computation - suggest/ # Autocomplete - connectors/ - webhook/ # Real-time sync - cron/ # Batch reindexing - durable-object/ - SearchIndexDO # Per-index coordination - QueryCoordinatorDO # Multi-index search - workers/ - search-edge # Edge search worker - index-ingest # Background indexing -``` - -### 3.3 Hybrid Search Strategy - -``` -Query: "comfortable wireless headphones under $100" - | - +-----------+-----------+ - | | - Keyword Path Semantic Path - | | - D1 FTS5 Workers AI - "wireless" embed(query) - "headphones" | - | Vectorize - | k-NN search - | | - +-----------+-----------+ - | - Rank Fusion - (RRF or Linear blend) - | - Metadata Filter - price < 100, in_stock - | - Final Results -``` - -### 3.4 Data Flow - -``` -Document Ingestion: - Client -> search.do/index - -> Queue (background) - -> Workers AI (embed) - -> Vectorize (upsert) - -> D1 (metadata + FTS) - -Search Query: - Client -> search.do (edge) - -> [D1 FTS + Vectorize] parallel - -> Rank fusion - -> D1 (enrich metadata) - -> Response -``` - ---- - -## 4. API Design - -### 4.1 Core Search API - -```typescript -// Natural language search -const results = await search.do` - Find wireless headphones under $100 - that have good reviews for travel -` - -// Structured search -const results = await search.search('products', 'wireless headphones', { - limit: 20, - offset: 0, - filters: [ - { field: 'price', op: 'lt', value: 100 }, - { field: 'inStock', op: 'eq', value: true } - ], - facets: ['category', 'brand'], - sort: [{ field: 'relevance', order: 'desc' }], - semantic: true, // Enable vector search - hybrid: { - alpha: 0.7 // 70% semantic, 30% keyword - } -}) -``` - -### 4.2 Vector Operations - -```typescript -// Generate embedding -const vector = await search.embed('comfortable headphones for travel') - -// Direct vector search -const similar = await search.searchVector('products', vector, { - limit: 10, - namespace: 'electronics', - filter: { brand: 'Sony' } -}) - -// Batch embeddings -const vectors = await search.embedBatch([ - 'wireless headphones', - 'bluetooth speakers', - 'usb cables' -]) -``` - -### 4.3 Index Management - -```typescript -// Create index with schema -await search.createIndex('products', { - semantic: true, - embeddingModel: '@cf/baai/bge-base-en-v1.5', - fields: [ - { name: 'title', type: 'text', searchable: true, boost: 2 }, - { name: 'description', type: 'text', searchable: true }, - { name: 'price', type: 'number', filterable: true, sortable: true }, - { name: 'category', type: 'keyword', filterable: true, facetable: true }, - { name: 'brand', type: 'keyword', filterable: true, facetable: true }, - { name: 'embedding', type: 'vector', dimensions: 768 } - ] -}) - -// Index documents -await search.index('products', [ - { id: 'p1', title: 'Wireless Headphones', price: 99, ... }, - { id: 'p2', title: 'Bluetooth Speaker', price: 49, ... } -]) -``` - -### 4.4 Facets & Suggestions - -```typescript -// Get facet values -const facets = await search.facets('products', ['category', 'brand'], { - query: 'headphones', - limit: 10 -}) -// { category: [{ value: 'audio', count: 150 }], brand: [...] } - -// Autocomplete suggestions -const suggestions = await search.suggest('products', 'wire', { - limit: 5, - fuzzy: true -}) -// ['wireless headphones', 'wireless speakers', 'wire cutters'] -``` - ---- - -## 5. Implementation Phases - -### Phase 1: Core Vector Search (MVP) -- [ ] Vectorize wrapper with Workers AI integration -- [ ] Basic upsert/query/delete operations -- [ ] Namespace support -- [ ] Metadata filtering -- [ ] SDK (`vectors.do` package) - -### Phase 2: Full-Text Search -- [ ] D1 FTS5 integration -- [ ] Tokenization and normalization -- [ ] BM25 scoring -- [ ] Typo tolerance (fuzzy matching) -- [ ] Synonym support - -### Phase 3: Hybrid Search -- [ ] Rank fusion (RRF + linear blend) -- [ ] Configurable alpha (keyword vs semantic) -- [ ] Query understanding (intent detection) -- [ ] Result deduplication - -### Phase 4: Advanced Features -- [ ] Faceted search with counts -- [ ] Autocomplete/suggestions -- [ ] Geosearch -- [ ] Analytics (query logs, zero-result tracking) -- [ ] Natural language search (`.do` template) - -### Phase 5: Enterprise Features -- [ ] Multi-tenancy (per-tenant namespaces) -- [ ] Index replication (global edge) -- [ ] Personalization -- [ ] A/B testing for ranking -- [ ] Query caching - ---- - -## 6. Cost Comparison - -### Scenario: 100K documents, 500K searches/month - -| Platform | Storage | Queries | Monthly Cost | -|----------|---------|---------|--------------| -| Algolia | $40 | $250 | ~$290 | -| Pinecone | $33 | $80 | ~$113 | -| Weaviate | $12 | $75 | ~$87 | -| **search.do** | $0.05 | $0.50 | **~$1** | - -**Advantage**: 100x+ cost reduction at scale - -### Latency Comparison - -| Platform | P50 Latency | Notes | -|----------|-------------|-------| -| Algolia | <20ms | Distributed edge | -| Pinecone | 50-100ms | Centralized regions | -| **search.do** | <10ms | Edge + cached | - ---- - -## 7. Technical Considerations - -### 7.1 Index Size Limitations - -**Challenge**: Vectorize has 5M vector limit per index - -**Solutions**: -1. **Sharding**: Partition by namespace/tenant -2. **Tiering**: Hot vectors in Vectorize, cold in R2 -3. **Multi-index**: Federated search across indexes - -### 7.2 Embedding Generation Costs - -**Challenge**: Workers AI costs per token - -**Mitigations**: -1. **Caching**: Hash-based embedding cache in KV -2. **Batching**: Batch document embeddings -3. **Model selection**: Use smaller models (bge-small) for suggestions - -### 7.3 Hybrid Search Latency - -**Challenge**: Parallel D1 + Vectorize adds latency - -**Optimizations**: -1. **Speculative execution**: Start both paths immediately -2. **Early termination**: Stop if one path has high-confidence results -3. **Caching**: Cache frequent queries at edge - -### 7.4 Full-Text Search Quality - -**Challenge**: D1 FTS5 is basic compared to Elasticsearch - -**Enhancements**: -1. **Stemming**: Porter stemmer for English -2. **Synonyms**: Synonym expansion before search -3. **Boosting**: Field-level relevance weights -4. **Typo tolerance**: Levenshtein distance matching - ---- - -## 8. Existing Code to Leverage - -### From `searches.do` SDK -- Complete TypeScript interface for search API -- Filter, facet, and sort type definitions -- Natural language `.do` template pattern - -### From `rewrites/mongo` Vector Search -- Vectorize type definitions -- Vector search stage translator -- Workers AI embedding integration - -### From `rewrites/convex` Full-Text -- Tokenization and normalization -- Fuzzy matching (Levenshtein) -- TF-IDF-like relevance scoring -- Search filter builder pattern - ---- - -## 9. Success Metrics - -### Performance -- P50 search latency < 10ms -- P99 search latency < 50ms -- Indexing throughput > 1000 docs/sec - -### Cost -- < $5/mo for 100K docs, 500K queries -- Zero egress costs (all Cloudflare) - -### Developer Experience -- Index creation < 1 minute -- First search < 5 minutes from signup -- SDK in npm, types in TypeScript - ---- - -## 10. Recommended Approach - -### Start with `vectors.do` -Focus on vector operations first since Vectorize is the most mature CF primitive. - -### Build on `searches.do` SDK -The SDK interface is already well-designed. Implement the backend to match. - -### Hybrid search is the differentiator -Combine D1 FTS5 + Vectorize for best-of-both-worlds search quality. - -### Edge caching is key -Use KV/Cache API for frequent queries to achieve sub-10ms latency. - ---- - -## Appendix A: Cloudflare Binding Example - -```typescript -// wrangler.toml -[[vectorize]] -binding = "VECTORS" -index_name = "products" - -[[d1_databases]] -binding = "DB" -database_id = "xxx" - -[[ai]] -binding = "AI" - -// worker.ts -export default { - async fetch(req: Request, env: Env) { - // Generate embedding - const { data } = await env.AI.run('@cf/baai/bge-base-en-v1.5', { - text: 'wireless headphones' - }) - - // Query vectors - const results = await env.VECTORS.query(data[0], { - topK: 10, - returnMetadata: 'all' - }) - - // Enrich from D1 - const ids = results.matches.map(m => m.id) - const docs = await env.DB.prepare( - `SELECT * FROM products WHERE id IN (${ids.map(() => '?').join(',')})` - ).bind(...ids).all() - - return Response.json({ results: docs.results }) - } -} -``` - ---- - -## Appendix B: Feature Parity Matrix - -| Feature | Algolia | Typesense | Meilisearch | Pinecone | Weaviate | Qdrant | search.do | -|---------|---------|-----------|-------------|----------|----------|--------|-----------| -| Keyword search | Y | Y | Y | - | Y | - | Y | -| Vector search | Y | Y | Y | Y | Y | Y | Y | -| Hybrid search | Y | Y | Y | - | Y | Y | Y | -| Typo tolerance | Y | Y | Y | - | - | - | Y | -| Faceting | Y | Y | Y | - | Y | Y | Y | -| Autocomplete | Y | Y | Y | - | - | - | Y | -| Geosearch | Y | Y | Y | - | Y | - | P2 | -| Multi-tenancy | Y | Y | Y | Y | Y | Y | Y | -| Analytics | Y | - | - | - | Y | - | Y | -| Edge deployment | Y | - | - | - | - | - | Y | -| Free tier | Y | - | - | Y | Y | Y | Y | -| Open source | - | Y | Y | - | Y | Y | - | - ---- - -*Document Version: 1.0* -*Last Updated: 2026-01-07* -*Author: Research Agent* diff --git a/rewrites/docs/research/workflows-scope.md b/rewrites/docs/research/workflows-scope.md deleted file mode 100644 index 4eab282b..00000000 --- a/rewrites/docs/research/workflows-scope.md +++ /dev/null @@ -1,718 +0,0 @@ -# Workflows Rewrite Scoping Document - -## Executive Summary - -This document scopes the `workflows.do` / `jobs.do` rewrite - a Cloudflare-native alternative to platforms like Inngest, Trigger.dev, Temporal, Windmill, Upstash QStash, and Defer. The goal is to provide a unified workflow orchestration and background job platform built entirely on Cloudflare primitives: Workflows, Queues, Durable Objects, and Cron Triggers. - ---- - -## Platform Competitive Analysis - -### 1. Inngest - Event-Driven Durable Execution - -**Core Value Proposition:** -- "Event-driven durable execution platform without managing queues, infra, or state" -- Abstracts queueing, scaling, concurrency, throttling, rate limiting, and observability - -**Key Features:** -| Feature | Implementation | -|---------|---------------| -| **Workflow Definition** | Code-based with TypeScript/Python/Go SDKs | -| **Durable Steps** | `step.run()`, `step.sleep()`, `step.sleepUntil()`, `step.waitForEvent()`, `step.invoke()` | -| **Triggers** | Events, cron schedules, webhooks | -| **Retry Policy** | Step-level automatic retries with memoization | -| **Concurrency** | Step-level limits with keys (per-user/per-tenant queues) | -| **Throttling** | Function-level rate limiting (e.g., 3 runs/minute) | -| **State** | Each step result saved, skipped on replay | - -**Architecture Insight:** -- Each step executes as a separate HTTP request -- Step IDs used for memoization across function versions -- Supports `onError: 'continue' | 'retry' | 'fail'` - -**Cloudflare Mapping:** -- `step.run()` -> CF Workflow step with DO state -- `step.sleep()` -> CF Workflow `step.sleep()` or DO Alarm -- `step.waitForEvent()` -> DO WebSocket hibernation or Queue consumer -- Concurrency keys -> DO sharding by key - ---- - -### 2. Trigger.dev - Background Jobs for AI/Long-Running Tasks - -**Core Value Proposition:** -- "Build AI workflows and background tasks in TypeScript" -- No execution timeouts, elastic scaling, zero infrastructure management - -**Key Features:** -| Feature | Implementation | -|---------|---------------| -| **Task Definition** | TypeScript functions with retry config | -| **Scheduling** | Cron-based, imperative via SDK | -| **Retry Policy** | Configurable attempts with exponential backoff | -| **Concurrency** | Queue-based concurrency management | -| **Checkpointing** | Automatic checkpoint-resume for fault tolerance | -| **Observability** | Real-time dashboard, distributed tracing, alerts | -| **Realtime** | Stream LLM responses to frontend | - -**Architecture Insight:** -- Workers poll task queues, execute code, report results -- Atomic versioning prevents code changes affecting in-progress tasks -- Human-in-the-loop via "waitpoint tokens" - -**Cloudflare Mapping:** -- Task queues -> Cloudflare Queues -- Long-running tasks -> CF Workflows (no timeouts) -- Checkpointing -> DO state persistence -- Realtime -> DO WebSocket broadcasts - ---- - -### 3. Temporal - Enterprise Workflow Orchestration - -**Core Value Proposition:** -- "Durable execution - guaranteeing code completion regardless of failures" -- Applications automatically recover from crashes by replaying execution history - -**Key Features:** -| Feature | Implementation | -|---------|---------------| -| **Workflows** | Define business logic with guaranteed completion | -| **Activities** | Encapsulate failure-prone operations with retries | -| **Workers** | Poll task queues, execute code, report results | -| **History** | Detailed event history for replay recovery | -| **Retry Policy** | Exponential backoff with `MaximumInterval` and `MaximumAttempts` | -| **Failure Types** | Transient, intermittent, permanent classification | - -**Error Recovery Strategies:** -- **Forward Recovery**: Retry until success -- **Backward Recovery**: Undo completed actions (compensating transactions) - -**Architecture Insight:** -- Workflow state = event sourcing (replay to reconstruct) -- Activities can run indefinitely with heartbeat monitoring -- Scale: 200+ million executions/second on Temporal Cloud - -**Cloudflare Mapping:** -- Event sourcing -> DO SQLite with event log table -- Workers -> CF Workers processing Queue messages -- Heartbeats -> DO Alarms for timeout detection -- Compensating transactions -> Step-level rollback handlers - ---- - -### 4. Windmill - Open-Source Workflows + Internal Tools - -**Core Value Proposition:** -- "Fast, open-source workflow engine and developer platform" -- Combines code flexibility with low-code speed - -**Key Features:** -| Feature | Implementation | -|---------|---------------| -| **Languages** | TypeScript, Python, Go, PHP, Bash, SQL, Rust, Docker | -| **Execution** | Scalable worker fleet for low-latency function execution | -| **Orchestration** | Assembles functions into efficient flows | -| **Integration** | Webhooks, open API, scheduler, Git sync | -| **Enterprise** | Built-in permissioning, secret management, OAuth | - -**Cloudflare Mapping:** -- Multi-language -> Worker Loader (sandboxed execution) -- Worker fleet -> Cloudflare's global network -- Secret management -> Workers Secrets + Vault integration - ---- - -### 5. Upstash QStash - Serverless Messaging - -**Core Value Proposition:** -- "Serverless messaging and scheduling solution" -- Guaranteed delivery without infrastructure management - -**Key Features:** -| Feature | Implementation | -|---------|---------------| -| **Messaging** | Publish messages up to 1MB (JSON, XML, binary) | -| **Scheduling** | Future-dated message delivery | -| **Retry Policy** | Automatic retries on failure | -| **Dead Letter Queue** | Manual intervention for persistently failed messages | -| **Fan-out** | URL Groups for parallel delivery | -| **FIFO** | Queue ordering for sequential processing | -| **Rate Limiting** | Parallelism controls | -| **Callbacks** | Delivery confirmations | -| **Deduplication** | Prevent duplicate delivery | - -**Upstash Workflow SDK:** -- Step-based architecture (each step = separate request) -- Failed step retries without re-executing prior steps -- Parallel processing with coordinated completion -- Extended delays (days, weeks, months) - -**Cloudflare Mapping:** -- Messaging -> Cloudflare Queues -- Dead Letter Queue -> Queue DLQ configuration -- FIFO -> Queue with single consumer -- Fan-out -> Multiple queue bindings -- Deduplication -> DO-based message tracking with TTL - ---- - -### 6. Defer (Acquired by Digger) - -**Core Value Proposition:** -- Background functions for existing codebases -- Zero-config deployment - -**Status:** Acquired - redirect to digger.tools - ---- - -## Cloudflare Native Primitives - -### Cloudflare Workflows (New!) - -**Key Capabilities:** -- Durable multi-step execution without timeouts -- Automatic retries and error handling -- Pause for external events or approvals -- State persistence for minutes, hours, or weeks -- `step.sleep()` and `step.sleepUntil()` for delays -- Built-in observability and debugging - -**API Pattern:** -```typescript -import { WorkflowEntrypoint } from 'cloudflare:workers' - -export class MyWorkflow extends WorkflowEntrypoint { - async run(event: WorkflowEvent, step: WorkflowStep) { - const result1 = await step.do('step-1', async () => { - return await this.env.SERVICE.doSomething() - }) - - await step.sleep('wait-period', '1 hour') - - const result2 = await step.do('step-2', async () => { - return await processResult(result1) - }) - - return result2 - } -} -``` - -### Cloudflare Queues - -**Key Capabilities:** -- Guaranteed delivery -- Batching, retries, and message delays -- Dead Letter Queues for failed messages -- Pull-based consumers for external access -- No egress charges - -**Configuration:** -- Delivery Delay: 0-43,200 seconds -- Message Retention: 60-1,209,600 seconds (default: 4 days) -- Max Retries: Default 3 -- Max Batch Size: Default 10 -- Max Concurrency: Autoscales when unset - -### Cloudflare Cron Triggers - -**Key Capabilities:** -- Five-field cron expressions with Quartz extensions -- Execute on UTC time -- Propagation delay: up to 15 minutes -- Can bind directly to Workflows -- Green Compute option (renewable energy only) - -**Examples:** -- `* * * * *` - Every minute -- `*/30 * * * *` - Every 30 minutes -- `0 17 * * sun` - 17:00 UTC Sundays -- `59 23 LW * *` - Last weekday of month at 23:59 UTC - -### Durable Objects - -**Key Capabilities:** -- Globally-unique named instances -- Transactional, strongly consistent SQLite storage -- In-memory state coordination -- WebSocket hibernation for client connections -- Alarms for scheduled compute - ---- - -## Architecture Vision - -``` -workflows.do / jobs.do -├── packages/ -│ ├── workflows.do/ # SDK (exists) -│ └── jobs.do/ # SDK (background jobs focus) -│ -├── workers/workflows/ # Worker (exists) -│ ├── src/ -│ │ ├── workflows.ts # WorkflowsDO implementation -│ │ ├── scheduler.ts # Cron + schedule management -│ │ ├── queue-consumer.ts # Queue message processing -│ │ └── observability.ts # Metrics + logging -│ └── wrangler.jsonc -│ -└── rewrites/workflows/ # Enhanced rewrite (new) - ├── src/ - │ ├── core/ - │ │ ├── workflow-engine.ts # CF Workflows integration - │ │ ├── step-executor.ts # Step execution with memoization - │ │ ├── event-router.ts # Event pattern matching - │ │ └── schedule-parser.ts # Natural schedule parsing - │ │ - │ ├── durable-object/ - │ │ ├── WorkflowOrchestrator.ts # Main DO for workflow state - │ │ ├── JobQueue.ts # Per-tenant job queuing - │ │ └── ScheduleManager.ts # Alarm-based scheduling - │ │ - │ ├── queue/ - │ │ ├── producer.ts # Queue message publishing - │ │ ├── consumer.ts # Queue message processing - │ │ └── dlq-handler.ts # Dead letter queue processing - │ │ - │ ├── triggers/ - │ │ ├── cron.ts # Cron trigger handling - │ │ ├── webhook.ts # Webhook trigger handling - │ │ └── event.ts # Event trigger handling - │ │ - │ ├── storage/ - │ │ ├── event-store.ts # Event sourcing for workflow history - │ │ ├── state-store.ts # Workflow state persistence - │ │ └── metrics-store.ts # Observability data - │ │ - │ └── api/ - │ ├── rest.ts # REST API (Hono) - │ ├── rpc.ts # RPC interface - │ └── websocket.ts # Real-time streaming - │ - ├── .beads/ # Issue tracking - └── CLAUDE.md # AI guidance -``` - ---- - -## Feature Comparison Matrix - -| Feature | Inngest | Trigger.dev | Temporal | workflows.do (Target) | -|---------|---------|-------------|----------|----------------------| -| **Step Functions** | Yes | Yes | Activities | Yes (CF Workflows) | -| **Durable Execution** | Yes | Yes | Yes | Yes (DO + CF Workflows) | -| **Event Triggers** | Yes | Yes | Yes | Yes | -| **Cron Schedules** | Yes | Yes | Via triggers | Yes (Cron Triggers) | -| **Webhooks** | Yes | Yes | Via signals | Yes | -| **Retry Policies** | Step-level | Task-level | Activity-level | Step-level | -| **Concurrency Keys** | Yes | Yes | Task queues | Yes (DO sharding) | -| **Rate Limiting** | Yes | Yes | Via config | Yes | -| **Dead Letter Queue** | Yes | Yes | Via config | Yes (Queue DLQ) | -| **Human-in-the-Loop** | waitForEvent | waitpoint | Signals | waitForEvent | -| **Long-Running** | Yes | Yes | Yes | Yes (no timeout) | -| **Observability** | Dashboard | Dashboard | Web UI | Dashboard | -| **Multi-Language** | TS/Py/Go | TypeScript | Many | TypeScript | -| **Self-Hosted** | No | No | Yes | Yes (CF Workers) | -| **Edge-Native** | No | No | No | Yes (global) | - ---- - -## Cloudflare-Native Advantages - -### 1. Zero Cold Start for Jobs -- Durable Objects maintain warm state -- Queue consumers always ready -- Cron triggers execute immediately - -### 2. Global Edge Execution -- Workflows run close to users -- Reduced latency for event processing -- Automatic geo-routing - -### 3. Integrated Primitives -- Queues for job distribution -- Workflows for long-running processes -- DO for state management -- Alarms for scheduling -- No external dependencies - -### 4. Cost Efficiency -- Pay per request, not per server -- No idle costs -- Free tier generous for testing - -### 5. Built-in Security -- Workers isolation -- Secrets management -- mTLS for service-to-service - ---- - -## Implementation Phases - -### Phase 1: Core Workflow Engine (Week 1-2) - -**Goals:** -- Integrate with Cloudflare Workflows primitive -- Implement step execution with memoization -- Add basic retry policies - -**Tasks:** -1. Create `WorkflowOrchestrator` DO -2. Implement `step.do()`, `step.sleep()`, `step.sleepUntil()` -3. Add step-level retry with exponential backoff -4. Persist step results for replay - -### Phase 2: Event & Trigger System (Week 2-3) - -**Goals:** -- Event pattern matching ($.on.Noun.event) -- Cron schedule parsing and execution -- Webhook trigger handling - -**Tasks:** -1. Implement event router with glob patterns -2. Parse natural language schedules (`$.every.5.minutes`) -3. Connect to Cron Triggers -4. Add webhook authentication - -### Phase 3: Queue Integration (Week 3-4) - -**Goals:** -- Job queuing with Cloudflare Queues -- Concurrency controls and rate limiting -- Dead letter queue handling - -**Tasks:** -1. Create queue producer/consumer -2. Implement concurrency keys (per-user queues) -3. Add rate limiting with DO counters -4. Configure DLQ with retry exhaustion - -### Phase 4: Observability & Dashboard (Week 4-5) - -**Goals:** -- Workflow execution history -- Real-time streaming -- Metrics and alerting - -**Tasks:** -1. Event sourcing for complete history -2. WebSocket streaming via DO -3. Metrics storage and querying -4. Alert webhook integration - -### Phase 5: Advanced Features (Week 5-6) - -**Goals:** -- Human-in-the-loop (waitForEvent) -- Parallel step execution -- Compensating transactions - -**Tasks:** -1. Implement `step.waitForEvent()` with token -2. Add parallel step execution with `Promise.all` -3. Define rollback handlers per step -4. Test complex workflow patterns - ---- - -## Data Model - -### Workflow Definition (SQLite) - -```sql -CREATE TABLE workflows ( - id TEXT PRIMARY KEY, - name TEXT NOT NULL, - description TEXT, - definition JSON NOT NULL, -- steps, events, schedules - timeout_ms INTEGER, - on_error TEXT DEFAULT 'fail', - version INTEGER DEFAULT 1, - created_at INTEGER NOT NULL, - updated_at INTEGER NOT NULL -); - -CREATE TABLE triggers ( - id TEXT PRIMARY KEY, - workflow_id TEXT NOT NULL, - type TEXT NOT NULL, -- 'event' | 'schedule' | 'webhook' - pattern TEXT, -- Event pattern or cron expression - config JSON, -- Additional trigger config - enabled INTEGER DEFAULT 1, - last_triggered_at INTEGER, - trigger_count INTEGER DEFAULT 0, - FOREIGN KEY (workflow_id) REFERENCES workflows(id) -); -``` - -### Execution State (SQLite) - -```sql -CREATE TABLE executions ( - id TEXT PRIMARY KEY, - workflow_id TEXT NOT NULL, - status TEXT NOT NULL, -- 'pending' | 'running' | 'paused' | 'completed' | 'failed' - input JSON, - state JSON, - current_step TEXT, - error TEXT, - started_at INTEGER NOT NULL, - completed_at INTEGER, - FOREIGN KEY (workflow_id) REFERENCES workflows(id) -); - -CREATE TABLE step_results ( - id TEXT PRIMARY KEY, - execution_id TEXT NOT NULL, - step_id TEXT NOT NULL, - status TEXT NOT NULL, - input JSON, - output JSON, - error TEXT, - retry_count INTEGER DEFAULT 0, - started_at INTEGER, - completed_at INTEGER, - FOREIGN KEY (execution_id) REFERENCES executions(id) -); - -CREATE TABLE events ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - execution_id TEXT NOT NULL, - type TEXT NOT NULL, -- 'step' | 'event' | 'schedule' | 'error' - name TEXT NOT NULL, - data JSON, - timestamp INTEGER NOT NULL, - FOREIGN KEY (execution_id) REFERENCES executions(id) -); -``` - ---- - -## API Design - -### REST API - -``` -POST /api/workflows # Create workflow -GET /api/workflows # List workflows -GET /api/workflows/:id # Get workflow -PUT /api/workflows/:id # Update workflow -DELETE /api/workflows/:id # Delete workflow - -POST /api/workflows/:id/start # Start execution -GET /api/executions # List executions -GET /api/executions/:id # Get execution -POST /api/executions/:id/pause # Pause execution -POST /api/executions/:id/resume # Resume execution -POST /api/executions/:id/cancel # Cancel execution -POST /api/executions/:id/retry # Retry execution -GET /api/executions/:id/history # Get execution history - -POST /api/events # Send event -POST /api/webhooks/:path # Webhook trigger -``` - -### RPC Interface (rpc.do pattern) - -```typescript -interface WorkflowsRPC { - // Workflow CRUD - createWorkflow(definition: WorkflowDefinition): Promise - getWorkflow(id: string): Promise - updateWorkflow(id: string, updates: Partial): Promise - deleteWorkflow(id: string): Promise - listWorkflows(options?: ListOptions): Promise - - // Execution - startWorkflow(id: string, input?: unknown): Promise - getExecution(id: string): Promise - pauseExecution(id: string): Promise - resumeExecution(id: string): Promise - cancelExecution(id: string): Promise - retryExecution(id: string): Promise - listExecutions(options?: ListOptions): Promise - - // Events & Triggers - sendEvent(type: string, data: unknown): Promise - registerTrigger(workflowId: string, trigger: Trigger): Promise - unregisterTrigger(workflowId: string): Promise - listTriggers(options?: TriggerListOptions): Promise - - // Observability - getHistory(executionId: string): Promise - getMetrics(workflowId?: string): Promise -} -``` - ---- - -## Key Design Decisions - -### 1. Workflow State Persistence - -**Decision:** Event sourcing with SQLite in Durable Objects - -**Rationale:** -- Complete replay capability (like Temporal) -- Step-level memoization (like Inngest) -- Strongly consistent (DO guarantees) -- Query-friendly for observability - -### 2. Exactly-Once Semantics - -**Decision:** Step ID-based deduplication - -**Implementation:** -- Each step has unique ID within workflow -- Step results cached in DO storage -- On replay, skip steps with cached results -- Idempotency requirement documented for handlers - -### 3. Long-Running Workflow Support - -**Decision:** CF Workflows + DO Alarms hybrid - -**Implementation:** -- Short workflows: CF Workflows primitive -- Long waits: DO Alarms for wake-up -- External events: DO WebSocket hibernation -- Timeouts: Configurable per workflow - -### 4. Error Handling Patterns - -**Decision:** Step-level configuration with sensible defaults - -**Options per step:** -- `onError: 'fail'` - Stop workflow (default) -- `onError: 'continue'` - Skip step, continue -- `onError: 'retry'` - Retry with backoff - -**Retry Policy:** -```typescript -{ - maxAttempts: 3, - initialIntervalMs: 1000, - backoffCoefficient: 2.0, - maxIntervalMs: 60000 -} -``` - -### 5. Concurrency Control - -**Decision:** DO sharding by concurrency key - -**Implementation:** -- Default: One DO per workflow -- With key: One DO per `{workflowId}:{key}` -- Rate limiting: Counter in DO with sliding window -- Throttling: Delay queue processing - ---- - -## Testing Strategy - -### Unit Tests -- Step execution logic -- Event pattern matching -- Schedule parsing -- Retry policy calculations - -### Integration Tests -- Queue producer/consumer -- Cron trigger execution -- Webhook authentication -- DO state persistence - -### E2E Tests -- Complete workflow execution -- Failure and retry scenarios -- Long-running workflow with sleeps -- Event-triggered workflows - ---- - -## Migration Path - -### From Inngest -```typescript -// Inngest -inngest.createFunction( - { id: 'sync-user' }, - { event: 'user/created' }, - async ({ event, step }) => { - await step.run('sync', () => syncUser(event.data)) - } -) - -// workflows.do -workflows.define($ => { - $.on.user.created(async (data, $) => { - await $.do('sync', () => syncUser(data)) - }) -}) -``` - -### From Trigger.dev -```typescript -// Trigger.dev -export const myTask = task({ - id: 'my-task', - retry: { maxAttempts: 3 }, - run: async (payload) => { ... } -}) - -// workflows.do -workflows.steps('my-task', { - steps: [ - { name: 'main', action: 'my-action', retry: { attempts: 3 } } - ] -}) -``` - ---- - -## Success Metrics - -1. **Reliability:** 99.99% execution completion rate -2. **Performance:** P99 step execution < 100ms -3. **Scale:** Support 10K concurrent workflows per account -4. **DX:** Workflow definition in < 10 lines of code -5. **Observability:** Full history available within 1s of event - ---- - -## Open Questions - -1. **Naming:** `workflows.do` vs `jobs.do` vs both? - - `workflows.do` - Complex multi-step orchestration - - `jobs.do` - Simple background job queue - - Recommendation: Both SDKs, shared infrastructure - -2. **Pricing Model:** Per execution? Per step? Per second? - - Align with CF pricing (requests + duration) - -3. **SDK Parity:** Support Python/Go like Inngest? - - Phase 1: TypeScript only - - Phase 2: Evaluate demand - -4. **Self-Hosted:** Allow customers to run own workers? - - Yes - CF Workers deploy anywhere - - Provide wrangler.jsonc template - ---- - -## References - -- [Cloudflare Workflows Docs](https://developers.cloudflare.com/workflows) -- [Cloudflare Queues Docs](https://developers.cloudflare.com/queues) -- [Cloudflare Durable Objects Docs](https://developers.cloudflare.com/durable-objects) -- [Inngest Documentation](https://www.inngest.com/docs) -- [Trigger.dev Documentation](https://trigger.dev/docs) -- [Temporal Documentation](https://docs.temporal.io) -- [Upstash QStash Documentation](https://upstash.com/docs/qstash) -- [Windmill Documentation](https://www.windmill.dev/docs) -- [Existing workflows.do Implementation](/Users/nathanclevenger/projects/workers/workers/workflows/src/workflows.ts) diff --git a/rewrites/docs/reviews/architectural-review.md b/rewrites/docs/reviews/architectural-review.md deleted file mode 100644 index f4020f16..00000000 --- a/rewrites/docs/reviews/architectural-review.md +++ /dev/null @@ -1,708 +0,0 @@ -# Architectural Review: workers.do Rewrites Platform - -**Date**: 2026-01-07 -**Scope**: fsx, gitx, kafka, nats, supabase rewrites -**Author**: Claude (Architectural Review) - ---- - -## Executive Summary - -The workers.do rewrites platform implements a consistent pattern for rebuilding popular databases and services on Cloudflare Durable Objects. After reviewing the mature rewrites (fsx, gitx, kafka, nats, supabase), I find the architecture to be **well-designed and scalable**, with strong consistency in core patterns but room for improvement in cross-rewrite integration and client SDK standardization. - -### Key Findings - -| Aspect | Rating | Notes | -|--------|--------|-------| -| DO Pattern Consistency | Good | All rewrites follow similar DO + Hono patterns | -| Storage Tier Implementation | Good | Hot/warm/cold tiers well-designed in fsx/gitx | -| Client/Server Split | Mixed | Inconsistent SDK patterns across rewrites | -| MCP Integration | Excellent | Comprehensive tool definitions in fsx/gitx | -| Cross-rewrite Dependencies | Needs Work | gitx -> fsx coupling is too tight | -| Scalability | Excellent | DO model supports millions of instances | - ---- - -## 1. Durable Objects Pattern Analysis - -### 1.1 Common DO Structure - -All reviewed rewrites follow a consistent pattern: - -``` - +-----------------------+ - | HTTP Entry Point | - | (Cloudflare Worker) | - +-----------------------+ - | - +---------------+---------------+ - | | | - +------------------+ +------------------+ +------------------+ - | {Service}DO (1) | | {Service}DO (2) | | {Service}DO (n) | - | SQLite + R2 | | SQLite + R2 | | SQLite + R2 | - +------------------+ +------------------+ +------------------+ -``` - -### 1.2 DO Implementation Comparison - -| Rewrite | DO Classes | Hono Routing | SQLite Schema | R2 Integration | -|---------|-----------|--------------|---------------|----------------| -| **fsx** | `FileSystemDO` | Yes (`/rpc`, `/stream/*`) | files, blobs tables | Tiered storage | -| **gitx** | `GitDO` (inferred) | Yes | objects, refs, hot_objects, wal | R2 packfiles | -| **kafka** | `TopicPartitionDO`, `ConsumerGroupDO`, `ClusterMetadataDO` | Yes | messages, watermarks | Not yet | -| **nats** | `NatsCoordinator` | Yes (RPC) | consumers | Not yet | -| **supabase** | `SupabaseDO` (planned) | TBD | Per-agent database | R2 storage | - -### 1.3 Architectural Diagram: FileSystemDO (Reference Implementation) - -``` -+------------------------------------------------------------------+ -| FileSystemDO | -+------------------------------------------------------------------+ -| Constructor | -| - Initialize Hono app | -| - Set up routes | -+------------------------------------------------------------------+ -| ensureInitialized() | -| - Run SQLite schema | -| - Create root directory | -+------------------------------------------------------------------+ -| Routes: | -| +------------------------------------------------------------+ | -| | POST /rpc -> handleMethod(method, params) | | -| | POST /stream/read -> streaming file read | | -| | POST /stream/write -> streaming file write | | -| +------------------------------------------------------------+ | -+------------------------------------------------------------------+ -| SQLite Tables: | -| +--------------------+ +--------------------+ | -| | files | | blobs | | -| +--------------------+ +--------------------+ | -| | id (PK) | | id (PK) | | -| | path (unique) | | data (BLOB) | | -| | name | | size | | -| | parent_id (FK) | | tier | | -| | type | | created_at | | -| | mode, uid, gid | +--------------------+ | -| | size | | -| | blob_id (FK) | | -| | timestamps... | | -| +--------------------+ | -+------------------------------------------------------------------+ -``` - -### 1.4 DO Initialization Pattern - -All DOs use lazy initialization: - -```typescript -// Common pattern across all rewrites -private initialized = false - -private async ensureInitialized() { - if (this.initialized) return - - // Run schema - await this.ctx.storage.sql.exec(SCHEMA) - - // Initialize data structures - // ... - - this.initialized = true -} - -async fetch(request: Request): Promise { - await this.ensureInitialized() - return this.app.fetch(request) -} -``` - -**Recommendation**: Extract this pattern into a base class in `objects/do/`: - -```typescript -// objects/do/base.ts -export abstract class BaseDO extends DurableObject { - protected app: Hono - private initialized = false - - protected abstract getSchema(): string - protected abstract setupRoutes(): void - protected abstract onInitialize(): Promise - - private async ensureInitialized() { - if (this.initialized) return - await this.ctx.storage.sql.exec(this.getSchema()) - await this.onInitialize() - this.initialized = true - } - - async fetch(request: Request): Promise { - await this.ensureInitialized() - return this.app.fetch(request) - } -} -``` - ---- - -## 2. Storage Tiers Analysis - -### 2.1 Tiered Storage Architecture - -``` -+-------------------+ -| REQUEST | -+-------------------+ - | - v -+-------------------+ miss +-------------------+ miss +-------------------+ -| HOT TIER | -----------> | WARM TIER | -----------> | COLD TIER | -| (DO SQLite) | | (R2) | | (R2 Archive) | -+-------------------+ +-------------------+ +-------------------+ -| Fast access | | Large files | | Infrequent access | -| < 1MB files | | < 100MB files | | Historical data | -| Single-threaded | | Object storage | | Compressed | -+-------------------+ +-------------------+ +-------------------+ - ^ | | - | promote on access | promote on access | - +----------------------------------+----------------------------------+ -``` - -### 2.2 Implementation Status by Rewrite - -| Rewrite | Hot (SQLite) | Warm (R2) | Cold (Archive) | Auto-Tier | Promotion | -|---------|--------------|-----------|----------------|-----------|-----------| -| **fsx** | Implemented | Implemented | Planned | By size | On access | -| **gitx** | Implemented | Pack files | Parquet | By access pattern | LRU | -| **kafka** | Implemented | Not yet | Not yet | N/A | N/A | -| **nats** | Implemented | Not yet | Not yet | N/A | N/A | -| **supabase** | Planned | Planned | Planned | TBD | TBD | - -### 2.3 fsx TieredFS Implementation - -```typescript -// rewrites/fsx/src/storage/tiered.ts - -class TieredFS { - private selectTier(size: number): 'hot' | 'warm' | 'cold' { - if (size <= hotMaxSize) return 'hot' // < 1MB -> SQLite - if (size <= warmMaxSize) return 'warm' // < 100MB -> R2 - return 'cold' // Archive - } - - async readFile(path: string): Promise<{ data: Uint8Array; tier: string }> { - // Check metadata for tier location - const metadata = await this.getMetadata(path) - - // Read from appropriate tier - if (metadata.tier === 'hot') { - return { data: await this.readFromHot(path), tier: 'hot' } - } - - if (metadata.tier === 'warm') { - const data = await this.warm.get(path) - - // Promote on access if under threshold - if (promotionPolicy === 'on-access' && data.length <= hotMaxSize) { - await this.promote(path, data, 'warm', 'hot') - } - - return { data, tier: 'warm' } - } - // ... cold tier handling - } -} -``` - -### 2.4 gitx Tiered Storage (Advanced) - -gitx has the most sophisticated tiered storage with CDC pipelines: - -``` -+-------------------+ +-------------------+ +-------------------+ -| Hot Objects | | R2 Packfiles | | Parquet Archive | -| (SQLite DO) | | (Git format) | | (Columnar) | -+-------------------+ +-------------------+ +-------------------+ - | | | - v v v -+-------------------+ +-------------------+ +-------------------+ -| object_index | | multi-pack idx | | partition files | -| tier: 'hot' | | pack_id, offset | | by date range | -+-------------------+ +-------------------+ +-------------------+ -``` - -**gitx Schema (rewrites/gitx/src/durable-object/schema.ts)**: -- `objects` - Main Git object storage -- `object_index` - Location index across tiers (tier, pack_id, offset) -- `hot_objects` - Frequently accessed cache with LRU -- `wal` - Write-ahead log for durability -- `refs` - Git references - ---- - -## 3. Client/Server Split Analysis - -### 3.1 Pattern Overview - -``` -+-------------------+ HTTP/RPC +-------------------+ -| Client SDK | --------------> | Durable Object | -| (npm package) | | (Worker) | -+-------------------+ +-------------------+ -| - Type-safe API | | - Hono routes | -| - DO stub wrap | | - SQLite storage | -| - Streaming | | - Business logic | -+-------------------+ +-------------------+ -``` - -### 3.2 Client Implementation Comparison - -#### fsx - FSx Client Class - -```typescript -// rewrites/fsx/src/core/fsx.ts -export class FSx { - private stub: DurableObjectStub - - constructor(binding: DurableObjectNamespace | DurableObjectStub) { - if ('idFromName' in binding) { - const id = binding.idFromName('global') - this.stub = binding.get(id) - } else { - this.stub = binding - } - } - - private async request(method: string, params: Record): Promise { - const response = await this.stub.fetch('http://fsx.do/rpc', { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ method, params }), - }) - // ... - } - - async readFile(path: string, encoding?: BufferEncoding): Promise { - return this.request<{ data: string }>('readFile', { path, encoding }) - } -} -``` - -#### kafka - KafkaClient (HTTP-based) - -```typescript -// rewrites/kafka/src/client/client.ts -export class KafkaClient { - private config: KafkaClientConfig - - constructor(config: KafkaClientConfig) { - this.config = { - baseUrl: config.baseUrl.replace(/\/$/, ''), - // ... - } - } - - producer(): KafkaProducerClient { /* ... */ } - consumer(options: ConsumerOptions): KafkaConsumerClient { /* ... */ } - admin(): KafkaAdminClient { /* ... */ } -} -``` - -### 3.3 Inconsistencies Identified - -| Aspect | fsx | kafka | gitx | nats | -|--------|-----|-------|------|------| -| Client Location | `src/core/fsx.ts` | `src/client/` | No SDK client | `src/rpc/` | -| Constructor Input | DO binding | HTTP config | N/A | RPC endpoint | -| RPC Format | JSON `{method, params}` | REST endpoints | N/A | JSON-RPC 2.0 | -| Streaming | Yes (`/stream/*`) | No | No | No | - -**Recommendation**: Standardize client SDK pattern: - -```typescript -// Proposed standard pattern for all rewrites -export interface ServiceClientConfig { - // Option 1: Internal (within workers.do) - binding?: DurableObjectNamespace - - // Option 2: External (HTTP client) - baseUrl?: string - apiKey?: string - - // Common - timeout?: number -} - -export function createClient( - serviceName: string, - config: ServiceClientConfig -): T { - if (config.binding) { - return new InternalClient(config.binding) as T - } - return new HTTPClient(config.baseUrl, config.apiKey) as T -} -``` - ---- - -## 4. MCP Integration Analysis - -### 4.1 MCP Tool Pattern - -All rewrites that implement MCP follow the same structure: - -```typescript -interface McpTool { - name: string - description: string - inputSchema: { - type: 'object' - properties: Record - required?: string[] - } - handler?: (params: Record) => Promise -} - -interface McpToolResult { - content: Array<{ type: 'text' | 'image'; text?: string; data?: string }> - isError?: boolean -} -``` - -### 4.2 Tool Coverage by Rewrite - -#### fsx MCP Tools (12 tools) -``` -fs_read, fs_write, fs_append, fs_delete, fs_move, fs_copy, -fs_list, fs_mkdir, fs_stat, fs_exists, fs_search, fs_tree -``` - -#### gitx MCP Tools (18 tools) -``` -git_status, git_log, git_diff, git_commit, git_branch, git_checkout, -git_push, git_pull, git_clone, git_init, git_add, git_reset, -git_merge, git_rebase, git_stash, git_tag, git_remote, git_fetch -``` - -#### nats MCP Tools (3+ tools) -``` -nats_publish, nats_consumer, nats_stream -``` - -### 4.3 MCP Architecture Diagram - -``` -+-------------------+ -| AI Agent | -| (Claude, etc.) | -+-------------------+ - | - | MCP Protocol - v -+-------------------+ -| MCP Server | -| (tool registry) | -+-------------------+ - | - | invokeTool(name, params) - v -+-------------------+ -| Tool Handler | -| (validateInput, | -| execute) | -+-------------------+ - | - v -+-------------------+ -| Repository/ | -| Service Context | -+-------------------+ -``` - -### 4.4 gitx MCP Implementation Quality - -The gitx MCP implementation is exemplary: - -1. **Security**: Comprehensive input validation - ```typescript - function validatePath(path: string): string { - if (path.includes('..') || path.startsWith('/') || /[<>|&;$`]/.test(path)) { - throw new Error('Invalid path: contains forbidden characters') - } - return path - } - ``` - -2. **Context Management**: Global repository context - ```typescript - let globalRepositoryContext: RepositoryContext | null = null - - export function setRepositoryContext(ctx: RepositoryContext | null): void { - globalRepositoryContext = ctx - } - ``` - -3. **Comprehensive Tools**: Full git workflow coverage - ---- - -## 5. Cross-Rewrite Dependencies - -### 5.1 Dependency Graph - -``` - +-------------------+ - | fsx | - | (File System DO) | - +-------------------+ - ^ - | - | imports CAS, compression, git-object - | - +-------------------+ - | gitx | - | (Git DO) | - +-------------------+ - ^ - | - | (potential future dependency) - | - +-------------------+ - | supabase | - | (Database DO) | - +-------------------+ -``` - -### 5.2 gitx -> fsx Coupling Analysis - -**Location**: `rewrites/gitx/src/storage/fsx-adapter.ts` - -```typescript -// Current (PROBLEMATIC) - Direct source imports -import { putObject as fsxPutObject } from '../../../fsx/src/cas/put-object' -import { getObject as fsxGetObject } from '../../../fsx/src/cas/get-object' -import { hashToPath } from '../../../fsx/src/cas/path-mapping' -import { sha1 } from '../../../fsx/src/cas/hash' -import { createGitObject, parseGitObject } from '../../../fsx/src/cas/git-object' -import { compress, decompress } from '../../../fsx/src/cas/compression' -``` - -**Issues**: -1. Path-based imports (`../../../`) are fragile -2. No npm package boundary -3. Changes in fsx can break gitx silently -4. Cannot version dependencies independently - -### 5.3 Recommended Dependency Pattern - -``` -Option A: Package Dependencies ------------------------------- -packages/ - fsx/ # npm: fsx.do - src/cas/ - index.ts # Public API exports - gitx/ # npm: gitx.do - package.json # "dependencies": { "fsx.do": "^1.0.0" } - -gitx imports: -import { putObject, sha1, compress } from 'fsx.do/cas' - - -Option B: Shared Primitives ---------------------------- -primitives/ - cas/ # npm: @workers.do/cas - put-object.ts - get-object.ts - hash.ts - compression.ts - -Both fsx and gitx import from: -import { sha1, compress } from '@workers.do/cas' -``` - -**Recommendation**: Option B (Shared Primitives) is cleaner for shared functionality like CAS operations. - ---- - -## 6. Scalability Analysis - -### 6.1 DO Scalability Characteristics - -| Factor | Analysis | Score | -|--------|----------|-------| -| **Horizontal Scale** | Each DO instance is independent; millions can run in parallel | Excellent | -| **Per-Instance Memory** | Limited to DO memory limits (128MB); SQLite helps | Good | -| **Storage Limits** | 10GB per DO (SQLite); unlimited via R2 | Excellent | -| **Global Distribution** | DOs automatically locate near users | Excellent | -| **Cost at Scale** | Pay per request/duration; efficient for sparse access | Excellent | - -### 6.2 Bottleneck Analysis - -``` -Potential Bottlenecks: -+------------------------------------------------------------------+ -| | -| 1. Hot Path Serialization (RPC JSON encoding) | -| - Impact: High for small frequent requests | -| - Mitigation: Batch operations, streaming | -| | -| 2. Single-Threaded DO Execution | -| - Impact: High for CPU-intensive operations | -| - Mitigation: Offload to Workers, use hibernation | -| | -| 3. SQLite Row Limits | -| - Impact: Medium for large datasets | -| - Mitigation: Tiered storage, pagination | -| | -| 4. R2 Request Latency | -| - Impact: Medium for cold reads | -| - Mitigation: Hot cache, prefetch | -| | -+------------------------------------------------------------------+ -``` - -### 6.3 Scale Projections - -| Scenario | DOs Required | Feasibility | -|----------|--------------|-------------| -| 1M AI agents, each with own filesystem | 1M FileSystemDO instances | Feasible | -| 1M AI agents, each with own git repo | 1M GitDO instances | Feasible | -| 100K topics, 10 partitions each | 1M TopicPartitionDO instances | Feasible | -| Global NATS mesh | Coordinator + per-region DOs | Feasible | - ---- - -## 7. Consistency Analysis - -### 7.1 Pattern Consistency Matrix - -| Pattern | fsx | gitx | kafka | nats | Consistent? | -|---------|-----|------|-------|------|-------------| -| DO extends DurableObject | Yes | Yes | Yes | Yes | Yes | -| Hono routing | Yes | Yes | Yes | Yes | Yes | -| Lazy init with `ensureInitialized` | Yes | Yes | Yes | Yes | Yes | -| SQLite for metadata | Yes | Yes | Yes | Yes | Yes | -| R2 for large data | Yes | Yes | No | No | Partial | -| MCP tools | Yes | Yes | No | Yes | Partial | -| Client SDK | Yes | No | Yes | Yes | Partial | -| Tiered storage | Yes | Yes | No | No | Partial | -| JSON-RPC format | Custom | Custom | REST | JSON-RPC 2.0 | No | - -### 7.2 Naming Convention Consistency - -| Rewrite | DO Class Name | Table Names | Route Prefix | -|---------|---------------|-------------|--------------| -| fsx | `FileSystemDO` | files, blobs | `/rpc`, `/stream/*` | -| gitx | (implicit) | objects, refs, hot_objects, wal | (various) | -| kafka | `TopicPartitionDO` | messages, watermarks | `/append`, `/read`, `/offsets` | -| nats | `NatsCoordinator` | consumers | `/` (RPC) | - -**Recommendation**: Standardize on: -- DO naming: `{Service}DO` (e.g., `FileSystemDO`, `GitDO`, `KafkaDO`, `NatsDO`) -- Route prefix: `/rpc` for main operations, `/stream/*` for streaming -- RPC format: JSON-RPC 2.0 standard - ---- - -## 8. Recommendations - -### 8.1 High Priority - -1. **Extract Base DO Class** - - Create `objects/do/base.ts` with common initialization, routing patterns - - All rewrites extend this base class - -2. **Standardize RPC Format** - - Adopt JSON-RPC 2.0 across all rewrites - - Create shared middleware in `middleware/rpc/` - -3. **Fix gitx -> fsx Coupling** - - Extract CAS operations to `primitives/cas/` or `packages/cas/` - - Both fsx and gitx import from shared package - -### 8.2 Medium Priority - -4. **Standardize Client SDKs** - - Create `sdks/{service}.do/` for each rewrite - - Consistent pattern: `new ServiceClient(binding | config)` - -5. **Add Tiered Storage to kafka/nats** - - kafka: Archive old messages to R2 - - nats: Archive old streams to R2 - -6. **Expand MCP Coverage** - - Add MCP tools to kafka (`kafka_produce`, `kafka_consume`, `kafka_admin`) - - Complete supabase MCP tools - -### 8.3 Low Priority - -7. **Add Health/Metrics Endpoints** - - Standard `/health` and `/metrics` routes - - Export to Cloudflare Analytics - -8. **Add OpenTelemetry Tracing** - - Trace requests across DO hops - - Integration with Cloudflare tracing - ---- - -## 9. Appendix: Quick Reference - -### 9.1 Key File Locations - -| Rewrite | DO Implementation | Client SDK | MCP Tools | -|---------|-------------------|------------|-----------| -| fsx | `src/durable-object/index.ts` | `src/core/fsx.ts` | `src/mcp/index.ts` | -| gitx | `src/durable-object/schema.ts` | N/A | `src/mcp/tools.ts` | -| kafka | `src/durable-objects/topic-partition.ts` | `src/client/client.ts` | N/A | -| nats | `src/durable-objects/nats-coordinator.ts` | `src/rpc/rpc-client.ts` | `src/mcp/tools/` | -| supabase | (planned) | (planned) | (planned) | - -### 9.2 SQLite Schema Quick Reference - -**fsx**: -```sql -CREATE TABLE files (id, path UNIQUE, name, parent_id, type, mode, ...); -CREATE TABLE blobs (id, data BLOB, size, tier, created_at); -``` - -**gitx**: -```sql -CREATE TABLE objects (sha PRIMARY KEY, type, size, data BLOB, created_at); -CREATE TABLE object_index (sha PRIMARY KEY, tier, pack_id, offset, ...); -CREATE TABLE hot_objects (sha PRIMARY KEY, type, data BLOB, accessed_at, ...); -CREATE TABLE wal (id AUTO, operation, payload BLOB, flushed); -CREATE TABLE refs (name PRIMARY KEY, target, type, updated_at); -``` - -**kafka**: -```sql -CREATE TABLE messages (offset AUTO PRIMARY KEY, key, value, headers, timestamp, ...); -CREATE TABLE watermarks (partition PRIMARY KEY, high_watermark, log_start_offset, ...); -``` - -**nats**: -```sql -CREATE TABLE consumers (stream_name, name, config, durable, created_at, ...); -``` - ---- - -## 10. Conclusion - -The workers.do rewrites platform demonstrates a solid architectural foundation with Durable Objects providing the per-instance isolation and SQLite/R2 providing the storage tiers. The patterns established in fsx and gitx should be formalized and applied consistently across all rewrites. - -Key strengths: -- Scalable DO-based architecture -- Sophisticated tiered storage in mature rewrites -- Comprehensive MCP integration for AI-native access - -Key areas for improvement: -- Cross-rewrite dependency management -- Client SDK standardization -- RPC format consistency - -With the recommended improvements, the platform will be well-positioned to support millions of concurrent AI agent instances, each with their own isolated infrastructure services. diff --git a/rewrites/docs/reviews/general-code-review.md b/rewrites/docs/reviews/general-code-review.md deleted file mode 100644 index 63e4a748..00000000 --- a/rewrites/docs/reviews/general-code-review.md +++ /dev/null @@ -1,418 +0,0 @@ -# General Code Review: rewrites/ Directory - -**Date:** 2026-01-07 -**Reviewer:** Claude (Opus 4.5) -**Scope:** Code quality and consistency across rewrites/ - ---- - -## Executive Summary - -The `rewrites/` directory contains reimplementations of popular services on Cloudflare Durable Objects. After reviewing fsx (gold standard), supabase, segment, sentry, posthog, launchdarkly, customerio, algolia, inngest, orb, kafka, and mongo, this review identifies a **significant maturity gap** between rewrites. - -### Key Findings - -| Category | Rating | Notes | -|----------|--------|-------| -| Code Quality (implemented) | **B+** | fsx, kafka, gitx, mongo have solid implementations | -| Documentation | **A** | Excellent READMEs with clear vision and API docs | -| Test Coverage | **B** | Good coverage where tests exist; many packages have none | -| Error Handling | **B+** | POSIX-style errors in fsx are exemplary | -| Code Organization | **A-** | Consistent structure when implemented | -| Implementation Progress | **C** | Most rewrites are documentation-only | - ---- - -## 1. Maturity Levels Across Rewrites - -### Tier 1: Production-Ready -These rewrites have substantial implementation with tests: - -| Package | Files | Tests | Notes | -|---------|-------|-------|-------| -| **fsx** | 40+ src files | 20+ test files | Gold standard, comprehensive | -| **gitx** | 50+ src files | 30+ test files | Wire protocol, pack files, MCP | -| **kafka** | 30+ src files | 10+ test files | Full producer/consumer/admin API | -| **mongo** | 50+ files (incl studio) | 15+ test files | Includes React-based studio app | - -### Tier 2: Initial Implementation -Have `src/` directory but minimal or empty: - -| Package | Status | -|---------|--------| -| segment | Empty src/ | -| customerio | Empty src/ | -| inngest | Empty src/ | -| orb | Empty src/ | - -### Tier 3: Documentation Only -No source code, only README.md and AGENTS.md: - -| Package | Status | -|---------|--------| -| supabase | Excellent README, no src | -| sentry | Good README, no src | -| posthog | Good README, no src | -| launchdarkly | Good README, no src | -| algolia | Good README, no src | - ---- - -## 2. Code Quality Analysis - -### 2.1 fsx (Gold Standard) - -**Strengths:** - -1. **POSIX-compliant error hierarchy** - ```typescript - // Excellent pattern - typed errors with errno codes - export class FSError extends Error { - code: string - errno: number - syscall?: string - path?: string - } - - export class ENOENT extends FSError { - constructor(syscall?: string, path?: string) { - super('ENOENT', -2, 'no such file or directory', syscall, path) - } - } - ``` - -2. **Comprehensive type definitions** - - `Stats`, `Dirent`, `FileHandle` classes with proper methods - - Option interfaces for every operation - - Clear separation of internal vs public types - -3. **Well-organized module structure** - ``` - src/ - core/ # Pure business logic, types, errors - cas/ # Content-addressable storage (git-like) - storage/ # SQLite + R2 tiered storage - durable-object/ # DO with Hono routing - mcp/ # AI tool definitions - grep/, glob/, find/, watch/ # Unix utilities - ``` - -4. **Thorough test coverage** - - Unit tests for every module - - Edge cases tested (large files, empty inputs, special bytes) - - Tests verify hash correctness against known values - -**Areas for Improvement:** - -1. `durable-object/index.ts` uses `any` type in `hydrateStats`: - ```typescript - private hydrateStats(stats: any): Stats { // Should be typed - ``` - -2. Error handling in DO could use the error classes instead of `Object.assign`: - ```typescript - // Current (inconsistent with core/errors.ts) - throw Object.assign(new Error('no such file or directory'), { code: 'ENOENT', path }) - - // Should be - throw new ENOENT('readFile', path) - ``` - -3. `watch()` method is a stub: - ```typescript - // Note: This is a simplified implementation - // Full implementation would use WebSocket or Server-Sent Events - ``` - -### 2.2 kafka - -**Strengths:** - -1. **Clean type definitions** with JSDoc comments - ```typescript - /** - * A record to be sent to a topic - */ - export interface ProducerRecord { - topic: string - key?: K - value: V - partition?: number - headers?: Record - timestamp?: number - } - ``` - -2. **Well-documented integration patterns** (MongoDB CDC, R2 Event Bridge) - -3. **Consistent API** following Kafka conventions - -**Areas for Improvement:** - -1. Type tests don't add much value: - ```typescript - it('ProducerRecord has required fields', () => { - const record: ProducerRecord = { topic: 'test-topic', value: {} } - expect(record.topic).toBe('test-topic') // Just validates TS compilation - }) - ``` - -### 2.3 Documentation Quality - -All READMEs follow an excellent template: - -1. **The Problem** - Why this rewrite matters for AI agents -2. **The Vision** - Code example showing the end state -3. **Architecture** - ASCII diagram of DO structure -4. **Quick Start** - Installation and basic usage -5. **API Overview** - Comprehensive method documentation -6. **The Rewrites Ecosystem** - How it fits with other packages - -This consistency is commendable and should be maintained. - ---- - -## 3. Pattern Consistency - -### 3.1 Consistent Patterns (Good) - -| Pattern | Location | Notes | -|---------|----------|-------| -| Durable Object + SQLite | All | Core architecture choice | -| Tiered storage (hot/warm/cold) | fsx, supabase docs | SQLite -> R2 -> Archive | -| MCP tools for AI access | All docs | `{package}.do/mcp` exports | -| Hono routing in DO | fsx, kafka | Single `/rpc` endpoint pattern | -| Package exports structure | fsx | `/core`, `/do`, `/mcp`, `/storage` | - -### 3.2 Inconsistent Patterns (Issues) - -1. **Package naming** - - Most: `{name}.do` (fsx.do, supabase.do) - - Exception: `@dotdo/sentry` (uses scoped npm name) - - **Recommendation:** Standardize on one pattern - -2. **AGENTS.md format varies** - - fsx: Brief, focused on beads workflow - - sentry: Includes architecture overview - - segment: Comprehensive with TDD workflow, epics table - - **Recommendation:** Create template combining best of all three - -3. **Test file naming** - - fsx: `src/cas/hash.test.ts` (co-located) - - gitx: `test/wire/pkt-line.test.ts` (separate test/) - - **Recommendation:** Co-locate tests with source for easier navigation - ---- - -## 4. Test Coverage Analysis - -### 4.1 Coverage by Package - -| Package | Test Files | Coverage Areas | -|---------|------------|----------------| -| fsx | 20+ | Core, CAS, grep, glob, find, watch, errors | -| gitx | 30+ | Wire protocol, pack format, MCP, DO | -| kafka | 10+ | Types, DO, schema, index | -| mongo | 15+ | Stores, utils, pipeline, E2E | -| supabase | 0 | - | -| segment | 0 | - | -| sentry | 0 | - | -| posthog | 0 | - | - -### 4.2 Test Quality Assessment - -**fsx tests (Excellent)** -```typescript -describe('SHA-256 Hash Computation', () => { - it('should hash empty Uint8Array to known SHA-256 value', async () => { - const data = new Uint8Array([]) - const hash = await sha256(data) - expect(hash).toBe('e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855') - }) - - it('should hash large data (>1MB) with SHA-256', async () => { - const size = 1.5 * 1024 * 1024 - const data = new Uint8Array(size) - // ...tests performance with realistic sizes - }) -}) -``` - -**fsx error tests (Excellent)** -- Tests every POSIX error class -- Verifies message formatting with/without optional params -- Validates inheritance chain - ---- - -## 5. Error Handling Patterns - -### 5.1 Best Practice (fsx/core/errors.ts) - -```typescript -export class FSError extends Error { - code: string - errno: number - syscall?: string - path?: string - dest?: string - - constructor(code: string, errno: number, message: string, syscall?: string, path?: string, dest?: string) { - const fullMessage = `${code}: ${message}${syscall ? `, ${syscall}` : ''}${path ? ` '${path}'` : ''}` - super(fullMessage) - this.name = 'FSError' - this.code = code - this.errno = errno - // ... - } -} -``` - -This pattern: -- Uses typed error classes (not just code strings) -- Includes standard POSIX errno values -- Provides informative error messages -- Supports `instanceof` checks - -### 5.2 Anti-Pattern (fsx/durable-object/index.ts) - -```typescript -// DON'T: Ad-hoc error objects -throw Object.assign(new Error('no such file or directory'), { code: 'ENOENT', path }) - -// DO: Use the error classes -throw new ENOENT('readFile', path) -``` - -**Recommendation:** Refactor DO to use error classes from core/errors.ts - ---- - -## 6. Code Organization - -### 6.1 Recommended Structure (from fsx) - -``` -{rewrite}/ - src/ - core/ # Pure business logic, types, errors - durable-object/ # DO class with Hono routing - storage/ # SQLite + R2 integration - mcp/ # AI tool definitions - client/ # HTTP client SDK (optional) - .beads/ # Issue tracking - test/ # Integration/E2E tests (or co-locate) - README.md # User documentation - AGENTS.md # AI agent instructions - package.json - tsconfig.json - vitest.config.ts -``` - -### 6.2 Current State - -| Package | Has src/ | Structure | -|---------|----------|-----------| -| fsx | Yes | Follows pattern | -| kafka | Yes | Follows pattern | -| gitx | Yes | Follows pattern | -| mongo | Yes | Has studio/ app too | -| supabase | No | Documentation only | -| segment | Yes (empty) | Placeholder | -| sentry | No | Documentation only | - ---- - -## 7. Recommendations - -### 7.1 Immediate Actions - -1. **Standardize error handling in DOs** - - Update `fsx/durable-object/index.ts` to use `ENOENT`, `EISDIR`, etc. - - Create shared error utilities in `@dotdo/common` - -2. **Create AGENTS.md template** - - Combine beads workflow from fsx - - Add TDD workflow from segment - - Include architecture section from sentry - -3. **Fix type safety issues** - - Remove `any` usage in `hydrateStats` - - Add proper return types to all public methods - -### 7.2 Short-term Improvements - -1. **Add integration tests to fsx** - - Test full DO lifecycle with miniflare - - Test streaming read/write with large files - - Test watch functionality (when implemented) - -2. **Implement stub methods** - - Complete `watch()` with WebSocket/SSE - - Add `createReadStream`/`createWriteStream` to FileHandle - -3. **Standardize package naming** - - Choose `{name}.do` or `@dotdo/{name}` consistently - -### 7.3 Long-term Strategy - -1. **Prioritize implementation** - - supabase and sentry have excellent docs, should be next - - Follow TDD workflow defined in segment/AGENTS.md - -2. **Create shared packages** - - `@dotdo/do-base` - Base Durable Object class with Hono - - `@dotdo/errors` - Shared error classes - - `@dotdo/mcp` - MCP tool utilities - -3. **Add code coverage reporting** - - Set up vitest coverage - - Add to CI/CD pipeline - - Require coverage thresholds for PRs - ---- - -## 8. Files Reviewed - -### Primary Review (In-depth) -- `/Users/nathanclevenger/projects/workers/rewrites/CLAUDE.md` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/AGENTS.md` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/package.json` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/core/index.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/core/fsx.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/core/errors.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/core/types.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/durable-object/index.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/cas/hash.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/cas/hash.test.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/fsx/src/core/errors.test.ts` - -### Secondary Review (Structure + Docs) -- `/Users/nathanclevenger/projects/workers/rewrites/supabase/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/supabase/AGENTS.md` -- `/Users/nathanclevenger/projects/workers/rewrites/segment/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/segment/AGENTS.md` -- `/Users/nathanclevenger/projects/workers/rewrites/sentry/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/sentry/AGENTS.md` -- `/Users/nathanclevenger/projects/workers/rewrites/posthog/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/kafka/README.md` -- `/Users/nathanclevenger/projects/workers/rewrites/kafka/src/types/records.ts` -- `/Users/nathanclevenger/projects/workers/rewrites/kafka/src/types/records.test.ts` - ---- - -## 9. Summary - -The rewrites/ directory shows **strong architectural vision** with excellent documentation, but has a **significant implementation gap**. The fsx package demonstrates what "done" looks like - it should be used as the template for all other rewrites. - -**Next steps:** -1. Use TDD workflow from segment/AGENTS.md -2. Prioritize supabase and sentry (have best docs) -3. Extract common patterns to shared packages -4. Add coverage requirements to CI - -The documentation-first approach has set up each rewrite for success. Now it's time to execute the implementations. diff --git a/rewrites/docs/reviews/product-roadmap-review.md b/rewrites/docs/reviews/product-roadmap-review.md deleted file mode 100644 index 9d248024..00000000 --- a/rewrites/docs/reviews/product-roadmap-review.md +++ /dev/null @@ -1,381 +0,0 @@ -# Product & Roadmap Review: workers.do Rewrites Platform - -**Review Date:** 2026-01-07 -**Reviewer:** Product/Vision Analysis - ---- - -## Executive Summary - -The workers.do platform represents an ambitious vision to reimagine enterprise SaaS infrastructure on Cloudflare's edge platform. The "rewrites" directory contains 70+ packages aiming to replace billion-dollar enterprise software with edge-native, AI-first alternatives. While the vision is compelling and market timing is optimal, execution is currently fragmented across too many initiatives with insufficient depth in core foundational services. - -**Overall Assessment:** Strong vision, solid architectural patterns, but needs focus. The platform is trying to boil the ocean when it should be proving the model with 3-4 deeply implemented showcases. - ---- - -## 1. Vision Alignment Analysis - -### Main Vision (CLAUDE.md) -The workers.do vision centers on: -- **Autonomous Startups** - Business-as-Code with AI agents -- **Named Agents** (Priya, Ralph, Tom, etc.) as tagged template functions -- **CapnWeb Pipelining** for multi-step agent workflows -- **Platform Services** - Identity, Payments, AI Gateway built-in - -### Rewrites Vision (VISION.md) -The rewrites vision focuses on: -- **Enterprise SaaS replacement** at 1/100th the cost -- **One-click deploy** via `npx create-dotdo salesforce` -- **AI-native operations** from day one -- **Edge-native global deployment** - -### Alignment Score: 7/10 - -**Alignments:** -- Both emphasize AI-native design -- Both target startup founders as the hero -- Both leverage Cloudflare edge infrastructure -- Both use Durable Objects as the core primitive - -**Gaps:** -- Main vision focuses on named agents (Priya, Ralph) but rewrites barely mention them -- CapnWeb pipelining is not visible in any rewrite implementation -- The "Supabase for every agent" story in supabase.do README doesn't connect to the main agent architecture -- No visible integration between agents/ and rewrites/ directories - -**Recommendation:** Create explicit integration points. Show how `tom.db` maps to a supabase.do instance. Demonstrate `ralph`'s filesystem is an fsx.do instance. - ---- - -## 2. Product Completeness Assessment - -### Tier 1: Production-Ready (Feature Complete with Tests) - -| Package | Files | Tests | Status | Notes | -|---------|-------|-------|--------|-------| -| **fsx** | 78 | 30+ | 80% | Core FS ops implemented, TDD in progress | -| **gitx** | 61 | 20+ | 70% | Pack format, wire protocol, MCP tools done | -| **mongo** | 100+ | 15+ | 75% | Full MongoDB API, vector search, MCP | -| **kafka** | 33 | 8 | 60% | Consumer groups, schema registry started | - -### Tier 2: Substantial Scaffold (Has Code, Needs Work) - -| Package | Files | Tests | Status | Notes | -|---------|-------|-------|--------|-------| -| **nats** | 34 | 18 | 50% | JetStream types, coordinator DO started | -| **redis** | 14 | - | 40% | Basic structure only | -| **neo4j** | 14 | - | 35% | Graph DB types defined | -| **convex** | 13 | - | 30% | React bindings, real-time | -| **firebase** | 14 | - | 30% | Auth + RTDB scaffold | - -### Tier 3: README Only (Vision Documented) - -| Package | Status | Market Cap Target | -|---------|--------|-------------------| -| **supabase** | README only | N/A (Supabase) | -| **salesforce** | README only | $200B | -| **hubspot** | README only | $30B | -| **servicenow** | README only | $150B | -| **zendesk** | README only | $10B | -| **workday** | README only | $60B | -| 60+ others | Empty/README | Various | - -### Summary Statistics - -``` -Total rewrites directories: 73 -With package.json: 10 -With substantial src/: 6 -With test coverage: 4 -Production-ready: 0 -``` - -**Verdict:** The project is in early-mid development. Core infrastructure (fsx, gitx, mongo) is progressing well, but the enterprise SaaS rewrites are aspirational. - ---- - -## 3. Beads Issues Roadmap Analysis - -### Issue Statistics by Repository - -| Repo | Total | Open | Closed | Blocked | Ready | Avg Lead Time | -|------|-------|------|--------|---------|-------|---------------| -| workers (parent) | 2,324 | 1,606 | 709 | 375 | 1,233 | 13.5 hrs | -| fsx | 294 | 236 | 58 | 136 | 100 | 1.0 hrs | -| gitx | 52 | 52 | 0 | 33 | 19 | 0 hrs | -| nats | 37 | 37 | 0 | 34 | 3 | 0 hrs | -| segment | - | - | - | - | - | - | -| supabase | 0 | 0 | 0 | 0 | 0 | - | - -### Key Themes from Ready Issues - -1. **TDD Cycles Dominate** - Most ready issues follow [RED]/[GREEN]/[REFACTOR] pattern -2. **MCP Migration** - Multiple packages migrating to standard DO MCP tools -3. **Core Foundation** - Types, constants, errors, path utils still in progress for fsx -4. **Glyphs Visual DSL** - Novel visual programming initiative (packages/glyphs) -5. **Event-Sourced Lakehouse** - Architecture epic for analytics layer - -### Implied Roadmap (Q1 2026) - -**Phase 1 - Foundation (Current)** -- Complete fsx core operations (file, directory, metadata) -- Finish gitx MCP migration -- Implement nats JetStream basics - -**Phase 2 - Integration** -- Connect fsx + gitx (git operations use virtual filesystem) -- Add CAS (Content-Addressable Storage) layer -- Implement DO base class standardization - -**Phase 3 - Database Layer** -- Port mongo.do production features to supabase.do -- Implement tiered storage across all data packages -- Add vector search to all applicable packages - -**Phase 4 - Enterprise Showcase** -- Pick ONE enterprise rewrite (recommend: zendesk or intercom) -- Build complete vertical with agents integration -- Create `npx create-dotdo` generator - ---- - -## 4. Market Positioning Analysis - -### Competitive Landscape - -| Competitor | What They Do | workers.do Advantage | -|------------|--------------|---------------------| -| **Supabase** | Postgres + Auth + Storage | Edge-native, per-agent isolation | -| **PlanetScale** | Serverless MySQL | SQLite simplicity, no connection limits | -| **Neon** | Serverless Postgres | True edge (DO), not just branching | -| **Convex** | Reactive backend | Same vision, but Cloudflare platform | -| **Firebase** | BaaS for mobile | AI-native, not bolted on | -| **Turso** | Edge SQLite | Integrated with DO ecosystem | - -### Unique Value Propositions - -1. **Per-Agent Databases** - No one else offers this model -2. **MCP-Native** - AI tool calling built into every service -3. **Edge-First** - True edge compute, not just edge CDN -4. **Unified Platform** - One account for DB, Git, Messaging, Storage -5. **Free Tier Optimization** - Designed for Cloudflare's free limits - -### Market Timing: Excellent - -- AI agents need isolated state (2025-2026 emerging need) -- Enterprise SaaS fatigue at all-time high -- Cloudflare continuously improving DO/R2/Vectorize -- MCP becoming industry standard for AI tool calling - -### Positioning Recommendation - -**Current Position:** "Enterprise SaaS rewrites for startups" -**Recommended Position:** "AI-native infrastructure where every agent has its own backend" - -The "every agent has its own Supabase" story is more compelling and differentiated than "Salesforce but cheaper." - ---- - -## 5. Target User Analysis - -### Primary Persona: "The AI-First Startup Founder" - -**Demographics:** -- Solo founder or 2-3 person team -- Building AI-powered product -- Technical enough to deploy to Cloudflare -- Budget-conscious but values quality - -**Jobs to Be Done:** -1. Give my AI agents persistent memory -2. Let agents collaborate without stepping on each other -3. Build without enterprise SaaS overhead -4. Scale from 0 to millions without architecture changes - -**Current Pain Points:** -- Supabase/Neon requires manual multi-tenancy for agents -- Firebase feels legacy, not AI-native -- Convex is close but not on Cloudflare -- Vector databases don't integrate with traditional DBs - -### Is the Story Compelling? 8/10 - -The "AI agent with its own database" story IS compelling because: - -1. **Real Pain Point** - AI agents DO need isolated state -2. **No Good Solution Exists** - No one optimizes for agent-per-DB -3. **Economic Model Works** - DO pricing allows millions of instances -4. **Technical Moat** - Hard to replicate without Cloudflare - -**But** the story needs clearer articulation: -- Show agents.do actually using these databases -- Demonstrate real workflows (Tom reviewing code stored in gitx) -- Prove the economic model with pricing calculator - ---- - -## 6. Feature Gap Analysis - -### Critical for MVP (P0) - -| Feature | Current Status | Gap | -|---------|---------------|-----| -| **Authentication** | Not implemented | Need agent identity + human OAuth | -| **CLI Generator** | Not started | `npx create-dotdo