feat(webhooks): add signed event delivery with retry handling

AUTODEV_REPORT.md

@@ -0,0 +1,61 @@
+# AUTODEV Report
+
+## Issue
+- Implemented: [Issue #77 - Webhook Event System](https://github.com/rohitdash08/FinMind/issues/77)
+
+## Changed Files
+- `README.md`
+- `packages/backend/app/__init__.py`
+- `packages/backend/app/db/schema.sql`
+- `packages/backend/app/models.py`
+- `packages/backend/app/routes/__init__.py`
+- `packages/backend/app/routes/auth.py`
+- `packages/backend/app/routes/bills.py`
+- `packages/backend/app/routes/expenses.py`
+- `packages/backend/app/routes/webhooks.py` (new)
+- `packages/backend/app/services/webhooks.py` (new)
+- `packages/backend/tests/test_webhooks.py` (new)
+
+## What Was Implemented
+- Added webhook domain models:
+  - `WebhookTarget`
+  - `WebhookDelivery`
+  - `WebhookEvent` enum (9 documented event types)
+- Added webhook service with:
+  - HMAC SHA-256 signed delivery (`X-FinMind-Signature` over `<timestamp>.<raw_json_payload>`)
+  - Delivery logging and status tracking
+  - Retry/failure handling with exponential backoff (1 minute to max 1 hour, up to 7 retries)
+  - Event type catalog with descriptions and payload examples
+- Added authenticated webhook routes:
+  - `POST/GET /webhooks/targets`
+  - `PATCH/DELETE /webhooks/targets/{id}`
+  - `GET /webhooks/deliveries`
+  - `POST /webhooks/deliveries/{id}/redeliver`
+  - `POST /webhooks/process-pending`
+  - `GET /webhooks/event-types`
+- Integrated webhook triggers into key flows:
+  - Expenses: created/updated/deleted
+  - Bills: created/updated/deleted
+  - Profile: updated (`/auth/me` patch)
+- Added PostgreSQL schema DDL for webhook tables/indexes and startup compatibility creation for existing Postgres deployments.
+- Documented webhook endpoints, signature contract, retry policy, and event types in `README.md`.
+
+## Validation Commands
+1. `cd packages/backend && ../../.venv/bin/python -m pytest -q tests/test_webhooks.py`
+2. `sh ./scripts/test-backend.sh tests/test_webhooks.py`
+3. `sh ./scripts/test-backend.sh tests/test_auth.py tests/test_expenses.py tests/test_bills.py tests/test_reminders.py tests/test_observability.py`
+4. `cd packages/backend && ../../.venv/bin/flake8 app/__init__.py app/routes/auth.py app/routes/bills.py app/routes/expenses.py app/routes/webhooks.py app/services/webhooks.py app/models.py tests/test_webhooks.py`
+5. `sh ./scripts/test-backend.sh`
+
+## Validation Results
+- Command 1: failed in local host context due missing `redis` hostname resolution (environment mismatch, not code failure).
+- Command 2: passed (`3 passed`).
+- Command 3: passed (`16 passed`).
+- Command 4: passed (no lint errors on changed backend files).
+- Command 5: passed (`25 passed`).
+
+## Risks / Follow-ups
+- Webhook delivery processing is synchronous on event trigger (and manual process endpoint), so slow/unstable webhook targets can increase API latency.
+- Retry processing is currently request-driven/manual (`trigger_event` and `/webhooks/process-pending`); there is no dedicated background worker/cron loop in this change.
+- Webhook secrets are stored in plaintext in DB (common but sensitive); encryption-at-rest or secret vault integration would improve security posture.
+- `bill.due` and `subscription.updated` are documented and supported as event types, but no automatic emitters were added yet because current code paths do not include dedicated due/subscription transition jobs.

README.md

@@ -66,6 +66,30 @@ OpenAPI: `backend/app/openapi.yaml`
 - Bills: CRUD `/bills`, pay/mark `/bills/{id}/pay`
 - Reminders: CRUD `/reminders`, trigger `/reminders/run`
 - Insights: `/insights/monthly`, `/insights/budget-suggestion`
+- Webhooks:
+  - targets: `POST/GET /webhooks/targets`, `PATCH/DELETE /webhooks/targets/{id}`
+  - deliveries: `GET /webhooks/deliveries`, `POST /webhooks/deliveries/{id}/redeliver`
+  - event catalog: `GET /webhooks/event-types`
+
+### Webhook Event Types
+- `expense.created`
+- `expense.updated`
+- `expense.deleted`
+- `bill.created`
+- `bill.updated`
+- `bill.deleted`
+- `bill.due`
+- `subscription.updated`
+- `profile.updated`
+
+Each webhook request is signed with HMAC SHA-256 and includes:
+- `X-FinMind-Event`
+- `X-FinMind-Timestamp`
+- `X-FinMind-Signature` (`sha256=<digest>`, signed over `<timestamp>.<raw_json_payload>`)
+
+Delivery retry policy:
+- exponential backoff starting at 1 minute and capped at 1 hour
+- up to 7 retries before marking a delivery as failed
 
 ## MVP UI/UX Plan
 - Auth screens: register/login.

packages/backend/app/__init__.py

@@ -110,10 +110,60 @@ def _ensure_schema_compatibility(app: Flask) -> None:
             NOT NULL DEFAULT 'INR'
             """
         )
+        cur.execute(
+            """
+            CREATE TABLE IF NOT EXISTS webhook_targets (
+              id SERIAL PRIMARY KEY,
+              user_id INT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+              url VARCHAR(500) NOT NULL,
+              secret VARCHAR(255) NOT NULL,
+              enabled BOOLEAN NOT NULL DEFAULT TRUE,
+              events JSONB NOT NULL,
+              created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+              updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+            )
+            """
+        )
+        cur.execute(
+            """
+            CREATE TABLE IF NOT EXISTS webhook_deliveries (
+              id SERIAL PRIMARY KEY,
+              target_id INT NOT NULL REFERENCES webhook_targets(id) ON DELETE CASCADE,
+              event_type VARCHAR(50) NOT NULL,
+              payload JSONB NOT NULL,
+              status VARCHAR(20) NOT NULL DEFAULT 'pending',
+              attempt_count INT NOT NULL DEFAULT 0,
+              last_attempt_at TIMESTAMP,
+              next_attempt_at TIMESTAMP,
+              response_status INT,
+              response_body TEXT,
+              created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+              updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+            )
+            """
+        )
+        cur.execute(
+            """
+            CREATE INDEX IF NOT EXISTS idx_webhook_targets_user
+            ON webhook_targets(user_id)
+            """
+        )
+        cur.execute(
+            """
+            CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_target
+            ON webhook_deliveries(target_id)
+            """
+        )
+        cur.execute(
+            """
+            CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_status_next_attempt
+            ON webhook_deliveries(status, next_attempt_at)
+            """
+        )
         conn.commit()
     except Exception:
         app.logger.exception(
-            "Schema compatibility patch failed for users.preferred_currency"
+            "Schema compatibility patch failed for webhook schema"
         )
         conn.rollback()
     finally:

packages/backend/app/db/schema.sql

@@ -123,3 +123,34 @@ CREATE TABLE IF NOT EXISTS audit_logs (
   action VARCHAR(100) NOT NULL,
   created_at TIMESTAMP NOT NULL DEFAULT NOW()
 );
+
+CREATE TABLE IF NOT EXISTS webhook_targets (
+  id SERIAL PRIMARY KEY,
+  user_id INT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+  url VARCHAR(500) NOT NULL,
+  secret VARCHAR(255) NOT NULL,
+  enabled BOOLEAN NOT NULL DEFAULT TRUE,
+  events JSONB NOT NULL,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+CREATE TABLE IF NOT EXISTS webhook_deliveries (
+  id SERIAL PRIMARY KEY,
+  target_id INT NOT NULL REFERENCES webhook_targets(id) ON DELETE CASCADE,
+  event_type VARCHAR(50) NOT NULL,
+  payload JSONB NOT NULL,
+  status VARCHAR(20) NOT NULL DEFAULT 'pending',
+  attempt_count INT NOT NULL DEFAULT 0,
+  last_attempt_at TIMESTAMP,
+  next_attempt_at TIMESTAMP,
+  response_status INT,
+  response_body TEXT,
+  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+  updated_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+CREATE INDEX IF NOT EXISTS idx_webhook_targets_user ON webhook_targets(user_id);
+CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_target ON webhook_deliveries(target_id);
+CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_status_next_attempt
+  ON webhook_deliveries(status, next_attempt_at);

packages/backend/app/models.py

@@ -133,3 +133,49 @@ class AuditLog(db.Model):
     user_id = db.Column(db.Integer, db.ForeignKey("users.id"), nullable=True)
     action = db.Column(db.String(100), nullable=False)
     created_at = db.Column(db.DateTime, default=datetime.utcnow, nullable=False)
+
+
+class WebhookEvent(str, Enum):
+    EXPENSE_CREATED = "expense.created"
+    EXPENSE_UPDATED = "expense.updated"
+    EXPENSE_DELETED = "expense.deleted"
+    BILL_CREATED = "bill.created"
+    BILL_UPDATED = "bill.updated"
+    BILL_DELETED = "bill.deleted"
+    BILL_DUE = "bill.due"
+    SUBSCRIPTION_UPDATED = "subscription.updated"
+    PROFILE_UPDATED = "profile.updated"
+
+
+class WebhookTarget(db.Model):
+    __tablename__ = "webhook_targets"
+    id = db.Column(db.Integer, primary_key=True)
+    user_id = db.Column(db.Integer, db.ForeignKey("users.id"), nullable=False)
+    url = db.Column(db.String(500), nullable=False)
+    secret = db.Column(db.String(255), nullable=False)
+    enabled = db.Column(db.Boolean, default=True, nullable=False)
+    events = db.Column(db.JSON, nullable=False)
+    created_at = db.Column(db.DateTime, default=datetime.utcnow, nullable=False)
+    updated_at = db.Column(
+        db.DateTime, default=datetime.utcnow, onupdate=datetime.utcnow, nullable=False
+    )
+
+
+class WebhookDelivery(db.Model):
+    __tablename__ = "webhook_deliveries"
+    id = db.Column(db.Integer, primary_key=True)
+    target_id = db.Column(
+        db.Integer, db.ForeignKey("webhook_targets.id"), nullable=False
+    )
+    event_type = db.Column(db.String(50), nullable=False)
+    payload = db.Column(db.JSON, nullable=False)
+    status = db.Column(db.String(20), default="pending", nullable=False)
+    attempt_count = db.Column(db.Integer, default=0, nullable=False)
+    last_attempt_at = db.Column(db.DateTime, nullable=True)
+    next_attempt_at = db.Column(db.DateTime, nullable=True)
+    response_status = db.Column(db.Integer, nullable=True)
+    response_body = db.Column(db.Text, nullable=True)
+    created_at = db.Column(db.DateTime, default=datetime.utcnow, nullable=False)
+    updated_at = db.Column(
+        db.DateTime, default=datetime.utcnow, onupdate=datetime.utcnow, nullable=False
+    )

packages/backend/app/routes/__init__.py

@@ -7,6 +7,7 @@
 from .categories import bp as categories_bp
 from .docs import bp as docs_bp
 from .dashboard import bp as dashboard_bp
+from .webhooks import bp as webhooks_bp
 
 
 def register_routes(app: Flask):
@@ -18,3 +19,4 @@ def register_routes(app: Flask):
     app.register_blueprint(categories_bp, url_prefix="/categories")
     app.register_blueprint(docs_bp, url_prefix="/docs")
     app.register_blueprint(dashboard_bp, url_prefix="/dashboard")
+    app.register_blueprint(webhooks_bp, url_prefix="/webhooks")

packages/backend/app/routes/auth.py

@@ -9,7 +9,8 @@
     get_jwt_identity,
 )
 from ..extensions import db, redis_client
-from ..models import User
+from ..models import User, WebhookEvent
+from ..services.webhooks import WebhookService
 import logging
 import time
 
@@ -94,6 +95,15 @@ def update_me():
             return jsonify(error="unsupported preferred_currency"), 400
         user.preferred_currency = cur
     db.session.commit()
+    WebhookService.trigger_event(
+        WebhookEvent.PROFILE_UPDATED,
+        {
+            "id": user.id,
+            "email": user.email,
+            "preferred_currency": user.preferred_currency or "INR",
+        },
+        user_id=uid,
+    )
     return jsonify(
         id=user.id,
         email=user.email,

packages/backend/app/routes/bills.py

@@ -2,8 +2,9 @@
 from flask import Blueprint, jsonify, request
 from flask_jwt_extended import jwt_required, get_jwt_identity
 from ..extensions import db
-from ..models import Bill, BillCadence, User
+from ..models import Bill, BillCadence, User, WebhookEvent
 from ..services.cache import cache_delete_patterns
+from ..services.webhooks import WebhookService
 import logging
 
 bp = Blueprint("bills", __name__)
@@ -59,6 +60,11 @@ def create_bill():
     db.session.add(b)
     db.session.commit()
     logger.info("Created bill id=%s user=%s name=%s", b.id, uid, b.name)
+    WebhookService.trigger_event(
+        WebhookEvent.BILL_CREATED,
+        _bill_to_dict(b),
+        user_id=uid,
+    )
     cache_delete_patterns(
         [f"user:{uid}:upcoming_bills*", f"user:{uid}:dashboard_summary:*"]
     )
@@ -85,7 +91,95 @@ def mark_paid(bill_id: int):
     cache_delete_patterns(
         [f"user:{uid}:upcoming_bills*", f"user:{uid}:dashboard_summary:*"]
     )
+    WebhookService.trigger_event(
+        WebhookEvent.BILL_UPDATED,
+        _bill_to_dict(b),
+        user_id=uid,
+    )
     logger.info(
         "Marked bill paid id=%s user=%s next_due_date=%s", b.id, uid, b.next_due_date
     )
     return jsonify(message="updated")
+
+
+@bp.patch("/<int:bill_id>")
+@jwt_required()
+def update_bill(bill_id: int):
+    uid = int(get_jwt_identity())
+    b = db.session.get(Bill, bill_id)
+    if not b or b.user_id != uid:
+        return jsonify(error="not found"), 404
+
+    data = request.get_json() or {}
+    if "name" in data:
+        b.name = str(data.get("name") or "").strip() or b.name
+    if "amount" in data:
+        b.amount = data.get("amount")
+    if "currency" in data:
+        b.currency = str(data.get("currency") or "INR")[:10]
+    if "next_due_date" in data:
+        try:
+            b.next_due_date = date.fromisoformat(str(data.get("next_due_date")))
+        except ValueError:
+            return jsonify(error="invalid next_due_date"), 400
+    if "cadence" in data:
+        try:
+            b.cadence = BillCadence(str(data.get("cadence")))
+        except ValueError:
+            return jsonify(error="invalid cadence"), 400
+    if "autopay_enabled" in data:
+        b.autopay_enabled = bool(data.get("autopay_enabled"))
+    if "channel_whatsapp" in data:
+        b.channel_whatsapp = bool(data.get("channel_whatsapp"))
+    if "channel_email" in data:
+        b.channel_email = bool(data.get("channel_email"))
+    if "active" in data:
+        b.active = bool(data.get("active"))
+
+    db.session.commit()
+    cache_delete_patterns(
+        [f"user:{uid}:upcoming_bills*", f"user:{uid}:dashboard_summary:*"]
+    )
+    WebhookService.trigger_event(
+        WebhookEvent.BILL_UPDATED,
+        _bill_to_dict(b),
+        user_id=uid,
+    )
+    return jsonify(_bill_to_dict(b)), 200
+
+
+@bp.delete("/<int:bill_id>")
+@jwt_required()
+def delete_bill(bill_id: int):
+    uid = int(get_jwt_identity())
+    b = db.session.get(Bill, bill_id)
+    if not b or b.user_id != uid:
+        return jsonify(error="not found"), 404
+
+    payload = {"id": b.id}
+    db.session.delete(b)
+    db.session.commit()
+    cache_delete_patterns(
+        [f"user:{uid}:upcoming_bills*", f"user:{uid}:dashboard_summary:*"]
+    )
+    WebhookService.trigger_event(
+        WebhookEvent.BILL_DELETED,
+        payload,
+        user_id=uid,
+    )
+    return jsonify(message="deleted"), 200
+
+
+def _bill_to_dict(bill: Bill) -> dict:
+    return {
+        "id": bill.id,
+        "name": bill.name,
+        "amount": float(bill.amount),
+        "currency": bill.currency,
+        "next_due_date": bill.next_due_date.isoformat(),
+        "cadence": bill.cadence.value,
+        "autopay_enabled": bill.autopay_enabled,
+        "channel_whatsapp": bill.channel_whatsapp,
+        "channel_email": bill.channel_email,
+        "active": bill.active,
+    }