Skip to content

Feat/sync foundation#238

Draft
MostafaKadry wants to merge 65 commits intodevelopfrom
feat/sync-foundation
Draft

Feat/sync foundation#238
MostafaKadry wants to merge 65 commits intodevelopfrom
feat/sync-foundation

Conversation

@MostafaKadry
Copy link
Copy Markdown
Collaborator

No description provided.

engahmed1190 and others added 30 commits April 5, 2026 21:13
Defines topology (two full ERPNext sites, branch-initiates-all-HTTPS),
Sync Site Config DocType (role-dependent cardinality, username+password
session auth), change capture (outbox for transactions + watermark with
tombstones for masters), pluggable adapter engine, per-entity conflict
rules, POS-client failover to central with three-source reconciliation,
reconciliation-gated IndexedDB flush, close-shift guard, observability
dashboard, security model, and test strategy.

Per-entity sync details (masters, stock, transactions, conflict
reconciliation) are deferred to separate sub-specs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
19 tasks covering DocTypes, sync module skeleton, custom fields,
role/permissions, seeds, and Test Sync Connection button. No data
flows yet — Plan 2 adds masters pull, Plan 3 adds transactions push
+ failover.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces the Sync Site Config DocType with Branch singleton enforcement,
Central multi-record support, HTTPS URL validation, branch_code format
validation, and a 4-test suite covering all four scenarios.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Local multi-site bench testing needs http:// central URLs between
sites on the same bench. Opt-in via env var; off by default so
production enforces https://. test_https_enforced skips when the
bypass is active.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Setup helpers to configure Sync Site Config on the two-bench
dev topology: frappe-bench (port 8000, central) and
frappe-bench-16 (port 8001, branch).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…l sections

Adds §15 to umbrella spec covering:
- Two-bench dev setup (frappe-bench port 8000 as central,
  frappe-bench-16 port 8001 as branch)
- Version-agnostic sync protocol design (single codebase,
  runtime v15/v16 detection, pos_next-owned endpoints)
- Bootstrap procedure and helper functions
- Running instructions for both benches

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…flict, log, dead letter, history)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…stom fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The export-fixtures command in Task 17 regenerated fixture files
from the dev site which was missing production roles. This restores:
- custom_docperm.json: all original role permissions
- role.json: POSNext Cashier + Nexus POS Manager + POS Next Sync Agent
- hooks.py: merged Role fixture filter into single entry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests SyncSession login from branch to central and authenticated
API calls across the two-bench dev environment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers central-side changes_since API, branch-side MastersPuller,
first adapters (Item, ItemPrice, Customer, GenericMaster),
tombstone hooks, and scheduler integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
10 tasks covering: changes_since API, health API, adapters (Item,
ItemPrice, Customer, GenericMaster), MastersPuller engine,
tombstone hooks, scheduler, test runner, and e2e integration test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
engahmed1190 and others added 30 commits April 6, 2026 16:13
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds on_trash tombstone hook for all synced master DocTypes and
cron scheduler entry for pull_if_due (runs every minute).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Synced data was already validated on central. Applying it on branch
with full Frappe validation causes failures when linked records
(Company, UOM) haven't been pulled yet or differ between v15/v16.

Adds _set_sync_flags() that sets ignore_validate, ignore_links,
ignore_mandatory on docs during sync apply.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Frappe's conflict detection rejects synced updates when the payload's
modified timestamp differs from the local record's. Adding
ignore_conflict=True fixes "has been modified after you opened it".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…etection

doc.update(payload) with central's modified timestamp causes Frappe
to think the record was edited by someone else. Fix: set fields
individually, skipping modified/modified_by/creation/owner.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
doc.save() triggers Company.update_default_account (v15-only method)
and NestedSet validation for Item Group. Using db_update() for the
update path bypasses all hooks/validations — synced data was already
validated on central. Insert path still uses insert() with sync flags.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows last pull time, outbox stats, conflict count, watermarks table,
and recent sync logs directly on the Sync Site Config form.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The scheduler worker doesn't import adapter modules by default,
leaving the registry empty. Add _ensure_adapters_loaded() call
before running the pull cycle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Code review findings addressed:
- Extract SKIP_ON_UPSERT constant, eliminate duplicate field-skip tuples
- ItemAdapter: delegate to super() instead of duplicating upsert logic
- Remove no-op serialize override from ItemAdapter
- Use try/except DoesNotExistError instead of exists()+get_doc() double-tap
- Auto-discover adapters via pkgutil instead of manual import list
- Remove per-record frappe.db.commit() — batch commit per page
- Add tombstone limit in changes_since to prevent unbounded response
- Use to_payload() instead of inline as_dict() in changes.py
- Cache branch_code in hooks_uuid.py (was DB query on every insert)
- Log tombstone errors instead of silently swallowing
- Add DIRECTIONS_PULL/DIRECTIONS_PUSH constants
- Fix health.py to use enabled=1 filter

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers outbox hooks, OutboxDrainer, central ingest API,
transaction adapters (Sales Invoice, Payment Entry, POS shifts,
SLE), docstatus-aware insert pattern, and scheduler integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
8 tasks: SubmittableAdapter, 5 transaction adapters, outbox hooks,
ingest API, OutboxDrainer with backoff/dead letter, scheduler,
test runner, and e2e integration test. Includes naming series
validation for branch-coded Sales Invoice names.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…OS shifts, SLE)

Sales Invoice includes naming series validation against origin_branch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds enqueue_to_outbox on: Sales Invoice (submit/cancel/update_after_submit),
Payment Entry (submit/cancel), POS Opening/Closing Shift (submit),
Stock Ledger Entry (after_insert), Customer (on_update).
Adds push_if_due to cron scheduler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers: glossary, architecture diagrams, data flows, configuration,
naming series convention, synced doctypes registry, adapter hierarchy,
all 12 DocTypes, 4 API endpoints, conflict resolution strategies,
custom fields, security, monitoring, troubleshooting, file structure,
and development setup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants