Skip to content

feat: transaction grpc server#531

Open
bas4r wants to merge 11 commits intomainfrom
feat--nwc-grpc-server
Open

feat: transaction grpc server#531
bas4r wants to merge 11 commits intomainfrom
feat--nwc-grpc-server

Conversation

@bas4r
Copy link
Copy Markdown

@bas4r bas4r commented Apr 20, 2026

Summary

  • Adds Blink Core transaction stream support for settled ledger transactions.
  • Replays missed transactions with after_transaction_id, then continues with Mongo change-
    stream live delivery.
  • Maps ledger transactions into TransactionEvent fields including direction, wallet/account
    IDs, payment hash, preimage, amounts, currency, settlement type, and timestamp.
  • Hardens the gRPC stream server behavior for explicit empty cursors, backpressure, client
    cancellation, and service-side stream errors.
  • Wires the standalone transactions gRPC stream service into Tilt/Bats e2e setup, including
    local Mongo replica-set support for change streams.

Story Coverage

E4-S1 Blink Core items addressed:

  • Standalone transactions-grpc-stream-server process.
  • transactions.proto as the Blink Core transaction stream contract.
  • Replay after reconnect/restart via after_transaction_id.
  • Both SENT and RECEIVED transaction events from Blink Core.
  • Local/e2e runtime support for the stream service.

Testing

Passed:

  • nix develop -c buck2 test //core/api:check-lint //core/api:check-type //core/api:check- yaml //core/api:check-circular-dependencies //core/api:unit-tests
  • nix develop -c buck2 build //core/api:prod_build //core/api-transactions-grpc-stream:api- transactions-grpc-stream
  • git diff --check

Note: full //core/api:test includes audit, which currently fails on the pre-existing
protobufjs advisory. This PR does not change dependency overrides.

@github-actions github-actions Bot added the core label Apr 20, 2026
@bas4r bas4r marked this pull request as draft April 20, 2026 13:23
@bas4r bas4r force-pushed the feat--nwc-grpc-server branch from 485fb12 to d04c67e Compare April 21, 2026 13:08
@bas4r bas4r changed the title feat: nwc grpc server feat: transaction grpc server Apr 21, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a standalone gRPC server and supporting domain/service layers to stream settled ledger transactions from Blink Core, with cursor-based replay and MongoDB change-stream live delivery. This fits alongside existing Core API services by exposing a new stream-oriented contract (transactions.proto) and wiring it into local dev + Bats e2e.

Changes:

  • Introduces a transactions stream domain + service, including mapping from ledger transactions to TransactionEvent and a gRPC server implementation with backpressure/cancellation handling.
  • Adds a ledger-side settled transaction streamer that replays after after_transaction_id and then tails MongoDB change streams with overlap de-dupe.
  • Wires the new service into build/dev/CI/e2e: Buck/Tilt targets, Bats helpers/tests, and local Mongo replica-set support for change streams.

Reviewed changes

Copilot reviewed 31 out of 36 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pnpm-lock.yaml Updates protobufjs override/resolution in the lockfile.
package.json Updates root resolutions for protobufjs.
flake.nix Adds Darwin-friendly buf/grpcurl selection, PATH shellHook additions, and a new build derivation.
dev/Tiltfile Adds a Tilt local_resource for api-transactions-grpc-stream with readiness probe + Mongo dependency.
dev/docker-compose.deps.yml Runs MongoDB as a replica set and adds a healthcheck to support change streams.
core/api/test/unit/services/transactions-stream/index.spec.ts Unit tests for TransactionsStreamService subscription behavior and errors.
core/api/test/unit/services/transactions-stream/helpers.spec.ts Unit tests for account/preimage resolvers and event mapper helpers.
core/api/test/unit/services/transactions-stream/grpc-server.spec.ts Unit tests for gRPC server mapping, invalid cursors, cancellation, and backpressure.
core/api/test/unit/services/ledger/stream-settled-transactions.spec.ts Unit tests for replay + change-stream delivery and error surfacing.
core/api/src/services/transactions-stream/proto/transactions.proto Defines the public gRPC contract (SubscribeTransactions / TransactionEvent).
core/api/src/services/transactions-stream/proto/transactions_pb.js Generated protobuf JS bindings for the new contract.
core/api/src/services/transactions-stream/proto/transactions_pb.d.ts Generated protobuf TS typings for the new contract.
core/api/src/services/transactions-stream/proto/transactions_grpc_pb.js Generated gRPC service definitions (JS).
core/api/src/services/transactions-stream/proto/transactions_grpc_pb.d.ts Generated gRPC service definitions (TS).
core/api/src/services/transactions-stream/proto/buf.gen.yaml Buf generation config for JS/gRPC/TS outputs.
core/api/src/services/transactions-stream/index.ts Implements TransactionsStreamService subscribe logic with cursor parsing and abort cleanup.
core/api/src/services/transactions-stream/helpers.ts Adds mapping helpers + account/preimage loading and caching.
core/api/src/services/transactions-stream/grpc-server.ts Implements server-streaming gRPC handler and backpressure handling.
core/api/src/services/transactions-stream/convert.ts Converts domain TransactionStreamEvent into protobuf TransactionEvent.
core/api/src/services/ledger/stream-settled-transactions.ts Streams settled transactions via replay query + Mongo change stream with dedupe.
core/api/src/services/ledger/index.ts Exposes streamSettledTransactions on LedgerService.
core/api/src/servers/transactions-grpc-stream-server.ts New standalone server process that starts gRPC + health endpoints.
core/api/src/domain/transactions-stream/index.types.d.ts Declares domain types for transaction stream events and enums.
core/api/src/domain/transactions-stream/index.ts Implements transaction stream enums and ledger-type mapping helpers.
core/api/src/domain/ledger/index.types.d.ts Adds streamSettledTransactions to the ledger service interface.
core/api/src/config/index.ts Exposes new env-configured ports for the stream server + health endpoint.
core/api/src/config/env.ts Adds env vars and defaults for TRANSACTIONS_GRPC_STREAM_PORT and health port.
core/api/package.json Ensures proto artifacts are copied on build; adds dev + codegen scripts.
core/api/BUCK Adds a Buck dev task binary for the new stream server.
core/api-transactions-grpc-stream/Dockerfile Adds container build for the standalone stream server artifact.
core/api-transactions-grpc-stream/BUCK Adds Buck targets/aliases for prod/dev builds of the new service.
bats/helpers/transactions-grpc-stream.bash Adds grpcurl wrapper helpers for the new stream endpoint.
bats/helpers/_common.bash Extends grpcurl_request to forward extra grpcurl args.
bats/core/api/transactions-grpc-stream.bats Adds e2e tests for invalid cursors and replay behavior.
bats/ci_setup_suite.bash Waits for the new stream service health endpoint in CI setup.
bats/ci_run.sh Adds the new service to CI Buck build set.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +181 to +186
healthcheck:
test:
[
"CMD-SHELL",
"mongosh --quiet --eval 'try { rs.status().ok } catch (e) { rs.initiate({ _id: \"rs0\", members: [{ _id: 0, host: \"localhost:27017\" }] }).ok }' | grep 1",
]
Comment on lines +60 to +66
call.removeListener("error", finish)
resolve()
}

call.once("drain", finish)
call.once("cancelled", finish)
call.once("error", finish)
Comment thread package.json
Comment on lines 14 to 18
"resolutions": {
"@types/ws": "^8.5.12",
"basic-ftp": "^5.2.0",
"protobufjs": "7.2.5",
"protobufjs": "7.5.5",
"http-cache-semantics": "4.1.1",
@github-actions github-actions Bot added the ci label May 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants