Skip to content

feat: replace grpc transport with fibp (fila binary protocol)#5

Merged
vieiralucas merged 4 commits intomainfrom
feat/fibp-transport
Mar 26, 2026
Merged

feat: replace grpc transport with fibp (fila binary protocol)#5
vieiralucas merged 4 commits intomainfrom
feat/fibp-transport

Conversation

@vieiralucas
Copy link
Copy Markdown
Member

@vieiralucas vieiralucas commented Mar 26, 2026

Summary

  • Rewrites the JS SDK transport layer from gRPC to FIBP (Fila Binary Protocol) — a length-prefixed binary framing protocol over raw TCP/TLS
  • Removes @grpc/grpc-js and @grpc/proto-loader dependencies entirely; adds protobufjs (previously transitive) for admin op payloads only
  • Adds new src/transport.ts with the full FIBP implementation: TCP/TLS connection, handshake, frame encode/decode, correlation ID multiplexing, auth frame, streaming push delivery
  • Public API is unchanged: Client, enqueue, consume, ack, nack, close, enqueueMany, all options

Changes

New files:

  • src/transport.ts — FIBP connection, frame encoding/decoding, all wire format helpers
  • test/transport.unit.test.ts — 18 unit tests covering frame format, payload round-trips, and wire format correctness (no server required)

Modified files:

  • src/client.ts — rewritten to use FibpConnection instead of gRPC channel; lazy connect on first operation
  • src/batcher.ts — adapted to use FibpConnection instead of FilaServiceClient
  • src/errors.ts — adds UnauthenticatedError, removes gRPC status code dependency
  • src/index.ts — exports UnauthenticatedError
  • test/helpers.ts — rewritten to use FIBP + protobufjs for admin ops (createQueue, listQueues readiness probe); removes gRPC admin client
  • test/auth.test.ts — updated to use UnauthenticatedError instead of RPCError with gRPC UNAUTHENTICATED code; removes gRPC imports
  • package.json — removes grpc deps, adds protobufjs, bumps to 0.3.0
  • tsconfig.json — removes generated/ from include
  • README.md — updated for FIBP transport, new error class, protocol section

Deleted:

  • generated/ — gRPC TypeScript type stubs, no longer needed

Test plan

  • Unit tests pass without a server: npm test — 25 tests (18 transport + 7 batcher)
  • npm run typecheck — clean
  • npm run lint — clean
  • Integration tests (client.test.ts, batch.test.ts, auth.test.ts) run when FILA_SERVER_BIN is set — CI provisions the server binary

🤖 Generated with Claude Code


Summary by cubic

Replaced the gRPC transport with FIBP (Fila Binary Protocol) over raw TCP/TLS to reduce deps and simplify the wire path. Public API stays the same; adds UnauthenticatedError for auth failures, preserves server error codes in RPCError (with detail), and sets all test server startup timeouts to 30s.

  • Migration

    • No code changes required; existing Client methods work as before.
    • If you checked gRPC UNAUTHENTICATED, catch UnauthenticatedError instead.
  • Dependencies

    • Removed @grpc/grpc-js and @grpc/proto-loader; added protobufjs.
    • Deleted generated/ stubs; version bumped to 0.3.0.

Written for commit ed658d1. Summary will update on new commits.

rewrite the javascript sdk transport layer from grpc to fibp:
- new src/transport.ts: tcp connection via net.socket/tls.connect,
  length-prefixed binary frame encoding/decoding, correlation id
  multiplexing (map + promises), auth frame for api key, stream
  support for consume push delivery
- client.ts: lazy connection, fibp request/response for enqueue/ack/nack,
  async iterable consume via openstream
- batcher.ts: adapted to fibp connection instead of grpc client
- errors.ts: adds unauthenticatederror, removes grpc status code dep
- test helpers rewritten to use fibp + protobufjs for admin ops
  (createqueue, listqueues readiness probe)
- new test/transport.unit.test.ts: 18 unit tests for frame encoding,
  payload round-trips, and wire format correctness
- removes @grpc/grpc-js and @grpc/proto-loader dependencies entirely;
  adds protobufjs (already transitive) for admin protobuf payloads
- removes generated/ grpc type stubs directory
- bumps version to 0.3.0
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 78 files

Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="test/helpers.ts">

<violation number="1" location="test/helpers.ts:187">
P2: Handle failure when creating the persistent admin connection; otherwise a connect error can leak the spawned server process and temp directory.</violation>
</file>

<file name="src/client.ts">

<violation number="1" location="src/client.ts:406">
P2: Inline error-frame parsing skips the `msg_len` field and reads to end of payload, which can include trailing bytes. Use the existing `decodeErrorPayload` helper from `transport.ts` that correctly bounds the message read.</violation>
</file>

<file name="src/transport.ts">

<violation number="1" location="src/transport.ts:602">
P1: The wire error `code` from the ERROR frame is discarded and replaced with `ErrCode.INTERNAL`. This means frame-level errors (e.g., authentication failures) always surface as generic internal errors, preventing the client from throwing typed errors like `UnauthenticatedError`.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread src/transport.ts Outdated
Comment thread test/helpers.ts Outdated
Comment thread src/client.ts Outdated
- transport: preserve wire error code in rpc errors (not hardcoded INTERNAL)
- client: add rpcerror catch+remap for transport-level errors on enqueue/ack/nack/consume
- errors: add rpcerror.detail to store raw message without code prefix
- batcher: remap transport-level rpcerror to typed errors before rejecting batch items
- helpers: handle connectadmin failure to avoid server process + tempdir leak
vieiralucas added a commit to faiscadev/fila that referenced this pull request Mar 26, 2026
@vieiralucas vieiralucas merged commit 277a556 into main Mar 26, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant