RvfBackend silently drops vectors with non-numeric string IDs

[dgtise25]

Bug Report: RVF Backend Silently Drops Vectors with Non-Numeric String IDs

Package: agentdb (alpha) — specifically RvfBackend in agentdb/backends/self-learning
Upstream dependency: @ruvector/rvf / @ruvector/rvf-node (N-API addon)
Severity: Critical — silent data loss
Date: 2026-02-22

Summary

RvfBackend silently drops all vectors whose IDs are non-numeric strings
(e.g. UUIDs, hex hashes). Only vectors with purely numeric string IDs (e.g.
"1", "42") are persisted. No error is thrown — the operation appears to
succeed but the data is lost.

Root Cause

The NodeBackend.ingestBatch() method in @ruvector/rvf/dist/backend.js
converts string IDs to numbers on line 118:

// @ruvector/rvf/dist/backend.js, line 118
const ids = entries.map((e) => Number(e.id));

The N-API layer expects i64[] (numeric labels). When e.id is a
non-numeric string like "da003664_2b0f6ff3747e", Number() returns NaN.
The native Rust HNSW layer silently ignores entries with NaN IDs — no error
is thrown, and ingestBatch returns { accepted: N } as if all entries were
ingested.

The same pattern appears in NodeBackend.delete() at line 147:

const numIds = ids.map((id) => Number(id));

And in NodeBackend.query() at line 136, results come back as numeric labels
converted to strings:

return results.map((r) => ({ id: String(r.id), distance: r.distance }));

This means even if vectors were ingested successfully (with numeric IDs),
search results would return the numeric label (e.g. "42") rather than the
original semantic ID the caller passed.

Reproduction

import { SelfLearningRvfBackend } from "agentdb/backends/self-learning";

// --- Non-numeric IDs: FAILS silently ---
const backend = await SelfLearningRvfBackend.create({
  dimension: 4, metric: "cosine",
  storagePath: "/tmp/test_string_ids.rvf",
  learning: false, maxElements: 1000,
});

for (let i = 0; i < 10; i++) {
  const vec = new Float32Array([Math.random(), Math.random(), Math.random(), Math.random()]);
  await backend.insertAsync(`chunk_${i}`, vec, { text: `test ${i}` });
}
await backend.flush();
await backend.save();
await backend.backend.db.close();

// Reopen — expect 10, get 1
const backend2 = await SelfLearningRvfBackend.create({
  dimension: 4, metric: "cosine",
  storagePath: "/tmp/test_string_ids.rvf",
  learning: false, maxElements: 1000,
});
console.log("Count:", backend2.getStats().count);  // → 1 (should be 10)
await backend2.backend.db.close();


// --- Numeric IDs: WORKS correctly ---
const backend3 = await SelfLearningRvfBackend.create({
  dimension: 4, metric: "cosine",
  storagePath: "/tmp/test_numeric_ids.rvf",
  learning: false, maxElements: 1000,
});

for (let i = 0; i < 10; i++) {
  const vec = new Float32Array([Math.random(), Math.random(), Math.random(), Math.random()]);
  await backend3.insertAsync(String(i + 1), vec, { text: `test ${i}` });
}
await backend3.flush();
await backend3.save();
await backend3.backend.db.close();

// Reopen — expect 10, get 10 ✓
const backend4 = await SelfLearningRvfBackend.create({
  dimension: 4, metric: "cosine",
  storagePath: "/tmp/test_numeric_ids.rvf",
  learning: false, maxElements: 1000,
});
console.log("Count:", backend4.getStats().count);  // → 10 ✓
await backend4.backend.db.close();

Contrast: HNSWLibBackend Handles This Correctly

HNSWLibBackend (agentdb/backends) maintains explicit string↔numeric
mappings:

// HNSWLibBackend.js
idToLabel = new Map();    // string ID → numeric label
labelToId = new Map();    // numeric label → string ID
metadata = new Map();     // string ID → metadata
nextLabel = 0;

insert(id, embedding, metadata) {
  // Assigns a numeric label, stores the mapping
  const label = this.nextLabel++;
  this.idToLabel.set(id, label);
  this.labelToId.set(label, id);
  // ...
}

These mappings are persisted to {path}.mappings.json alongside the HNSW
index file, and restored on load().

RvfBackend has no such mapping layer. It passes string IDs directly to
the N-API layer, which expects integers.

Affected Components

Method	File	Line	Issue
ingestBatch()	@ruvector/rvf/dist/backend.js	118	Number(e.id) → NaN for non-numeric strings
delete()	@ruvector/rvf/dist/backend.js	147	Number(id) → NaN for non-numeric strings
query()	@ruvector/rvf/dist/backend.js	136	Returns numeric labels, not original string IDs

Impact

• Silent data loss: Vectors are reported as accepted but never persisted
• No error signal: ingestBatch() returns { accepted: N } even though
  0 vectors were actually stored
• Metadata loss: Even with numeric IDs, metadata is not round-tripped
  through the N-API layer (it's stored in-memory only by AgentDB's
  RvfBackend wrapper, not persisted to the .rvf file)
• Search returns wrong IDs: Query results use numeric labels, not the
  original semantic IDs

Proposed Fix

Option A: Add ID mapping to RvfBackend (AgentDB layer)

Add idToLabel/labelToId maps to RvfBackend, identical to
HNSWLibBackend. Persist the mappings as a sidecar JSON file
({path}.mappings.json). This is the simplest fix and keeps the N-API layer
unchanged.

// RvfBackend additions
idToLabel = new Map();
labelToId = new Map();
metadata = new Map();
nextLabel = 1;  // RVF uses 1-based labels

insert(id, embedding, metadata) {
  let label = this.idToLabel.get(id);
  if (label === undefined) {
    label = this.nextLabel++;
    this.idToLabel.set(id, label);
    this.labelToId.set(label, id);
  }
  this.metadata.set(id, metadata);
  // Queue with numeric label
  this.pending.push({ id: String(label), vector: ..., metadata });
}

Option B: Support string IDs natively in @ruvector/rvf-node

Modify the Rust N-API layer to accept string IDs and handle the mapping
internally (e.g. store a string→u64 hash table in the RVF file). This is a
larger change but eliminates the sidecar file.

Option C: Hash string IDs to numeric labels (quick but lossy)

Use a deterministic hash (e.g. FNV-1a or xxHash) to convert string IDs to
numeric labels. Risk: hash collisions cause silent overwrites. Not
recommended for production use.

Recommended Fix

Option A — it's what HNSWLibBackend already does successfully, requires
no changes to the Rust N-API layer, and can be implemented entirely in the
AgentDB TypeScript code.

Environment

• agentdb@alpha (npm)
• @ruvector/rvf@0.x (npm, N-API addon)
• Node.js v22.19.0
• Linux 6.17.0-14-generic, x86_64
• Tested on RTX 4090, 24 GB VRAM

[ruvnet]

Fix Implemented

Root cause confirmed: RuVectorBackend passed string IDs directly to the @ruvector/core N-API layer, which internally converts them via Number(). Non-numeric string IDs (UUIDs, prefixed IDs, hex hashes) became NaN and were silently dropped by the Rust HNSW layer.

Fix (Option A from the issue): Added idToLabel/labelToId mapping layer to RuVectorBackend, matching the pattern already used by HNSWLibBackend.

Changes

packages/agentdb/src/backends/ruvector/RuVectorBackend.ts:

• Added idToLabel: Map<string, number>, labelToId: Map<number, string>, and nextLabel counter (1-based)
• insert(): Assigns a numeric label for each string ID before passing to the N-API layer. Reuses existing label if the same ID is re-inserted.
• search(): Maps numeric labels from results back to original string IDs, and resolves metadata using original IDs.
• remove(): Looks up the numeric label for the string ID, removes the mapping, then passes the numeric label to the N-API layer.
• save(): Persists idToLabel, labelToId, nextLabel, and metadata together in the .meta.json sidecar file.
• load(): Restores ID mappings from the new format, with backward compatibility for the legacy metadata-only format.
• close(): Clears all mapping state.

Validation

Added packages/agentdb/tests/backends/ruvector-id-mapping.test.ts — 14 tests, all passing:

• UUID-style IDs, prefix-style IDs (chunk_0), hex hashes — all inserted and counted correctly
• Numeric string IDs continue to work
• Mixed numeric and non-numeric IDs work
• Search returns original string IDs (not numeric labels)
• Metadata correctly associated with original string IDs
• Remove works with string IDs
• Batch insert works with string IDs
• ID mappings persist through save/load cycle
• Legacy .meta.json format loads without error
• Re-inserting same ID reuses the existing numeric label
• close() fully resets mapping state

No regressions in existing backend tests (pre-existing hnswlib test failures are unchanged).

[ruvnet]

Resolved

Fix summary

Layer	Package	Version	What changed
Upstream	@ruvector/rvf	0.2.0	NodeBackend now maps string IDs to sequential i64 labels natively (ruvnet/RuVector#200, merged)
AgentDB	agentdb	2.0.0-alpha.2.21	Defense-in-depth idToLabel/labelToId mapping in RuVectorBackend + ruvector bumped to ^0.1.99

What ships

Package	Version	Registry
@ruvector/rvf	0.2.0	npm
agentdb	2.0.0-alpha.2.21	npm

PR

• fix(agentdb): String ID mapping for RuVectorBackend — silent data loss #115

Validation

14 unit tests covering UUID, prefix, hex, mixed, and numeric string IDs — all passing. No regressions in existing backend tests.