Feature: Storage Anchor #164

sephynox · 2026-01-26T23:34:57Z

Summary

This PR introduces a new storage anchor service:

API with PUT, GET, DELETE, and SEARCH operations
Path-based namespacing (/user//...) with owner-enforced access control
Client-side encryption via EncryptedContainer with optional author signing (RFC 5652 CMS SignerInfo)
Public/private visibility with pre-signed URLs for anonymous public access
Quota enforcement with configurable limits per object size, object count, and total storage
Namespace validators for content validation (e.g., user icons)

Key Changes

Encrypted Container (src/lib/encrypted-container.ts)

Added RFC 5652 CMS SignerInfo support for author signatures
Added structured error handling with EncryptedContainerError and typed error codes

Storage Service (src/services/storage/)

Server (server.ts): HTTP server implementation with authenticated endpoints
Client (client.ts): Client SDK with provider discovery
Memory Backend (drivers/memory.ts): In-memory backend with atomic operations (copy-on-write)

…at/storage-anchor

Copilot

Pull request overview

Introduces a new Storage Anchor service with HTTP server + client support, including path-based object storage, quota reporting, namespace validation, and public access via pre-signed URLs. It also extends the resolver metadata model and enhances EncryptedContainer with structured error handling and optional signing.

Changes:

Added Storage Anchor server routes (put/get/delete/search/quota/public) with quota/validator hooks and service metadata publishing.
Added Storage Anchor client/provider for discovery via resolver, CRUD operations, search/quota, and pre-signed public URLs.
Extended core libraries: resolver types/lookup, error deserialization mapping, and EncryptedContainer error/signing support (with tests).

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/services/storage/server.ts	Implements Storage Anchor HTTP API endpoints and publishes storage service metadata
src/services/storage/server.test.ts	Adds basic server/serviceMetadata and error-path tests
src/services/storage/lib/validators.ts	Adds namespace validator framework + built-in icon validator
src/services/storage/lib/validators.test.ts	Tests validator matching and icon validation logic
src/services/storage/common.ts	Defines storage types, requests/responses, signing data, and storage-specific errors
src/services/storage/common.test.ts	Adds path utility tests and an in-memory backend for test usage
src/services/storage/client.ts	Adds resolver-driven client/provider with signed requests and public URL generation
src/services/storage/client.test.ts	Adds integration-style tests for provider discovery and CRUD/search/quota/public flows
src/lib/utils/tests/node.ts	Adds helper to publish resolver metadata in tests
src/lib/resolver.ts	Extends resolver metadata schema and lookup logic for `storage` services
src/lib/resolver.test.ts	Refactors to reuse shared resolver-metadata helper
src/lib/error.ts	Registers storage service errors for JSON deserialization
src/lib/encrypted-container.ts	Adds structured error type/codes and optional signing (v3) support
src/lib/encrypted-container.test.ts	Updates tests for buffer conversions + adds error/signing coverage
src/client/index.ts	Exposes Storage client entrypoint alongside other service clients

src/services/storage/server.ts

src/services/storage/client.ts

src/services/storage/common.ts

src/services/storage/server.ts

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 6 comments.

src/services/storage/server.ts

src/services/storage/lib/validators.ts

src/services/storage/client.ts

src/services/storage/server.ts

Copilot

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 5 comments.

src/services/storage/common.ts

src/services/storage/server.ts

src/lib/encrypted-container.ts

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 7 comments.

src/services/storage/drivers/memory.ts

src/services/storage/server.ts

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 1 comment.

src/services/storage/server.ts

Copilot

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 6 comments.

src/services/storage/server.ts

src/services/storage/client.ts

src/services/storage/drivers/memory.ts

src/lib/encrypted-container.ts

src/lib/resolver.ts

rkeene · 2026-01-28T03:03:38Z

package-lock.json

Why is this file changing while package.json is not in this changeset ?

rkeene · 2026-01-28T03:07:52Z

src/services/storage/common.ts

+ * Interface for atomic storage operations.
+ * Provides the same operations as StorageBackend but within an atomic scope.
+ */
+export interface StorageAtomicInterface {


This seems like it will be a VERY difficult interface to implement for a real world provider, and I'm not sure we need an atomic interface for multiple storage operations.

What's the thinking behind requiring every provider to take a snapshot of what a storage system looks like at a given moment so that a search doesn't return any newer documents, for example ?

Quota management was the original issue. You could exceed or be prevented from using the limit under certain circumstances.

It seems like there should be tests for this then ?

rkeene · 2026-01-28T03:14:42Z

src/services/storage/server.ts

+
+			// Validate encrypted container if needed
+			if (needsAnchorDecryption) {
+				if (!anchorAccount?.hasPrivateKey) {


We could check this at startup

rkeene · 2026-01-28T16:27:46Z

src/services/storage/server.ts

+		};
+
+		// GET /api/object - Retrieve an object
+		routes['GET /api/object'] = async function(_params, _postData, _headers, url) {


Ideally the resource would be part of the URL pathname and not as query parameter

routeMatch does not support wildcards. Suggestions?

rkeene · 2026-01-28T16:29:22Z

src/services/storage/server.ts

+				const signatureBuffer = Buffer.from(signature, 'base64');
+				const messageBuffer = Buffer.from(message, 'utf-8');
+
+				const validSig = ownerAccount.verify(


Why not use the utils/signing library ?

rkeene · 2026-01-28T16:29:54Z

src/services/storage/server.ts

+			}
+
+			// Decrypt using anchor account
+			if (!anchorAccount?.hasPrivateKey) {


Again, this could be checked at startup instead of at random places throughout the code

rkeene · 2026-01-28T16:40:06Z

src/services/storage/server.ts

+			}
+
+			// Decode base64 data to get actual stored object size
+			const data = Buffer.from(request.data, 'base64');


Why would the data we're asking to be stored come to us in base64 encoding ? It should be the body of the request which can be encoded safely in a more efficient way

rkeene · 2026-01-28T17:35:52Z

src/services/storage/client.test.ts

+	}) => Promise<StorageObjectMetadata>;
+}
+
+type ClientTestFn = (ctx: ClientTestContext) => Promise<void>;


We're not starving for space, we can use real words like context and Function when writing a type that is meant to describe what things are...

rkeene · 2026-01-28T17:37:48Z

src/services/storage/client.test.ts

+	const makePath = (relativePath: string): StoragePath =>
+		makeStoragePath(account.publicKeyString.get(), relativePath);


Suggested change

const makePath = (relativePath: string): StoragePath =>

makeStoragePath(account.publicKeyString.get(), relativePath);

const makePath = function(relativePath: string): StoragePath {

return(makeStoragePath(account.publicKeyString.get(), relativePath));

};

rkeene · 2026-01-28T17:39:43Z

src/services/storage/client.test.ts

+/**
+ * Helper to run a test with a fully configured client and provider.
+ */
+async function withClient(seed: string | ArrayBuffer, fn: ClientTestFn, options: WithClientOptions = {}): Promise<void> {


We don't need to describe the method we're calling as fn -- for one, we already know its type and shape an Fn (or as some might say, a function) from the type it can have a better name, one meant for humans who are the only ones who care about names.

rkeene · 2026-01-28T17:40:50Z

src/services/storage/client.test.ts

+
+// #endregion
+
+describe('Storage Client - Provider Discovery', () => {


Suggested change

describe('Storage Client - Provider Discovery', () => {

describe('Storage Client - Provider Discovery', function() {

rkeene · 2026-01-28T17:44:13Z

src/services/storage/client.test.ts

+		const path = makePath('test.txt');
+
+		// PUT
+		const putResult = await provider.put(path, testData, {


Maybe the client should have a beginSession() method with a given provider which lets you have a session that has some state like:

account

working directory

default visibility (optional)

sonarqubecloud · 2026-01-28T23:38:00Z

Quality Gate passed

Issues
39 New issues
0 Accepted issues

Measures
0 Security Hotspots
76.1% Coverage on New Code
3.1% Duplication on New Code

See analysis details on SonarQube Cloud

sephynox added 2 commits January 26, 2026 12:36

Encrypted container updates.

4f894c2

Storage anchor initial.

b39abd0

sephynox self-assigned this Jan 26, 2026

sephynox added 2 commits January 26, 2026 15:36

Merge branch 'main' of https://github.com/KeetaNetwork/anchor into fe…

71ffb3c

…at/storage-anchor

Fix test issues.

5da6d6c

sephynox added the enhancement New feature or request label Jan 26, 2026

sephynox added 2 commits January 26, 2026 15:44

Package lock issues.

89775f2

Fix lock file again...

4d9bd99

sephynox requested a review from Copilot January 26, 2026 23:51

Copilot started reviewing on behalf of sephynox January 26, 2026 23:51 View session

Copilot AI reviewed Jan 26, 2026

View reviewed changes

Fixes - first pass.

42bea5b

sephynox requested a review from Copilot January 27, 2026 00:57

Copilot started reviewing on behalf of sephynox January 27, 2026 00:58 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

Additional fixes.

ce779ad

sephynox requested a review from Copilot January 27, 2026 21:12

Copilot started reviewing on behalf of sephynox January 27, 2026 21:13 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

sephynox added 2 commits January 27, 2026 14:22

Atomic storage driver.

85c9276

Fixes.

8ead9d9

sephynox requested a review from Copilot January 27, 2026 22:31

Copilot started reviewing on behalf of sephynox January 27, 2026 22:31 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

More fixes.

2d7beb9

sephynox requested a review from Copilot January 27, 2026 23:01

Copilot started reviewing on behalf of sephynox January 27, 2026 23:01 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

src/services/storage/server.ts Outdated Show resolved Hide resolved

Add CORS.

ad5cb4a

sephynox requested a review from Copilot January 27, 2026 23:46

Copilot started reviewing on behalf of sephynox January 27, 2026 23:46 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

Add missing resolver properties and other fixes.

92eef54

sephynox marked this pull request as ready for review January 28, 2026 00:30

rkeene reviewed Jan 28, 2026

View reviewed changes

package-lock.json

Copy link

Member

rkeene Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this file changing while package.json is not in this changeset ?

rkeene reviewed Jan 28, 2026

View reviewed changes

rkeene requested changes Jan 28, 2026

View reviewed changes

sephynox added 3 commits January 28, 2026 11:26

PR comments first pass.

ea1155b

PR comments, second pass.

75d99ab

Require an anchor account.

a83aff6

		const makePath = (relativePath: string): StoragePath =>
		makeStoragePath(account.publicKeyString.get(), relativePath);


		// #endregion

		describe('Storage Client - Provider Discovery', () => {

Feature: Storage Anchor #164

Are you sure you want to change the base?

Feature: Storage Anchor #164

Uh oh!

Conversation

sephynox commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Changes

Encrypted Container (src/lib/encrypted-container.ts)

Storage Service (src/services/storage/)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

sephynox commented Jan 26, 2026 •

edited

Loading