Skip to content

Build a WASM-compatible object_store alternative #41

@johnbchron

Description

@johnbchron

We need to be able to access arbitrary object storage from a Cloudflare worker.

Our access pattern for the crate object_store can be found in the first-party storage crate.

Basically, we use a models::StorageCredentials to create some struct that can access the storage that our StorageCredentials are semantically pointing to. Sometimes that's local storage, in which object_store is not involved, but in the case of R2, we do use object_store. Right now, we use get() with streaming, multipart methods, and head() for testing.

The problem is that object_store use reqwest, which uses sockets for networking, i.e. the normal approach. In WASM, we can't really do that. Usually you use browser APIs, but we don't have those either. We have the Fetch API in the worker crate. This is non-standard partially because they couldn't provide browser APIs, and partially because they have to translate it back to JS and send it back through their proxy network as an outbound request.

I think it would be best if you could fork object_store to allow plugging in a different request backend. If reqwest has native support for this, that's the optimal situation. If not, it may work to make the interface sub out reqwest, though that might breach the complexity barrier. If not that, you could just recreate a minified version of object_store in the Nika repo.

Metadata

Metadata

Assignees

Labels

campaignNecessary for the long-term goal of the project

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions