Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 63 additions & 18 deletions adaptors/collections.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ It is designed for high performance over a large volume of data.

Collections are secure, private datastores which are only accessible to
Workflows within a particular OpenFn Project. They can be created, managed and
destroyed from the OpenFn Admin page.
destroyed from the OpenFn Settings page.

The Collections Adaptor provides an interface to workflows to use the
Collections API.
Expand Down Expand Up @@ -278,28 +278,69 @@ fn(state => {

## CLI usage

Workflows which use Collections can be run through the CLI. You will need to:
Since version 1.21.0 (Jan 2026), `@openfn/cli` fully supports the use of
collections in workflows.

- Get a Personal Access Token (PAT)
- Update the `workflow.json` with your PAT and the OpenFn endpoint
- Set the step to use the Collections adaptor
Prior to 1.21.0, special setup was needed. So this guide is split into two
sections: we recommend using the latest CLI and following first guide below.
It's shorter, cleaner and easier.

But if you're unable to update your CLI, you can still configure collections
manually with the [Older CLI guide](#older-cli).

Remember that a Collection must be created from the Settings page before it can
be used!

:::tip

You can also call the Collections API directly from the CLI, without using a
Workflow. See the [CLI Collections Guide](/documentation/collections-cli)

:::

### Latest CLI 1.20+

The CLI includes full native support for collections within a workflow.

If you've pulled a project from the app (Lightning), all you'll need to do is
provide an API Token when you run the workflow. You can either do this by
passing the `--api-key` argument, or by setting the `OPENFN_API_KEY` env var.

```bash
openfn my-workflow.json --api-key $MY_OPENFN_PAT
```

By default, the CLI will use our cloud platform at app.openfn.org to find any
collections referenced in the workflow. To use a different deployment server,
you can pass `--endpoint` or set the `OPENFN_ENDPOINT` env var. Note that this
should point to the server root - so `http://app.openfn.org`, not
`http://app.openfn.org/collections`.

:::tip

You can also call the Collections API directly from the CLI. See the
[CLI Collections Guide](/documentation/collections-cli)
Since v1.19.0 The CLI fully supports `.env` files. If you run multiple projects
locally, you may want to create a `.env` file in your project root and configure
your endpoint and API token in there:

```
OPENFN_API_KEY="eyJhbGc..."
OPENFN_ENDPOINT="https://app.openfn.org"
```

The `.env` file will be automatically loaded by the CLI. Check debug output for
details.

:::

Collections are designed for close integration with the platform app, but can be
used from the CLI too.
### Older CLI

You can get a Personal Access Token from any v2 deployment.
The rest of this guide only applies when using older CLI versions, prior to
1.21.0.

Remember that a Collection must be created from the Admin page before it can be
used!
Remember that a Collection must be created from the Settings page before it can
be used!

### For a single job
#### For a single job

You can pass multiple adaptors from the CLI:

Expand All @@ -318,7 +359,7 @@ You'll need to set configuration on the state.json:
}
```

### For a workflow
#### For a workflow

If you're using `workflow.json`, set the token and endpoint on
`workflow.credentials`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong key, this should be workflow.configuration

Expand All @@ -327,7 +368,7 @@ If you're using `workflow.json`, set the token and endpoint on
{
"workflow": {
"steps": [ ... ],
"credentials": {
"configuration": {
"collections_endpoint": "http://localhost:4000/collections",
"collections_token": "...paste your Personal Access Token..."
}
Expand All @@ -350,8 +391,12 @@ And make sure that any steps which use collections have multiple adaptors set:
}
```

### I've noticed a problem with this Adaptor, or something is out of date, what can I do?
## Contributing

Thanks for asking! We are a fully Open Source Digital Public Good, and we welcome contributions from our community. Check out our [Adaptors Wiki](https://github.com/OpenFn/adaptors/blob/main/wiki/index.md) for more information on how you can update Adaptors!
We are a fully Open Source Digital Public Good, and we welcome contributions
from our community. Check out our
[Adaptors Wiki](https://github.com/OpenFn/adaptors/blob/main/wiki/index.md) for
more information on how you can update Adaptors!

Or, you can always reach out to the Community through our [Community Forum here](https://community.openfn.org/).
Or, you can always reach out to the Community through our
[Community Forum here](https://community.openfn.org/).