Refactor C2D docs. by mariacarmina · Pull Request #1513 · oceanprotocol/docs

mariacarmina · 2025-05-12T18:19:13Z

Fixes #1515 .

Changes proposed in this PR:

adding sequence flow diagrams for free + paid compute
explainin the entire free and paid flow diagrams
refactor existing documentation for c2d
create documentation for escrow contract
update code example for c2d in ocean.js

…xists.

giurgiur99

Some small changes 🙏

giurgiur99 · 2025-05-23T06:14:16Z

.gitbook/assets/vscode/setup.png

Can we update with latest screenshot without paid resources? Thanks.

giurgiur99 · 2025-05-23T06:16:04Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+In the C2D workflow, the following steps are performed:
+
+1. The consumer initiates a compute-to-data job by selecting the desired data asset and algorithm, and then, the orders are validated via the dApp used.
+2. A dedicated and isolated execution pod is created for the C2D job.


Container not pod for now

giurgiur99 · 2025-05-23T06:17:11Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+3. The execution pod loads the specified algorithm into its environment.
+4. The execution pod securely loads the selected dataset for processing.
+5. The algorithm is executed on the loaded dataset within the isolated execution pod.
+6. The results and logs generated by the algorithm are securely returned to the user.


We can mention that we use a web3 auto-generated/custom PK and this is how we ensure private access to results

we can mention the signature generation that it is required on get results endpoint.

giurgiur99 · 2025-05-23T06:18:00Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+
+Now, let's delve into the inner workings of the Provider. Initially, it verifies whether the Consumer has sent the appropriate datatokens to gain access to the desired data. Once validated, the Provider interacts with the Operator-Service, a microservice responsible for coordinating the job execution. The Provider submits a request to the Operator-Service, which subsequently forwards the request to the Operator-Engine, the actual compute system in operation.
+
+The Operator-Engine, equipped with functionalities like running Kubernetes compute jobs, carries out the necessary computations as per the requirements. Throughout the computation process, the Operator-Engine informs the Operator-Service of the job's progress. Finally, when the job reaches completion, the Operator-Engine signals the Operator-Service, ensuring that the Provider receives notification of the job's successful conclusion.


No kubernetes for now

This is old stack, will remove these lines.

giurgiur99 · 2025-05-23T06:19:02Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `GetComputeEnvironments` - returns list of environments that can be selected to run the algorithm on
+- `InitializeCompute` - generates provider fees necessary for asset's ordering
+- `FreeStartCompute` - runs algorithms without necessary publish the assets on-chain (dataset and algorithm), using free resources from the selected environment
+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.


startCompute

COMPUTE_START: 'startCompute',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:19:33Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `FreeStartCompute` - runs algorithms without necessary publish the assets on-chain (dataset and algorithm), using free resources from the selected environment
+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.
+- `ComputeGetStatus` - retrieves compute job status.
+- `ComputeStop` - stops compute job execution when the job is `Running`.


COMPUTE_STOP: 'stopCompute',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:19:44Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+- `PaidStartCompute` - runs algorithms with on-chain assets (dataset and algorithm), using paid resources from the selected environment. The payment is requested at every start compute call, being handled by `Escrow` contract.
+- `ComputeGetStatus` - retrieves compute job status.
+- `ComputeStop` - stops compute job execution when the job is `Running`.
+- `ComputeGetResult` - returns compute job results when job is `Finished`.


COMPUTE_GET_RESULT: 'getComputeResult',

This list represents the handlers from Ocean Node, not the command names

giurgiur99 · 2025-05-23T06:20:25Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+
+One of its responsibility revolves around fetching and preparing the required assets and files, ensuring a smooth and seamless execution of the job. By meticulously handling the environment configuration, the **C2D Engine** guarantees that all necessary components are in place, setting the stage for a successful job execution.
+
+1. **Fetching Dataset Assets**: It downloads the files corresponding to datasets and saves them in the location `/data/inputs/DID/`. The files are named based on their array index ranging from 0 to X, depending on the total number of files associated with the dataset.


Dataset can be did/url/arweave/ipfs

And also need to specify for the algorithm as well, good catch!

giurgiur99 · 2025-05-23T06:21:30Z

developers/compute-to-data-v2.0/free-compute-to-data-flow.md

+
+## Prerequisites
+
+The prerequisite for this flow is the algorithm code which can be input for consumers components: Ocean CLI and it is open for integration with other systems (e.g. Ocean Enterprise Marketplace).


CLI and vscode extension, might be useful to add links

There are linked in Setup , but I will link as well in the Prerequisites, thank you.

giurgiur99 · 2025-05-23T06:24:26Z

developers/vscode/README.md

New dataset here will be url/did/ipfs/arweave.

We will now use the c2d_examples not algo_dockers repo

giurgiur99

Some small comments. Thanks!

giurgiur99 · 2025-07-17T06:15:23Z

developers/compute-to-data-v2.0/README.md

Broken links for Architecture & Overview Guides

I referenced statically .md files for each section, in md works fine. How does it look for you?

giurgiur99 · 2025-07-17T06:18:44Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

PaidStartCompute -> startCompute
ComputeGetStatus -> getComputeStatus
ComputeStop -> stopCompute
ComputeGetResult -> getComputeResult

The rest should be lowercase to follow the names from P2P handler

Here I targeted only handler names which are valid for both protocols as specified "which includes handlers for operations that can be called via HTTP and P2P protocols". If we want to ref commands and HTTP endpoint names, we can link the README for Ocean Node API.md.
I can add the link for it and keep the handler names as they are currently described in the documentation.

The rest should be lowercase to follow the names from P2P handler

The handlers (which include the core functionality) are used for both protocols in the code.

giurgiur99 · 2025-07-17T06:20:02Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

We don't use kubernetes at the moment, therfore no pods

I specified that currently only Docker engine is supported and in the future we can extend to support Kubernetes as well.

They are still called pods in some place here in the .md. Thanks!

giurgiur99 · 2025-07-17T06:25:29Z

developers/vscode/README.md

Can we add a screenshot from the latest version of vscode extension? Also for now in extension we only use the rawAlgorithm and send the algo in the request, no did, ipfs, ...

Yes, sure, I have added 2 screenhots for VS code extension and updated algorithms supported formats. Thanks!

giurgiur99 · 2025-07-21T05:28:58Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

+
+1. The consumer initiates a compute-to-data job by selecting the desired data asset and algorithm, and then, the orders are validated via the dApp used.
+2. A dedicated and isolated execution container is created for the C2D job.
+3. The execution pod loads the specified algorithm into its environment.


Yep but here and below they are still called "pods". Thanks!

giurgiur99 · 2025-07-21T05:30:03Z

developers/compute-to-data-v2.0/compute-to-data-architecture.md

They are still called pods in some place here in the .md. Thanks!

ndrpp · 2025-10-08T11:22:22Z

Whilst checking the docs in the onboarding phase I found these links from the docs that don't work:

FAQ page discover/faq.md, Data marketplaces & Ocean Market, https://autobotocean.com website returns 502
user-guides/wallets/README.md and data-scientists/ocean.py/remote-setup.md: https://oceanprotocol.com/token - link returns 404
https://oceanprotocol.com/templates grammar mistake "Buidl using.."

C2D refactor for docs.

26d91a8

mariacarmina self-assigned this May 12, 2025

mariacarmina added 5 commits May 22, 2025 11:00

Updated architecture document for C2D v2. Updated flow diagrams.

df3ae44

detail free compute flow.

79f2f76

Updated flow diagrams and finished detailing free compute flow.

9decebd

Add content for paid compute detailed flow.

feb2b83

Finished paid compute flow. Added introduction for escrow contract.

56dd872

mariacarmina marked this pull request as ready for review May 22, 2025 20:28

mariacarmina added 2 commits May 22, 2025 23:49

Update escrow content. Deeleted compute file from ocean.js, already e…

46d8d8d

…xists.

Updated old compute example for ocean.js with the paid compute version.

7cbd13a

mariacarmina changed the title ~~C2D refactor for docs.~~ Refactor C2D docs. May 22, 2025

giurgiur99 reviewed May 23, 2025

View reviewed changes

mariacarmina added 7 commits May 29, 2025 07:51

Add API specification for compute stack from Ocean Node.

4790083

Updated payloads.

b972e97

Added payloads examples.

14a4c8b

Fix review.

6e6f9c5

Updated formats for ddo.

c739649

Add links.

1ab0b90

Fix tweaks.

f4e5812

mariacarmina mentioned this pull request Jul 16, 2025

Updated C2D main page.md #1521

Open

mariacarmina added 2 commits July 16, 2025 14:53

Merge branch 'main' into feature/c2d-docs

de48cd2

Updated compute options regarding trusted algorithms.

95a5ab4

giurgiur99 reviewed Jul 17, 2025

View reviewed changes

mariacarmina added 2 commits July 19, 2025 15:42

Mention Ocean Node's API.md.

ad47c67

Updated vscode screenshots and description for algorithms.

13ae634

giurgiur99 approved these changes Oct 8, 2025

View reviewed changes


		Now, let's delve into the inner workings of the Provider. Initially, it verifies whether the Consumer has sent the appropriate datatokens to gain access to the desired data. Once validated, the Provider interacts with the Operator-Service, a microservice responsible for coordinating the job execution. The Provider submits a request to the Operator-Service, which subsequently forwards the request to the Operator-Engine, the actual compute system in operation.

		The Operator-Engine, equipped with functionalities like running Kubernetes compute jobs, carries out the necessary computations as per the requirements. Throughout the computation process, the Operator-Engine informs the Operator-Service of the job's progress. Finally, when the job reaches completion, the Operator-Engine signals the Operator-Service, ensuring that the Provider receives notification of the job's successful conclusion.


		One of its responsibility revolves around fetching and preparing the required assets and files, ensuring a smooth and seamless execution of the job. By meticulously handling the environment configuration, the C2D Engine guarantees that all necessary components are in place, setting the stage for a successful job execution.

		1. Fetching Dataset Assets: It downloads the files corresponding to datasets and saves them in the location `/data/inputs/DID/`. The files are named based on their array index ranging from 0 to X, depending on the total number of files associated with the dataset.


		## Prerequisites

		The prerequisite for this flow is the algorithm code which can be input for consumers components: Ocean CLI and it is open for integration with other systems (e.g. Ocean Enterprise Marketplace).

Conversation

mariacarmina commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giurgiur99 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

giurgiur99 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndrpp commented Oct 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mariacarmina commented May 12, 2025 •

edited

Loading