Skip to content

Commit 3c6e472

Browse files
MasterPtatoMasterPtato
committed
chore: write/update docs
1 parent eb75b3a commit 3c6e472

File tree

10 files changed

+557
-3
lines changed

10 files changed

+557
-3
lines changed

‎CLAUDE.md‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,6 @@ cargo test -- --nocapture
4040

4141
# Check for linting issues
4242
cargo clippy -- -W warnings
43-
44-
# When adding a new package to the workspace
45-
deno run -A scripts/cargo/update_workspace.ts
4643
```
4744

4845
### Docker Development Environment

‎docs/engine/ACTOR_KV.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Actor KV Storage
2+
3+
Each actor has its own private KV store which can be manipulated or accessed with the various provided KV operations.
4+
5+
## Keys and Values
6+
7+
A KV key is a byte array, aka a blob. A KV value is also a byte array/blob.
8+
9+
Every set KV value contains metadata which includes the version and create timestamp of the key (version being a string byte array denoting the version of the Rivet Engine).
10+
11+
## Operations
12+
13+
### Get
14+
15+
- Input
16+
- List of keys
17+
- Output
18+
- List of keys
19+
- List of values
20+
- List of metadata
21+
22+
Keys that don't exist aren't included in the output so it is important to read the output's list of keys.
23+
24+
### List
25+
26+
- Input
27+
- Query mode
28+
- All - Lists all keys up to the given limit
29+
- Range - Lists all keys between the two given keys (exclusivity toggleable)
30+
- Prefix - Lists all keys with the given key as a prefix
31+
- Reverse - Whether to iterate keys in descending order instead of ascending
32+
- Limit - how maximum returned keys
33+
- Output
34+
- List of keys
35+
- List of values
36+
- List of metadata
37+
38+
### Put
39+
40+
- Input
41+
- List of keys
42+
- List of values
43+
- Output
44+
- Empty
45+
46+
### Delete
47+
48+
- Input
49+
- List of keys
50+
- Output
51+
- Empty
52+
53+
### Drop
54+
55+
- Input
56+
- Empty
57+
- Output
58+
- Empty
59+
60+
This operation deletes all keys in the entire actor's KV store. Use cautiously.
61+
62+
## Errors
63+
64+
Every operation can return an error instead of its regular response. The error includes a message string.
65+
66+
## Implementation Details
67+
68+
Each KV request has a u32 request ID which is to be provided by the user (handled internally by RivetKit). Rivet makes no attempt to order or deduplicate the responses to KV requests, it is up to the client to match the responses to the requests via the request ID.

‎docs/engine/ACTOR_LIFECYCLE.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Actor Lifecycle Flow Diagram
2+
3+
```mermaid
4+
---
5+
config:
6+
theme: mc
7+
look: classic
8+
---
9+
sequenceDiagram
10+
participant A as API
11+
participant U as User
12+
participant G as Gateway
13+
14+
participant R as Runner
15+
participant RWS as Runner WS
16+
participant RWF as Runner Workflow
17+
participant AWF as Actor Workflow
18+
19+
critical runner connection
20+
R->>RWS: Connect
21+
RWS->>RWF: Create runner workflow
22+
end
23+
24+
critical actor creation
25+
U->>A: POST /actors
26+
27+
A->>AWF: Create actor workflow
28+
A->>U:
29+
end
30+
31+
critical initial request
32+
U->>G: Request to actor
33+
note over G: Await actor ready
34+
35+
critical actor allocation
36+
note over AWF: Allocate
37+
38+
AWF->>RWF: Send StartActor
39+
40+
RWF->>RWS:
41+
RWS->>R:
42+
note over R: Start actor
43+
R->>RWS: Actor state update: Running
44+
RWS->>RWF:
45+
RWF->>AWF:
46+
note over AWF: Publish Ready msg
47+
end
48+
AWF->>G: Receive runner ID
49+
50+
G->>RWS: Tunnel ToClientRequestStart
51+
RWS->>R:
52+
note over R: Handle request
53+
R->>RWS: ToServerResponseStart
54+
RWS->>G:
55+
G->>U:
56+
end
57+
58+
critical second request
59+
U->>G: Request to actor
60+
note over G: Actor already connectable
61+
G->>RWS: Tunnel ToClientRequestStart
62+
RWS->>R:
63+
note over R: Handle request
64+
R->>RWS: ToServerResponseStart
65+
RWS->>G:
66+
G->>U:
67+
end
68+
69+
note over A, AWF: Time passes
70+
71+
critical actor sleep
72+
R->>RWS: Actor intent: Sleep
73+
RWS->>RWF:
74+
RWF->>AWF:
75+
note over AWF: Mark as sleeping
76+
AWF->>RWF: Send StopActor
77+
RWF->>RWS:
78+
RWS->>R:
79+
note over R: Stop actor
80+
R->>RWS: Actor state update: Stopped
81+
RWS->>RWF:
82+
RWF->>AWF:
83+
note over AWF: Sleep
84+
end
85+
86+
critical request to sleeping actor
87+
U->>G: Request to actor
88+
note over G: Actor sleeping
89+
G->>AWF: Wake
90+
note over G: Await actor ready
91+
critical actor allocation
92+
note over AWF: Allocate
93+
AWF->>RWF: Send StartActor
94+
RWF->>RWS:
95+
RWS->>R:
96+
note over R: Start actor
97+
R->>RWS: Actor state update: Running
98+
RWS->>RWF:
99+
RWF->>AWF:
100+
note over AWF: Publish Ready msg
101+
end
102+
AWF->>G: Receive runner ID
103+
104+
G->>RWS: Tunnel ToClientRequestStart
105+
RWS->>R:
106+
note over R: Handle request
107+
R->>RWS: ToServerResponseStart
108+
RWS->>G:
109+
G->>U:
110+
end
111+
```

‎docs/engine/GASOLINE.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Gasoline
2+
3+
Gasoline (at engine/packages/gasoline) is the durable execution engine running most persistent things on Rivet Engine.
4+
5+
Gasoline consists of:
6+
- Workflows - Similar to the concept of actors (not Rivet Actors) which can sleep (be removed from memory) when not in use
7+
- Signals - Facilitates intercommunication between workflow <-> workflow and other services (such as api) -> workflow
8+
- Messages - Ephemeral "fire-and-forget" communication between workflows -> other services
9+
- Activities - Individual steps in a workflow, each can be individually retried upon failure and "replayed" instead of re-executed with every workflow run
10+
- Operations - Thin wrappers around native rust functions. Provided for clean interop with the Gasoline ecosystem
11+

‎docs/engine/GUARD.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Rivet Guard
2+
3+
Guard facilitates HTTP communication between the public internet and various internal rivet services, as well as tunnelling connections to actors.
4+
5+
## Routing
6+
7+
Guard uses request path and/or headers to route requests.
8+
9+
### Actors
10+
11+
Guard routes requests to actors when:
12+
- the path matches `/gateway/{actor_id}/{...path}`
13+
- the path matches `/gateway/{actor_id}@{token}/{...path}`
14+
- when connecting a websocket, `Sec-Websocket-Protocol` consists of comma delimited dot separated pairs like `rivet_target.actor,rivet_actor.{actor_id}`
15+
- otherwise, when the `X-Rivet-Target` header is set to `actor` and `X-Rivet-Actor` header is set to the actor id
16+
17+
### Runners
18+
19+
Guard accepts runner websocket connections when:
20+
- the path matches `/runners/connect`
21+
- `Sec-Websocket-Protocol` consists of comma delimited dot separated pairs like `rivet_target.runner`
22+
23+
### API Requests
24+
25+
Guard routes requests to the API layer when the `X-Rivet-Target` header is set to `api-public` or is unset.
26+
27+
## Proxying (Gateway)
28+
29+
The Gateway (a portion of Guard) acts as a proxy for requests and websockets to actors:
30+
31+
- Internally, the websocket connects to a websocket listener running on the Rivet Engine
32+
- Rivet Engine transmits HTTP requests and websocket messages via the runner protocol to the actor's corresponding runner's websocket
33+
- The runner has a single websocket connection open to Guard which is independent from any client websocket connection
34+
- This single connection multiplexes all actor requests and websocket connections
35+
- The runner delegates requests and websockets to actors
36+
- The runner sends HTTP responses and websocket messages back to Rivet through is websocket via the runner protocol
37+
- Rivet transforms the runner protocol messages into HTTP responses and websocket messages
38+
39+
### Websocket Hibernation
40+
41+
The Gateway allows us to implement hibernatable websockets (see HIBERNATING_WS.md) for actors. We can keep a client's websocket connection open while simultaneously allowing for actors to sleep, resulting in 0 usage when there is no traffic over the websocket. The actor is automatically awoken when a websocket message is transmitted to the Gateway.

‎docs/engine/HIBERNATING_WS.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Hibernating Websockets
2+
3+
## Lifecycle
4+
5+
1. Client establishes a websocket connection to an actor via Rivet, which is managed by Guard (see GUARD.md)
6+
2. Guard checks to see if the actor is awake. If it is, skip step 3
7+
3. If the actor is not awake, send a Wake signal to its workflow. This will make the actor allocate to an existing runner, or in the case of serverless, start a new runner and allocate to that
8+
4. Guard sends the runner a ToClientWebSocketOpen message a via the runner protocol
9+
5. The runner sends back ToServerWebSocketOpen to acknowledge the connection
10+
- The runner must set `.canHibernate = true` for hibernation to work
11+
6. At this point the websocket connection is fully established and any websocket messages sent by the client are proxied through Guard to the runner to be delegated to the actor
12+
7. Should the actor go to sleep, the runner will close the websocket by sending ToServerWebSocketClose with `.hibernate = true` via the runner protocol
13+
8. Guard receives that the websocket has closed on the runner side and starts hibernating. During hibernation nothing happens.
14+
9.
15+
- If the actor is awoken from any other source, go to step 6. We do not send a ToClientWebSocketOpen message in this case
16+
- If the client sends a websocket message during websocket hibernation, go to step 2
17+
- If the client closes the websocket, the actor is rewoken (if not already running) and sent a ToClientWebSocketClose
18+
19+
## State
20+
21+
To facilitate state management on the runner side (specifically via RivetKit), each hibernating websocket runs a keepalive loop which periodically stores a value to UDB marking it as active.
22+
23+
When a client websocket closes during hibernation, this value is cleared.
24+
25+
When a runner receives a CommandStartActor message via the runner protocol, it contains information about which hiberating requests are still active.

‎docs/engine/RUNNER_SHUTDOWN.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Runner Shutdown
2+
3+
## 1. Self initiated shutdown
4+
5+
1. Runner sends ToServerStopping to runner WS
6+
2. Runner WS proxies ToServerStopping to runner WF
7+
3. Runner WF sets itself as "draining", preventing future actor allocations to it
8+
4. Runner WF sends GoingAway signal to all actor WFs
9+
5. Once the runner lost threshold is passed, runner WF sends ToClientClose to runner WS
10+
6. Runner WS closes connection to runner, informing it not to attempt reconnection
11+
12+
## 2. Rivet initiated shutdown
13+
14+
1. Runner WF receives Stop signal
15+
2. Runner WF sends GoingAway signal to all actor WFs
16+
3. Once the runner lost threshold is passed, runner WF sends ToClientClose to runner WS
17+
4. Runner WS closes connection to runner, informing it not to attempt reconnection

0 commit comments

Comments
 (0)