Open
Conversation
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
The responses API wraps user text in OpenAI multimodal array format
([{"type":"text","text":"..."}]) but Near.AI E2EE only encrypted
string content, leaving array content as plaintext. Near.AI's model node
then failed to decrypt with 'Failed to decrypt field'.
Added try_flatten_text_content_array() to detect text-only arrays and
flatten them to plain strings before encryption. Genuinely multimodal
content (with images) is still left as-is.
Also added debug logging across the E2EE pipeline and new live
integration tests (hyper-flow, streaming, multi-turn, repeated).
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds Near.AI integration: attestation (Intel TDX + NVIDIA NRAS), client-side E2EE (ECIES/HKDF/AES-GCM), NearAiVerifier with caching and periodic checks, wiring into ProxyRouter/AppState and request/response flows, startup proxy host/forwarders, appended PCR history entries, and new crypto/attestation dependencies. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Proxy as Proxy Server
participant NearAI as NearAI Cloud API
participant PCCS as Intel PCCS
participant NRAS as NVIDIA NRAS
participant JWKS as JWKS Endpoint
Client->>Proxy: POST /chat/completions (NearAI model)
Proxy->>Proxy: generate nonce
Proxy->>NearAI: GET /attestation?nonce=...
NearAI-->>Proxy: AttestationReport (TDX quote, nvidia_payload)
rect rgba(100, 150, 200, 0.5)
Note over Proxy,PCCS: TDX verification
Proxy->>PCCS: Fetch collateral / TCB info
PCCS-->>Proxy: Collateral / TCB
Proxy->>Proxy: Verify TDX quote, extract report_data
end
rect rgba(150, 100, 200, 0.5)
Note over Proxy,NRAS: GPU attestation
Proxy->>NRAS: POST /verify (nvidia_payload, nonce)
NRAS-->>Proxy: JWT response
Proxy->>JWKS: GET JWKS
JWKS-->>Proxy: JWKs
Proxy->>Proxy: Verify NRAS JWT (ES384, kid, nonce, verdict)
end
Proxy->>Proxy: prepare_e2ee_request (derive ephemeral keys, encrypt body)
Proxy->>NearAI: POST /chat/completions (encrypted body + x-client-pub-key headers)
NearAI-->>Proxy: Streamed/response (encrypted)
Proxy->>Proxy: decrypt_chat_completion_json_in_place (AES-GCM via shared secret)
Proxy-->>Client: Decrypted chat response
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord. Comment |
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…separate chunks Near.AI sends finish_reason and usage on separate SSE chunks, unlike Tinfoil/vLLM which combine them. Track stream_finished state across chunks so billing fires when usage arrives after finish_reason. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Avoid verification failures if Near upgrades to TD15 quotes. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
NVIDIA NRAS may return the attestation verdict as either a boolean or a string. Previously only the boolean form was handled, which would silently reject a string "true" and fail model node verification. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
… leaks in logs Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Comment on lines
+699
to
+701
| nearai_verifier | ||
| .clone() | ||
| .spawn_periodic_verification(vec!["zai-org/GLM-5-FP8".to_string()]); |
There was a problem hiding this comment.
📝 Info: Periodic verification hardcodes model name, creating coupling with proxy_config
The spawn_periodic_verification call at src/main.rs:699-701 hardcodes "zai-org/GLM-5-FP8", while the canonical-to-provider mapping lives in proxy_config.rs:72 (get_model_name_for_provider). If someone adds a new Near.AI model or renames the mapping in proxy_config.rs, they must also update the hardcoded list in main.rs. There's no single source of truth for which Near.AI models should be pre-verified. Consider deriving the pre-verification list from the routing table.
Was this helpful? React with 👍 or 👎 to provide feedback.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@src/nearai/attestation.rs`:
- Around line 31-50: The code currently only enforces TDX status checks when
tdx_require_up_to_date() is true, but per Intel DCAP we must unconditionally
reject "REVOKED" and "OUT_OF_DATE"; modify the post-verify_tdx handling in the
attestation flow (around verify_tdx and the subsequent checks on verified.status
/ verified.platform_status / verified.qe_status) to first check if
verified.status == "REVOKED" or verified.status == "OUT_OF_DATE" and immediately
return Err(NearAiError::Tdx(...)) with a clear message including verified.status
and advisory_ids, and then keep the existing conditional check for
tdx_require_up_to_date() to enforce UpToDate when configured.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix: require UpToDate TDX status by default Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> fix: require UpToDate TDX status Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…ssing plaintext Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/nearai/e2ee.rs (1)
85-97: Avoid redundant clone when encrypting string content.The
s.clone()on line 87 is unnecessary sinces.as_bytes()can be called directly on the borrowed string.♻️ Proposed fix
Value::String(s) => { let plaintext_len = s.len(); - let plaintext = s.clone(); - let encrypted_hex = encrypt_ecies_hex(plaintext.as_bytes(), &model_pubkey)?; + let encrypted_hex = encrypt_ecies_hex(s.as_bytes(), &model_pubkey)?; trace!( "Near.AI E2EE: encrypted messages[{}] role={} plaintext_len={} ciphertext_hex_len={}", i, role, plaintext_len, encrypted_hex.len() );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/nearai/e2ee.rs` around lines 85 - 97, The Value::String arm currently does an unnecessary clone (s.clone()) before encryption; instead keep the borrowed string `s`, set `plaintext_len = s.len()`, and pass `s.as_bytes()` directly into `encrypt_ecies_hex` (the call site of encrypt_ecies_hex) and assign the resulting hex into `*content_val`; remove the `plaintext` local clone to eliminate the redundant allocation while leaving the logging and `encrypted_count` update unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@src/nearai/e2ee.rs`:
- Around line 223-244: The reviewer confirms the newline join behavior in
try_flatten_text_content_array is acceptable and no code change is required;
leave the function as-is (the function try_flatten_text_content_array should
still validate array elements are objects with "type" equal to "text" or
"input_text", extract "text" strings, and join them with "\n").
---
Nitpick comments:
In `@src/nearai/e2ee.rs`:
- Around line 85-97: The Value::String arm currently does an unnecessary clone
(s.clone()) before encryption; instead keep the borrowed string `s`, set
`plaintext_len = s.len()`, and pass `s.as_bytes()` directly into
`encrypt_ecies_hex` (the call site of encrypt_ecies_hex) and assign the
resulting hex into `*content_val`; remove the `plaintext` local clone to
eliminate the redundant allocation while leaving the logging and
`encrypted_count` update unchanged.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/nitro-deploy.md`:
- Around line 1352-1393: The headings "Near.AI Cloud API", "NVIDIA NRAS (GPU
Attestation)" and the following "Intel PCS (DCAP Collateral)" are using '####'
which creates a two-level jump under the enclosing '##' section; change those
headers from '####' to '###' so they are proper subsections (update the lines
containing "Near.AI Cloud API", "NVIDIA NRAS (GPU Attestation)", and "Intel PCS
(DCAP Collateral)" to use '###' heading markers) to restore correct hierarchy.
- Around line 1341-1399: The fenced code blocks in this section are missing
language tags (triggers MD040); update the blocks around the three address list,
the restart command, and each systemd unit snippet
(vsock-near-cloud-api-proxy.service, vsock-near-nras-proxy.service,
vsock-near-intel-pcs-proxy.service) by adding appropriate language identifiers
(e.g., yaml for the address list, sh for the sudo systemctl restart command, and
ini for the systemd unit files) so tooling and rendering treat them correctly.
Comment on lines
+1341
to
+1399
| ``` | ||
| - {address: cloud-api.near.ai, port: 443} | ||
| - {address: nras.attestation.nvidia.com, port: 443} | ||
| - {address: api.trustedservices.intel.com, port: 443} | ||
| ``` | ||
|
|
||
| Restart the nitro vsock proxy service: | ||
| ``` | ||
| sudo systemctl restart nitro-enclaves-vsock-proxy.service | ||
| ``` | ||
|
|
||
| #### Near.AI Cloud API | ||
| ```sh | ||
| sudo vim /etc/systemd/system/vsock-near-cloud-api-proxy.service | ||
| ``` | ||
|
|
||
| Add the following content: | ||
| ``` | ||
| [Unit] | ||
| Description=Vsock Near.AI Cloud API Proxy Service | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| User=root | ||
| ExecStart=/usr/bin/vsock-proxy 8042 cloud-api.near.ai 443 | ||
| Restart=always | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target | ||
| ``` | ||
|
|
||
| #### NVIDIA NRAS (GPU Attestation) | ||
| ```sh | ||
| sudo vim /etc/systemd/system/vsock-near-nras-proxy.service | ||
| ``` | ||
|
|
||
| Add the following content: | ||
| ``` | ||
| [Unit] | ||
| Description=Vsock NVIDIA NRAS Proxy Service | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| User=root | ||
| ExecStart=/usr/bin/vsock-proxy 8043 nras.attestation.nvidia.com 443 | ||
| Restart=always | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target | ||
| ``` | ||
|
|
||
| #### Intel PCS (DCAP Collateral) | ||
| ```sh | ||
| sudo vim /etc/systemd/system/vsock-near-intel-pcs-proxy.service | ||
| ``` | ||
|
|
||
| Add the following content: | ||
| ``` | ||
| [Unit] |
There was a problem hiding this comment.
Add language tags to new fenced code blocks.
The new fenced blocks in this section omit language identifiers, which triggers MD040 and makes rendering/tooling less reliable. Add appropriate tags (e.g., sh, ini).
Proposed change
-```
+```yaml
- {address: cloud-api.near.ai, port: 443}
- {address: nras.attestation.nvidia.com, port: 443}
- {address: api.trustedservices.intel.com, port: 443}...
- +sh
sudo systemctl restart nitro-enclaves-vsock-proxy.service
...
-```
+```ini
[Unit]
Description=Vsock Near.AI Cloud API Proxy Service
After=network.target
[Service]
User=root
ExecStart=/usr/bin/vsock-proxy 8042 cloud-api.near.ai 443
Restart=always
[Install]
WantedBy=multi-user.target
...
- +ini
[Unit]
Description=Vsock NVIDIA NRAS Proxy Service
After=network.target
[Service]
User=root
ExecStart=/usr/bin/vsock-proxy 8043 nras.attestation.nvidia.com 443
Restart=always
[Install]
WantedBy=multi-user.target
...
-```
+```ini
[Unit]
Description=Vsock Intel PCS Proxy Service
After=network.target
[Service]
User=root
ExecStart=/usr/bin/vsock-proxy 8044 api.trustedservices.intel.com 443
Restart=always
[Install]
WantedBy=multi-user.target
</details>
<details>
<summary>🧰 Tools</summary>
<details>
<summary>🪛 markdownlint-cli2 (0.21.0)</summary>
[warning] 1341-1341: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
---
[warning] 1348-1348: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
---
[warning] 1352-1352: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4
(MD001, heading-increment)
---
[warning] 1358-1358: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
---
[warning] 1378-1378: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
---
[warning] 1398-1398: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
</details>
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
Verify each finding against the current code and only fix it if needed.
In @docs/nitro-deploy.md around lines 1341 - 1399, The fenced code blocks in
this section are missing language tags (triggers MD040); update the blocks
around the three address list, the restart command, and each systemd unit
snippet (vsock-near-cloud-api-proxy.service, vsock-near-nras-proxy.service,
vsock-near-intel-pcs-proxy.service) by adding appropriate language identifiers
(e.g., yaml for the address list, sh for the sudo systemctl restart command, and
ini for the systemd unit files) so tooling and rendering treat them correctly.
</details>
<!-- fingerprinting:phantom:triton:eagle -->
<!-- This is an auto-generated comment by CodeRabbit -->
Comment on lines
+1352
to
+1393
| #### Near.AI Cloud API | ||
| ```sh | ||
| sudo vim /etc/systemd/system/vsock-near-cloud-api-proxy.service | ||
| ``` | ||
|
|
||
| Add the following content: | ||
| ``` | ||
| [Unit] | ||
| Description=Vsock Near.AI Cloud API Proxy Service | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| User=root | ||
| ExecStart=/usr/bin/vsock-proxy 8042 cloud-api.near.ai 443 | ||
| Restart=always | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target | ||
| ``` | ||
|
|
||
| #### NVIDIA NRAS (GPU Attestation) | ||
| ```sh | ||
| sudo vim /etc/systemd/system/vsock-near-nras-proxy.service | ||
| ``` | ||
|
|
||
| Add the following content: | ||
| ``` | ||
| [Unit] | ||
| Description=Vsock NVIDIA NRAS Proxy Service | ||
| After=network.target | ||
|
|
||
| [Service] | ||
| User=root | ||
| ExecStart=/usr/bin/vsock-proxy 8043 nras.attestation.nvidia.com 443 | ||
| Restart=always | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target | ||
| ``` | ||
|
|
||
| #### Intel PCS (DCAP Collateral) | ||
| ```sh |
There was a problem hiding this comment.
Fix heading level jump (use ### under the ## section).
#### Near.AI Cloud API introduces a two‑level jump from ##, which breaks heading hierarchy. Switch these #### headers to ### in this section.
Proposed change
-#### Near.AI Cloud API
+### Near.AI Cloud API
...
-#### NVIDIA NRAS (GPU Attestation)
+### NVIDIA NRAS (GPU Attestation)
...
-#### Intel PCS (DCAP Collateral)
+### Intel PCS (DCAP Collateral)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| #### Near.AI Cloud API | |
| ```sh | |
| sudo vim /etc/systemd/system/vsock-near-cloud-api-proxy.service | |
| ``` | |
| Add the following content: | |
| ``` | |
| [Unit] | |
| Description=Vsock Near.AI Cloud API Proxy Service | |
| After=network.target | |
| [Service] | |
| User=root | |
| ExecStart=/usr/bin/vsock-proxy 8042 cloud-api.near.ai 443 | |
| Restart=always | |
| [Install] | |
| WantedBy=multi-user.target | |
| ``` | |
| #### NVIDIA NRAS (GPU Attestation) | |
| ```sh | |
| sudo vim /etc/systemd/system/vsock-near-nras-proxy.service | |
| ``` | |
| Add the following content: | |
| ``` | |
| [Unit] | |
| Description=Vsock NVIDIA NRAS Proxy Service | |
| After=network.target | |
| [Service] | |
| User=root | |
| ExecStart=/usr/bin/vsock-proxy 8043 nras.attestation.nvidia.com 443 | |
| Restart=always | |
| [Install] | |
| WantedBy=multi-user.target | |
| ``` | |
| #### Intel PCS (DCAP Collateral) | |
| ```sh | |
| ### Near.AI Cloud API |
🧰 Tools
🪛 markdownlint-cli2 (0.21.0)
[warning] 1352-1352: Heading levels should only increment by one level at a time
Expected: h3; Actual: h4
(MD001, heading-increment)
[warning] 1358-1358: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
[warning] 1378-1378: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/nitro-deploy.md` around lines 1352 - 1393, The headings "Near.AI Cloud
API", "NVIDIA NRAS (GPU Attestation)" and the following "Intel PCS (DCAP
Collateral)" are using '####' which creates a two-level jump under the enclosing
'##' section; change those headers from '####' to '###' so they are proper
subsections (update the lines containing "Near.AI Cloud API", "NVIDIA NRAS (GPU
Attestation)", and "Intel PCS (DCAP Collateral)" to use '###' heading markers)
to restore correct hierarchy.
…-deploy Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…tion Intel get_collateral and NVIDIA verify_gpu_attestation calls had no retry handling, unlike fetch_attestation_report which already retried 3 times. Transient network failures (e.g. vsock proxy not ready at boot) caused immediate verification failure with a 10-minute wait for recovery. Also use alternate Display format for get_collateral errors to log the full error chain instead of just the top-level message. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
…aves dcap-qvl's get_collateral builds its own reqwest client with hickory-dns enabled, which bypasses /etc/hosts and requires a real DNS server. Nitro enclaves have no DNS -- all hosts resolve via /etc/hosts to local vsock traffic forwarders. Add nearai/collateral.rs that replicates the HTTP fetching logic from dcap-qvl 0.3.12 using a plain reqwest client. All cryptographic verification still uses dcap_qvl::verify::ring::verify unchanged. Also add certificates.trustedservices.intel.com vsock proxy for the Intel root CA CRL distribution point used by Intel PCS. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Most production TDX deployments lag behind Intel's latest firmware updates, resulting in OutOfDate or SWHardeningNeeded TCB status. Accept these statuses (the quote is still cryptographically valid) and log a warning with the advisory IDs. Only Revoked is rejected. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit