Conversation
…ma-name chore: return schema
* Fix CI envs for system tests. * Update addresses env. * Remove address file env. * use this branch of cli. * use cli main branch. * Fix cli branch. * replace branch.
* routes and unit tests * unit and integration tests * add jwt * add jwt types * fix unit tests * refactor to handler * remove unused typesense schema * remove console logs * revert ddo file * refactor * use port 8000 tests * test with handler * revert aquarius * decorator used for valdiation * fix validation tests * support for new node instance * force refresh node * new instance test * fix bad enum * force refresh handler * cli tests paid compute * revert main cli * specify message in handler * remove duplicate routes * implement skipValidation params * override env tests * add header in handler * add docker comput envs in ci * reorder priority * add nonce * check nonce * name routes * env example and use config
…vent fix: update indexedMetadata event
| } | ||
| } catch (err) { | ||
| HTTP_LOGGER.error(err.message) | ||
| res.status(500).send(err.message) |
Check warning
Code scanning / CodeQL
Exception text reinterpreted as HTML Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 7 days ago
General approach: Avoid returning raw exception messages directly to HTTP clients in a way that can be rendered as HTML. Instead, send a generic, non-sensitive message and log the detailed error server-side. If the client needs details, prefer structured JSON with sanitized content, and explicitly set a non-HTML content type.
Best targeted fix here:
- In the
catchblock ofdirectCommandRoute.post, replaceres.status(500).send(err.message)with a safe, fixed response body (e.g., “Internal server error”) and optionally a JSON shape that does not expose internals. - Keep logging
err.message(or the full error) toHTTP_LOGGERso server operators still see the details. - Optionally, ensure we treat
errasunknownand coerce to string safely for logging, but avoid using that string in the response.
Concretely, in src/components/httpRoutes/commands.ts:
- Modify lines 136–139 (the
catchblock) to:- Type
errasanyorunknown(we can keepanyto avoid broader changes). - Log something like
HTTP_LOGGER.error(err instanceof Error ? err.message : String(err)). - Respond with
res.status(500).json({ error: 'Internal server error' })or a similar generic message, instead of sendingerr.message.
- Type
- No changes are needed in
validateCommands.tsfor this issue; it only affects log messages and standard 400 responses that are not derived from thrown exceptions.
This single change ensures all the alert variants—each concerned with err.message being sent—are addressed, because the exception text will no longer be reinterpreted as HTML in the client.
| @@ -134,8 +134,10 @@ | ||
| res.end() | ||
| } | ||
| } catch (err) { | ||
| HTTP_LOGGER.error(err.message) | ||
| res.status(500).send(err.message) | ||
| // Log detailed error on the server side | ||
| HTTP_LOGGER.error(err instanceof Error ? err.message : String(err)) | ||
| // Return a generic error message to the client to avoid exposing internal details | ||
| res.status(500).json({ error: 'Internal server error' }) | ||
| } | ||
| } | ||
| ) |
| if (!database || !database.logs) { | ||
| res.status(503).send('Logs database is not available') | ||
| return | ||
| } |
Check warning
Code scanning / CodeQL
Exception text reinterpreted as HTML Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 7 days ago
In general, to fix this type of vulnerability, you should avoid returning raw exception messages directly to the client, especially when they may contain user input. Instead, return a generic, constant error message to the client (e.g., “Internal Server Error”) and log the detailed error message server-side. If you do need to expose some error details, encode or escape the message appropriately for the output context (HTML, JSON, etc.).
For this specific code, the best minimal fix that doesn’t change core functionality is to stop concatenating error.message into the HTTP response and send a static error message instead, while still logging the detailed error via HTTP_LOGGER. This preserves current behavior from the client’s perspective (still a 500 with an “Internal Server Error” message) but removes the XSS vector. Concretely, in src/components/httpRoutes/logs.ts, in the /log/:id route’s catch block, replace res.status(500).send('Internal Server Error' + error.message) with res.status(500).send('Internal Server Error'). No new imports or helper methods are needed because detailed error info is already captured by HTTP_LOGGER.error.
| @@ -87,6 +87,6 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(`Error retrieving log: ${error.message}`) | ||
| res.status(500).send('Internal Server Error' + error.message) | ||
| res.status(500).send('Internal Server Error') | ||
| } | ||
| }) |
chore: sync 02-04
| const response = await axios({ | ||
| method: 'get', | ||
| url: input, | ||
| headers, | ||
| responseType: 'stream', | ||
| timeout: 30000 | ||
| }) |
Check failure
Code scanning / CodeQL
Server-side request forgery Critical
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
To fix the problem, user-controlled URLs must be validated/normalized before being used as the url of outgoing axios requests. In particular, the hostname (and optionally port) should be constrained to a safe allow‑list defined in configuration, and requests to private/loopback/multicast/link-local IP ranges should be rejected even if the hostname is allowed. This should be done inside the UrlStorage.validate and/or getDownloadUrl logic, because that is the central point through which all URL-based storages pass.
The best low-impact fix is:
-
Extend
UrlStorage.validateto:- Parse
file.urlusing the standardURLclass. - Ensure the protocol is
http:orhttps:(reject others). - If the node configuration (
this.config) defines a list of allowed URL patterns or explicit allowed hostnames (e.g.,allowedURLsorallowedHosts), validate the parsed URL against that allow‑list. Since we cannot assume new config fields, we can repurposeunsafeURLsmore safely by:- Optionally rejecting non-absolute URLs early.
- Adding a hard check that hostnames must not be obvious local addresses (e.g.
localhost,127.0.0.1) and that schemes are restricted.
- Resolve the hostname to an IP address and block private/loopback ranges if possible. However, DNS resolution requires additional libraries or async code; to keep this change minimal and synchronous inside
validate, we will at least block obvious internal hostnames/IPs by pattern.
- Parse
-
Strengthen
getDownloadUrlto returnnull(or throw) when validation fails, and ensureUrlStorage.getReadableStreamand the baseStorage.getReadableStreamhandle anullURL defensively by throwing an error instead of passingnull/undefinedintoaxios. -
Add a shared helper method in
UrlStorage(private static/instance) that performs host validation (e.g., rejectinglocalhost,127.*,10.*,192.168.*,172.16.*–172.31.*,::1) using simple regex checks. This lives in the same file and requires no new imports.
Concretely in src/components/storage/index.ts:
- Modify
UrlStorage.validate()to:- Parse the URL with
new URL(file.url). - Ensure protocol is
http:/https:. - Reject known internal hosts/IPs via a new private method
isPrivateHost.
- Parse the URL with
- Add that
isPrivateHost(url: URL)method toUrlStorage. - Modify
UrlStorage.getDownloadUrl()so that if validation fails, it throws instead of returningnull, to avoid accidental usage of an invalid URL. - Optionally, add an extra check in the base
Storage.getReadableStream()to throw ifinputis falsy, to guard any future subclass misuse.
These changes keep existing external behaviour (accepting valid public HTTP/HTTPS URLs) while adding strong guards against SSRF.
| @@ -235,6 +235,19 @@ | ||
| if (!['get', 'post'].includes(file.method?.toLowerCase())) { | ||
| return [false, 'Invalid method for URL'] | ||
| } | ||
| // Basic scheme and host validation to mitigate SSRF | ||
| let parsedUrl: URL | ||
| try { | ||
| parsedUrl = new URL(file.url) | ||
| } catch (e) { | ||
| return [false, 'Invalid URL format'] | ||
| } | ||
| if (parsedUrl.protocol !== 'http:' && parsedUrl.protocol !== 'https:') { | ||
| return [false, 'Invalid URL protocol, only HTTP/HTTPS are allowed'] | ||
| } | ||
| if (this.isPrivateHost(parsedUrl)) { | ||
| return [false, 'URL host is not allowed'] | ||
| } | ||
| if (this.config && this.config.unsafeURLs) { | ||
| for (const regex of this.config.unsafeURLs) { | ||
| try { | ||
| @@ -253,6 +266,29 @@ | ||
| return [true, ''] | ||
| } | ||
|
|
||
| // Detect obvious local/private hosts to reduce SSRF risk | ||
| private isPrivateHost(url: URL): boolean { | ||
| const hostname = url.hostname.toLowerCase() | ||
| // localhost and loopback | ||
| if (hostname === 'localhost' || hostname === '127.0.0.1' || hostname === '::1') { | ||
| return true | ||
| } | ||
| // IPv4 private ranges | ||
| if ( | ||
| /^10\./.test(hostname) || // 10.0.0.0 – 10.255.255.255 | ||
| /^192\.168\./.test(hostname) || // 192.168.0.0 – 192.168.255.255 | ||
| /^172\.(1[6-9]|2[0-9]|3[0-1])\./.test(hostname) || // 172.16.0.0 – 172.31.255.255 | ||
| /^169\.254\./.test(hostname) // link-local | ||
| ) { | ||
| return true | ||
| } | ||
| // Simple check for IPv6 private/link-local addresses | ||
| if (/^(fc00:|fd00:|fe80:)/i.test(hostname)) { | ||
| return true | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| isFilePath(): boolean { | ||
| const regex: RegExp = /^(.+)\/([^/]*)$/ // The URL should not represent a path | ||
| const { url } = this.getFile() | ||
| @@ -263,10 +299,11 @@ | ||
| } | ||
|
|
||
| getDownloadUrl(): string { | ||
| if (this.validate()[0] === true) { | ||
| const [isValid, message] = this.validate() | ||
| if (isValid === true) { | ||
| return this.getFile().url | ||
| } | ||
| return null | ||
| throw new Error(`Invalid download URL: ${message}`) | ||
| } | ||
|
|
||
| async fetchSpecificFileMetadata( |
| response.stream.pipe(res) | ||
| } else { | ||
| HTTP_LOGGER.error(response.status.error) | ||
| res.status(response.status.httpStatus).send(response.status.error) |
Check warning
Code scanning / CodeQL
Information exposure through a stack trace Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
In general, to fix this kind of issue, you should never send raw Error objects (or their stacks) back to the client. Instead, log the detailed error on the server and return a generic, user-friendly error message (optionally with a non-sensitive error code) to the client. This preserves observability for developers while avoiding exposing stack traces or implementation details.
For this specific file, the minimal, behavior-preserving change is to keep the current logging (HTTP_LOGGER.error(error.message)) and replace res.status(500).send(error) with a response that sends a safe, generic payload (string or JSON). Do this in each catch block:
- Around line 36: change
res.status(500).send(error)to something likeres.status(500).send('Internal server error')or a small JSON object such as{ error: 'Internal server error' }. - Around line 69: same replacement.
- Around line 101 (the line flagged by CodeQL): same replacement.
No new imports are strictly needed; Express already supports sending strings/JSON. We should also keep the logging as-is so error details remain available on the server side. This approach avoids altering existing business logic or handler behavior, only the shape of the error response.
| @@ -33,7 +33,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -66,7 +66,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -98,7 +98,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } |
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) |
Check warning
Code scanning / CodeQL
Information exposure through a stack trace Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
In general, the fix is to avoid sending the raw error object to the client and instead return a generic, user-safe error message while logging the full details on the server. This preserves observability for developers without leaking implementation details to users or attackers.
Concretely, in src/components/httpRoutes/policyServer.ts:
- Keep
HTTP_LOGGER.error(error.message)(or extend it to log the full error) to preserve diagnostics. - Replace each
res.status(500).send(error)with something likeres.status(500).send('Internal server error')or a minimal JSON{ error: 'Internal server error' }. - Optionally, tighten typings for the
errorparameter tounknownand narrow it before logging, but that is not strictly required to fix the information exposure.
The specific changes:
- Around lines 34–37: change
res.status(500).send(error)to send a generic message. - Around lines 67–70: same change.
- Around lines 99–102: same change.
No new imports are required; we only change the response payloads.
| @@ -33,7 +33,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -66,7 +66,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -98,7 +98,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } |
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) |
Check warning
Code scanning / CodeQL
Exception text reinterpreted as HTML Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
In general, the safest fix is to avoid sending raw exception objects or messages back to the client, especially when those messages may include user-controlled input. Instead, return a generic, constant error response and log the full error server‑side. If you do need to expose error details, you should at minimum HTML-encode or otherwise escape meta-characters before including them in any context that might be rendered as HTML.
For this specific code, the minimal, backward-compatible improvement is to change the catch blocks in src/components/httpRoutes/policyServer.ts so that they no longer call res.send(error). Instead, they should send a simple, fixed JSON object (or plain string) describing an internal error, and keep detailed information only in logs via HTTP_LOGGER.error. That way, even if the Error object or its message contains tainted data derived from req.caller, no user-controlled content is reflected in the HTTP response body. We should also avoid depending on the .message property if error might not be an Error instance; we can still keep the current logging pattern, but we will guard it with basic type checks to avoid runtime issues.
Concretely:
- In the first
PolicyServerPassthroughRoute.posthandler, replace:with something like:} catch (error) { HTTP_LOGGER.error(error.message) res.status(500).send(error) }
} catch (error) { const message = error instanceof Error ? error.message : String(error) HTTP_LOGGER.error(message) res.status(500).json({ error: 'Internal server error' }) }
- Do the same for the two subsequent
catchblocks in the same file. This preserves logging while ensuring the client only ever gets a static, non‑tainted message, eliminating the XSS vector.
No other files need to change for this fix.
| @@ -32,8 +32,9 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| const message = error instanceof Error ? error.message : String(error) | ||
| HTTP_LOGGER.error(message) | ||
| res.status(500).json({ error: 'Internal server error' }) | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -65,8 +66,9 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| const message = error instanceof Error ? error.message : String(error) | ||
| HTTP_LOGGER.error(message) | ||
| res.status(500).json({ error: 'Internal server error' }) | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -97,8 +99,9 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| const message = error instanceof Error ? error.message : String(error) | ||
| HTTP_LOGGER.error(message) | ||
| res.status(500).json({ error: 'Internal server error' }) | ||
| } | ||
| // res.sendStatus(200) | ||
| } |
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) |
Check warning
Code scanning / CodeQL
Exception text reinterpreted as HTML Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
In general, to fix this problem we should avoid sending raw exception objects or messages directly in HTTP responses. Instead, we should return a generic, non-sensitive error message to clients and log the detailed error server-side. If we must send details, they must be safely encoded/escaped and free of user-controlled content.
The best minimal fix here is to change the catch blocks in src/components/httpRoutes/policyServer.ts so that they:
- Log the full error using
HTTP_LOGGER.error(error)(orerror.message). - Send a generic string such as
'Internal server error'or a similarly safe message viares.status(500).send('Internal server error')instead of sending the rawerrorobject.
This preserves existing functionality in terms of status codes and logging, while removing the possibility that attacker-controlled content inside error will be reinterpreted as HTML by the browser. No changes are required to src/index.ts, src/components/core/handler/handler.ts, or src/components/core/handler/policyServer.ts for this specific sink; they can continue to pass caller through for rate limiting.
Concretely:
- In
src/components/httpRoutes/policyServer.ts, update the threecatch (error)blocks (for the/PolicyServerPassthroughroute and the two/initializePSVerificationroutes) so that line 36, 69, and 101 (theres.status(500).send(error)calls) instead send a constant, non-tainted message string. Optionally, also log the full error object instead of justerror.message. No new imports are needed.
| @@ -32,8 +32,8 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| HTTP_LOGGER.error(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -65,8 +65,8 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| HTTP_LOGGER.error(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -97,8 +97,8 @@ | ||
| res.status(response.status.httpStatus).send(response.status.error) | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| HTTP_LOGGER.error(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } |
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) |
Check warning
Code scanning / CodeQL
Information exposure through a stack trace Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 3 days ago
In general terms, the fix is to stop sending the raw error object (or other detailed stack/diagnostic data) in HTTP responses. Instead, we should log the error on the server (which is already being done) and return a sanitized, generic error message and/or an error code to the client.
The best minimal change here is to modify each catch block so that:
- We continue logging details with
HTTP_LOGGER.error(error.message)as already present. - We replace
res.status(500).send(error)with a generic string or simple JSON payload that does not include the error object or stack trace, such asres.status(500).send('Internal server error')or a small JSON{ error: 'Internal server error' }.
Concretely, in src/components/httpRoutes/policyServer.ts:
- Around line 36–37: change
res.status(500).send(error)tores.status(500).send('Internal server error'). - Around line 69–70: change
res.status(500).send(error)tores.status(500).send('Internal server error'). - Around line 101–102: change
res.status(500).send(error)tores.status(500).send('Internal server error').
No new methods or imports are required; we just adjust these response calls. This preserves the existing behavior (500 status on errors and server-side logging) while removing the information exposure.
| @@ -33,7 +33,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -66,7 +66,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } | ||
| @@ -98,7 +98,7 @@ | ||
| } | ||
| } catch (error) { | ||
| HTTP_LOGGER.error(error.message) | ||
| res.status(500).send(error) | ||
| res.status(500).send('Internal server error') | ||
| } | ||
| // res.sendStatus(200) | ||
| } |
enterprise node