Skip to content

Conversation

@nchapagain001
Copy link
Contributor

@nchapagain001 nchapagain001 commented Oct 15, 2025

This PR adds new default behavior in Virtual Client. Virtual Client will split its output for process telemetry logging to handle cases where standard output and standard error exceed Event Hub’s 1 MB message size limit.

Background

Event Hub enforces a 1 MB limit on telemetry events. Previously, when process outputs exceeded the configured character threshold (default 125,000 chars), Virtual Client would truncate the data, resulting in incomplete logs that made debugging/reporting difficult.

Changes

• Added a default behavior to split StandardOutput and StandardError in VirtualClientLoggingExtensions.LogProcessDetails.
• When StandardOutput and StandardError does not exceed maxchar threshold (125000), it will output everything in a single event.

• When StandardOutput and StandardError's combined length exceed maxchar threshold (125000),
• StandardOutput will split into multiple chunked telemetry events instead of being truncated.
• StandardError will split into multiple chunked telemetry events instead of being truncated.
• Each chunk is emitted as its own *.ProcessDetails event.
• Each event still carries the same OperationId/ExperimentId so chunks can be reassembled downstream.

Reconstructing chunked output in KQL

If you enable TelemetrySplit and want to reconstruct the original StandardOutput from chunked telemetry, the following Kusto query shows a way to do that:

// KQL Query
// Purpose: Reconstructs chunked standard output emitted by ExecuteCommandMonitor when TelemetrySplit is enabled.
// - Step 1: Filter to the relevant ProcessDetails events for the profile.
// - Step 2: Order chunks by standardOutputChunkPart.
// - Step 3: Concatenate the per-chunk standardOutput values back into a single string.
Traces_Dev01
| where Timestamp > now(-5d) and Message == "ExecuteCommandMonitor.ProcessDetails"
| where ProfileName == "MONITORS-DEFAULT1.json" //For testing only.
// Step 1: ensure proper ordering of chunks
| sort by toint(CustomDimensions.standardOutputPart) asc
// Step 2: assemble the full standardOutput per operation/experiment
| summarize
    count(), standardOutput_Final = array_strcat(make_list(tostring(CustomDimensions.process.standardOutput)), ""), take_any(*) by OperationId, ExperimentId
// Step 3: project useful fields + reconstructed output
| project Timestamp, ExperimentId, Message, OperationId, ProfileName,count_, stringLength = strlen(standardOutput_Final)//, standardOutput_Final
image

Notes:

• standardOutputChunkPart is only present on chunked events; for non-chunked events, the existing single-event behavior remains unchanged.
• A similar pattern can be used for StandardError with standardErrorChunkPart if needed.

@nchapagain001 nchapagain001 marked this pull request as draft October 23, 2025 21:05
@nchapagain001 nchapagain001 marked this pull request as ready for review December 11, 2025 23:02
@nchapagain001 nchapagain001 requested a review from brdeyo December 11, 2025 23:33
@nchapagain001 nchapagain001 merged commit 57fcce3 into main Dec 13, 2025
5 checks passed
@nchapagain001 nchapagain001 deleted the users/nchapagain/splittingStandardStreams branch December 13, 2025 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants