Python: Bug: Streaming replies error – final ResponseOutputMessage missing .delta on large runs

[ltwlf]

Describe the bug
Invoking an AzureResponseAgent that returns a function‑calling answer whose total context grows to many tokens (exact threshold still unclear) causes the run to terminate with

AttributeError: 'ResponseOutputMessage' object has no attribute 'delta'

To Reproduce

1. Use the latest main branch of Semantic Kernel.
2. Configure an Azure OpenAI GPT‑4 model that supports tool/function calling (e.g. gpt‑4o).
3. Create an AzureResponseAgent with at least one function/tool; keep all defaults.
4. Prompt the agent so that the response invokes the tool and the combined context becomes very large (e.g. ask for a deep recursive JSON structure).
5. Observe the run end with the AttributeError above.

Expected behavior
The agent run should finish cleanly and deliver the complete assistant message (including tool_calls) without raising an exception, regardless of response length.

Screenshots
N/A – console traceback available upon request.

Platform

• Language: Python 3.13.3
• Source: main branch
• AI model: Azure OpenAI gpt‑4.1
• IDE: VS Code
• OS: Windows 11 

Additional context

• Bug has surfaced in two separate apps under active development.
• Root cause seems related to OpenAI’s switch from streaming …MessageDeltaEvent objects to a final ResponseOutputMessage (which lacks .delta) for very large tool‑calling completions—but the precise size boundary still needs investigation.

[ltwlf]

Digging a bit deeper

Root cause – Since March 2025 the OpenAI/Azure Responses API appends a terminal object of type ResponseOutputMessage to every streamed run.

This object exposes its payload in .content and deliberately has no .delta.

By contrast, the incremental events SK was written for (ResponseTextDeltaEvent, ResponseMessageDeltaEvent, etc.) all carry their chunk under .delta.

Why we hit it mostly with tool calls – Whenever the model returns a tool_calls array, the SDK is required (per the Assistants spec) to emit the consolidated ResponseOutputMessage so you get a valid, schema-complete assistant message — see the “Final message” note in the official Assistants streaming docs. Plain-text replies may skip that step when the answer is short, which is why non-tool completions often survive.

Trigger pattern – Any long response (many tokens) or any function-calling response, because both reliably produce the final ResponseOutputMessage.

Effect in SK – In responses_agent_thread_actions.py the stream loop still assumes every event owns .delta; when the last object doesn’t, Python raises
AttributeError: 'ResponseOutputMessage' object has no attribute 'delta'.

Impact – Hard stop: the completed assistant message (and any tool_calls) never reaches callers, breaking both user UX and planner logic.

I’m putting together a concise fix and will open a PR shortly. Just wanted to document the investigation so everyone sees it’s a general streaming issue rather than something unique to function calls.