Serialize and restore ParsedChatRequest in chat sessions

[lucas-koehler]

What it does

Fixes #16561

Implement serialization of ParsedChatRequest data to preserve structured information (agent references, variable references, tool requests) across session persistence and restoration.

Previously, only raw text was preserved after session restoration, causing loss of parsed structure data. This affected UI rendering and prevented proper display of agent names, variables, and tool references in restored chat sessions.

Main code changes:

• Add serialization interfaces for ParsedChatRequest components in chat-model-serialization.ts (SerializableParsedRequestPart, SerializableToolRequest, SerializableResolvedVariable, etc.)
• Implement toSerializable() methods on ParsedChatRequestTextPart, ParsedChatRequestVariablePart, ParsedChatRequestFunctionPart, and ParsedChatRequestAgentPart classes
• Update MutableChatRequestModel to serialize/deserialize parsedRequest data via toSerializable() and deserializeParsedRequest() methods
• Serialize tool functions as id only and restore from the registry. For this, MutableChatModel and MutableChatRequestModel now require the tool registry as constructor parameter
• Add comprehensive test coverage for serialization/deserialization of all part types, including edge cases and complex mixed requests

16561-pr.webm

How to test

• Use the AI chat and chat using an agent and variable reference
• Restart theia
• Restore the session
• Observe: The agent, variable and tool function are now highlighted and provide additional information on hover

NOTE: While the serialization also supports functions/tool requests, highlighting them does not work yet in general. See #16722

Follow-ups

Breaking changes

• This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines
• User-facing text is internationalized using the nls service (for details, please see the Internationalization/Localization section in the Coding Guidelines)

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[sdirix]

High-level feedback:

• The ChatModel is restored via the ChatService. At the moment we already avoid handing in registries into the chat model by restoring registry-related data later, see here and here.
• Could we follow this pattern here too? This would overall be a simpler process and continue with the established pattern, avoiding this registry dependency

[lucas-koehler]

@sdirix That is a very good point! Thanks for catching that. I refactored the code accordingly to restore the tool requests from the registry in the chat service.
In the current refactoring, the chat service uses the registry directly as the deserializer registry is currently only meant for response content but tool requests are part of the request.
Is this okay with you or do you think we should extend the registry now to also support request content?

[sdirix]

I will review at the latest next week.

[sdirix]

Works for me and could be merged as is 👍

I have some comments, please have a look.

[sdirix]

I think if there is no id we should not restore it? And during storing we should make sure they have an id?

[lucas-koehler]

I think the problem lies earlier: variable ids are only available if the variable is resolved during serialization (see packages/ai-chat/src/common/parsed-chat-request.ts). Currently we serialize the variable in any case.
We could think about serializing unresolved variables as text parts but there could be value in knowing this is a variable even if it was not/could not be resolved at the time of serialization.

[lucas-koehler]

For now, I moved adding a fallback value of the id to the serialization function.

[lucas-koehler]

@sdirix I added serialization of the description. I also removed the 2 changelog commits during rebase: They cancelled each other and caused conflicts.
I also moved the initialization of the variable id fallback value to the serialization function of a variable part.
For now I left serializing and deserializing variable parts without an id. IMO the core question is whether we want to serialize variable parts with an unresolved variable at all. IMO this can be a follow up issue. WDYT?