diff --git a/A2UI_TECHNICAL_ARCHITECTURE.md b/A2UI_TECHNICAL_ARCHITECTURE.md new file mode 100644 index 00000000..91e2592d --- /dev/null +++ b/A2UI_TECHNICAL_ARCHITECTURE.md @@ -0,0 +1,350 @@ +# A2UI Technical Architecture Diagram + +## Complete System Flow + +```mermaid +graph TB + subgraph "Agent Side" + LLM[LLM/Agent
Gemini/Claude/etc] + A2A_EXT[A2A Extension
A2uiExtension.java] + JSON_GEN[A2UI JSON Generator
Declarative UI Description] + end + + subgraph "Transport Layer" + A2A_PROTO[A2A Protocol
JSONL Messages] + MSG_TYPES[Message Types:
• surfaceUpdate
• dataModelUpdate
• beginRendering
• deleteSurface] + end + + subgraph "Client: Message Processing" + MSG_REC[Message Receiver
Stream Reader] + PROC[A2uiMessageProcessor
Signal-based Model Builder] + SIGNALS[Reactive Signals
SignalArray/Map/Object/Set] + DATA_MODEL[Data Model
JSONPath-based] + end + + subgraph "Client: Rendering System" + ROOT[Root Component
a2ui-root] + REGISTRY[Component Registry
Custom Components] + RENDER[renderComponentTree
Component Tree Builder] + LIT[Lit Framework
Template Rendering] + end + + subgraph "Component System" + BASE[Base: Root Class
Extends LitElement] + BUILTIN[Built-in Components
Button, Card, Text, etc.] + CUSTOM[Custom Components
Registered via Registry] + TYPES[Type System
A2UITagNameMap] + end + + subgraph "Data Binding & Reactivity" + CONTEXT[Data Context Path
e.g., /reservation] + BINDING[Property Binding
Two-way Data Flow] + EFFECTS[Signal Effects
Auto Re-render] + WATCHER[SignalWatcher
Lit Integration] + end + + subgraph "Event System" + USER_INTERACT[User Interaction
Click, Input, etc.] + STATE_EVENT[StateEvent
a2ui.action] + EVENT_DISPATCH[Event Dispatch
to Client App] + AGENT_FEEDBACK[Agent Feedback
User Actions] + end + + subgraph "DOM Output" + HTML_OUT[HTML Elements
a2ui-button, etc.] + STYLES[Theme System
CSS Styling] + DOM[Browser DOM
Rendered UI] + end + + %% Agent Flow + LLM -->|Generates UI Intent| JSON_GEN + JSON_GEN -->|Creates A2UI JSON| A2A_EXT + A2A_EXT -->|Wraps in A2A Part| A2A_PROTO + + %% Transport + A2A_PROTO -->|JSONL Stream| MSG_TYPES + MSG_TYPES -->|Messages| MSG_REC + + %% Message Processing + MSG_REC -->|Process Messages| PROC + PROC -->|Builds Reactive Model| SIGNALS + SIGNALS -->|Stores Data| DATA_MODEL + + %% Rendering + DATA_MODEL -->|Component Tree| ROOT + ROOT -->|Looks up Components| REGISTRY + REGISTRY -->|Custom Components| CUSTOM + ROOT -->|Standard Components| RENDER + RENDER -->|Creates Templates| LIT + + %% Component System + LIT -->|Instantiates| BASE + BASE -->|Inherits from| BUILTIN + BASE -->|Can use| CUSTOM + BUILTIN -->|Type-checked via| TYPES + CUSTOM -->|Registered in| REGISTRY + + %% Data Binding + DATA_MODEL -->|Reads from| CONTEXT + CONTEXT -->|Binds to| BINDING + BINDING -->|Updates via| EFFECTS + EFFECTS -->|Watched by| WATCHER + WATCHER -->|Triggers| LIT + + %% Events + DOM -->|User Clicks| USER_INTERACT + USER_INTERACT -->|Creates| STATE_EVENT + STATE_EVENT -->|Dispatches| EVENT_DISPATCH + EVENT_DISPATCH -->|Sends to| AGENT_FEEDBACK + AGENT_FEEDBACK -.->|Updates UI| LLM + + %% DOM Output + LIT -->|Renders| HTML_OUT + HTML_OUT -->|Styled by| STYLES + STYLES -->|Displays in| DOM + + style LLM fill:#e1f5ff + style PROC fill:#fff4e1 + style ROOT fill:#e8f5e9 + style DATA_MODEL fill:#f3e5f5 + style DOM fill:#ffebee +``` + +## Component Rendering Pipeline + +```mermaid +sequenceDiagram + participant Agent + participant Transport + participant Processor + participant Root + participant Component + participant Lit + participant DOM + + Agent->>Transport: Generate A2UI JSON + Note over Agent: surfaceUpdate message
with components array + + Transport->>Processor: Stream JSONL Messages + Processor->>Processor: Parse Messages + Processor->>Processor: Build Signal-based Model + + Processor->>Root: Set childComponents (Signal) + Root->>Root: renderComponentTree() + + alt Custom Component + Root->>Root: Check componentRegistry + Root->>Component: Instantiate Custom Element + else Standard Component + Root->>Root: Switch on component.type + Root->>Component: Create Lit Template + end + + Component->>Lit: html`` + Lit->>DOM: Render to DOM + + DOM->>Component: User Interaction + Component->>Root: Dispatch StateEvent + Root->>Transport: Send Event to Agent + Transport->>Agent: User Action Data +``` + +## Data Binding Flow + +```mermaid +graph LR + subgraph "Agent Updates Data" + AGENT_UPDATE["Agent sends
dataModelUpdate"] + PATH["Path: /reservation/guests"] + VALUE["Value: 2"] + end + + subgraph "Signal System" + SIGNAL_UPDATE[Signal Updates
Reactive Model] + SUBSCRIBE[Components Subscribe
to Data Paths] + end + + subgraph "Component Rendering" + READ_DATA[Component Reads
from Context Path] + RENDER_UPDATE[Lit Re-renders
Affected Components] + end + + subgraph "User Interaction" + USER_CHANGE[User Changes Value
in TextField] + WRITE_DATA[Component Writes
to Data Model] + SIGNAL_NOTIFY[Signal Notifies
Subscribers] + end + + AGENT_UPDATE -->|Updates| PATH + PATH -->|Sets| VALUE + VALUE -->|Triggers| SIGNAL_UPDATE + SIGNAL_UPDATE -->|Notifies| SUBSCRIBE + SUBSCRIBE -->|Reads| READ_DATA + READ_DATA -->|Updates| RENDER_UPDATE + + USER_CHANGE -->|Modifies| WRITE_DATA + WRITE_DATA -->|Updates| SIGNAL_NOTIFY + SIGNAL_NOTIFY -->|Triggers| RENDER_UPDATE + + style AGENT_UPDATE fill:#e1f5ff + style SIGNAL_UPDATE fill:#fff4e1 + style RENDER_UPDATE fill:#e8f5e9 + style WRITE_DATA fill:#f3e5f5 +``` + +## Component Type System + +```mermaid +graph TD + subgraph "Type Definitions" + TAG_MAP[A2UITagNameMap
Type-safe Tag Mapping] + COMP_TYPES[Component Types
Button, Card, Text, etc.] + CUSTOM_TYPES[Custom Types
Registered Components] + end + + subgraph "Component Classes" + ROOT_CLASS[Root Class
Base Component] + BUILTIN_CLASS[Built-in Classes
Button, Card, etc.] + CUSTOM_CLASS[Custom Classes
User-defined] + end + + subgraph "Registration" + REGISTRY["ComponentRegistry
Type to Constructor Map"] + REGISTER["register Function
Type-safe Registration"] + end + + subgraph "Instantiation" + INSTANCE["instanceOf Function
Tag to Instance"] + CTOR["Constructor Lookup
customElements.get"] + NEW["new Constructor
Type-safe Creation"] + end + + TAG_MAP -->|Maps| COMP_TYPES + TAG_MAP -->|Extends| CUSTOM_TYPES + COMP_TYPES -->|Implemented by| BUILTIN_CLASS + CUSTOM_TYPES -->|Implemented by| CUSTOM_CLASS + BUILTIN_CLASS -->|Extends| ROOT_CLASS + CUSTOM_CLASS -->|Extends| ROOT_CLASS + + CUSTOM_CLASS -->|Registered via| REGISTER + REGISTER -->|Stores in| REGISTRY + REGISTRY -->|Looked up by| INSTANCE + INSTANCE -->|Finds| CTOR + CTOR -->|Creates| NEW + + style TAG_MAP fill:#e1f5ff + style REGISTRY fill:#fff4e1 + style INSTANCE fill:#e8f5e9 +``` + +## Message Processing Details + +```mermaid +graph TB + subgraph "Input: JSONL Messages" + MSG1["surfaceUpdate message"] + MSG2["dataModelUpdate message"] + MSG3["beginRendering message"] + end + + subgraph "A2uiMessageProcessor" + PARSE[Parse JSON Messages] + VALIDATE[Validate Schema] + BUILD[Build Component Map] + DATA_BUILD[Build Data Model] + end + + subgraph "Signal-based Storage" + COMP_MAP["Component Map
id to ComponentNode"] + DATA_STORE["Data Store
SignalMap/Object"] + SURFACE_MAP["Surface Map
surfaceId to Surface"] + end + + subgraph "Output: Reactive Model" + GET_SURFACES[getSurfaces
Returns Surface Map] + GET_COMPONENTS[getComponents
Returns Component Array] + GET_DATA[getData
Returns Signal-based Data] + end + + MSG1 -->|Process| PARSE + MSG2 -->|Process| PARSE + MSG3 -->|Process| PARSE + + PARSE -->|Validate| VALIDATE + VALIDATE -->|If surfaceUpdate| BUILD + VALIDATE -->|If dataModelUpdate| DATA_BUILD + + BUILD -->|Store| COMP_MAP + DATA_BUILD -->|Store| DATA_STORE + BUILD -->|Store| SURFACE_MAP + + COMP_MAP -->|Accessed via| GET_COMPONENTS + DATA_STORE -->|Accessed via| GET_DATA + SURFACE_MAP -->|Accessed via| GET_SURFACES + + style PARSE fill:#e1f5ff + style DATA_STORE fill:#fff4e1 + style GET_SURFACES fill:#e8f5e9 +``` + +## Event Flow: User Interaction to Agent + +```mermaid +sequenceDiagram + participant User + participant Button + participant Root + participant EventSystem + participant ClientApp + participant Transport + participant Agent + + User->>Button: Clicks Button + Button->>Button: @click Handler + Button->>Root: Create StateEvent + Note over Button,Root: eventType: a2ui.action
dataContextPath: /reservation
sourceComponentId: btn1 + + Root->>EventSystem: dispatchEvent(StateEvent) + EventSystem->>ClientApp: Event Listener Receives + ClientApp->>ClientApp: Extract Action Data + ClientApp->>ClientApp: Include Data Context + + ClientApp->>Transport: Send to Agent + Note over ClientApp,Transport: A2A Message with
User Action Data + + Transport->>Agent: Deliver Action + Agent->>Agent: Process User Action + Agent->>Agent: Generate Response + + Agent->>Transport: Send Updated UI + Transport->>ClientApp: A2UI Messages + ClientApp->>Root: Update Component Tree + Root->>Button: Re-render if Needed +``` + +## Key Technical Concepts + +### 1. **Declarative UI Generation** +- Agent generates JSON describing UI intent, not implementation +- Components are referenced by type (e.g., "Button") not code +- Client maps types to native implementations + +### 2. **Reactive Data Model** +- Signal-based reactivity for efficient updates +- JSONPath-like data context paths +- Automatic re-rendering when data changes + +### 3. **Type Safety** +- TypeScript ensures tag names map to correct classes +- Component properties are type-checked +- Custom components follow interface contracts + +### 4. **Extensibility** +- Component registry for custom components +- Framework-agnostic (same JSON works across renderers) +- Open catalog system for component definitions + +### 5. **Security** +- No code execution from agent +- Only pre-approved components can be rendered +- Data binding is declarative, not executable