feat: Address review comments on Guard/GuardrailProvider CRDs

[Copilot]

• Fix: Add runtime.agentic-layer.ai_guardrailproviders.yaml and runtime.agentic-layer.ai_guards.yaml to config/crd/kustomization.yaml so they are installed during make install (fixes e2e test failures)
• API changes: use corev1.ObjectReference for AiGateway.spec.guardrails and Guard.spec.providerRef
• API changes: add ApiKeySecretRef, ExternalUrl fields to GuardrailProvider; embed corev1.ObjectReference in GuardrailBackendRef
• Updated sample YAMLs and added e2e test for GuardrailProvider/Guard
• Regenerated CRDs and deepcopy functions

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Introduce Guardrails</issue_title>
<issue_description># Guardrails Concept

---

What is a Guardrail?

• A mechanism that checks inputs and outputs against specific criteria
• When anomalies are detected, the call can be stopped, reported, and/or modified
• Guardrails can be deployed in various components of an agentic platform:
  • AI Gateway (Model Router): Check LLM calls
  • Agent Gateway: Check incoming requests to agents or outgoing final responses from agents
  • Tool Gateway: Check data passed from LLMs/agents to tools
• Examples of Guardrails
  • PII Protection: Detects when personal data is present
  • Secrets
  • Toxic Language
  • More Guardrails: https://guardrailsai.com/hub

sequenceDiagram
    autonumber
    participant A as Agent
    box DarkGreen Guardrails
      participant IV as InputGuard
      participant OF as OutputGuard
    end
    participant MR as AI Gateway (Model Router)

    A->>IV: Raw input
    activate IV
    IV->>MR: Validated and filtered input
    deactivate IV

    activate MR
    MR-->>OF: Raw output
    deactivate MR

    activate OF
    OF-->>A: Validated and filtered output
    deactivate OF

Loading

How is a Guardrail defined?

• Guardrails consist of multiple Guards
• There are InputGuards and OutputGuards
• For a request, there is exactly one Guardrail, consisting of a list of Guards. The list should be processed in order.
• A Guard can return a modified (sanitized) input
• A Guard can throw an error that aborts the LLM call
  • TODO: How do we cleanly return the error?
• Input and output is always text for now, possibly streamed

Solution Approaches

Proxy

graph LR

      Agent <--> Guardrails
      Guardrails <--> MR

    Guardrails["AI Gateway
    (Guardrail Proxy)"]
		MR["AI Gateway
		(Model Router)"]

    %% Styling
    style Agent fill:#fff4e1
    style MR fill:#ffe1f5
    style Guardrails fill:#D2FCD6

Loading

• Separate deployment instances for separate responsibilities (Single Responsibility Principle)
• Independent of the gateway implementation used
• Dependent on the protocol
• Additional resource usage
• More complex infrastructure

Possible Implementations in Kubernetes

• Separate reverse proxy in front of the gateway (own deployment)
  • This is how we already implemented the Agent Gateway (KrakenD-based)
• Sidecar container
  • This is how solutions like Envoy do it
  • A reverse proxy is still needed, but it runs alongside each container (Agent? or next to a Gateway?)
  • Resource usage is correspondingly high if a container runs for each agent
  • Challenge: If a service mesh is installed simultaneously that also wants to use sidecar containers, things get complicated
  • Consideration: If such a solution is targeted, one could also build on Envoy. This is how kgateway does it.
  • One could potentially also build on kgateway. agentgateway integrates there (though they are not fully independent of each other)
• eBPF filter -> not suitable, as processing the entire request/response body cannot easily run in the kernel due to limited resources

Gateway-Integrated

Using the AI Gateway as an example: Guardrails are configured within the AI Gateway.

graph LR

      Agent <--> MR

		MR[AI Gateway]

    %% Styling
    style Agent fill:#fff4e1
    style MR fill:#ffe1f5

Loading

• There is one AI Gateway (KrakenD/LiteLLM/...) with one config for model routing and guardrails
• Simpler debugging and maintenance; Lower complexity for users; No network hop
• High dependency on the specific implementation. Requires that the AI Gateway supports guardrails and that they can be configured as generically as possible.

Existing Solutions

Guardrails

• https://github.com/guardrails-ai/guardrails
• https://protectai.github.io/llm-guard/get_started/quickstart/

Gateways with Guardrail Support

• LiteLLM - https://docs.litellm.ai/docs/proxy/guardrails/quick_start
• agentgateway - Add Guardrails: Centrally configure and prevent harmful outcomes and ensure safe user interactions agentgateway/agentgateway#378
• LiteLLM Docs Guardrails (configured via config file)
• Traefik AI Gateway Docs Guardrails (configured via CRD)
• Example Kong AI Gateway Guardrail Config (configured via HTTP request)
• PortKey Guardrail Config Docs (configured via HTTP request)

Implementation

Requirements

• Guardrails from various vendors / libraries can be used
  • We explicitly do not want to (only) offer our own guardrails, but rather build on existing solutions that can be integrated
• Guardrails / Guards can be defined via CRDs independently of the impleme...

• Fixes Introduce Guardrails #61

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[g3force]

@copilot the e2e test fail now, fix them

[Copilot]

@copilot the e2e test fail now, fix them

Fixed in commit 5d379a0. The Guard and GuardrailProvider CRD YAML files were generated but missing from config/crd/kustomization.yaml, so make install didn't install them and the e2e tests failed with "no matches for kind GuardrailProvider". Added both entries to the kustomization.