-
Notifications
You must be signed in to change notification settings - Fork 71
添加评估器-优化器模式实现 #34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
添加评估器-优化器模式实现 #34
Changes from 51 commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
2ba64e6
feat: add vector store redis simple
sunknightzy 6b92b99
feat: add README.md
sunknightzy 4cf14a6
feat: optimiz README.md
sunknightzy 736ddfb
add vector store redis simple
glmapper 8081276
feat: build_agent_with_workflow
Gepeng18 db90529
Merge pull request #20 from Gepeng18/feature/build_agent_with_workflow
spring-ai-tech 2cb6ea9
checkstyle: update RouterSelectorWorkflowStep class
Gepeng18 ff0fc6d
Merge pull request #21 from Gepeng18/checkstyle/build_agent_with_work…
glmapper aae73e5
Update README.md
spring-ai-tech b7d75bc
chat memory readme
glmapper 74666ec
update readme
glmapper 0630cc1
Merge pull request #22 from java-ai-tech/update-readme
spring-ai-tech 357c4ca
update readme
glmapper bf72fce
update readme
glmapper 753d20d
update readme
glmapper 37bab47
update readme_en
glmapper 9dad16d
add doc for tool-calling and evaluation
glmapper 38f94dd
add doc for milvus
glmapper 2d8d766
add orchestrator workflow
glmapper 8e5b2b3
add orchestrator workflow readme
glmapper 6947c6d
add blod link with https://www.readme-i18n.com/FareedKhan-dev/all-rag…
glmapper 7d00e68
add ChatMemoryController for quick test
glmapper f9ed8be
start history
glmapper a3007d6
支持使用 MariaDB 作为向量库 (#23)
siyuan-sy b703f1d
update contact information
glmapper 53da17e
update contact information
glmapper d201b9e
add Spring AI 框架原理与实战
glmapper 3c69cf7
show visitor
glmapper d918e6b
show visitor
glmapper 11a7b93
fix(spring-ai-chat-qwen): 修复 API 密钥配置问题
liu-657667 aa32453
add vedio 马克的技术工作坊
glmapper ea1282a
fix(spring-ai-chat-qwen): 修复 API 密钥配置问题 (#26)
liu-657667 028cd4a
Merge branch 'java-ai-tech:main' into main
liu-657667 1986033
feat(spring-ai-observability): 添加可观察性模块
liu-657667 bdef12a
Merge remote-tracking branch 'origin/main'
liu-657667 5251783
refactor(spring-ai-observability-metric): 优化 image 控制器的实现
liu-657667 c1aa8c6
docs(spring-ai-observability): 更新快速启动指南中的配置文件路径
liu-657667 126c4ce
feat(spring-ai-observability): 添加可观察性模块 (#28)
liu-657667 0338c85
fix mcp-sample bugs (#30)
hyz1994121 383cbc8
Merge branch 'java-ai-tech:main' into main
liu-657667 54a026c
feat(spring-ai-mcp): 添加 Nacos MCP 客户端和服务端模块
liu-657667 d06c48e
refactor(nacos): 简化 Nacos 应用代码结构
liu-657667 469bf75
refactor(nacos-mcp-client): 优化 OpenAI API 密钥配置
liu-657667 d1ee647
refactor(nacos): 修正应用主类名称
liu-657667 2113451
添加 Nacos MCP 客户端和服务端模块 (#32)
liu-657667 63028f2
Merge branch 'java-ai-tech:main' into main
liu-657667 d046b72
add spring-ai-alibaba-repository
glmapper 379fee4
fix issue 33
glmapper 8818948
Merge branch 'java-ai-tech:main' into main
liu-657667 0051d4c
Merge remote-tracking branch 'origin/main'
liu-657667 7f88499
feat(agent): 添加评估器-优化器模式实现
liu-657667 905583d
Merge branch 'java-ai-tech:main' into main
liu-657667 9d2ea74
refactor(spring-ai-agent): 重构 evaluator-optimizer 模块
liu-657667 a129a92
Merge remote-tracking branch 'origin/main'
liu-657667 d92bcb9
docs:移除 CLAUDE.md 文件
liu-657667 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
205 changes: 205 additions & 0 deletions
205
spring-ai-agent/spring-ai-evaluator-optimizer/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| # Spring AI 评估器-优化器 智能体 | ||
|
|
||
| 这个模块实现了**评估器-优化器模式**,通过生成、评估和优化循环对AI生成的解决方案进行迭代改进。 | ||
|
|
||
| ## 快速开始 🚀 | ||
|
|
||
| ### 1. 启动应用 | ||
| ```bash | ||
| cd spring-ai-evaluator-optimizer | ||
| mvn spring-boot:run | ||
| ``` | ||
|
|
||
| 应用将在端口 8084 启动。 | ||
|
|
||
| ### 2. 测试健康检查 | ||
| ```bash | ||
| curl http://localhost:8084/api/evaluator-optimizer/health | ||
| ``` | ||
|
|
||
| ### 3. 基本任务求解示例 | ||
| ```bash | ||
| curl -X POST "http://localhost:8084/api/evaluator-optimizer/solve?task=创建一个计算斐波那契数列的Java方法" | ||
| ``` | ||
|
|
||
| ### 4. 高级任务求解示例 | ||
| ```bash | ||
| curl -X POST "http://localhost:8084/api/evaluator-optimizer/solve-advanced" \ | ||
| -d "task=创建一个用户管理的REST API端点" \ | ||
| -d "context=Spring Boot应用,使用Spring Security和JPA" \ | ||
| -d "criteria=必须包含输入验证、错误处理和合适的HTTP状态码" \ | ||
| -H "Content-Type: application/x-www-form-urlencoded" | ||
| ``` | ||
|
|
||
| ## 概述 | ||
|
|
||
| 评估器-优化器模式适用于以下场景: | ||
| - 有明确评估标准的任务 | ||
| - 迭代改进能提供可衡量价值的场景 | ||
| - 需要多轮评估和改进的复杂任务 | ||
|
|
||
| ## 架构设计 | ||
|
|
||
| ### 核心组件 | ||
|
|
||
| 1. **GeneratorService(生成器服务)**: 生成初始解决方案并基于反馈进行改进 | ||
| 2. **EvaluatorService(评估器服务)**: 对生成的解决方案进行批判性评估和评分 | ||
| 3. **EvaluatorOptimizerService(评估优化服务)**: 编排迭代改进循环 | ||
| 4. **模型类**: 为每个阶段提供类型安全的请求/响应对象 | ||
|
|
||
| ### 工作流程 | ||
|
|
||
| 1. **生成阶段**: 为给定任务创建初始解决方案 | ||
| 2. **评估阶段**: 对生成的解决方案进行评分和批评 | ||
| 3. **改进循环**: 基于反馈迭代改进解决方案 | ||
| 4. **收敛判断**: 当达到可接受质量或超过最大迭代次数时停止 | ||
|
|
||
| ## 配置说明 | ||
|
|
||
| ### 环境变量 | ||
|
|
||
| 应用支持多种AI模型配置,当前使用阿里云兼容模式: | ||
liu-657667 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ```yaml | ||
| spring: | ||
| ai: | ||
| openai: | ||
| api-key: 你的API密钥 | ||
| base-url: https://dashscope.aliyuncs.com/compatible-mode | ||
| chat: | ||
| options: | ||
| model: qwen-plus | ||
| temperature: 0.7 | ||
| deepseek: | ||
| api-key: 你的API密钥 | ||
| base-url: https://dashscope.aliyuncs.com/compatible-mode | ||
| chat: | ||
| options: | ||
| model: qwen-max | ||
| temperature: 0.3 | ||
| ``` | ||
| ### 模型配置 | ||
| - **生成器**: Qwen-Plus (temperature: 0.7) 用于创造性的解决方案生成 | ||
| - **评估器**: Qwen-Max (temperature: 0.3) 用于一致性评估 | ||
| ## 使用方法 | ||
| ### API 接口 | ||
| #### 基本任务求解 | ||
| ```bash | ||
| curl -X POST "http://localhost:8084/api/evaluator-optimizer/solve?task=创建一个计算斐波那契数列的Java方法" | ||
| ``` | ||
|
|
||
| #### 带上下文的高级任务求解 | ||
| ```bash | ||
| curl -X POST "http://localhost:8084/api/evaluator-optimizer/solve-advanced" \ | ||
| -d "task=创建一个用户管理的REST端点" \ | ||
| -d "context=Spring Boot应用,使用JPA和安全框架" \ | ||
| -d "criteria=必须包含适当的验证和错误处理" \ | ||
| -H "Content-Type: application/x-www-form-urlencoded" | ||
| ``` | ||
|
|
||
| #### 健康检查 | ||
| ```bash | ||
| curl http://localhost:8084/api/evaluator-optimizer/health | ||
| ``` | ||
|
|
||
| ### 响应示例 | ||
|
|
||
| ```json | ||
| { | ||
| "finalSolution": "public class FibonacciCalculator {...}", | ||
| "chainOfThought": [ | ||
| "开始为任务执行评估器-优化器循环: 创建斐波那契计算器", | ||
| "第1次迭代 - 生成解决方案: public class FibonacciCalculator...", | ||
| "第1次迭代 - 评分: 9.5, 可接受: true", | ||
| "在第1次迭代收敛,评分9.5" | ||
| ], | ||
| "evaluationHistory": [...], | ||
| "totalIterations": 1, | ||
| "finalScore": 9.5, | ||
| "converged": true | ||
| } | ||
| ``` | ||
|
|
||
| ## 配置参数 | ||
|
|
||
| - **最大迭代次数**: 3次(最大改进循环次数) | ||
| - **收敛阈值**: 8.5分(可接受解决方案的评分阈值) | ||
| - **接受阈值**: 7.0分(接受的最低评分) | ||
|
|
||
| ## 测试 | ||
|
|
||
| 运行测试套件: | ||
|
|
||
| ```bash | ||
| mvn test | ||
| ``` | ||
|
|
||
| 测试包括: | ||
| - 基本评估器-优化器循环功能 | ||
| - 上下文感知的任务求解 | ||
| - 与实际AI模型的集成测试 | ||
|
|
||
| ## 使用场景 | ||
|
|
||
| ### 理想应用 | ||
| - 代码生成和优化 | ||
| - 技术文档创建 | ||
| - 多种解决方案的问题求解 | ||
| - 迭代改进的创意写作 | ||
|
|
||
| ### 示例任务 | ||
| - "创建一个线程安全的Java缓存实现" | ||
| - "设计一个图书管理系统的REST API" | ||
| - "为排序算法编写全面的单元测试" | ||
| - "创建电商平台的数据库模式" | ||
|
|
||
| ## 实现说明 | ||
|
|
||
| ### 生成策略 | ||
| - 初始生成专注于完整性和功能性 | ||
| - 改进生成针对特定反馈要点 | ||
| - 保持成功要素的同时改进问题区域 | ||
|
|
||
| ### 评估标准 | ||
| - **正确性**: 解决方案是否正确解决了问题? | ||
| - **完整性**: 是否满足了所有需求? | ||
| - **清晰度**: 解决方案是否易于理解? | ||
| - **最佳实践**: 是否遵循良好的编码/设计实践? | ||
| - **效率**: 解决方案是否合理高效? | ||
|
|
||
| ### 收敛逻辑 | ||
| - 当解决方案评分 ≥ 8.5 且标记为可接受时停止 | ||
| - 最多3次迭代以防止无限循环 | ||
| - 提供演化轨迹以保证透明度 | ||
|
|
||
| ## 依赖项 | ||
|
|
||
| - Spring Boot 3.3.6 | ||
| - Spring AI 1.0.0 | ||
| - Spring AI OpenAI Starter | ||
| - Spring AI DeepSeek Starter | ||
| - Lombok 用于减少样板代码 | ||
|
|
||
| ## 测试结果 ✅ | ||
|
|
||
| 基于实际功能测试,该模块表现优异: | ||
|
|
||
| - **✅ 应用启动**: Java 21环境下成功启动 | ||
| - **✅ 健康检查**: 所有端点正常响应 | ||
| - **✅ 基本求解**: 斐波那契计算示例,评分9.5/10,1次迭代收敛 | ||
| - **✅ 高级求解**: 支持上下文和自定义标准,评分8.5/10 | ||
| - **✅ 评估循环**: Generator和Evaluator协同工作正常 | ||
| - **✅ 配置管理**: 阿里云兼容模式配置成功 | ||
|
|
||
| ## 未来增强 | ||
|
|
||
| - 可配置的评估标准 | ||
| - 自定义评分模型 | ||
| - 多评估器共识机制 | ||
| - 解决方案比较和排名 | ||
| - 性能指标和分析 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <parent> | ||
| <groupId>com.glmapper</groupId> | ||
| <artifactId>spring-ai-agent</artifactId> | ||
| <version>0.0.1</version> | ||
| </parent> | ||
| <artifactId>spring-ai-evaluator-optimizer</artifactId> | ||
| <name>spring-ai-evaluator-optimizer</name> | ||
| <description>Evaluator Optimizer Agent Pattern Implementation</description> | ||
|
|
||
| <properties> | ||
liu-657667 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| <java.version>21</java.version> | ||
| <spring-ai.version>1.0.0</spring-ai.version> | ||
| </properties> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>org.springframework.ai</groupId> | ||
| <artifactId>spring-ai-starter-model-openai</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework.ai</groupId> | ||
| <artifactId>spring-ai-starter-model-deepseek</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-web</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-actuator</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-starter-test</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.projectlombok</groupId> | ||
| <artifactId>lombok</artifactId> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.springframework.boot</groupId> | ||
| <artifactId>spring-boot-maven-plugin</artifactId> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
liu-657667 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| </project> | ||
20 changes: 20 additions & 0 deletions
20
...izer/src/main/java/com/glmapper/ai/evaluator/optimizer/EvaluatorOptimizerApplication.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| package com.glmapper.ai.evaluator.optimizer; | ||
|
|
||
| import org.springframework.boot.SpringApplication; | ||
| import org.springframework.boot.autoconfigure.SpringBootApplication; | ||
|
|
||
| /** | ||
| * Evaluator Optimizer Agent Pattern Application | ||
| * | ||
| * This application demonstrates the Evaluator-Optimizer pattern for iterative | ||
| * refinement of AI-generated solutions through generation, evaluation, and optimization cycles. | ||
| * | ||
| * @author glmapper | ||
liu-657667 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| */ | ||
| @SpringBootApplication | ||
| public class EvaluatorOptimizerApplication { | ||
|
|
||
| public static void main(String[] args) { | ||
| SpringApplication.run(EvaluatorOptimizerApplication.class, args); | ||
| } | ||
| } | ||
29 changes: 29 additions & 0 deletions
29
...-optimizer/src/main/java/com/glmapper/ai/evaluator/optimizer/config/ChatClientConfig.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| package com.glmapper.ai.evaluator.optimizer.config; | ||
|
|
||
| import org.springframework.ai.chat.client.ChatClient; | ||
| import org.springframework.ai.deepseek.DeepSeekChatModel; | ||
| import org.springframework.ai.openai.OpenAiChatModel; | ||
| import org.springframework.beans.factory.annotation.Qualifier; | ||
| import org.springframework.context.annotation.Bean; | ||
| import org.springframework.context.annotation.Configuration; | ||
|
|
||
| /** | ||
| * Chat client configuration for Evaluator Optimizer | ||
| * | ||
| * @author glmapper | ||
| */ | ||
| @Configuration | ||
| public class ChatClientConfig { | ||
|
|
||
| @Bean | ||
| @Qualifier("generatorChatClient") | ||
| public ChatClient generatorChatClient(OpenAiChatModel openAiChatModel) { | ||
| return ChatClient.builder(openAiChatModel).build(); | ||
| } | ||
|
|
||
| @Bean | ||
| @Qualifier("evaluatorChatClient") | ||
| public ChatClient evaluatorChatClient(DeepSeekChatModel deepSeekChatModel) { | ||
| return ChatClient.builder(deepSeekChatModel).build(); | ||
| } | ||
| } |
57 changes: 57 additions & 0 deletions
57
...ain/java/com/glmapper/ai/evaluator/optimizer/controller/EvaluatorOptimizerController.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| package com.glmapper.ai.evaluator.optimizer.controller; | ||
|
|
||
| import com.glmapper.ai.evaluator.optimizer.model.RefinedResponse; | ||
| import com.glmapper.ai.evaluator.optimizer.service.EvaluatorOptimizerService; | ||
| import org.springframework.web.bind.annotation.*; | ||
|
|
||
| /** | ||
| * REST controller for the Evaluator-Optimizer pattern | ||
| * | ||
| * @author glmapper | ||
| */ | ||
| @RestController | ||
| @RequestMapping("/api/evaluator-optimizer") | ||
| public class EvaluatorOptimizerController { | ||
|
|
||
| private final EvaluatorOptimizerService evaluatorOptimizerService; | ||
|
|
||
| public EvaluatorOptimizerController(EvaluatorOptimizerService evaluatorOptimizerService) { | ||
| this.evaluatorOptimizerService = evaluatorOptimizerService; | ||
| } | ||
|
|
||
| /** | ||
| * Execute the evaluator-optimizer loop for a given task | ||
| * | ||
| * @param task The task description | ||
| * @return RefinedResponse with the final solution and evolution trace | ||
| */ | ||
| @PostMapping("/solve") | ||
| public RefinedResponse solve(@RequestParam String task) { | ||
| return evaluatorOptimizerService.loop(task); | ||
| } | ||
|
|
||
| /** | ||
| * Execute the evaluator-optimizer loop with context and custom criteria | ||
| * | ||
| * @param task The task description | ||
| * @param context Additional context (optional) | ||
| * @param criteria Custom evaluation criteria (optional) | ||
| * @return RefinedResponse with the final solution and evolution trace | ||
| */ | ||
| @PostMapping("/solve-advanced") | ||
| public RefinedResponse solveAdvanced(@RequestParam String task, | ||
| @RequestParam(required = false) String context, | ||
| @RequestParam(required = false) String criteria) { | ||
| return evaluatorOptimizerService.loop(task, context, criteria); | ||
| } | ||
|
|
||
| /** | ||
| * Simple health check endpoint | ||
| * | ||
| * @return status message | ||
| */ | ||
| @GetMapping("/health") | ||
| public String health() { | ||
| return "Evaluator-Optimizer service is running"; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.