[feature] [sub feature 1] Support model judge in evaluation#149
[feature] [sub feature 1] Support model judge in evaluation#149SJTUyh wants to merge 4 commits intoAISBench:masterfrom
Conversation
Summary of ChangesHello @SJTUyh, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the evaluation pipeline by introducing a robust framework for model judging using Large Language Models. It allows for a new stage in the workflow where an LLM can assess the correctness of another model's predictions, providing a more nuanced and automated evaluation capability. The changes include new worker classes, dataset types, and utility functions to support this new judging mechanism, making the evaluation process more flexible and powerful. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
SJTUyh
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new JudgeInfer worker to support model-judged evaluation, enhancing the evaluation workflow. However, it introduces potential security vulnerabilities, including prompt injection in the LLM judge templates and path traversal in file operations within the worker classes. Model and dataset abbreviations should be sanitized before being used in file paths, and LLM prompts should be designed to be resilient against injection from untrusted model outputs. Additionally, there are issues related to prompt consistency, error handling, and implementation robustness that need to be addressed.
|
|
||
| def load(self, predictions_path: str, **kwargs): | ||
|
|
||
| dataset_content = self.dataset_instance.dataset["test"] |
There was a problem hiding this comment.
This line will raise a TypeError if self.dataset_instance.dataset is a datasets.Dataset object, as it is not subscriptable with a string key like "test". The load method of Aime2025Dataset, which is used to create self.dataset_instance, returns a Dataset object, not a DatasetDict.
The logic should be changed to handle both Dataset and DatasetDict correctly. You should get the dataset object first and then check its type.
| dataset_content = self.dataset_instance.dataset["test"] | |
| dataset_content = self.dataset_instance.dataset |
| key = ( | ||
| task["models"][0]["abbr"] # same model | ||
| + "_" | ||
| + str(task['datasets'][0][0]['type']) # same dataset type | ||
| + "_" | ||
| + str(task["datasets"][0][0]["infer_cfg"]["inferencer"]) # same inferencer with the same args | ||
| ) |
There was a problem hiding this comment.
Using str() on a dictionary to generate a grouping key is unreliable. The string representation of a dictionary can vary depending on key insertion order (in older Python versions) or how it's constructed. This could lead to tasks not being merged correctly when they should be.
To create a stable and canonical representation of the dictionary, I recommend using json.dumps with sort_keys=True. Using a tuple for the key is also generally safer than string concatenation.
key = (
task["models"][0]["abbr"], # same model
str(task['datasets'][0][0]['type']), # same dataset type
json.dumps(task["datasets"][0][0]["infer_cfg"]["inferencer"], sort_keys=True) # same inferencer with the same args
)| match = re.search(r'[AB]', pred[-1:]) | ||
| return match.group(0) if match else 'B' |
There was a problem hiding this comment.
Extracting the judge's decision by only looking at the last character (pred[-1:]) is very brittle. LLMs may not always follow formatting instructions perfectly and might add extra characters, newlines, or punctuation.
Consider using a more robust regular expression to find the first occurrence of 'A' or 'B' as a standalone character in the model's output. This will make the parsing much more reliable.
| match = re.search(r'[AB]', pred[-1:]) | |
| return match.group(0) if match else 'B' | |
| match = re.search(r'\b[AB]\b', pred) | |
| return match.group(0) if match else 'B' |
| preds = load_jsonl(prediction_path) | ||
| preds.sort(key=lambda x: x.get('id',0)) |
There was a problem hiding this comment.
If prediction_path does not exist, the preds variable is never initialized. This will cause an UnboundLocalError on the next line when preds.sort() is called.
To fix this, initialize preds to an empty list before the if block.
| preds = load_jsonl(prediction_path) | |
| preds.sort(key=lambda x: x.get('id',0)) | |
| preds = [] | |
| if os.path.exists(prediction_path): | |
| preds = load_jsonl(prediction_path) | |
| preds.sort(key=lambda x: x.get('id', 0)) |
|
|
||
| # update judge cfgs to model cfgs and data | ||
| for task in tasks: | ||
| task["datasets"][0][0]["predictions_path"] = osp.join(cfg.judge_infer.partitioner.out_dir, task["models"][0]["abbr"], f'{task["datasets"][0][0]["abbr"]}.jsonl') |
There was a problem hiding this comment.
File paths are constructed using abbr values (model and dataset abbreviations) taken directly from the configuration without sanitization. This allows for path traversal attacks where an attacker providing a malicious configuration could cause the application to read, write, or delete arbitrary files on the system. For example, if a model abbreviation is set to ../../etc, the application might attempt to operate on files outside the intended output directory.
| summarizer = build_from_cfg(summarizer_cfg) | ||
| summarizer.summarize(time_str=self.args.cfg_time_str) | ||
|
|
||
| def _cfg_pre_process(self, cfg: ConfigDict) -> None: |
There was a problem hiding this comment.
The function's type hint specifies a return type of None, but the function actually returns cfg. The calling code in do_work also uses this return value. Please update the type hint to match the implementation and usage.
def _cfg_pre_process(self, cfg: ConfigDict) -> ConfigDict:|
|
||
| dataset_content = self.dataset_instance.dataset["test"] | ||
|
|
||
| # 加载被测模型的推理结果(排序后) |
There was a problem hiding this comment.
This comment is in Chinese. To maintain consistency and improve readability for all contributors, please write comments in English.
| # 加载被测模型的推理结果(排序后) | |
| # Load the inference results of the model under test (sorted) |
1 participant
Thanks for your contribution; we appreciate it a lot. The following instructions will make your pull request healthier and help you get feedback more easily. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
感谢您的贡献，我们非常重视。以下说明将使您的拉取请求更健康，更易于获得反馈。如果您不理解某些项目，请不要担心，只需提交拉取请求并从维护人员那里寻求帮助即可。
PR Type / PR类型
Related Issue | 关联 Issue
Fixes #(issue ID / issue 编号) / Relates to #(issue ID / issue 编号)
🔍 Motivation / 变更动机
Please describe the motivation of this PR and the goal you want to achieve through this PR.
请描述您的拉取请求的动机和您希望通过此拉取请求实现的目标。
📝 Modification / 修改内容
Please briefly describe what modification is made in this PR.
请简要描述此拉取请求中进行的修改。
📐 Associated Test Results / 关联测试结果
Please provide links to the related test results, such as CI pipelines, test reports, etc.
请提供相关测试结果的链接，例如 CI 管道、测试报告等。
Does the modification introduce changes that break the backward compatibility of the downstream repositories? If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
是否引入了会破坏下游存储库向后兼容性的更改？如果是，请描述它如何破坏兼容性，以及下游项目应该如何修改其代码以保持与此 PR 的兼容性。
If the modification introduces performance degradation, please describe the impact of the performance degradation and the expected performance improvement.
如果引入了性能下降，请描述性能下降的影响和预期的性能改进。
🌟 Use cases (Optional) / 使用案例（可选）
If this PR introduces a new feature, it is better to list some use cases here and update the documentation.
如果此拉取请求引入了新功能，最好在此处列出一些用例并更新文档。
✅ Checklist / 检查列表
Before PR:
After PR:
👥 Collaboration Info / 协作信息
🌟 Useful CI Command / 实用的CI命令
/gemini review/gemini summary/gemini help/readthedocs build