Questions on SWE-bench Verified and AST-retrieval

[liangyuchen0014]

作者你好，在你们发表的论文中有写到“The results show consistent improvements over strong agent-based and pipeline-based baselines and establish a new state-of-the-art (SOTA) (79.4%) on SWE-bench Verified.”然而在最新的leaderboard中，还没有展示出贵工作的成绩。如果这不是因为审核时间过长的原因的话，个人感觉会对贵工作的真实性造成一些动摇。

此外，在机器之心宣传贵工作的文章当中（https://mp.weixin.qq.com/s/3vVYnAy51-XMOsH6S7YAvg ） ，有提到“InfCode 自研了基于抽象语法树（AST）的结构化检索引擎。”然而，我在论文中没有找到有关AST检索的相关论述。

论文中仅在“2.4 工具集与执行环境（Tool Suite and Execution Environment）”部分提到了用于代码检索的工具——Searcher，其功能描述为“通过ripgrep执行高效的迭代目录搜索，支持正则表达式以定位相关代码或测试片段（Performs efficient iterative directory searches using ripgrep, with support for regular expressions to locate relevant code or test fragments）”。该工具依赖正则化文本检索，未涉及AST结构化分析或基于AST的检索设计，也未在其他章节（如研究背景、方法创新、实验细节等）提及AST相关的结构化检索引擎。
所以我想请问，贵工作中是如何使用AST检索的？如果没有使用到的话，那么机器之心的那篇文章很有可能是AI生成的😂

希望作者能解答我的困惑，谢谢！

Hello authors, in your published paper, it is written that "The results show consistent improvements over strong agent-based and pipeline-based baselines and establish a new state-of-the-art (SOTA) (79.4%) on SWE-bench Verified." However, in the latest leaderboard, your work's results have not been displayed. If this is not due to a prolonged review process, personally, it may cast some doubt on the authenticity of your work.

In addition, in the article promoting your work by Machine Heart (https://mp.weixin.qq.com/s/3vVYnAy51-XMOsH6S7YAvg), it is mentioned that "InfCode has independently developed a structured retrieval engine based on Abstract Syntax Trees (AST)." However, I did not find any discussion about AST retrieval in the paper.

The paper only mentions a tool for code retrieval called Searcher in the section "2.4 Tool Suite and Execution Environment". Its function is described as "Performs efficient iterative directory searches using ripgrep, with support for regular expressions to locate relevant code or test fragments". This tool relies on regularized text retrieval, does not involve AST structural analysis or AST-based retrieval design, and there is no mention of an AST-related structured retrieval engine in other chapters (such as research background, methodological innovations, experimental details, etc.).

So I would like to ask, how is AST retrieval used in your work? If it is not used, then the article from Machine Heart is likely AI-generated 😂

I hope the authors can answer my questions. Thank you!

[qadong]

感谢您的细致观察与反馈。关于 SWE-bench Verified 的 79.4% 成绩，这是我们基于 Python 场景使用文本检索方案（Searcher）取得的真实结果，目前榜单更新正处于审核流程中（PR 链接：Add result for InfCode(Tokfinity) by PeteWongggg · Pull Request #356 · SWE-bench/experiments）；而媒体报道中提到的“基于 AST 的结构化检索”实际上是指我们专门针对 C++ 场景开发的后续工作 "InfCode-C++"（相关基准：https://multi-swe-bench.github.io/#/）。正如我们在论文（https://arxiv.org/abs/2511.16005）中详述的，C++ 复杂的特性（如模版元编程、标识符重载及前置声明）导致传统正则检索（grep）难以区分前置声明与实际定义 ，文件级定位准确率仅约 40%；为此，我们引入了基于 Tree-sitter 的静态分析并构建了结构化索引，有效解决了文本检索无法处理的语义歧义问题。此外，得益于 Tree-sitter 强大的多语言解析能力，该架构在理论上同样支持 Python 和 Java，目前的论文则优先聚焦于定位痛点最为显著的 C++ 场景进行验证，我们会在ISSTA26的流程结束后，合入infcode 主仓库。

Thank you for your detailed feedback and close attention to our work. Regarding the 79.4% score on SWE-bench Verified, this is an authentic result achieved on Python tasks using our text-based retrieval scheme (Searcher), and the leaderboard update is currently undergoing the review process (PR Link:Add result for InfCode(Tokfinity) by PeteWongggg · Pull Request #356 · SWE-bench/experiments ). The "AST-based structured retrieval" mentioned in media reports actually refers to our separate, follow-up work specifically tailored for C++ scenarios, titled "InfCode-C++" (Benchmark: https://multi-swe-bench.github.io/#/). As detailed in our paper (https://arxiv.org/abs/2511.16005), complex C++ features—such as template metaprogramming, identifier overloading, and forward declarations—make it difficult for traditional regex-based retrieval (grep) to distinguish between forward declarations and actual definitions , resulting in a file-level localization accuracy of only ~40%. To address this, we introduced Tree-sitter-based static analysis and constructed a structural index, which effectively resolves semantic ambiguities that text-based retrieval cannot handle. Furthermore, thanks to Tree-sitter's robust multi-language parsing capabilities, this architecture theoretically supports Python and Java as well; however, our current paper prioritizes C++ to validate the system where localization challenges are most acute. We plan to merge this work into the main InfCode repository following the conclusion of the ISSTA 2026 review process.

[liangyuchen0014]

谢谢作者的详细解答，我会再看一下你提到的“InfCode-C++”这篇文章。
Thank you for thedetailed answer. I will take a look at the paper "InfCode-C++" you mentioned.