Automated_Codebase_Modification

Overview

Large Language Models (LLMs) possess significant capabilities for debugging and generating code. However, users must still possess a foundational understanding of programming and be able to integrate the generated code into the existing codebase to fulfill specific objectives. To address this challenge, my teammate and I undertook an investigation into the potential of LLMs to directly modify the codebase in accordance with user requirements. We put our focus on a Warehouse Management System codebase, developed using Django and Vue.js.

We began with an in-depth analysis of the stages and workflows within the WMS codebase, aiming to understand its architecture and operational dynamics. Subsequently, we explored the SWE-agent, a tool designed to interface with large language models (LLMs), to enable automated codebase modifications based on user-generated prompts. To provide accurate overview of the codebase, we developed both human-written and machine-generated overview, which was generated by the Repo-agent. A significant portion of our work focused on developing and optimizing prompt inputs and configurations to enhance the accuracy and efficacy of these automated modifications. This study explores the potential for LLM-driven automation in complex codebases, and offers insights into the optimization of LLM interactions for precise and reliable software modifications.

Run Code

1. Install and set up SWE-agent by reading README.md in SWE-agent directory. (Directly use the directory got from this repo is OK, but still need to set up local environment)
2. Install and set up Repo-agent by reading README.md in Repo-agent directory. (Directly use the directory got from this repo is OK, but still need to set up local environment)
3. Run Repo-agent on GreaterWMS (or other directorys or codebases you want to make changes to)
4. Update instructions_v3.md in SWE-agent with focus on absolute positions of the files so that LLM can successfully access the file information. For instance, change /home/Automated_Codebase_Modification/SWE-agent/markdown_docs to the absolute file position of the output of the Repo-agent.
5. In instructions_v3.md, put your request at "Task to do" to let LLM know your request.
6. Run the following terminal commands

      cd SWE-agent \
      python run.py \
       --model_name gpt4 \
       --data_path "path to your instruction" \
       --repo_path "path to your codebase" \
       --config_file "path to config file" \
       --apply_patch_locally \
       --per_instance_cost_limit 2.00
    ```

e.g.

   python run.py \
     --model_name gpt4o \
     --data_path "/home/Automated_Codebase_Modification/SWE-agent/instructions_v3.md" \
     --repo_path "/home/Automated_Codebase_Modification/GreaterWMS" \
     --config_file "/home/Automated_Codebase_Modification/SWE-agent/config/wms_updated.yaml" \
     --apply_patch_locally \
     --per_instance_cost_limit 2.00

Note: The configurations and prompts are only work for GreaterWMS. If you want to explore this method on other codebases, you need to optimize configurations and prompts yourself.

Contributors

• Ziang Gu
• Juyuan Huang

References:

1. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024). SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. arXiv link.
2. Luo, Q., Ye, Y., Liang, S., Zhang, Z., Qin, Y., Lu, Y., Wu, Y., Cong, X., Lin, Y., Zhang, Y., Che, X., & Liu, Z., Sun, M. (2024). RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation. arXiv:2402.16667 [cs.CL]. Available at: arXiv link.