Application Exploration and Summarization

This set of scripts helps you:

1. Scan a Ruby on Rails application to identify key files (models, controllers, services, views).
2. Generate a summary JSON file (app_summary.json) that lists all discovered files along with a brief description.
3. Use OpenAI's API to generate detailed summaries of each file and augment app_summary.json with richer information.

Prerequisites

• OpenAI API Key: You need a valid OPENAI_API_KEY with access to the chosen model (e.g., gpt-4 or gpt-3.5-turbo).
• jq: A command-line JSON processor. On macOS, install with brew install jq.
• Bash Shell: Tested on Linux and macOS. If on macOS, you may need to install GNU utilities if you make changes to find or sed commands.

Files Overview

• app_explore.sh: The main entry point. This script:

  • Prompts for OpenAI API key and model name.
  • Runs generate_summary.sh to create app_summary.json.
  • Runs orchestrate_summaries_in_json.sh to call OpenAI API via summarize_file.sh and add detailed summaries to app_summary.json.

• generate_summary.sh:

  • Scans the Rails app/models, app/controllers, app/services, and app/views directories.
  • Finds .rb files (for models, controllers, services) and template files (e.g., .erb, .haml, .html for views).
  • Extracts a short description from the top of each file.
  • Produces app_summary.json with an array of files and their basic info.

• summarize_file.sh:

  • Takes a file path as input.
  • Sends the file’s content to the OpenAI API (using the model specified in OPENAI_MODEL).
  • Returns a short, AI-generated summary of the file’s purpose and main functions.

• orchestrate_summaries_in_json.sh:

  • Reads app_summary.json generated by generate_summary.sh.
  • Iterates through each file listed in app_summary.json.
  • Calls summarize_file.sh to get a detailed summary.
  • Updates app_summary.json by adding a detailed_summary field for each file.

Setup

1. Ensure you have jq installed:

   # macOS (Homebrew)
   brew install jq

   On Linux, jq is often available in package managers:

   # Debian/Ubuntu
   sudo apt-get install jq

2. Set Executable Permissions:

   chmod +x app_explore.sh
   chmod +x generate_summary.sh
   chmod +x summarize_file.sh
   chmod +x orchestrate_summaries_in_json.sh

3. Prepare Your Rails App Directory: Run these scripts at the root of your Rails application so that app/models, app/controllers, app/services, and app/views are in the expected locations.

Usage

1. Run the Main Entry Script:

   ./app_explore.sh

   • If OPENAI_API_KEY is not set, it will prompt you to enter one.
   • It will ask you for your preferred model (default gpt-4).

2. Process:

   • app_explore.sh runs generate_summary.sh to create app_summary.json.
   • It then runs orchestrate_summaries_in_json.sh to add detailed summaries to each file in app_summary.json.

3. Result:

   • After completion, check the app_summary.json file.
   • Each file entry will now have a detailed_summary field with the AI-generated description.

Troubleshooting

• Empty app_summary.json:
  Check if your Rails directories (app/models, app/controllers, app/services, app/views) contain files. If empty, no entries will be generated.

• Invalid Model or API Key:
  If detailed_summary is null, ensure your OPENAI_API_KEY is valid and that the model you chose (e.g., gpt-4) is accessible to your API key.
  You can also print the raw response in summarize_file.sh to debug any API errors:

  echo "$RESPONSE" >&2

• Parse Errors in JSON:
  Make sure not to modify generate_summary.sh to add trailing commas or incomplete braces. The provided version ensures well-formed JSON. If you run into JSON parse errors, validate the file with jq . app_summary.json to locate the issue.

Customization

• Additional Directories:
  To include more directories (e.g., app/helpers), add another call to process_directory or create a similar function if needed.

• Adjusting Prompt or Temperature:
  In summarize_file.sh, you can tweak the max_tokens, temperature, or prompt to get different styles or lengths of summaries.

Conclusion

These scripts streamline the process of generating an overview and detailed summaries of a Rails codebase using the OpenAI API. Just run app_explore.sh, follow the prompts, and review the enriched app_summary.json file.