Skip to content

Optimize version suggester, fix bugs and optimize the timezone, add WhlRepacker in utils#28

Merged
TongWu merged 8 commits intomainfrom
dev
Jun 26, 2025
Merged

Optimize version suggester, fix bugs and optimize the timezone, add WhlRepacker in utils#28
TongWu merged 8 commits intomainfrom
dev

Conversation

@TongWu
Copy link
Owner

@TongWu TongWu commented Jun 26, 2025

  1. Fix safe minor upgrade when all minors vulnerable #27 Optimize version suggester, if all minor versions are vulnerable, check major version instead.
  2. Fix when packages appear in NotUsed.txt but doesn't have remark in report.
  3. Change timezone for monthly report filename to UTC+8 SG time in order to tally with weekly report.
  4. Add WhlRepacker in utils, which is an ad-hoc script allows to repack the whl file to metadata 2.3 version in order to satisfy private feeds metadata version control.

Summary by CodeRabbit

  • New Features

    • Added the ability to repack Python wheel files to update their metadata version, including batch processing via a command-line interface.
  • Improvements

    • Made the path for the "not used packages" file configurable through an environment variable.
    • Unified timestamp handling for report filenames to use Singapore time.
  • Refactor

    • Enhanced the logic and clarity of the safe minor upgrade suggestion process, with improved documentation and error handling.

@TongWu TongWu changed the title Optimize version suggester, fix bugs and optimize the timezone Optimize version suggester, fix bugs and optimize the timezone, add WhlRepacker in utils Jun 26, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jun 26, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

The changes introduce a new module for repacking Python wheel files to update their metadata, refactor the logic for suggesting safe package upgrades, and enhance configurability in report generation by using environment variables and unified timestamp handling. No changes were made to exported or public entity declarations outside of internal logic.

Changes

File(s) Change Summary
GenerateReport.py Uses environment variable NOTUSED_PACKAGES for not-used packages file path, unifies timestamp source for reports.
utils/VersionSuggester.py Refactors suggest_safe_minor_upgrade for clarity, improves upgrade logic, updates docstring and error handling.
utils/WhlRepacker.py Adds new module to repack .whl files, update METADATA version, batch process wheels, and provide CLI interface.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant WhlRepacker
    participant FileSystem

    User->>CLI: Run repack command with input/output dirs
    CLI->>WhlRepacker: process_all_wheels(input_dir, output_dir)
    WhlRepacker->>FileSystem: Find all .whl files in input_dir
    loop For each wheel
        WhlRepacker->>FileSystem: Extract wheel to temp dir
        WhlRepacker->>FileSystem: Locate METADATA file
        alt METADATA found
            WhlRepacker->>FileSystem: Update Metadata-Version to 2.3
            WhlRepacker->>FileSystem: Repack wheel to output_dir
        else METADATA missing
            WhlRepacker->>CLI: Log error
        end
        WhlRepacker->>FileSystem: Cleanup temp dir
    end
    WhlRepacker->>CLI: Done
Loading

Possibly related PRs

Suggested labels

codex

Poem

A hop, a skip, a wheel repacked,
Metadata fresh, no detail lacked.
Reports now read the path you choose,
And upgrades find the safest news.
With every change, this rabbit grins—
For tidy code is where joy begins! 🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3a1ab37 and e3d1733.

📒 Files selected for processing (1)
  • utils/WhlRepacker.py (1 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@TongWu TongWu self-assigned this Jun 26, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (5)
utils/VersionSuggester.py (1)

77-79: Fix formatting issue flagged by static analysis.

The function definition needs proper spacing according to PEP 8 standards.

+
 async def suggest_safe_minor_upgrade(
     pkg: str, current_version: str, all_versions: list
 ) -> str:
utils/WhlRepacker.py (4)

6-6: Fix formatting issues flagged by static analysis.

The function definitions need proper spacing according to PEP 8 standards.

+
 def repack_single_wheel(wheel_path: Path, output_dir: Path):

6-53: Consider refactoring to reduce complexity.

The function has 16 local variables which exceeds the recommended limit. Consider extracting helper functions for metadata processing and wheel creation.

Here's a suggested refactor:

 def repack_single_wheel(wheel_path: Path, output_dir: Path):
     """Repack a single wheel to use Metadata-Version: 2.3"""
     temp_dir = output_dir / (wheel_path.stem + '_unpacked')
 
+    try:
+        _extract_wheel(wheel_path, temp_dir)
+        metadata_file = _find_metadata_file(temp_dir)
+        if not metadata_file:
+            print(f"[ERROR] METADATA not found in: {wheel_path.name}")
+            return
+        
+        _update_metadata_version(metadata_file)
+        _create_repacked_wheel(temp_dir, wheel_path, output_dir)
+        print(f"[DONE] Repacked: {wheel_path.name}")
+    finally:
+        if temp_dir.exists():
+            shutil.rmtree(temp_dir)
+
+def _extract_wheel(wheel_path: Path, temp_dir: Path):
+    with zipfile.ZipFile(wheel_path, 'r') as zip_ref:
+        zip_ref.extractall(temp_dir)
+
+def _find_metadata_file(temp_dir: Path) -> Path:
+    for root, _, files in os.walk(temp_dir):
+        for file in files:
+            if file == 'METADATA':
+                return Path(root) / file
+    return None
+
+def _update_metadata_version(metadata_file: Path):
+    with open(metadata_file, 'r', encoding='utf-8') as f:
+        lines = f.readlines()
+    
+    with open(metadata_file, 'w', encoding='utf-8') as f:
+        for line in lines:
+            if line.startswith("Metadata-Version:"):
+                f.write("Metadata-Version: 2.3\n")
+            else:
+                f.write(line)
+
+def _create_repacked_wheel(temp_dir: Path, original_wheel: Path, output_dir: Path):
+    new_wheel_path = output_dir / f"{original_wheel.stem}.whl"
+    with zipfile.ZipFile(new_wheel_path, 'w', zipfile.ZIP_DEFLATED) as zipf:
+        for root, _, files in os.walk(temp_dir):
+            for file in files:
+                file_path = Path(root) / file
+                arcname = file_path.relative_to(temp_dir)
+                zipf.write(file_path, arcname)

54-54: Fix formatting issue flagged by static analysis.

Add proper spacing before function definition.

+
 def process_all_wheels(input_dir: str, output_dir: str = None):

72-80: Fix formatting issue and improve script name in usage message.

The script reference in the usage message doesn't match the actual filename.

+
 if __name__ == "__main__":
     import sys
     if len(sys.argv) < 2:
-        print("Usage: python repack_wheels_batch.py <input_dir> [<output_dir>]")
+        print("Usage: python WhlRepacker.py <input_dir> [<output_dir>]")
     else:
         input_dir = sys.argv[1]
         output_dir = sys.argv[2] if len(sys.argv) > 2 else None
         process_all_wheels(input_dir, output_dir)
+
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 95c9bf1 and 3a1ab37.

⛔ Files ignored due to path filters (2)
  • MonthlyReport/2025-06/MonthlyReport-202506-26-0227.xlsx is excluded by !**/*.xlsx
  • WeeklyReport/2025-06-23/WeeklyReport_20250626_101759.csv is excluded by !**/*.csv
📒 Files selected for processing (3)
  • GenerateReport.py (2 hunks)
  • utils/VersionSuggester.py (1 hunks)
  • utils/WhlRepacker.py (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
utils/VersionSuggester.py (1)
Learnt from: TongWu
PR: TongWu/PythonPackageManager#14
File: utils/VersionSuggester.py:136-137
Timestamp: 2025-06-24T03:17:27.141Z
Learning: In utils/VersionSuggester.py, the suggest_upgrade_version function has been intentionally disabled as it's outdated. The preferred approach is to use suggest_safe_minor_upgrade function instead for version suggestions.
🧬 Code Graph Analysis (2)
utils/VersionSuggester.py (1)
utils/VulnChecker.py (1)
  • fetch_osv (30-102)
GenerateReport.py (1)
utils/SGTUtils.py (1)
  • now_sg (25-27)
🪛 Flake8 (7.2.0)
utils/WhlRepacker.py

[error] 6-6: expected 2 blank lines, found 1

(E302)


[error] 54-54: expected 2 blank lines, found 1

(E302)


[error] 72-72: expected 2 blank lines after class or function definition, found 1

(E305)

utils/VersionSuggester.py

[error] 77-77: expected 2 blank lines, found 1

(E302)

🪛 Pylint (3.3.7)
utils/WhlRepacker.py

[refactor] 6-6: Too many local variables (16/15)

(R0914)

🔇 Additional comments (7)
utils/VersionSuggester.py (4)

80-85: LGTM! Improved documentation.

The expanded docstring clearly explains the function's behavior, including the new logic for checking the next major release when all minor versions are vulnerable. This aligns well with the PR objective.


90-106: LGTM! Cleaner logic structure.

The refactored approach of separating candidate versions into minor_candidates and higher_major lists makes the code more readable and maintainable compared to the previous implementation.


117-127: LGTM! Excellent implementation of PR objective.

This new logic successfully implements the optimization described in the PR objectives: "if all minor versions of a package are found to be vulnerable, the system will now check the major version instead." The implementation correctly finds the next major version and checks for non-vulnerable versions there.


131-133: LGTM! Appropriate exception handling adjustment.

Changing from logger.error to logger.warning is appropriate since this function gracefully handles failures by returning "unknown" rather than causing the program to fail. The pragma comment correctly indicates this is hard to test in unit tests.

utils/WhlRepacker.py (1)

24-27: Improve error handling and cleanup.

The current implementation doesn't clean up the temp directory when METADATA is not found. Consider using a try-finally block or context manager.

-    if not metadata_file or not metadata_file.exists():
-        print(f"[ERROR] METADATA not found in: {wheel_path.name}")
-        shutil.rmtree(temp_dir)
-        return
+    if not metadata_file or not metadata_file.exists():
+        print(f"[ERROR] METADATA not found in: {wheel_path.name}")
+        if temp_dir.exists():
+            shutil.rmtree(temp_dir)
+        return

Likely an incorrect or invalid review comment.

GenerateReport.py (2)

119-130: LGTM! Excellent improvement in configurability.

The change from hardcoded file path to using the NOTUSED_PACKAGES environment variable significantly improves configurability. The enhanced logging that shows the actual file path being used is also helpful for debugging. This change addresses the PR objective for fixing the NotUsed.txt handling.


339-344: LGTM! Consistent timezone handling implementation.

Using now_sg() instead of datetime.now() ensures that the monthly report filenames use Singapore time (UTC+8), which aligns with the PR objective to make monthly report filenames consistent with weekly report filenames. Storing the timestamp in a separate variable also improves code readability.

@TongWu TongWu temporarily deployed to WT_WeeklyTriggerEnv June 26, 2025 03:05 — with GitHub Actions Inactive
@TongWu TongWu merged commit 3979097 into main Jun 26, 2025
3 checks passed
@TongWu TongWu had a problem deploying to WT_WeeklyTriggerEnv June 26, 2025 03:05 — with GitHub Actions Error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant