feat: Use uv and Python 3.12 #4
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
imports on multilines to help debug
refactor names to ignore case as that's how they're written in CFG
This change containerizes the application using Docker and updates the video download logic to use yt-dlp instead of youtube-dl. Key changes: - Adds a Dockerfile to create a container image for the application. - Adds an entrypoint.sh script that updates yt-dlp on container startup, using a persistent cache. - Refactors the `twl_downloader.py` script to be Python 3 compatible, use environment variables for configuration, and use the `yt-dlp` library directly. - Removes the interactive `setup.py` script, as configuration is now handled via environment variables. - Updates the README.md with instructions on how to build and run the Docker container.
This change updates the project to use the `uv` package manager and upgrades the Python version to 3.12. - The `Dockerfile` is updated to use the `python:3.12-slim` base image and installs `uv` for dependency management. - The `entrypoint.sh` script is updated to use `uv`. - The `Makefile` is updated to use `uv` for local dependency installation, and adds the `--system` flag as required by `uv`.
Author
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll acknowledge your comments with a 👀 emoji and then get to work. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! I will automatically address your feedback. For any comments you don't want me to act on, just include (aside). For security, I will only act on instructions from the user who triggered this task for this pull request. |
This change replaces flake8 with ruff for linting and formatting, and adds VSCode integration for a smoother development experience. - Replaces flake8 with ruff in development dependencies. - Adds a `pyproject.toml` to configure ruff with black-compatible formatting. - Updates the `Makefile` to use `ruff` for the `lint` and `format` targets. - Adds a `.vscode/settings.json` file to configure the editor to use ruff for formatting on save and displaying linting errors. - The codebase has been reformatted using `ruff format`.
This change incorporates advanced features from the twl-dl-server project, enhancing the downloader's capabilities. - The `yt-dlp` download options have been updated to prefer 4K resolution and the MKV container format. - Adds support for a YouTube cookies file, allowing downloads of age-restricted or private videos. This is configured via the `YOUTUBE_COOKIES_FILE` environment variable. - The `README.md` and test suite have been updated to reflect these new features.
This change refactors the Dockerfile to use a multi-stage build pattern, resulting in a smaller and more efficient final container image. - The Dockerfile is now split into a "builder" stage and a final stage. - The builder stage creates a virtual environment and installs dependencies. - The final stage copies only the necessary artifacts (the venv, the uv binary, and app scripts) from the builder, excluding build-time dependencies and intermediate files.
This change adds several advanced features to improve the quality of the downloaded videos and their associated metadata. - The script now embeds chapters into the final MKV file from both YouTube's native chapters and SponsorBlock data. The SponsorBlock categories can be configured via an environment variable. - NFO file generation is enhanced to include richer metadata, such as the video's upload date and the date it was downloaded. - The video file's modification time is now set to match the video's original upload date, allowing for chronological sorting in media libraries. - The `yt-dlp` format selection is set to `bestvideo+bestaudio/best` to ensure the highest quality download. - Documentation and tests have been updated to reflect these new features.
This change adds several advanced features to improve the quality of the downloaded videos and their associated metadata, ensuring compatibility with Jellyfin. - NFO file generation is updated to be compliant with the Jellyfin specification, including tags like `<uniqueid>`, `<releasedate>`, `<dateadded>`, genres, and tags. - The script now embeds chapters into the final MKV file from both YouTube's native chapters and SponsorBlock data. The SponsorBlock categories can be configured via an environment variable. - The video file's modification time is now set to match the video's original upload date, allowing for chronological sorting in media libraries. - The `yt-dlp` format selection is set to `bestvideo+bestaudio/best` to ensure the highest quality download. - Documentation and tests have been updated to reflect these new features.
This change improves the project's testing methodology and documentation based on user feedback. - Testing is now performed inside a Docker container by default using a new `tester` stage in the Dockerfile. The `make test` command now orchestrates this. The previous local test runner is available as `make test-local`. - The `README.md` has been updated with a more concrete `docker run` example, using the specific paths requested by the user. - A new unit test was added to verify that the Kodi notification logic is correctly skipped when not configured. - A bug in the `Makefile`'s `build` target was fixed to ensure it builds the final production image, not an intermediate stage.
This change corrects an error in the Dockerfile that caused the build to fail. The `uv pip install` command was not correctly targeting the virtual environment. - The `RUN` commands in the `builder` and `tester` stages now use the `--python` flag to specify the virtual environment's Python interpreter, ensuring dependencies are installed in the correct location.
This change adds an end-to-end integration test to verify the core download functionality with a real-world example. - Adds a new `test_integration.py` file with a test that downloads a specific YouTube video and uses `ffprobe` to verify its duration and resolution. - The `Dockerfile` is updated with a `tester` stage that includes `ffmpeg` as a dependency for video analysis. - The `Makefile` is updated with a more granular testing strategy (`test`, `test-integration`, `test-all`) to separate fast unit tests from slow integration tests. - A check for `ffmpeg` was added to the `test-local` target to provide a better error message if it's not installed. - The `README.md` is updated to document the new local dependency and testing targets.
This change updates the integration test to be more accurate, based on user-provided ffprobe output.
- The `ffprobe` command in `test_integration.py` is updated to match the user's example.
- The assertions now check for the exact duration string ("213.068000") and height (2160), removing the previous tolerance-based check.
This change updates the documentation to be more user-friendly and consistent. - Updates the `Makefile` and `README.md` to use the standardized `twl-downloader` image name. - Updates the `README.md` `docker run` example to use the specific host paths provided by the user. - Adds a new section to the `README.md` with detailed instructions on how to acquire and use a `cookies.txt` file for YouTube.
This change adds more flexibility to the script's configuration based on user feedback. - Replaces the `TWL_DOWNLOAD_TO_TMP` boolean with a `TWL_TMP_DOWNLOAD_LOCATION` path. If this variable is set, downloads will go to the specified temporary directory before being moved. If it is not set, downloads go directly to the final destination. - Adds a `TWL_LOOKBACK_DAYS` environment variable to allow easy configuration of the time window for fetching new videos. - Updates the `.env.example` file and tests to reflect these new configuration options.
This change corrects a `KeyError` in the integration test that occurred after refactoring the temporary download path configuration. - The `config` dictionary in `test_integration.py` is updated to use the `tmp_download_location` key instead of the old `download_to_tmp` key, aligning it with the latest script logic.
This change refactors the application to use a strict container-first path management strategy, resolving inconsistencies between the code and the documented Docker workflow. - The Python script is updated to use hardcoded paths (`/downloads`, `/tmp`) for all file operations, removing the ability to configure these via environment variables. - The `Makefile` is simplified by removing the `run-local` target, reinforcing that the primary execution method is via Docker. - The `README.md` and `.env.example` files are updated to remove the obsolete path configuration variables, simplifying the setup process for the user. - The test suite is updated to align with the new hardcoded path logic.
This change corrects several critical runtime errors identified in user feedback. - Installs `ffmpeg` in the final Docker image stage, which is required by `yt-dlp` for merging video and audio formats. - Removes the non-root `appuser` from the Dockerfile. The container now runs as root, which resolves the `PermissionError` when writing to mounted volumes. - Updates the `Makefile` and `.gitignore` with the user's provided versions. - Adds detailed instructions to the `README.md` on how to acquire and use a `cookies.txt` file for YouTube downloads.
…DRM downloads; update README and tests accordingly
…te configuration and tests
nickv2002
force-pushed
the
feat/containerize-yt-dlp
branch
from
3ac1b12 to
1ce192e
Compare
- Refactor Dockerfile with parallel build stages for faster builds * Split python-builder and deno-installer into independent parallel stages * Add BuildKit cache mounts for /root/.cache/pip and /var/cache/apt for faster rebuilds * Install ffmpeg directly in final stage instead of copying selectively * Consolidate related RUN commands to reduce layers - Ensure ffmpeg is properly available in final image * ffmpeg required by yt-dlp for video processing * Direct apt installation is more reliable than library copying - Fix test configurations to include missing keys * Add js_runtimes: ["deno"] to all test configs * Add remote_components: ["ejs:github"] to all test configs * These keys are required by get_video_metadata() for EJS support Result: - All 11 unit tests passing - Faster rebuilds with parallel stages and BuildKit cache mounts - Slimmer final image by removing curl/unzip (only in deno-installer) - More reliable ffmpeg detection in container
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change updates the project to use the
uvpackage manager for faster dependency installation and upgrades the environment to Python 3.12.PR created automatically by Jules for task 10630908006539006988