Lint, Test, and Build PRs using github workflows. #27
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ripariancommit
added a commit
to ripariancommit/rule34Py
that referenced
this pull request
Jun 3, 2025
Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
Owner
|
Merged the master back to develop, tested it, works so far |
The 'all' make target builds the 'html' documentation target. According to the GNU coding standards, the 'all' target "need not rebuild any documentation". Since building the docs implies installing the .[docs] optional python dependencies, it can cause unexpected errors for builders. So remove the 'html' target dependency. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
In several places in rule34Py, we use a type hinting syntax to declare a union of types, with the pipe character. This is only a valid syntax in >py3.10. It causes a TypeError in versions prior to that. Import and use the 'Union' type from the typing module, to provide backwards compatibility. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
Unless we have some specific information that a dependency prior to some version does not work in the project, we should not arbitrarily restrict the dep version. Remove all '>=' dependency constraints in the pyproject.toml. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The tomllib module was added to the standard python distribution starting in version 3.11. Trying to build the docs on versions prior to that will throw an import error. Conditionally prefer to use the third-party 'tomli' module for python runtimes prior to 3.11. Tomli provides a similar interface and intentional backport support for TOML parsing on old pythons. It is licensed under MIT. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The project claims to support python versions back to 3.9, but these aren't necessarily the versions that developers test their changes on. Upgrade the PR check pipeline to use matrix testing of python 3.9 and the latest python runtime on both windows and linux. Also move linting into its own workflow stage, prior to the build/testing stages. Only upload artifacts from the latest-python build/test jobs, since they are probably the most interesting. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
Python 3.9 is the earliest python version that is still supported by python upstream. And it is the earliest version that we reliably test on GH runners (without going to great lengths to manually install an earlier python runtime). So upgrade the minimum supported python version in the package metadata to 3.9. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The upstream rule34Py repo requires that PRs opened against it target the 'develop' ref. Add a pr-quailty check stage to validate the above assertion. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The PR Check pipeline should contain only the logic that we want to run on PR checks - like linting and policy checking. Then we should have a CI pipeline that gates commits into the 'develop' branch based on whether they pass the test suite. Then we should have a CD pipeline that does things like deploy the GH pages.
This project still supports python versions old enough to be before general support for TOML files, and the pyproject.toml specification. builders running python 3.9 might have difficulty following the tool setup instructions without upgrading their setuptools version. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The ruff linter generates a .ruff_cache directory. It is a temporary file that should be ignored. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
This project is now well-tested by unit tests and autotesting. The debug.py script can be removed in favor of the new testing. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The dist target should only build the source distribution, per the GNU standard. To get both the wheel and sdist, simply call ``` make all make dist ``` Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The requests-ratelimiter module "requires" python >=3.7,<4.0. Because it is a dependency of this project, when you try to process the project dependencies with poetry, you get a satisfaction warning that the python versions required by this project (>3.9) technically could include runtimes 4.0+, breaking compat. It's an odd rqeuirement, that honestly should be fixed in upstream requests-ratelimiter. Until we can get that upstreamed, just add '<4.0' to our python requirement to resolve the error. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
The Twine utility is now used to publish the project artifacts, but it is not included in the pyproject dependencies. Add it to the deps. Use a small python script to programmatically extract the project name and version from the pyproject.toml file, since that is the source of the artifact names. Generally clean up the Makefile targets. Signed-off-by: Riparian Commit <ripariancommit@protonmail.com>
ripariancommit
force-pushed
the
dev/autotesting
branch
from
d7bb334 to
54c0fb4
Compare
Contributor
Author
|
Rebased the branch to |
b3yc0d3
added a commit
that referenced
this pull request
Jun 9, 2025
### Added - Added captcha-compliance capabilities. You can now pass the client your captcha clearance token by completing the captcha in your browser, opening the response header, and writing your `cf_clearance` token into the client's `rule34Py.captcha_clearance` attribute and setting the client's `user_agent` attribute to match your browser. - Alternatively, set your execution environment's `R34_CAPTCHA_CLEARANCE` and `R34_USER_AGENT` variables to the appropriate values. The client will read form them during initialization. - See [this user guide](https://b3yc0d3.github.io/rule34Py/guides/captcha-clearance.html) for more information. (#24) - Added a rate limiter for requests to the rule34.xxx base site (the PHP endpoint). By default, the client will now limit API calls that use this endpoint to 1 per second. - This behavior can be disabled by setting `rule34Py.set_base_site_rate_limit(False)`. - There is no rate limit on the api.rule34.xxx endpoint, which is assumed to handle rate-limiting on the server-side. - This new feature requires the `requests-ratelimiter` module. - Added project documentation via sphinx docs. See: https://b3yc0d3.github.io/rule34Py/. It can be built by calling `make html`. (#24) ### Changed - Changed the behavior of the `rule34Py.random_post()` method to function more like the website. The method now accepts no `tag` parameters, and returns a random post ID from all posts on the site. Users who want to use the old behavior of returning a random post from the first 1000 tag-search results are directed to do something like `random.choice(rule34Py.search([tags...]))`. - Changed the `rule34Py.get_pool()` method to return a `Pool` object containing more complete information about a Pool. - The Pool's post ids can be accessed via the `Pool.posts` attribute. - Moved the `:examples/` python recipes into tutorial documentation in the new sphinx documentation (`:docs/tutorials`). (#24) ### Deprecated - Deprecated support for python runtimes 3.5 to 3.8, as they are EOL. The minimum supported python version is now 3.9. (#27) ### Removed - Removed the deprecated `legacy.setup.py` file. Users are instructed to upgrade their setuptools installation and use `pyproject.toml` to build the project. - Removed the `debug.py` file. Users are instructed to use the `make check` Makefile target to run the project's test suite. (#27)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR extends the github workflows to run PR checks, autotest changes, and auto-build distribution artifacts and documentation. It also changes the project's supported python versions (see below).
Changes
setup.legacy.pydebug.pyfiles, since they are replaced by newerpyproject.tomland pytesting.developbranch, and performs linting and commit policy checking (NOOP for now).developref. It runs the project test suite on a matrix of Windows and Ubuntu GH runners, each with python 3.9 and the latest python (currently 3.13). If any environment fails, it will show on the PR.masterbranch. It builds the project distribution and package files, and builds and publishes the project documentation to GH pages.pyproject.toml.pyproject.tomldependencies. I found some guidance that it is best to let these versions float, unless you know for a fact that your project requires a particular version or range..gitignore.htmldocs target from theallMakefile target.Testing
Process
This PR is based on adevelopref that has the recentmasterchanges merged into it. I will rebase this PR and mark it as ready for review once the develop branch is merged.