Added E2E testing setup using Playwright

[nusje2000]

Summary

Add end-to-end test suite and CI pipeline for Djehuty. The e2e tests use Playwright to validate core platform workflows (authentication, dataset CRUD, file management, publishing, collaboration, collections, search, admin, and more) against a fully running instance with a SPARQL endpoint.

When a PR is opened or commits are pushed, CI spins up a Djehuty instance with a Virtuoso SPARQL service, then runs the test suite in parallel using a matrix strategy. This strategy spawns one job per pytest marker (smoke, auth, dataset, files, etc.). Each job collects coverage data from the running application. Once all test jobs complete, a final job combines the coverage artifacts and posts a coverage report as a PR comment. Concurrency groups ensure that pushing new commits automatically cancels any in-progress runs for the same branch.

A minor bug was found and fixed during test development: the dataset search API defaulted to no scopes instead of all scopes, causing empty search results.

Changes

• .github/workflows/e2e.yml: CI workflow that spins up the full stack via Docker Compose and runs Playwright e2e tests.
• .github/workflows/ci.yml: Basic CI workflow triggered on push and PR.
• .github/workflows/coverage.yml: Pytest workflow with code coverage reporting.
• tests/e2e/: Playwright test suite covering core platform workflows (authentication, dataset CRUD, file management, publishing, collaboration, collections, search, admin, and more).
• tests/api/: API-level tests for dataset endpoints.
• docker/, .github/ci-djehuty.xml, test-data/: Test infrastructure and seed data for the CI environment.
• src/djehuty/web/database.py: Fix dataset search to default to all scopes.
• README.md: Added testing setup instructions.

Approval Checklist

• I agree to follow Djehuty's code of conduct.
• I have read and I have follow the code contribution workflow.
• Code style and conventions were respected.
• Documentation has been updated where needed (README, docs, or examples).
• Review approved by at least one maintainer.
• Merge readiness (PR is squashed into a single commit and follows the commit template).

[nusje2000]

You can find the test results here: https://github.com/riotbyte-com/djehuty/actions/runs/23766168922. Since this is coming from our fork, the Github actions are not being executed until this PR gets merged.

[gabrielakuhn]

Hi @nusje2000 , thank you for opening this PR! I added a fell comments besides the reviews we already did in person. I have some tests failing in my local setup and I believe I need to improve the test data a bit. I will contact you by email 😉

[k-f-a]

That is an impressive PR @nusje2000! 🎉

nit: This PR has significant overlap with #57 (uv/just/Docker dev setup). I'd suggest rebasing on top of #57 once it merges if @gabrielakuhn agrees

- #57 introduces just dev with Docker Compose, Virtuoso, and auto-initialization the CI workflows here should build on that rather than creating a parallel Docker setup.
- #57 replaces pyproject.toml.in with pyproject.toml and uses uv.lock. The requirements-dev.txt here should become [project.optional-dependencies] in pyproject.toml instead.
- #57 rewrites the README — the testing instructions should go into that new structure.

[k-f-a]

LGTM.

[k-f-a]

Hi @nusje2000,
There is a easy conflict, just the .gitignore as I could see :)
Looking forward to merging this PR :)

[nusje2000]

Conflicts have been fixed, comments have been resolved and you can find a successfull build in our fork repository: https://github.com/riotbyte-com/djehuty/actions/runs/24664054040

I also created #76 for changing the default search scope in the API so it will start returning results when not providing a scope for the search (defaulting to all scopes if non are passed). I've updated the test in this PR to expect a 500 request so this does not need that change and when this PR gets merged, I'll make the change to the test in the PR containing the fix.

[gabrielakuhn]

Thank you @nusje2000 for applying the changes! All good now.

Please note that you need to squash the commit following the commit message template before merge it. As you have quite a bit of files instead of reference the files you can reference the folder and do a brief explanation of what kind of files is maintained in new introduced folder each folder.