A state-of-the-art, production-ready automation ecosystem built with Playwright, Python, and Local AI (Ollama). This framework is designed to behave like an autonomous QA engineer—capable of running tests, understanding failures, and providing intelligent reporting with zero manual intervention.
This framework features a "Brain" layer that integrates directly with local AI models (like TinyLlama) via Ollama. Here is the autonomous workflow:
- Detection: When a test fails, the framework immediately captures the Full Trace, Console Logs, and a High-Resolution Screenshot (POC).
- Analysis: The error context is fed into the local AI model. The AI "reads" the failure just like a human engineer would.
- Diagnosis: The AI determines if the failure is a Functional Bug, a Synchronization Issue (Timeout), or a UI Change.
- Reporting: The AI writes a human-readable investigation report, including a Root Cause and a Suggested Fix, which is injected directly into your HTML Dashboard.
Zero-Key Architecture: Unlike other tools, this uses Local AI. It requires No API Keys (OpenAI/DeepSeek), ensuring 100% data privacy and $0 operational costs.
- What it is: A system that replaces manual log checking.
- How it works: Uses NLP to categorize errors and provide debugging recommendations automatically in every report.
- What it is: A stakeholder-ready HTML dashboard.
- How it works: Generates a self-contained report with embedded screenshots, AI insights, and color-coded categorizations (Smoke, Security, API, Regression).
- OWASP Validation: Built-in checks for XSS (Cross-Site Scripting) and SQL Injection (SQLi) vulnerabilities.
- SSL Monitoring: Automatically alerts the team if SSL certificates are expiring within 30 days.
- Security Headers: Validates the presence of critical security headers to protect against clickjacking and MIME-sniffing.
- How it works: On failure, the AI analyzes the page's HTML structure to suggest stable alternative selectors, reducing maintenance time.
This framework is engineered to provide 360-degree quality assurance. Below are the testing types implemented and supported:
| Category | Testing Types Covered | Description |
|---|---|---|
| Functional | Smoke, Sanity, Regression, Retesting, Integration, System, E2E, UAT | Validates core business flows and ensure no regressions occur after code changes. |
| Structural | Black Box, White Box, Gray Box, Boundary Value, Equivalence Partitioning | Uses mathematical validation techniques to ensure input field integrity. |
| Non-Functional | Performance, Load, Stress, Benchmark, Scalability, Stability | Monitors system behavior under load and measures action latency. |
| Security | Vulnerability, Penetration, SQL Injection, XSS, Auth/Authz, SSL | Automated scans for OWASP Top 10 vulnerabilities and secure header validation. |
| Quality of Experience | Usability, UX, Accessibility (WCAG), Localization (i18n/l10n) | Ensures the app is accessible to all users and works across different locales. |
| Compatibility | Cross-Browser, Cross-Platform, Responsive, Mobile Emulation | Validates UI consistency across Chrome, Firefox, Safari, and Mobile viewports. |
| Specialized | API Testing, Exploratory, Ad-hoc, Monkey, Chaos, Resilience | Backend validation and stability testing under unpredictable conditions. |
This ecosystem has achieved Level 5 Autonomy, functioning as a Digital QA Engineer rather than a simple test runner.
-
🕵️ Explorer Agent: Independently crawls the UI, builds relationship graphs, and learns business flows.
-
🤖 Autonomous QA Engineer: NEW! Scans your existing repository to identify testing gaps, then dynamically generates and executes missing scenarios (Security, Edge-case, UX) in real-time.
-
🪄 Generator Agent: Dynamically creates test strategies, scenarios, and mutated test data in real-time.
-
🛡️ Security Agent: Performs offensive audits, predicting and testing for complex vulnerabilities (IDOR, Race conditions, SQLi).
-
🩹 Healing Agent: Automatically recovers broken locators and regenerates selector repositories.
-
🧠 RCA Agent: Performs deep-root cause analysis with historical context.
-
📈 Learning Agent: Powered by a Long-Term Memory System to predict bugs based on historical failure patterns.
-
📝 Reporting Agent: Generates stakeholder dashboards with visual POCs and technical audits.
- Playwright + Python: Core automation engine.
- Brain Layer: Qwen / DeepSeek / TinyLlama (via Ollama).
- Memory Layer: Persistent JSON-based intelligence (Expandable to Vector DB).
- Vision AI: Visual layout understanding and OCR-based bug detection.
-
Clone the repository:
git clone https://github.com/mejbaurbahar/demo.git cd demo -
Install dependencies:
pip install -r requirements.txt playwright install
-
Run All Tests:
pytest
-
Run Specific Suite:
# Run Smoke tests pytest -m smoke # Run Security & Performance tests pytest -m "security or performance" # Run all Functional tests pytest tests/functional/
The framework is fully integrated with GitHub Actions. On every push:
- It provisions a Linux environment.
- It installs and launches Ollama locally.
- It executes the full test suite.
- It Emails a detailed HTML Report to the stakeholders using your secure Gmail bridge.
The report includes:
- Status Summary: Pass/Fail/Skip counts.
- AI Investigation: Root cause analysis per failure.
- Screenshots: Visual proof for every error.
- Trace Files: Step-by-step recording of the failure.