Skip to content

SanemSarioglu/pentesting-api

BranchesTags
 
 

Repository files navigation

🔍 Internal Vulnerability Scanner API

Professional-grade web application vulnerability scanning API powered by Nuclei and OWASP ZAP

API Version Python Docker License

📋 Table of Contents

🚀 Quick Start

Start the API

git clone <repository_url>
cd pentesting-api
docker-compose up -d

Run Your First Scan

# Quick Nuclei scan (safe for any target)
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://testphp.vulnweb.com",
    "engine": "nuclei",
    "options": {"type": "quick"}
  }'

# Or try ZAP passive scan (also safe)
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://testphp.vulnweb.com",
    "engine": "zap-passive",
    "enhanced_passive": true
  }'

Check Results

# Get scan status
curl http://localhost:5000/scan/{scan_id}/status

# View HTML report
open http://localhost:5000/report/{scan_id}/html

🔧 Installation

Prerequisites

  • Docker & Docker Compose
  • 4GB+ RAM recommended
  • Network access to target systems

Deploy with Docker

# Clone repository
git clone <repository_url>
cd pentesting-api

# Start services
docker-compose up -d

# Verify installation
curl http://localhost:5000/health

Services Overview

Service Port Description
API Server 5000 Main vulnerability scanner API
OWASP ZAP 8080 Web application security scanner

📊 Complete API Reference

Base URL

http://localhost:5000

Authentication

Currently no authentication required. ⚠️ Add authentication in production!

🔗 All Available Endpoints

1. Health Check

Check API status and available engines

GET /health

Response:

{
  "status": "healthy",
  "service": "Internal Vulnerability Scanner",
  "version": "1.1.0",
  "engines_available": ["nuclei", "zap"]
}

cURL Example:

curl http://localhost:5000/health

2. Start Vulnerability Scan

Initiate a new security scan

POST /scan
Content-Type: application/json

Request Body:

{
  "url": "string (required)",
  "engine": "string (required)",
  "options": "object (optional)",
  "auth": "object (optional, ZAP only)",
  "enable_parallel": "boolean (optional, ZAP active only)",
  "max_workers": "integer (optional, ZAP active only)",
  "enhanced_passive": "boolean (optional, ZAP passive only)"
}

Parameters:

  • url: Target URL to scan (must include http/https)
  • engine: Scanner engine (nuclei, zap, zap-passive, zap-active)
  • options: Engine-specific configuration
  • auth: Authentication configuration (ZAP only)
    • username: Username for form-based authentication
    • password: Password for authentication
    • login_url: Login form URL
  • enable_parallel: Enable parallel scanning (ZAP active only, default: true)
  • max_workers: Number of parallel scan workers (ZAP active only, default: 3)
  • enhanced_passive: Enable enhanced passive scanning features (ZAP passive only, default: true)

Response:

{
  "message": "Scan started successfully",
  "scan_id": "123e4567-e89b-12d3-a456-426614174000",
  "target": "http://example.com",
  "engine": "nuclei",
  "status_url": "/scan/123e4567-e89b-12d3-a456-426614174000/status",
  "report_url": "/report/123e4567-e89b-12d3-a456-426614174000",
  "html_report_url": "/report/123e4567-e89b-12d3-a456-426614174000/html"
}

cURL Examples:

Basic Nuclei Scan:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://example.com",
    "engine": "nuclei",
    "options": {"type": "quick"}
  }'

Custom Nuclei Scan:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://example.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["tech", "cve", "sqli", "xss"],
      "severity": ["critical", "high"],
      "intensity": "aggressive"
    }
  }'

ZAP Enhanced Passive Scan:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://example.com",
    "engine": "zap-passive",
    "enhanced_passive": true
  }'

ZAP Advanced Active Scan with Authentication (FIXED):

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://testphp.vulnweb.com/login.php",
    "engine": "zap-active",
    "auth": {
      "username": "test",
      "password": "test",
      "login_url": "http://testphp.vulnweb.com/login.php"
    },
    "enable_parallel": false,
    "max_workers": 1
  }'

PowerShell Example (Windows):

$body = '{"url": "http://testphp.vulnweb.com", "engine": "zap-active", "enable_parallel": false, "max_workers": 1}'
Invoke-RestMethod -Uri "http://localhost:5000/scan" -Method POST -ContentType "application/json" -Body $body

3. Get Scan Status

Check scan progress and basic results

GET /scan/{scan_id}/status

Response:

{
  "status": "completed",
  "target": "http://example.com",
  "engine": "nuclei",
  "duration": 45.23,
  "result": {
    "success": true,
    "scan_id": "123e4567-e89b-12d3-a456-426614174000",
    "vulnerabilities_found": 5
  }
}

Status Values:

  • in_progress: Scan is running
  • completed: Scan finished successfully
  • failed: Scan encountered an error

cURL Example:

curl http://localhost:5000/scan/123e4567-e89b-12d3-a456-426614174000/status

4. Get JSON Report

Retrieve detailed scan results in JSON format

GET /report/{scan_id}

Response:

{
  "scan_id": "123e4567-e89b-12d3-a456-426614174000",
  "target": "http://example.com",
  "engine": "nuclei",
  "engine_version": "v3.4.7",
  "scan_date": "2025-07-25 17:30:41",
  "scan_intensity": "comprehensive",
  "status": "completed",
  "summary": {
    "total": 5,
    "critical": 1,
    "high": 2,
    "medium": 1,
    "low": 0,
    "info": 1
  },
  "vulnerabilities": [
    {
      "title": "SQL Injection",
      "severity": "critical",
      "description": "SQL injection vulnerability detected",
      "url": "http://example.com/login.php",
      "template": "sql-injection",
      "template_path": "/templates/sql-injection.yaml",
      "cve": ["CVE-2021-1234"],
      "reference": ["https://owasp.org/..."],
      "type": "http",
      "matcher_status": true,
      "curl_command": "curl -X POST ...",
      "host": "example.com",
      "port": "80",
      "scheme": "http",
      "timestamp": "2025-07-25T17:30:20Z",
      "extracted_results": []
    }
  ],
  "scan_statistics": {
    "duration_seconds": 300,
    "templates_used": ["tech", "cve", "sqli"],
    "severity_filter": ["critical", "high", "medium", "info"],
    "scan_command": "/usr/local/bin/nuclei ..."
  },
  "scan_warnings": []
}

cURL Example:

curl http://localhost:5000/report/123e4567-e89b-12d3-a456-426614174000

5. Get HTML Report

View formatted vulnerability report in browser

GET /report/{scan_id}/html

Returns a styled HTML page with:

  • Executive summary
  • Vulnerability breakdown by severity
  • Detailed findings with remediation steps
  • Technical details and proof-of-concept

Browser Example:

http://localhost:5000/report/123e4567-e89b-12d3-a456-426614174000/html

cURL Example:

curl http://localhost:5000/report/123e4567-e89b-12d3-a456-426614174000/html > report.html

6. List All Scans

Get overview of all performed scans

GET /scans

Response:

{
  "scans": [
    {
      "scan_id": "123e4567-e89b-12d3-a456-426614174000",
      "target": "http://example.com",
      "engine": "nuclei",
      "status": "completed"
    },
    {
      "scan_id": "456e7890-e89b-12d3-a456-426614174001",
      "target": "http://testsite.com",
      "engine": "zap-active",
      "status": "in_progress"
    }
  ]
}

cURL Example:

curl http://localhost:5000/scans

7. List Available Engines

Get scanner engines and their capabilities

GET /engines

Response:

{
  "engines": {
    "nuclei": {
      "name": "Nuclei",
      "description": "Fast vulnerability scanner based on simple YAML-based DSL",
      "capabilities": ["passive", "template-based"],
      "scan_types": ["quick", "comprehensive"]
    },
    "zap": {
      "name": "OWASP ZAP",
      "description": "The OWASP Zed Attack Proxy",
      "capabilities": ["passive", "active", "authentication", "parallel"],
      "scan_types": ["passive", "active"]
    }
  }
}

cURL Example:

curl http://localhost:5000/engines

8. API Documentation

Get API overview and example requests

GET /

Response:

{
  "service": "Internal Vulnerability Scanner API",
  "version": "1.1.0",
  "endpoints": {
    "POST /scan": "Start vulnerability scan",
    "GET /scan/<id>/status": "Get scan status",
    "GET /report/<id>": "Get JSON report",
    "GET /report/<id>/html": "Get HTML report",
    "GET /scans": "List all scans",
    "GET /engines": "List available engines",
    "GET /health": "Health check"
  },
  "example_scan_requests": {
    "quick_scan": {
      "url": "http://example.com",
      "engine": "nuclei",
      "options": {"type": "quick"}
    },
    "comprehensive_scan": {
      "url": "http://example.com",
      "engine": "nuclei",
      "options": {"type": "comprehensive"}
    },
    "deep_scan": {
      "url": "http://example.com",
      "engine": "nuclei",
      "options": {"type": "deep"}
    },
    "custom_scan": {
      "url": "http://example.com",
      "engine": "nuclei",
      "options": {
        "type": "custom",
        "tags": ["tech", "cve", "sqli", "xss", "rce"],
        "severity": ["critical", "high", "medium"],
        "intensity": "aggressive"
      }
    },
    "zap_advanced_scan": {
      "url": "http://example.com",
      "engine": "zap-active",
      "auth": {
        "username": "user",
        "password": "pass",
        "login_url": "http://example.com/login"
      },
      "enable_parallel": true,
      "max_workers": 3
    }
  },
  "scan_types": {
    "quick": "Fast scan with basic templates (3 min)",
    "comprehensive": "Thorough scan with many templates (10 min)", 
    "deep": "Maximum coverage scan (15 min)",
    "custom": "User-defined scan parameters"
  },
  "available_intensities": ["light", "normal", "aggressive", "maximum"],
  "common_tags": ["tech", "cve", "misconfig", "sqli", "xss", "rce", "file-upload", "lfi", "ssrf", "idor", "auth-bypass", "owasp-top-10", "exposure"]
}

cURL Example:

curl http://localhost:5000/

🛡️ Security Engines

Nuclei Scanner

Fast, template-based vulnerability scanner

Scan Types & Actual Timing

Type Duration Rate Limit Templates Description
quick 4 min 30/sec tech, cve, misconfig Light reconnaissance scan
comprehensive 12 min 35/sec 10+ categories Aggressive vulnerability assessment
deep 20 min 20/sec All templates Maximum template coverage
custom Variable User-defined User-defined Fully customizable scan

All Available Nuclei Options

Quick Scan:

{
  "url": "http://example.com",
  "engine": "nuclei",
  "options": {
    "type": "quick"
  }
}

Comprehensive Scan:

{
  "url": "http://example.com",
  "engine": "nuclei",
  "options": {
    "type": "comprehensive"
  }
}

Deep Scan:

{
  "url": "http://example.com",
  "engine": "nuclei",
  "options": {
    "type": "deep"
  }
}

Custom Scan with All Options:

{
  "url": "http://example.com",
  "engine": "nuclei",
  "options": {
    "type": "custom",
    "tags": ["tech", "cve", "sqli", "xss", "rce", "file-upload", "lfi", "ssrf", "idor", "auth-bypass", "owasp-top-10", "exposure", "misconfig"],
    "severity": ["critical", "high", "medium", "low", "info"],
    "intensity": "maximum"
  }
}

Complete Template Tags List

Vulnerability Categories:

  • tech - Technology detection and fingerprinting
  • cve - CVE vulnerabilities
  • misconfig - Configuration issues and misconfigurations
  • sqli - SQL injection vulnerabilities
  • xss - Cross-site scripting
  • rce - Remote code execution
  • file-upload - File upload vulnerabilities
  • lfi - Local file inclusion
  • ssrf - Server-side request forgery
  • idor - Insecure direct object reference
  • auth-bypass - Authentication bypass
  • owasp-top-10 - OWASP Top 10 vulnerabilities
  • exposure - Information exposure and leaks

Severity Levels

  • critical - Critical vulnerabilities requiring immediate attention
  • high - High-impact vulnerabilities
  • medium - Medium-risk vulnerabilities
  • low - Low-impact vulnerabilities
  • info - Information gathering and fingerprinting

Intensity Levels & Actual Settings

Intensity Rate Limit Timeout Retries Use Case
light 30/sec 8s 0 Fast reconnaissance
normal 50/sec 12s 0 Balanced scanning
aggressive 35/sec 18s 2 Thorough assessment
maximum 20/sec 25s 3 Deep vulnerability hunting

OWASP ZAP Scanner

🆕 Advanced web application security testing with enhanced features

ZAP Scan Types & New Capabilities

Enhanced Passive Scan (Safe for Production):

{
  "url": "http://example.com",
  "engine": "zap-passive",
  "enhanced_passive": true
}
  • Duration: 2-5 minutes (enhanced mode)
  • Method: Advanced spider + AJAX spider + passive analysis
  • Safety: ✅ Safe for production systems
  • Coverage: Comprehensive URL discovery with modern web app support
  • Features:
    • Enhanced spider configuration (depth: 8, children: 30)
    • AJAX spider for SPA applications
    • Robots.txt and sitemap parsing
    • Comment and form parameter discovery

Advanced Active Scan (⚠️ Test Systems Only):

{
  "url": "http://example.com",
  "engine": "zap-active",
  "enable_parallel": true,
  "max_workers": 3
}
  • Duration: 10-20 minutes (depending on site size)
  • Method: Multi-phase comprehensive security testing
  • Safety: ⚠️ Only for test environments
  • Coverage: Maximum vulnerability detection with actual exploits
  • Features:
    • Advanced scan policies with 20+ vulnerability rules
    • Parallel scanning for performance
    • Deep spider crawling (depth: 10, children: 50)
    • AJAX spider for modern web applications
    • Comprehensive attack simulation

🔐 Advanced Active Scan with Authentication:

{
  "url": "http://example.com",
  "engine": "zap-active",
  "auth": {
    "username": "testuser",
    "password": "testpass123",
    "login_url": "http://example.com/login"
  },
  "enable_parallel": true,
  "max_workers": 5
}
  • Authentication: Form-based authentication support
  • Session Management: Maintains authenticated sessions
  • Coverage: Tests authenticated areas of the application
  • Security: Credentials handled securely and cleaned up after scan

🆕 New ZAP Advanced Features

1. Authentication Support 🔐
  • Form-based authentication configuration
  • Session management for authenticated scanning
  • Automatic credential cleanup after scan completion
  • Support for complex login flows
2. Parallel Active Scanning ⚡
  • Configurable parallel workers (default: 3, max recommended: 5)
  • Intelligent URL chunking for optimal performance
  • Reduces scan time by 40-60% for large applications
  • Automatic load balancing across scan workers
3. Enhanced Spider Configuration 🕷️
  • Deep Crawling: Up to 10 levels deep
  • Comprehensive Coverage: 50 URLs per page
  • Smart Parsing: Robots.txt, sitemap.xml, comments
  • Form Handling: Automatic form discovery and processing
  • Parameter Detection: Advanced parameter identification
4. AJAX Spider Support 🌐
  • SPA Applications: Full JavaScript rendering support
  • Dynamic Content: Discovers AJAX-loaded content
  • Modern Web Apps: React, Angular, Vue.js compatibility
  • Complementary Discovery: Works alongside traditional spider
5. Advanced Scan Policies ⚙️
  • 20+ Vulnerability Rules: Comprehensive coverage
  • High Attack Strength: Maximum detection capability
  • Low Alert Threshold: Catches subtle vulnerabilities
  • Custom Payloads: Advanced injection techniques
  • Specific Rules Include:
    • SQL Injection (MySQL, PostgreSQL, Oracle, SQLite)
    • Cross-Site Scripting (Reflected & Persistent)
    • Remote Code Execution
    • CRLF Injection
    • Server-Side Includes
    • Parameter Tampering
6. Enhanced Reporting 📊
  • Detailed Statistics: URL discovery metrics, scan phases
  • Performance Metrics: Duration breakdown by phase
  • Coverage Analysis: Forms, parameters, authentication status
  • Enhanced Vulnerability Details: Evidence, payloads, confidence levels
  • Scan Configuration Tracking: Authentication, parallel settings

ZAP Scan Process Overview

🔍 Enhanced Passive Scan Process:

  1. Advanced Spider Configuration (30 sec)
  2. Comprehensive Spider Crawling (2-3 min)
  3. AJAX Spider for Modern Content (1-2 min)
  4. Passive Security Analysis (30 sec)
  5. Report Generation with Statistics (15 sec)

💥 Advanced Active Scan Process:

  1. 🔐 Authentication Setup (30 sec)
  2. 🕷️ Advanced Spider Configuration (30 sec)
  3. ⚙️ Advanced Scan Policy Creation (45 sec)
  4. 🔍 Comprehensive Spider Crawling (3-5 min)
  5. 🌐 AJAX Spider for Modern Web Apps (2-3 min)
  6. 🔍 Passive Scan Processing (15 sec)
  7. 💥 Advanced Active Vulnerability Scanning (5-15 min)
  8. 📊 Results Collection and Analysis (30 sec)
  9. 🧹 Resource Cleanup (15 sec)

ZAP Performance Optimization

📈 Performance by Site Size:

Site Size Enhanced Passive Advanced Active Advanced Active + Auth
Small (5-20 pages) 1-2 min 5-8 min 6-10 min
Medium (20-100 pages) 2-5 min 10-20 min 12-25 min
Large (100+ pages) 5-10 min 20-40 min 25-45 min

⚡ Parallel Scanning Benefits:

  • 2 Workers: 30-40% faster than single scan
  • 3 Workers: 40-50% faster (recommended)
  • 4-5 Workers: 50-60% faster (for large sites only)

💡 Complete Usage Examples

1. Technology Stack Discovery

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["tech"],
      "severity": ["info"],
      "intensity": "light"
    }
  }'

2. Critical Vulnerability Hunt

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["cve", "sqli", "xss", "rce", "auth-bypass"],
      "severity": ["critical", "high"],
      "intensity": "aggressive"
    }
  }'

3. OWASP Top 10 Assessment

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["owasp-top-10", "sqli", "xss", "rce", "idor", "auth-bypass", "exposure"],
      "intensity": "maximum"
    }
  }'

4. Complete Security Audit

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "nuclei",
    "options": {
      "type": "deep"
    }
  }'

5. Safe Production Scan (Enhanced)

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://production-site.com",
    "engine": "zap-passive",
    "enhanced_passive": true
  }'

6. 🆕 Advanced Test Environment Scan

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://test-environment.com",
    "engine": "zap-active",
    "enable_parallel": true,
    "max_workers": 3
  }'

7. 🆕 Authenticated Application Security Testing

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://webapp.test.com",
    "engine": "zap-active",
    "auth": {
      "username": "testuser",
      "password": "testpass123",
      "login_url": "http://webapp.test.com/login"
    },
    "enable_parallel": true,
    "max_workers": 4
  }'

8. 🆕 High-Performance Parallel Scanning

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://large-application.test.com",
    "engine": "zap-active",
    "enable_parallel": true,
    "max_workers": 5
  }'

9. Batch Scanning Multiple Targets

#!/bin/bash
targets=("http://site1.com" "http://site2.com" "http://site3.com")
scan_ids=()

echo "Starting batch scan of ${#targets[@]} targets..."

for target in "${targets[@]}"; do
  echo "Scanning: $target"
  
  response=$(curl -s -X POST http://localhost:5000/scan \
    -H "Content-Type: application/json" \
    -d "{
      \"url\": \"$target\",
      \"engine\": \"nuclei\",
      \"options\": {\"type\": \"comprehensive\"}
    }")
  
  scan_id=$(echo $response | jq -r '.scan_id')
  scan_ids+=("$scan_id")
  echo "Scan ID: $scan_id"
done

echo "Monitoring ${#scan_ids[@]} scans..."

for scan_id in "${scan_ids[@]}"; do
  echo "Monitoring scan: $scan_id"
  
  while true; do
    status=$(curl -s http://localhost:5000/scan/$scan_id/status | jq -r '.status')
    target=$(curl -s http://localhost:5000/scan/$scan_id/status | jq -r '.target')
    
    echo "[$target] Status: $status"
    
    if [[ "$status" == "completed" ]]; then
      vulns=$(curl -s http://localhost:5000/report/$scan_id | jq '.summary.total')
      echo "[$target] ✅ Completed - Found $vulns vulnerabilities"
      echo "[$target] Report: http://localhost:5000/report/$scan_id/html"
      break
    elif [[ "$status" == "failed" ]]; then
      echo "[$target] ❌ Failed"
      break
    fi
    
    sleep 30
  done
done

echo "All scans completed!"

10. Real-time Monitoring Script

#!/bin/bash
scan_id="$1"

if [ -z "$scan_id" ]; then
    echo "Usage: $0 <scan_id>"
    exit 1
fi

echo "Monitoring scan: $scan_id"
echo "Press Ctrl+C to stop monitoring"

while true; do
    response=$(curl -s http://localhost:5000/scan/$scan_id/status)
    status=$(echo $response | jq -r '.status')
    target=$(echo $response | jq -r '.target')
    duration=$(echo $response | jq -r '.duration')
    
    timestamp=$(date '+%Y-%m-%d %H:%M:%S')
    echo "[$timestamp] $target - Status: $status - Duration: ${duration}s"
    
    if [[ "$status" == "completed" ]]; then
        vulns=$(echo $response | jq -r '.result.vulnerabilities_found // 0')
        echo "🎉 Scan completed! Found $vulns vulnerabilities"
        echo "📊 JSON Report: http://localhost:5000/report/$scan_id"
        echo "📋 HTML Report: http://localhost:5000/report/$scan_id/html"
        break
    elif [[ "$status" == "failed" ]]; then
        error=$(echo $response | jq -r '.error // "Unknown error"')
        echo "❌ Scan failed: $error"
        break
    fi
    
    sleep 10
done

📈 Advanced Features

🆕 ZAP Advanced Authentication

Enterprise Application Authentication:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://enterprise-app.com",
    "engine": "zap-active",
    "auth": {
      "username": "security_tester",
      "password": "complex_password123!",
      "login_url": "https://enterprise-app.com/auth/login"
    },
    "enable_parallel": true,
    "max_workers": 3
  }'

Benefits:

  • Tests authenticated functionality
  • Discovers auth-protected vulnerabilities
  • Session management for complex apps
  • Automatic credential cleanup

🆕 High-Performance Parallel Scanning

Large Application Optimization:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://large-ecommerce-site.test",
    "engine": "zap-active",
    "enable_parallel": true,
    "max_workers": 5
  }'

Performance Gains:

  • 40-60% faster than sequential scanning
  • Intelligent URL chunking
  • Load-balanced worker distribution
  • Optimized for large applications (100+ pages)

Custom Scan Configurations

Targeted SQL Injection Testing:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://webapp.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["sqli"],
      "severity": ["critical", "high", "medium"],
      "intensity": "maximum"
    }
  }'

XSS Vulnerability Assessment:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://webapp.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["xss"],
      "severity": ["critical", "high", "medium"],
      "intensity": "aggressive"
    }
  }'

Configuration and Exposure Audit:

curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://webapp.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "tags": ["misconfig", "exposure"],
      "intensity": "normal"
    }
  }'

API Integration Examples

Python SDK Example:

import requests
import json
import time

class VulnScanner:
    def __init__(self, base_url="http://localhost:5000"):
        self.base_url = base_url
    
    def start_scan(self, url, engine="nuclei", options=None, **kwargs):
        """Start a vulnerability scan with advanced options."""
        payload = {"url": url, "engine": engine}
        if options:
            payload["options"] = options
        
        # Add ZAP-specific parameters
        for key in ['auth', 'enable_parallel', 'max_workers', 'enhanced_passive']:
            if key in kwargs:
                payload[key] = kwargs[key]
        
        response = requests.post(f"{self.base_url}/scan", json=payload)
        if response.status_code == 202:
            return response.json()
        else:
            raise Exception(f"Scan failed: {response.text}")
    
    def start_authenticated_scan(self, url, username, password, login_url, 
                                engine="zap-active", parallel=True, workers=3):
        """Start an authenticated ZAP scan."""
        return self.start_scan(
            url=url,
            engine=engine,
            auth={
                "username": username,
                "password": password,
                "login_url": login_url
            },
            enable_parallel=parallel,
            max_workers=workers
        )
    
    def wait_for_completion(self, scan_id, poll_interval=10, timeout=1800):
        """Wait for scan completion with timeout."""
        start_time = time.time()
        
        while time.time() - start_time < timeout:
            status = self.get_status(scan_id)
            
            if status["status"] == "completed":
                return self.get_report(scan_id)
            elif status["status"] == "failed":
                raise Exception(f"Scan failed: {status.get('error', 'Unknown error')}")
            
            print(f"Scan {scan_id[:8]}... - Status: {status['status']} - Duration: {status.get('duration', 0):.1f}s")
            time.sleep(poll_interval)
        
        raise Exception(f"Scan timeout after {timeout} seconds")
    
    def get_status(self, scan_id):
        """Get scan status."""
        response = requests.get(f"{self.base_url}/scan/{scan_id}/status")
        return response.json()
    
    def get_report(self, scan_id):
        """Get detailed JSON report."""
        response = requests.get(f"{self.base_url}/report/{scan_id}")
        return response.json()
    
    def list_scans(self):
        """List all scans."""
        response = requests.get(f"{self.base_url}/scans")
        return response.json()

# 🆕 Advanced Usage Examples
scanner = VulnScanner()

# Enhanced passive scan
print("🔍 Starting enhanced passive vulnerability scan...")
result = scanner.start_scan(
    "http://testphp.vulnweb.com",
    engine="zap-passive",
    enhanced_passive=True
)
print(f"✅ Scan started: {result['scan_id']}")

# Advanced authenticated scan
print("🔐 Starting authenticated security assessment...")
auth_result = scanner.start_authenticated_scan(
    url="http://webapp.test.com",
    username="testuser",
    password="testpass123",
    login_url="http://webapp.test.com/login",
    parallel=True,
    workers=4
)

# Wait for completion and get results
report = scanner.wait_for_completion(auth_result["scan_id"])
print(f"📊 Authenticated scan completed! Found {report['summary']['total']} vulnerabilities")
print(f"   Critical: {report['summary']['critical']}")
print(f"   High: {report['summary']['high']}")
print(f"   Medium: {report['summary']['medium']}")

# Enhanced vulnerability analysis
if 'statistics' in report:
    stats = report['statistics']
    print(f"\n📈 Enhanced Statistics:")
    if 'total_urls_discovered' in stats:
        print(f"   - Total URLs discovered: {stats['total_urls_discovered']}")
    if 'authentication_enabled' in stats:
        print(f"   - Authentication used: {stats['authentication_enabled']}")
    if 'parallel_scanning_enabled' in stats:
        print(f"   - Parallel scanning: {stats['parallel_scanning_enabled']}")
    print(f"   - Scan duration: {stats.get('total_scan_duration', 'N/A')}")

# Custom comprehensive scan
print("\n🔍 Starting comprehensive vulnerability scan...")
comprehensive_result = scanner.start_scan(
    "http://testphp.vulnweb.com",
    options={
        "type": "custom",
        "tags": ["tech", "cve", "sqli", "xss", "rce", "file-upload"],
        "severity": ["critical", "high", "medium"],
        "intensity": "aggressive"
    }
)

comprehensive_report = scanner.wait_for_completion(comprehensive_result["scan_id"])
print(f"📊 Comprehensive scan completed! Found {comprehensive_report['summary']['total']} vulnerabilities")

# List all scans
all_scans = scanner.list_scans()
print(f"\n📋 Total scans performed: {len(all_scans['scans'])}")

JavaScript/Node.js Example:

const axios = require('axios');

class VulnScanner {
    constructor(baseUrl = 'http://localhost:5000') {
        this.baseUrl = baseUrl;
    }

    async startScan(url, engine = 'nuclei', options = null, extraParams = {}) {
        const payload = { url, engine, ...extraParams };
        if (options) payload.options = options;

        try {
            const response = await axios.post(`${this.baseUrl}/scan`, payload);
            return response.data;
        } catch (error) {
            throw new Error(`Scan failed: ${error.response?.data || error.message}`);
        }
    }

    // 🆕 Advanced authenticated scan method
    async startAuthenticatedScan(url, username, password, loginUrl, options = {}) {
        return this.startScan(url, 'zap-active', null, {
            auth: { username, password, login_url: loginUrl },
            enable_parallel: options.parallel !== false,
            max_workers: options.workers || 3
        });
    }

    // 🆕 Enhanced passive scan method  
    async startEnhancedPassiveScan(url) {
        return this.startScan(url, 'zap-passive', null, {
            enhanced_passive: true
        });
    }

    async waitForCompletion(scanId, pollInterval = 10000, timeout = 1800000) {
        const startTime = Date.now();

        while (Date.now() - startTime < timeout) {
            const status = await this.getStatus(scanId);

            if (status.status === 'completed') {
                return await this.getReport(scanId);
            } else if (status.status === 'failed') {
                throw new Error(`Scan failed: ${status.error || 'Unknown error'}`);
            }

            console.log(`Scan ${scanId.substring(0, 8)}... - Status: ${status.status} - Duration: ${status.duration || 0}s`);
            await new Promise(resolve => setTimeout(resolve, pollInterval));
        }

        throw new Error(`Scan timeout after ${timeout / 1000} seconds`);
    }

    async getStatus(scanId) {
        const response = await axios.get(`${this.baseUrl}/scan/${scanId}/status`);
        return response.data;
    }

    async getReport(scanId) {
        const response = await axios.get(`${this.baseUrl}/report/${scanId}`);
        return response.data;
    }
}

// 🆕 Advanced Usage Examples
async function runAdvancedScans() {
    const scanner = new VulnScanner();
    
    try {
        // Enhanced passive scan
        console.log('🔍 Starting enhanced passive scan...');
        const passiveResult = await scanner.startEnhancedPassiveScan('http://testphp.vulnweb.com');
        console.log(`✅ Enhanced passive scan started: ${passiveResult.scan_id}`);
        
        // Authenticated active scan
        console.log('🔐 Starting authenticated active scan...');
        const authResult = await scanner.startAuthenticatedScan(
            'http://webapp.test.com',
            'testuser',
            'testpass123',
            'http://webapp.test.com/login',
            { parallel: true, workers: 4 }
        );
        
        console.log(`✅ Authenticated scan started: ${authResult.scan_id}`);
        
        // Wait for authenticated scan completion
        const report = await scanner.waitForCompletion(authResult.scan_id);
        
        console.log(`📊 Scan completed! Found ${report.summary.total} vulnerabilities`);
        console.log(`   Critical: ${report.summary.critical}`);
        console.log(`   High: ${report.summary.high}`);
        console.log(`   Medium: ${report.summary.medium}`);
        
        // 🆕 Enhanced statistics analysis
        if (report.statistics) {
            const stats = report.statistics;
            console.log('\n📈 Enhanced Statistics:');
            console.log(`   - URLs discovered: ${stats.total_urls_discovered || 'N/A'}`);
            console.log(`   - Authentication: ${stats.authentication_enabled ? 'Enabled' : 'Disabled'}`);
            console.log(`   - Parallel scanning: ${stats.parallel_scanning_enabled ? 'Enabled' : 'Disabled'}`);
            console.log(`   - Scan duration: ${stats.total_scan_duration || 'N/A'}`);
            
            if (stats.scan_coverage) {
                console.log(`   - Forms found: ${stats.scan_coverage.forms_found || 0}`);
                console.log(`   - Parameters tested: ${stats.scan_coverage.parameters_tested || 0}`);
            }
        }
        
    } catch (error) {
        console.error('❌ Scan failed:', error.message);
    }
}

runAdvancedScans();

Direct Nuclei Integration

# Run Nuclei directly in container for custom workflows
docker exec pentesting-api-vuln-scanner-1 \
  nuclei -target http://example.com \
  -tags tech,cve \
  -severity critical,high \
  -rate-limit 30 \
  -timeout 10 \
  -jsonl -silent

ZAP Advanced Configuration

Access ZAP UI at http://localhost:8080 for:

  • Manual security testing
  • Custom scan policies
  • Authentication configuration
  • Advanced reporting

⚙️ Configuration

Environment Variables

# docker-compose.yml
environment:
  - FLASK_ENV=development
  - FLASK_DEBUG=1
  - SECRET_KEY=your-secret-key-change-in-production
  - ZAP_HOST=zap
  - ZAP_PORT=8080
  - NUCLEI_PATH=/usr/local/bin/nuclei
  - REPORTS_DIR=reports
  - MAX_SCAN_DURATION=300
  - RATE_LIMIT_PER_MINUTE=10

🆕 Advanced ZAP Configuration Parameters

Parameter Default Description
ZAP_MAX_SPIDER_DEPTH 10 Maximum spider crawling depth
ZAP_MAX_SPIDER_CHILDREN 50 Maximum URLs per page
ZAP_SPIDER_TIMEOUT 600 Spider timeout in seconds
ZAP_AJAX_TIMEOUT 300 AJAX spider timeout
ZAP_MAX_PARALLEL_SCANS 5 Maximum parallel scan workers
ZAP_SCAN_POLICY_STRENGTH High Attack strength level
ZAP_ALERT_THRESHOLD Low Alert detection threshold

Volume Mounts

volumes:
  - ./app/reports:/app/reports        # Scan reports storage
  - ./app:/app                        # Application code
  - nuclei_templates:/root/.nuclei    # Nuclei templates

Port Configuration

ports:
  - "5000:5000"  # API Server
  - "8080:8080"  # ZAP Proxy UI

Performance Tuning

# Increase memory for large scans
deploy:
  resources:
    limits:
      memory: 4G
      cpus: '2'
    reservations:
      memory: 2G
      cpus: '1'

🔒 Security Considerations

Production Deployment Checklist

  • Add Authentication: Implement API key or OAuth
  • Rate Limiting: Configure per-IP rate limits
  • HTTPS Only: Use TLS encryption
  • Network Isolation: Restrict scanner network access
  • Input Validation: Whitelist target domains
  • Logging: Enable comprehensive audit logging
  • Monitoring: Set up alerting for scan activities
  • 🆕 Credential Security: Secure handling of authentication data
  • 🆕 Resource Limits: Configure parallel scan limits

Safe Scanning Practices

  • Get Permission: Only scan systems you own or have written permission to test
  • Test Environment First: Always test on non-production systems
  • Rate Limiting: Configure appropriate scan speeds to avoid DoS
  • Monitoring: Monitor resource usage during scans
  • ZAP Active Scans: Use only in dedicated test environments
  • 🆕 Authentication Data: Use test credentials only, never production credentials
  • 🆕 Parallel Scanning: Monitor server resources when using multiple workers

Example Production Configuration

# production-docker-compose.yml
version: '3.8'
services:
  vuln-scanner:
    build: .
    ports:
      - "5000:5000"
    environment:
      - FLASK_ENV=production
      - FLASK_DEBUG=0
      - SECRET_KEY=${SECRET_KEY}
      - RATE_LIMIT_PER_MINUTE=5
      - MAX_SCAN_DURATION=600
      # 🆕 Advanced ZAP security settings
      - ZAP_MAX_PARALLEL_SCANS=2
      - ZAP_SCAN_TIMEOUT=900
    volumes:
      - /secure/reports:/app/reports
    restart: unless-stopped
    
  zap:
    image: owasp/zap:stable
    command: zap.sh -daemon -host 0.0.0.0 -port 8080 -config api.disablekey=true
    ports:
      - "127.0.0.1:8080:8080"  # Bind to localhost only
    restart: unless-stopped

🐛 Troubleshooting

Common Issues & Solutions

"Scan not found" Error

# Check if scan ID is correct
curl http://localhost:5000/scans | jq '.scans[] | select(.scan_id | contains("partial-id"))'

# List all recent scans
curl http://localhost:5000/scans | jq '.scans | sort_by(.scan_id) | reverse'

ZAP Connection Failed

# Check ZAP container status
docker-compose ps zap
docker-compose logs zap

# Test ZAP API directly
curl http://localhost:8080/JSON/core/view/version/

# Restart ZAP if needed
docker-compose restart zap

🆕 Authentication Issues (Updated)

# Run comprehensive authentication debug script
python debug_zap_auth.py

# Check authentication configuration in logs
docker-compose logs vuln-scanner | grep "Authentication"

# Check for userid parameter errors (FIXED in v1.1.1)
docker-compose logs vuln-scanner | grep "userid"

# Verify ZAP forced user mode setup
docker-compose logs vuln-scanner | grep "forced user"

# Check authentication verification process
docker-compose logs vuln-scanner | grep "Verifying authentication"

# Verify login URL accessibility
curl -I http://your-app.com/login

# Test authentication manually
curl -X POST http://your-app.com/login \
  -d "username=testuser&password=testpass123" \
  -c cookies.txt -v

# Test with working examples
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://testphp.vulnweb.com",
    "engine": "zap-active",
    "enable_parallel": false,
    "max_workers": 1
  }'

🆕 Parallel Scanning Issues

# Check system resources during parallel scans
docker stats

# Reduce parallel workers if needed
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "zap-active",
    "enable_parallel": true,
    "max_workers": 2
  }'

# Monitor active scan progress
docker-compose logs vuln-scanner | grep "parallel"

Nuclei Templates Outdated

# Update Nuclei templates manually
docker exec pentesting-api-vuln-scanner-1 nuclei -update-templates

# Verify template update
docker exec pentesting-api-vuln-scanner-1 nuclei -stats

Memory/Performance Issues

# Check container resource usage
docker stats

# Monitor during scan
docker exec pentesting-api-vuln-scanner-1 top

# Increase memory limits in docker-compose.yml
deploy:
  resources:
    limits:
      memory: 4G

Scan Timeouts

# Check scan configuration in logs
docker-compose logs vuln-scanner | grep "Enhanced" 

# Adjust scan intensity for faster results
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://target.com",
    "engine": "nuclei",
    "options": {
      "type": "custom",
      "intensity": "light"
    }
  }'

Debug Mode

# Enable verbose logging
docker-compose logs -f vuln-scanner

# Check specific scan logs
docker-compose logs vuln-scanner | grep "scan_id_here"

# ZAP debug logs
docker-compose logs zap | grep ERROR

# 🆕 Advanced ZAP debugging
docker-compose logs vuln-scanner | grep "Phase"
docker-compose logs vuln-scanner | grep "Authentication"
docker-compose logs vuln-scanner | grep "parallel"

🆕 Authentication Debug Script

New comprehensive debugging tool for authentication issues:

# Run the authentication debug script
python debug_zap_auth.py

This script automatically tests:

  • ✅ API health and connectivity
  • ✅ ZAP service direct connection
  • ✅ Simple scan without authentication
  • ✅ Authentication scan (where issues typically occur)

Example output:

🐛 ZAP AUTHENTICATION DEBUG SCRIPT
==================================================
⏰ Started at: 2025-07-30 14:30:00

==================== TEST 1: API HEALTH ====================
🏥 Testing API health...
✅ API is healthy: {'status': 'healthy', 'service': 'Internal Vulnerability Scanner'}

==================== TEST 2: ZAP DIRECT ====================
🔍 Testing ZAP service directly...
📡 Testing ZAP at: http://localhost:8080/JSON/core/view/version/
✅ ZAP is running! Version: {'version': '2.16.1'}

==================== TEST 3: SIMPLE SCAN ====================
🧪 Testing simple scan without auth...
✅ Simple scan started: a15fb705-9aa7-4857-a3c0-030131abda80

==================== TEST 4: AUTH SCAN ====================
🔐 Testing auth scan (the problematic one)...
✅ Auth scan completed successfully!

==================== SUMMARY ====================
api_health     : ✅ PASS
zap_direct     : ✅ PASS  
simple_scan    : ✅ PASS
auth_scan      : ✅ PASS

Health Checks

# Comprehensive system check
curl http://localhost:5000/health && echo "✅ API healthy"
curl http://localhost:5000/engines && echo "✅ Engines available"  
curl http://localhost:8080/JSON/core/view/version/ && echo "✅ ZAP healthy"

# Test basic scan functionality
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://httpbin.org/ip",
    "engine": "nuclei",
    "options": {"type": "quick"}
  }' && echo "✅ Scan endpoint working"

# 🆕 Test advanced ZAP features (FIXED)
curl -X POST http://localhost:5000/scan \
  -H "Content-Type: application/json" \
  -d '{
    "url": "http://testphp.vulnweb.com",
    "engine": "zap-active",
    "enable_parallel": false,
    "max_workers": 1
  }' && echo "✅ ZAP active scanning working"

📊 API Response Codes

Code Status Description Example Response
200 OK Request successful {"status": "healthy"}
202 Accepted Scan started (async operation) {"scan_id": "123...", "message": "Scan started"}
400 Bad Request Invalid request parameters {"error": "URL is required"}
404 Not Found Scan ID not found {"error": "Scan not found"}
500 Internal Server Error Server error occurred {"error": "Scan engine failed"}

📚 Additional Resources

Documentation Links

Community Resources

Learning Resources

🆕 What's New in v1.1.1 (Latest)

🔧 Critical Bug Fixes

  • ✅ FIXED: ZAP Authentication Issues: Resolved userid parameter errors in spider and active scans
  • ✅ FIXED: Forced User Mode: Proper authentication context management for all scan types
  • ✅ FIXED: API Parameter Handling: Correct boolean and string parameter formatting for ZAP API
  • ✅ FIXED: Spider Authentication: Removed unsupported parameters, using forced user mode correctly
  • ✅ FIXED: Active Scan Authentication: Proper context and user management for authenticated scanning

🚀 Enhanced Features

  • 🔐 Multi-Type Authentication: Basic, Digest, and Form-based authentication support
  • 🔍 Authentication Verification: Advanced verification with multiple fallback methods
  • ⚡ Improved Error Handling: Better error messages and debugging information
  • 📊 Enhanced Logging: Detailed phase-by-phase scan progress tracking
  • 🧪 Debug Tools: New debug script for troubleshooting authentication issues

🛠️ New Testing Tools

  • debug_zap_auth.py: Comprehensive authentication debugging script
  • test_scan_comparison.py: Compare passive vs active scan results
  • test_auth_attacks.py: Authentication attack test cases and examples

🔧 Working Examples (Tested & Verified)

{
  "url": "http://testphp.vulnweb.com",
  "engine": "zap-active",
  "enable_parallel": false,
  "max_workers": 1
}
{
  "url": "http://testphp.vulnweb.com/login.php",
  "engine": "zap-active",
  "auth": {
    "username": "test",
    "password": "test",
    "login_url": "http://testphp.vulnweb.com/login.php"
  },
  "enable_parallel": false,
  "max_workers": 1
}

Previous Major Enhancements (v1.1.0)

  • 🔐 Authentication Support: Form-based authentication for ZAP scans
  • ⚡ Parallel Scanning: Multi-worker active scanning for improved performance
  • 🕷️ Enhanced Spider: Advanced crawling with AJAX support for modern web apps
  • 📊 Detailed Statistics: Comprehensive scan metrics and performance data
  • 🌐 SPA Support: AJAX spider for Single Page Applications
  • ⚙️ Advanced Policies: 20+ specific vulnerability detection rules
  • 🔍 Enhanced Passive: Improved passive scanning with better coverage

Performance Improvements

  • 40-60% faster scanning with parallel workers
  • Better coverage for modern web applications
  • Enhanced discovery with AJAX spider support
  • Optimized resource usage with intelligent load balancing
  • ✅ Stable Authentication: Reliable authenticated scanning without API errors

See Also

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🤝 Contributing

We welcome contributions! Please see our contributing guidelines:

  1. Fork the repository
  2. Create feature branch: git checkout -b feature/amazing-feature
  3. Commit changes: git commit -m 'Add amazing feature'
  4. Push to branch: git push origin feature/amazing-feature
  5. Open Pull Request

Development Setup

# Clone for development
git clone <repository_url>
cd pentesting-api

# Install development dependencies  
pip install -r requirements-dev.txt

# Run tests
python -m pytest tests/

# Start development server
python app/main.py

🔍 Happy Hunting! Stay ethical, scan responsibly.

Remember: Always obtain proper authorization before scanning any systems you don't own.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Python 97.0%
  • Shell 2.4%
  • Dockerfile 0.6%