feat: add in-scope/out-of-scope CLI flags and workspace config

[Copilot]

Adds --in-scope/-is and --out-of-scope/-os CLI flags for HTTP crawler tasks, with workspace-level configuration support via workspace.yaml.

Changes

• Definitions: Added IN_SCOPE and OUT_OF_SCOPE constants
• Meta Options: Added scope options to HTTP crawler tasks (OPTS_HTTP_CRAWLERS)
• Katana Integration: Mapped to native -crawl-scope and -crawl-out-scope flags
• Workspace Config:
  • Added Workspaces config class with in_scope/out_of_scope list fields
  • Auto-loads from <workspace>/workspace.yaml if present
  • CLI options override workspace config
• Tests: Added unit tests for CLI flags and workspace config loading

Usage

# Via CLI
secator task katana https://example.com --in-scope ".*example\.com.*" --out-of-scope ".*admin.*"

# Via workspace config (~/.secator/reports/myworkspace/workspace.yaml)
in_scope:
  - '.*example\.com.*'
out_of_scope:
  - '.*admin.*'

secator task katana https://example.com --workspace myworkspace

Notes

• gospider and cariddi marked scope options as OPT_NOT_SUPPORTED (no native equivalents)
• Scope patterns are comma-separated regex strings when passed via CLI
• Workspace YAML format supports list notation for better readability

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repos/projectdiscovery/katana/releases/tags/v1.3.0
  • Triggering command: /home/REDACTED/.local/bin/secator secator task katana --workspace test_workspace REDACTED --sync -q (http block)
  • Triggering command: /home/REDACTED/.local/bin/secator secator task katana --workspace test_workspace REDACTED --in-scope .*override\.com.* --sync -q (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>feat: add in-scope / out-of-scope cli flags</issue_title>
<issue_description>- Add in-scope / out-of-scope cli flags (for instance --in-scope / -is and --out-of-scope / -os)

• Add a meta option for in-scope / out-of-scope for all tasks
• Make sure to search and implement appropriate options for tasks that natively can take inscope / outoscope lists. For instance: katana has the following options:
  -cs, -crawl-scope string[] in scope url regex to be followed by crawler
  -cos, -crawl-out-scope string[] out of scope url regex to be excluded by crawler
• Add a way to add a YAML config for workspaces where these things can be defined</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes feat: add in-scope / out-of-scope cli flags #869

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}