CTI Scraper & Dashboard

This project is a Cyber Threat Intelligence (CTI) tool designed to scrape Dark Web (Tor) sources, store the data, and visualize it in a web dashboard.

Features

• Tor Scraper: Interactive scraping using Chromedp over Tor SOCKS5 proxy.
• Data Collection: Captures HTML content, screenshots, generating titles and criticality scores automatically.
• Dashboard: Web interface to view, search, and analyze collected intelligence.
• Dockerized: specific container setup for seamless deployment.

Technologies Used

• Go (Golang): Core logic for scraping and web server.
• Chromedp: For headless browser automation and rendering.
• GORM (PostgreSQL): Database ORM.
• Gin: Web framework.
• React + Tailwind CSS: Modern Frontend UI.
• Docker & Docker Compose: Containerization.

Frontend Setup (Required for Dashboard)

The interface has been upgraded to a modern React Single Page Application. You must build the UI assets before running the Go server (unless using Docker).

cd ui
npm install
npm run build
cd ..

How to Run with Docker (Recommended)

This is the easiest way to run the full application (Scraper + New UI + Database + Tor).

1. Ensure Docker and Docker Compose are installed.
2. Edit targets.yaml to add Tor URLs (.onion) you want to monitor.
3. Run the following command:

docker-compose up --build

4. Access the dashboard at http://localhost:8080.
5. Default credentials to login dashboard is admin/admin

The Docker build automatically compiles the React Frontend and the Go Backend.

Criticality Scoring

Each content is analyzed for specific keywords (e.g., 'leak', 'database', 'hack', 'password').

• Base Score: 1
• +1 for each matching keyword found in the content.
• Max Score: 10