Skip to content

[DEPRECIATED] 🧠 NerdCrawler is a fully self-hosted search engine built from scratch using React, TailwindCSS, Flask, and Python. Everything β€” from the crawler to the UI β€” was custom-made without third-party APIs.

License

Notifications You must be signed in to change notification settings

KingNixon20/WebCrawlerProject

Repository files navigation

🧠 NerdCrawler ---[DEPRECIATED}---- still works, will always work, but is not longer receiving updates :(

Static Badge Website

alt text NerdCrawler is a fully self-hosted search engine built entirely from scratch β€” including both the frontend and backend. No external APIs, no third-party engines β€” just raw crawling power.

πŸš€ Features

  • 🌐 Self-hosted backend using Python Flask
  • βš›οΈ Frontend built with React and TailwindCSS
  • 🧭 Custom crawler indexing real websites
  • πŸ—ƒοΈ Database support coming in a future update
  • 🎨 Modern UI with support for theming

πŸ› οΈ Tech Stack

Layer Tech
Frontend React, TailwindCSS, Vite
Backend Flask (Python)
Crawler Custom Python-based system
Hosting Fully self-hosted

βš™οΈ Getting Started

To run NerdCrawler locally, you’ll need to:

  1. Clone the repository
  2. Set up your environment
  3. Create a config.py file (see the example included)
  4. Start the Flask server and React frontend with app.py

The Crawler can be run with main.py


πŸ“ config.py Example

Make sure you create your own config.py file in the backend directory. Use the included example as a reference for necessary fields like API keys, port numbers, or crawler settings.


πŸ§ͺ Status

Version 1 is up and running and actively being improved!
Expect bugs, layout changes, and big feature drops.


🀝 Contributing

While this project is currently solo-developed, contributors and testers are always welcome. Feel free to fork the repo or drop suggestions in issues!


About

[DEPRECIATED] 🧠 NerdCrawler is a fully self-hosted search engine built from scratch using React, TailwindCSS, Flask, and Python. Everything β€” from the crawler to the UI β€” was custom-made without third-party APIs.

Resources

License

Stars

Watchers

Forks

Packages

No packages published