Name		Name	Last commit message	Last commit date
Latest commit History 33 Commits
CrawlerApp.Client		CrawlerApp.Client
CrawlerApp.Console		CrawlerApp.Console
CrawlerApp.DataStore		CrawlerApp.DataStore
CrawlerApp.Test		CrawlerApp.Test
.gitignore		.gitignore
CrawlerApp.sln		CrawlerApp.sln
README.md		README.md

Repository files navigation

Final Project Web Crawler

Requirements

Use of the Gearman model.
Use a MySQL database for the storage system.
Web crawler has a ReST API for all features and functions.
Web crawler has a web page for reports.
Scalable
Polite

Definition

Architecture

Web Crawler Architecture

About

A polite web crawler that scrapes links from websites.

Report repository

Releases

No releases published

Packages

Contributors

Languages