Skip to content

Robot detection #1

@wnsite

Description

@wnsite

Hi! Cool code, using it all the time. One thing, some websites can detect robots and shoot a 403: Forbidden or something like that causing the dump to fail. I have verified that if you go to the website in a normal browser, it does not send that error

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions