Skip to content

feat: support per-blog User-Agent overrides for feeds#7

Open
Sowevo wants to merge 4 commits intoHyaxia:mainfrom
Sowevo:feat/custom-user-agent
Open

feat: support per-blog User-Agent overrides for feeds#7
Sowevo wants to merge 4 commits intoHyaxia:mainfrom
Sowevo:feat/custom-user-agent

Conversation

@Sowevo
Copy link
Copy Markdown

@Sowevo Sowevo commented Mar 1, 2026

Summary

  • add per-blog --user-agent on blogwatcher add (similar to --scrape-selector)
  • remove global --user-agent to keep behavior simple and explicit
  • scanning now uses per-blog UA only (or default UA if not configured)

Why

Different sites may require different User-Agent values. Per-blog settings are clearer and avoid confusing precedence rules.

Changes

  • model/storage:
    • add user_agent field to Blog
    • add user_agent column in DB schema (with migration via ALTER TABLE ... ADD COLUMN)
  • CLI:
    • blogwatcher add ... --user-agent "..." to persist per-blog UA
    • blogwatcher blogs now prints configured per-blog UA
    • removed global --user-agent flag
  • scanner:
    • use only blog-specific UA when present
    • pass effective UA to feed discovery, feed fetch, and scraper fetch
  • docs:
    • README examples for per-blog UA

Verification

  • go test ./... passes
  • blogwatcher --help no longer shows global --user-agent

@Sowevo
Copy link
Copy Markdown
Author

Sowevo commented Mar 1, 2026

Sorry for the PR churn — I moved too fast 🙇
I’ve simplified it to per-blog --user-agent only and removed the global fallback.
Thanks for the patience 🐾

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant