Summary
The scraper parsing jobs have minimal test coverage. ParseMeetingPageJob and ParseAgendaJob have no tests. DiscoverMeetingsJob has partial coverage limited to committee resolution.
What's already covered
- Committee resolution during discovery — 3 tests in
test/jobs/scrapers/discover_meetings_committee_test.rb (exact name match, alias match, unrecognized name). These test Committee.resolve, not the full discovery/parsing flow.
Jobs still needing tests
Scrapers::ParseMeetingPageJob (app/jobs/scrapers/parse_meeting_page_job.rb)
- Parses detail page HTML for a single meeting
- Extracts agenda items, motions, voting records
- Downloads linked documents
Test scenarios:
- Parse a fixture detail page → creates correct
AgendaItem records
- Identifies and enqueues document downloads
- Handles pages with no agenda/documents gracefully
- Idempotent re-run doesn't duplicate agenda items
Scrapers::ParseAgendaJob (app/jobs/scrapers/parse_agenda_job.rb)
- Parses agenda HTML documents
Test scenarios:
- Parse agenda HTML → creates structured agenda items
- Handles various agenda formats
- Idempotent behavior
Scrapers::DiscoverMeetingsJob — full flow (optional expansion)
- Parse a fixture HTML listing page with multiple meetings → creates correct Meeting records
- Handles duplicate meetings (idempotent re-run)
- Enqueues downstream
ParseMeetingPageJob for each meeting
Approach
- Create HTML fixture files (
test/fixtures/files/) from real Two Rivers pages (sanitized if needed)
- Stub HTTP requests with
WebMock or Minitest::Mock
- Test both happy path and edge cases (missing data, malformed HTML)
Related
Summary
The scraper parsing jobs have minimal test coverage.
ParseMeetingPageJobandParseAgendaJobhave no tests.DiscoverMeetingsJobhas partial coverage limited to committee resolution.What's already covered
test/jobs/scrapers/discover_meetings_committee_test.rb(exact name match, alias match, unrecognized name). These testCommittee.resolve, not the full discovery/parsing flow.Jobs still needing tests
Scrapers::ParseMeetingPageJob(app/jobs/scrapers/parse_meeting_page_job.rb)Test scenarios:
AgendaItemrecordsScrapers::ParseAgendaJob(app/jobs/scrapers/parse_agenda_job.rb)Test scenarios:
Scrapers::DiscoverMeetingsJob— full flow (optional expansion)ParseMeetingPageJobfor each meetingApproach
test/fixtures/files/) from real Two Rivers pages (sanitized if needed)WebMockorMinitest::MockRelated