Entity/Stance tracking pipeline: extraction, services, and UI

[AndreRobitaille]

Summary

The data models for entity and stance tracking are fully implemented in the schema (entities, entity_facts, entity_mentions, stance_observations) but there is no extraction pipeline, no services, and no UI to populate or display this data.

Current State

Models (Exist)

• Entity — name, entity_type, aliases (jsonb), status, notes
• EntityFact — entity_id, fact_text, source_type, source_ref, status, sensitive, verified_on, verification_notes
• EntityMention — entity_id, meeting_id, meeting_document_id, page_number, raw_name, quote, context
• StanceObservation — entity_id, meeting_id, meeting_document_id, page_number, position, quote, sentiment, confidence, topic

Missing Components

1. Extraction Jobs

• ExtractEntitiesJob — Identify council members, department heads, public commenters from meeting documents
• ExtractStancesJob — Identify public comment positions, voting patterns, stated positions on topics

2. Services

• Entity resolution service — Match mentions to known entities (alias handling, fuzzy matching)
• Stance aggregation — Summarize entity positions across meetings for a topic

3. Admin UI

• Entity management (view, edit, merge duplicates, manage aliases)
• Entity fact management (add/verify/flag facts)
• Stance observation review

4. Public UI

• Member page (/members/:id) — Currently basic; should show:
  • Voting record across topics
  • Meeting attendance
  • Stance observations (public comments, stated positions)
• Topic page — Add entity/stance section showing who has spoken on a topic
• Meeting page — Link mentions to entity profiles

Design Considerations

• Privacy: EntityFact has a sensitive flag — sensitive facts must be gated in public views
• TOPIC_GOVERNANCE.md: Stance tracking is limited to public commenters per governance rules
• Verification: Facts and stances should flow through admin review before public display
• Confidence scoring: StanceObservation has a confidence field — low-confidence observations should be flagged for review

Suggested Implementation Order

1. Entity extraction from meeting minutes (identify speakers, voters)
2. Link existing Member + Vote data to Entity records
3. Admin UI for entity management
4. Stance extraction from public comment sections
5. Public member page enrichment
6. Topic page entity/stance section

Priority

Medium — The models are ready and the data is flowing through meetings. This is a significant feature addition that would enhance accountability tracking, which is a core project goal.

[AndreRobitaille]

Status (2026-04-14): partially superseded, scope should be narrowed.

Since this issue was filed, the Member + MeetingAttendance + CommitteeMembership + Vote models and the ExtractCommitteeMembersJob / ExtractVotesJob pipeline have absorbed the 'council member / department head / voter' slice of this:

• Member resolution via Member.resolve (alias lookup, auto-alias)
• Attendance tracking per meeting, with departure detection
• Votes + motions linked to topics (via agenda_item_ref as of the topic-page overhaul)
• Admin UI for members, committees, and aliases

Entity / EntityFact / EntityMention / StanceObservation models still exist but remain empty stubs (37-122 bytes each) with no data flowing through them.

Remaining scope worth preserving under this issue:

• Stance observations from public-comment sections (with governance constraints — public commenters only, not officials)
• Public commenter tracking as non-member entities
• Entity facts / verified relational data (though Track official relationships, business interests, and family ties #98 now covers officials' relationships specifically)

Related: #98 (official relationships) is the narrower, higher-priority cut of this. Recommend treating this as the 'public commenter stance' work and letting #98 handle officials.