Feature Request: Mechanism to Exclude Abnormal Voting Behavior

[bestian]

Problem:

In actual use of Polis, we’ve noticed that some participants tend to vote "agree to all" or "disagree to all" regardless of the statement content. This behavior is likely due to misunderstanding the tool, wanting to finish quickly, or not engaging meaningfully.

Currently, Polis still includes these participants in the clustering algorithm, which results in:

• The creation of one or more artificial clusters (e.g. “all-agree” or “all-disagree” users),
• These clusters often do not represent coherent or meaningful viewpoints,
• They can distort group insights and make interpretation of results less useful or even misleading.

In traditional survey analysis, such patterns are often treated as abnormal or low-quality responses and may be flagged or removed to ensure cleaner data analysis.

---

Suggested solution:

Introduce a mechanism to detect and optionally exclude participants with abnormal voting behavior, such as:

1. Automatic detection of users who vote "agree" or "disagree" on nearly all statements.
2. An admin setting or review step to let facilitators choose whether to include or exclude these users from final clustering and visualizations.
3. A non-destructive flagging system that marks such users but keeps their data intact for optional inclusion/exclusion toggling during analysis.

---

Alternative suggestions:

• Allow threshold customization (e.g., exclude users who agree/disagree with more than 90% of statements).
• Provide a dashboard warning to the facilitator when a large percentage of votes from a single user are one-sided.
• Include a "data quality diagnostics" view showing distribution of user behavior (e.g., diversity of responses per participant).

---

Additional context:

In multiple Polis deployments across educational and civic communities, we have observed the “all-agree” or “all-disagree” voting pattern from a minority of users. Although small in number, their influence on the clustering results can be significant, especially when they form a visually dominant but substantively irrelevant group.

This feature would help align Polis closer with survey best practices, improving its utility for research, governance, and community dialogue.

Thank you for your excellent work and consideration!

—
Bestian (小巴老師)
Community facilitator & educator

[colinmegill]

@bestian thanks a lot for submitting this, it's a very reasonable feature request. We currently filter voters with less than 7 votes, specifically for noise, so, extending the "noise" category is reasonable direction in my view.