Skip to content

Fix user scoring bugs — inverted spam_label, wrong quotient scale #674

@realproject7

Description

@realproject7

Problem

The user scoring script (`scripts/score-users.mjs`) from PR #659 has critical logic errors that produced wrong results for the majority of users.

Bugs to fix

Bug 1: `spam_label` logic is INVERTED (CRITICAL)

Current (wrong):
```javascript
const isSpam = spamRaw.length > 0 && spamRaw !== "0";
if (isSpam) return { score: 0, tag: "None" };
```

Reality:

  • `spam_label = 2` → non-spammer (good user) — 4,848 users
  • `spam_label = 0` → not verified / potential spam — 2,435 users
  • `spam_label = ""` (blank) → unknown — 52 users

Fix:
```javascript
// spam_label=2 means non-spammer. 0 or blank = not verified.
// Don't disqualify based on spam_label — use as a scoring signal instead.
const spamVerified = spamRaw === "2"; // verified non-spammer
// Add bonus for verified non-spammer, slight penalty for unverified
```

Impact: 4,848 legitimate users (including project7 with 8,978 followers) were incorrectly scored 0.

Bug 2: `quotient_score` scale is wrong

Current (wrong):
```javascript
const quotientScore = Math.min(1, num("quotient_score") / 100);
```

Reality: Quotient score range is 0 to ~0.95 (already 0-1 scale). Dividing by 100 makes it effectively zero.

Fix:
```javascript
const quotientScore = Math.min(1, Math.max(0, num("quotient_score")));
```

Impact: Quotient signal contributed ~0% to scores for all 5,775 users with quotient data.

Re-run requirements

After fixing the logic:

  1. Re-run the script: `node scripts/score-users.mjs`
  2. Output to same file: `archive/dropcast-users-scored-20260330.csv`
  3. Post updated summary stats as ticket comment (tag counts, score distribution, top 20)
  4. Verify project7 (FID 8106) scores appropriately high (8,978 followers, 0.99 neynar, 0.706 quotient, X verified, pro subscriber)

Verification reference

project7 expected signals:

  • FC followers: 8,978 → log-scaled to 100K = ~0.79
  • X followers: 2,389 → log-scaled to 500K = ~0.59
  • X verified: true → 1.0
  • Neynar score: 0.99
  • Quotient score: 0.706
  • Pro subscriber: true
  • spam_label: 2 (verified non-spammer)
  • Bio: 100+ chars, contains "designer", "Coder"
  • Has Twitter: yes

Expected score should be 60+, not 0.

Branch

`task/673-fix-user-scoring`

Self-Verification (T3)

  • project7 (FID 8106) scores 60+ with tag "Writer"
  • spam_label=2 users are NOT zeroed out
  • spam_label=0 users get a penalty (not instant zero)
  • Quotient scores used as-is (0-1), not divided by 100
  • Updated summary stats posted as comment
  • Score distribution looks reasonable (top users 70-90 range)
  • `npm run build` passes

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/T3Assigned to T3 builder agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions