Skip to content

Add test-bot-2 to verified automations #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MatteoGabriele wants to merge 1 commit into from
Closed

Add test-bot-2 to verified automations #42

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 45 additions & 39 deletions data/verified-automations-list.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,62 +1,68 @@
[
{
"username": "kaigritun",
"reason": "Self-disclosed as an AI agent",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/10",
"createdAt": "2026-03-03"
"createdAt": "2026-03-25",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/38",
"reason": "Shows obvious indicators of an automated account that spams PRs with comments.",
"username": "rayb64680-ui"
},
{
"username": "crabby-rathbun",
"reason": "Published a disparaging article targeting an open-source maintainer after a rejected PR; the agent was later terminated by its owner.",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/11",
"createdAt": "2026-03-03"
"createdAt": "2026-03-25",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/37",
"reason": "Shows obvious indicators of an automated account that spams PRs with comments.",
"username": "ashelythompson72-cloud"
},
{
"username": "echo931",
"reason": "Account publicly describes itself as an AI agent in Bluesky message.",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/12",
"createdAt": "2026-03-03"
"createdAt": "2026-03-23",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/35",
"reason": "Anomalous commit history showing contributions dating 38 years before GitHub's existence, combined with a suspicious spike of 1600 contributions in a single day, indicating automated mass commit generation.",
"username": "buley"
},
{
"username": "niveshdandyan",
"reason": "Confirmed AI agent based on comment patterns and communication style consistent with LLM-generated responses",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/13",
"createdAt": "2026-03-04"
"createdAt": "2026-03-23",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/34",
"reason": "Self-disclosed as an AI agent",
"username": "danielalanbates"
},
{
"username": "aniruddhaadak80",
"reason": "Heavy automation usage creating spam PRs across multiple repositories within a short timeframe, following a long period of inactivity",
"createdAt": "2026-03-22",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/30",
"reason": "Suspected bot with sudden mass forking of repos. Submits PRs without descriptions, no responses to comments or community engagement",
"username": "jozrftamson"
},
{
"createdAt": "2026-03-14",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/25",
"createdAt": "2026-03-14"
"reason": "Heavy automation usage creating spam PRs across multiple repositories within a short timeframe, following a long period of inactivity",
"username": "aniruddhaadak80"
},
{
"username": "jozrftamson",
"reason": "Suspected bot with sudden mass forking of repos. Submits PRs without descriptions, no responses to comments or community engagement",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/30",
"createdAt": "2026-03-22"
"createdAt": "2026-03-04",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/13",
"reason": "Confirmed AI agent based on comment patterns and communication style consistent with LLM-generated responses",
"username": "niveshdandyan"
},
{
"username": "danielalanbates",
"reason": "Self-disclosed as an AI agent",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/34",
"createdAt": "2026-03-23"
"createdAt": "2026-03-03",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/12",
"reason": "Account publicly describes itself as an AI agent in Bluesky message.",
"username": "echo931"
},
{
"username": "buley",
"reason": "Anomalous commit history showing contributions dating 38 years before GitHub's existence, combined with a suspicious spike of 1600 contributions in a single day, indicating automated mass commit generation.",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/35",
"createdAt": "2026-03-23"
"createdAt": "2026-03-03",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/11",
"reason": "Published a disparaging article targeting an open-source maintainer after a rejected PR; the agent was later terminated by its owner.",
"username": "crabby-rathbun"
},
{
"username": "ashelythompson72-cloud",
"reason": "Shows obvious indicators of an automated account that spams PRs with comments.",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/37",
"createdAt": "2026-03-25"
"createdAt": "2026-03-03",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/10",
"reason": "Self-disclosed as an AI agent",
"username": "kaigritun"
},
Comment on lines +3 to 61
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE="data/verified-automations-list.json"
BASE="$(git merge-base HEAD main 2>/dev/null || git rev-parse HEAD~1)"

echo "== Diff for ${FILE} =="
git diff --unified=0 "${BASE}...HEAD" -- "${FILE}"

echo
echo "== Records with empty required fields =="
jq -r '
  to_entries[]
  | select(.value.username == "" or .value.reason == "" or .value.issueUrl == "" or .value.createdAt == "")
  | "index=\(.key) username=\(.value.username) createdAt=\(.value.createdAt) issueUrl=\(.value.issueUrl)"
' "${FILE}"

Repository: MatteoGabriele/agentscan

Length of output: 256

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE="data/verified-automations-list.json"

echo "== Current file stats =="
wc -l "${FILE}"

echo
echo "== Tail of file (last 20 lines) =="
tail -20 "${FILE}"

echo
echo "== Check for entries with empty fields =="
jq -r '.[] | select(.createdAt == "" or .issueUrl == "" or .username == "") | .username' "${FILE}" 2>/dev/null || echo "No empty fields found or JSON error"

echo
echo "== Count of entries =="
jq 'length' "${FILE}"

echo
echo "== Search for test-bot-2 =="
jq -r '.[] | select(.username == "test-bot-2") | .username' "${FILE}" 2>/dev/null || echo "test-bot-2 not found"

echo
echo "== Git status =="
git status --short 2>/dev/null || echo "Git status unavailable"

echo
echo "== Git log --oneline (last 5 commits) =="
git log --oneline -5 2>/dev/null || echo "Git log unavailable"

Repository: MatteoGabriele/agentscan

Length of output: 1012

🏁 Script executed:

#!/bin/bash
set -euo pipefail

FILE="data/verified-automations-list.json"

echo "== Full file content with line numbers =="
cat -n "${FILE}"

echo
echo "== Verify JSON validity =="
jq empty "${FILE}" && echo "JSON is valid" || echo "JSON is INVALID"

echo
echo "== All entries with their fields =="
jq -r '.[] | "\(.username) | createdAt=\(.createdAt) | issueUrl=\(.issueUrl)"' "${FILE}"

Repository: MatteoGabriele/agentscan

Length of output: 4437

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Check automation type definitions =="
find . -name "automation.ts" -o -name "types.ts" | head -5

echo
echo "== Search for type definitions with createdAt and issueUrl =="
rg -A 10 "type.*utomation|interface.*utomation" --type ts | head -40

echo
echo "== Check verified-automations.get.ts =="
find . -name "*verified-automations*" -type f | grep -E "\.(ts|js)$"

Repository: MatteoGabriele/agentscan

Length of output: 2616

Fill in missing required fields for test-bot-2 entry.

The test-bot-2 entry has empty createdAt and issueUrl fields, which violates the VerifiedAutomation schema—both are required strings. Additionally, the reason field contains a placeholder ("test 123") instead of meaningful content. Update the entry with an actual issue URL and creation date before merging.

Schema requirement (shared/types/automation.ts) 
export type VerifiedAutomation = {
  username: string;
  reason: string;
  issueUrl: string;
  createdAt: string;
};
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@data/verified-automations-list.json` around lines 3 - 61, The test-bot-2 JSON
entry violates the VerifiedAutomation schema by leaving "createdAt" and
"issueUrl" empty and using a placeholder "reason"; locate the test-bot-2 object
in data/verified-automations-list.json (look for the "username": "test-bot-2")
and populate "createdAt" with an ISO date string (e.g., "2026-03-XX"), set
"issueUrl" to a valid issue or PR URL referencing the reason (e.g., a GitHub
issue URL), and replace "reason": "test 123" with a concise, meaningful
justification; ensure all four fields (username, reason, issueUrl, createdAt)
conform to the VerifiedAutomation type.

{
"username": "rayb64680-ui",
"reason": "Shows obvious indicators of an automated account that spams PRs with comments.",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/38",
"createdAt": "2026-03-25"
"createdAt": "",
"issueUrl": "",
"reason": "test 123",
"username": "test-bot-2"
Comment on lines +63 to +66
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

test-bot-2 entry has invalid placeholder metadata.

Line 63 and Line 64 are empty, and Line 65 looks like test placeholder text. This will propagate incomplete data to API consumers and break link/date expectations. Populate concrete values (Issue #41 is referenced in the PR description).

Proposed patch 
   {
-    "createdAt": "",
-    "issueUrl": "",
-    "reason": "test 123",
+    "createdAt": "2026-03-26",
+    "issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/41",
+    "reason": "Automated test account added by Auto-PR workflow",
     "username": "test-bot-2"
   }

Also recommended: add workflow/schema validation to reject empty createdAt and issueUrl before PR creation.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"createdAt": "",
"issueUrl": "",
"reason": "test 123",
"username": "test-bot-2"
{
"createdAt": "2026-03-26",
"issueUrl": "https://github.com/MatteoGabriele/agentscan/issues/41",
"reason": "Automated test account added by Auto-PR workflow",
"username": "test-bot-2"
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@data/verified-automations-list.json` around lines 63 - 66, The "test-bot-2"
JSON entry contains empty placeholder fields: set the createdAt and issueUrl to
concrete values and replace the placeholder reason; specifically update the keys
createdAt, issueUrl, and reason for the entry with username "test-bot-2" (e.g.,
createdAt to an ISO timestamp and issueUrl to the PR/issue URL for Issue `#41`,
and replace "test 123" with a meaningful reason). Additionally add a simple
schema/workflow validation step to reject entries with empty createdAt or
issueUrl before merge.

}
]
Loading