feat: add ai to detect clankers by huang-julien · Pull Request #1 · unveil-project/identity

huang-julien · 2026-04-02T19:47:33Z

Extract the logic of MatteoGabriele/agentscan-action#15 to unveil-project

Copilot

Pull request overview

This PR adds an optional AI-assisted analysis module to @unveil/identity, allowing heuristic identity signals to be augmented with an LLM-based classification via GitHub Models, and exposes it as a new @unveil/identity/ai subpath.

Changes:

Introduces src/ai/* (prompt builder, types, and a GitHub Models-backed getAIAnalysis implementation).
Updates build/packaging to emit and export the new ./ai entrypoint, and adds the voight-kampff-compactor dependency.
Adds a CLI script to run heuristic + AI analysis and documents the new API in the README.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
tsdown.config.ts	Adds an explicit multi-entry build including the new AI entrypoint.
src/ai/types.ts	Defines input/output types for AI analysis.
src/ai/prompt.ts	Adds the system prompt and prompt-construction logic (with event slimming/compaction).
src/ai/index.ts	Re-exports the AI module surface as a subpath entry.
src/ai/analysis.ts	Implements `getAIAnalysis` calling GitHub Models and parsing the response.
scripts/ai-analyser-user.ts	Adds a runnable script to fetch user data/events and invoke the AI analysis.
README.md	Documents AI-enhanced usage and supported models.
package.json	Exposes `./ai`, adds script, and adds runtime dependency.
pnpm-lock.yaml	Locks the new dependency and related lock updates.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/ai/analysis.ts

src/ai/types.ts

Copilot · 2026-04-03T19:51:10Z

src/ai/prompt.ts

+export function buildUserPrompt(input: AIAnalysisInput): string {
+
+  const events = slimEvents(input.events);
+  const eventDates = input.events
+    .map(e => e.created_at)


buildUserPrompt is typed to require token and model via AIAnalysisInput, but it doesn't use either field. Consider changing the parameter type to a narrower shape (e.g., a dedicated AIPromptInput or a Pick excluding auth/provider fields) to keep prompt construction decoupled from transport/auth concerns.

Copilot · 2026-04-03T19:51:10Z

tsdown.config.ts

+    'src/index.ts',
+    'src/ai/index.ts',


This config file otherwise uses double quotes, but the new entry paths use single quotes. For consistency with the established style in this repo, switch these strings to double quotes.

Suggested change

'src/index.ts',

'src/ai/index.ts',

"src/index.ts",

"src/ai/index.ts",

Copilot · 2026-04-03T19:51:10Z

src/ai/index.ts

+export * from './analysis'
+export * from './types'
+export * from './prompt'


The repo consistently uses double quotes and semicolons (e.g., src/index.ts), but this barrel file uses single quotes and omits semicolons. Aligning with the existing style improves consistency and reduces churn in future edits.

Suggested change

export * from './analysis'

export * from './types'

export * from './prompt'

export * from "./analysis";

export * from "./types";

export * from "./prompt";

Copilot · 2026-04-03T19:51:10Z

scripts/ai-analyser-user.ts

+    for (let page = 1; page <= 2; page++) {
+        const res = await fetch(
+            `https://api.github.com/users/${username}/events?per_page=300&page=${page}`,


GitHub’s REST API caps per_page at 100; using per_page=300 is ignored and makes the script’s intent unclear. Use per_page=100 (matching scripts/fetch-github-events.js) and adjust paging logic accordingly.

Suggested change

for (let page = 1; page <= 2; page++) {

const res = await fetch(

`https://api.github.com/users/${username}/events?per_page=300&page=${page}`,

const perPage = 100;

const maxEvents = 600;

const maxPages = Math.ceil(maxEvents / perPage);

for (let page = 1; page <= maxPages; page++) {

const res = await fetch(

`https://api.github.com/users/${username}/events?per_page=${perPage}&page=${page}`,

Copilot · 2026-04-03T19:51:11Z

README.md

+`getAIAnalysis` accepts any model available on GitHub Models (e.g. `openai/gpt-4o`, `deepseek/DeepSeek-R1`). It returns `null` if the model produces no usable response.
+


The README says getAIAnalysis “returns null if the model produces no usable response,” but the current implementation throws on HTTP errors and JSON parse failures. Either update the README to describe the throwing behavior, or change the implementation to catch/return null for these failure modes.

Copilot · 2026-04-03T19:51:11Z

src/ai/analysis.ts

+export async function getAIAnalysis(input: AIAnalysisInput): Promise<AIAnalysisResult | null> {
+  const { token, model = 'openai/gpt-4o-mini', username, analysis, accountCreatedAt, publicRepos, events } = input;
+  const prompt = buildUserPrompt({ token, model, username, analysis, accountCreatedAt, publicRepos, events });
+


New AI prompt construction / response parsing logic is introduced here, but there are no tests covering key behaviors (e.g., prompt includes orgs/heuristic summary, response sanitization/JSON parsing). Adding a small vitest suite with mocked fetch and snapshot/unit assertions would help prevent regressions.

huang-julien added 8 commits April 2, 2026 21:44

feat: add ai to detect clankers

ac2a620

chore: udpate prompt

8eac1cd

fix: types

b05ca50

fix: prompt and add type jsdoc

f39148d

chore: add manual run script

c4bd250

fix: comment

f59d4f1

improve prompt

14b9bc1

feat: update prompt and test

e10c844

huang-julien requested a review from Copilot April 3, 2026 19:46

Copilot started reviewing on behalf of huang-julien April 3, 2026 19:47 View session

Copilot AI reviewed Apr 3, 2026

View reviewed changes

huang-julien added 6 commits April 3, 2026 22:48

feat: use zod

91b482b

update script

62ca24d

fix: types

602a17d

fix: forgot to send org

e2d154b

fix: get gh token from env by default

a58b9f3

test: add tests

4079fdd

huang-julien requested a review from MatteoGabriele April 7, 2026 16:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add ai to detect clankers#1

feat: add ai to detect clankers#1
huang-julien wants to merge 14 commits intounveil-project:mainfrom
huang-julien:feat/anti_clanker_ai

huang-julien commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Copilot AI Apr 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-    for (let page = 1; page <= 2; page++) {
-        const res = await fetch(
-            `https://api.github.com/users/${username}/events?per_page=300&page=${page}`,
+    const perPage = 100;
+    const maxEvents = 600;
+    const maxPages = Math.ceil(maxEvents / perPage);
+    for (let page = 1; page <= maxPages; page++) {
+        const res = await fetch(
+            `https://api.github.com/users/${username}/events?per_page=${perPage}&page=${page}`,

		`getAIAnalysis` accepts any model available on GitHub Models (e.g. `openai/gpt-4o`, `deepseek/DeepSeek-R1`). It returns `null` if the model produces no usable response.

Conversation

huang-julien commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants