Skip to content

V2 skill#1

Draft
apreshill wants to merge 13 commits intopixeltable:mainfrom
apreshill:v2-skill
Draft

V2 skill#1
apreshill wants to merge 13 commits intopixeltable:mainfrom
apreshill:v2-skill

Conversation

@apreshill
Copy link

@apreshill apreshill commented Mar 12, 2026

Generated by Claude

Refactor core skill: mental model + anti-patterns

Background

Ran the Anthropic skill-creator evaluation against the v1 skill. The refactor follows Anthropic's skill authoring best practices — particularly wrong/right pattern framing, problem-first design, and keeping skills under 500 lines. Key findings:

  • Structured as API reference: Code examples showed what Pixeltable can do, but no mental model for the agent to reason from — nothing about persistence, data referencing, or "everything flows through the table"
  • Procedural, not prescriptive: Documented what you can do but not what you should do and when — the agent had no basis for choosing between options or steering users toward the right pattern
  • No wrong/right framing: The skill showed correct code but never showed what incorrect code looks like, so the agent couldn't recognize common mistakes
  • Context cost: 1383-line API_REFERENCE.md loaded alongside the skill

Developed a 4-phase plan: fix core skill structure, eliminate API_REFERENCE.md, create specialized skills (RAG, media, agents, app), sweep imports.

What changed (Phase 1 — core skill)

Restructured from flat API sections into three that build on each other:

  • Mental Model — what Pixeltable is (persistent, versioned, everything through the table), how data referencing works, the type system, schema mutability, and what the table handles automatically (caching, rate limits, retries). Includes wrong/right code examples so the agent can recognize mistakes, not just generate correct code.
  • How to Use This Skill — 4 behavioral modes (new user, scaffold, code review, debug) with a 10-row anti-pattern table the agent scans for automatically (SDK bypass, wrong types for media, if_exists misuse, async FastAPI, single-row tables, custom UDFs duplicating built-ins, defensive wrappers, UDFs called outside the table, external tools that duplicate built-in capabilities, manual JSON parsing).
  • Pixeltable Patterns — intent-matched code ("I need X" → use Y) for tables, views, computed columns, CRUD operations, querying, embedding indexes, versioning, JSON path expressions, 4 export paths, and sampling.

The biggest conceptual additions are things the v1 didn't need to say because they're obvious if you already know Pixeltable, but an agent doesn't know them:

  • Caching, rate limits, retries are automatic. The agent was generating retry loops and caching layers because nothing told it Pixeltable handles this. Now "What the table handles for you" explains why bypassing computed columns means losing all three.
  • Everything is incremental. Views, computed columns, and embedding indexes all update on insert — no re-indexing, no re-processing of existing data.
  • Versioning means stored, retrievable outputs. Not just change tracking — every computed result is stored and traceable. The agent needs this to recommend t.describe() for lineage and time-travel queries for comparison.
  • if_exists has three distinct use cases. Now explained as its own section: 'error' for safety, 'ignore' for setup scripts, 'replace' for iteration.
  • CRUD operations for directories, tables, and columns. The v1 had no coverage of update, delete, drop, move, list, or column management — agents hit a wall on basic operations like renaming a column or listing tables.
  • Zero-friction onboarding. Default "get started" example uses hosted images and built-in functions — no API key needed. Goal→pattern routing table maps 7 common user goals (document search, image catalog, video understanding, etc.) to the right Pixeltable building blocks.

Also added: test-then-commit workflow (.head() before add_computed_column), JSON path expressions for navigating AI API responses, sampling with stratification, and a problem-first skill description for marketplace discoverability.

Moved to specialized skills (future PRs):

  • Tool-calling agent pipeline, @pxt.query, provider table, companion resources
  • export_sql(), .to_pytorch_dataset(), .to_coco_dataset() (specialized export paths)

Still to do (future PRs)

  • Eliminate API_REFERENCE.md (1383 lines)
  • Create specialized skills: pixeltable-rag first, then media, agents, app
  • Sweep all examples to pxtf.* imports

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the Pixeltable skill documentation from a mostly API-reference layout into a more agent-actionable structure centered on a mental model, usage modes, and common anti-pattern detection.

Changes:

  • Reorganized SKILL.md into “Mental Model”, “How to Use This Skill”, and “Pixeltable Patterns”.
  • Added wrong/right examples, an anti-pattern scan table, and debugging guidance to steer agents toward Pixeltable-native patterns.
  • Expanded pattern coverage (schema evolution, views/iterators, computed columns workflow, embedding search, JSON path usage, export options, versioning concepts).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

if_exists='ignore'
)
# Find rows with errors
t.where(t.analysis.errortype != None).select(t.analysis.errortype, t.analysis.errormsg).collect()
Comment on lines +321 to +325
t.update({'score': 1.0}, where=t.category == 'important')
t.delete(where=t.is_active == False)
pxt.move('dir1.my_table', 'dir2.my_table') # move between directories
pxt.drop_table('dir.my_table') # drop by catalog path
pxt.drop_table(t) # drop by handle
**Views** are derived from a table in three ways:
- With an **iterator** — splits rows into sub-rows (document chunks, video frames, audio segments)
- As a **filtered subset** — `pxt.create_view('dir.active', t.where(t.is_active == True))`
- As a **sample** — `pxt.create_view('dir.sample', t.sample(n=100, seed=42))`
)
```python
# --- Directories ---
pxt.create_dir('my_dir/sub_dir')
Comment on lines +366 to +368
@pxt.udf
def make_thumbnail(img: PIL.Image.Image) -> PIL.Image.Image:
return img.resize((320, 320))
```
| Anti-pattern | What to look for | Fix |
|---|---|---|
| SDK call outside computed column | `openai.OpenAI()`, `anthropic.Anthropic()`, `requests.get(...)` in app/endpoint code | Move to `add_computed_column` using `pxtf.<provider>.*` |
Comment on lines +255 to +259
t = pxt.create_table('my_project.items', {
'content': pxt.String,
'uuid': pxtf.uuid.uuid7(),
'created_at': pxt.Timestamp,
}, primary_key=['uuid'])
t.add_computed_column(cleaned=clean_text(t.content))
# From CSV/Parquet — auto-infer schema, override media columns
t = pxt.create_table('my_project.data', source='data.csv',
schema_overrides={'image_path': pxt.Image, 'audio_path': pxt.Audio})
Comment on lines +274 to +278
chunks = pxt.create_view('project.chunks', docs,
iterator=pxtf.document.document_splitter(docs.doc, separators='sentence', limit=300))

results = search_documents('machine learning').collect()
frames = pxt.create_view('project.frames', videos,
iterator=pxtf.video.frame_iterator(videos.video, fps=1))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants