feat: CLI migration + progressive disclosure redesign for ultimate-scraper

[lukas-bekr]

Summary

Major upgrade to the apify-ultimate-scraper skill: migrates from REST API scripts to Apify CLI, restructures the information architecture using progressive disclosure, and enriches all workflow guides with 58 research-backed data pipeline patterns.

Phase 1: CLI migration

• Replaced 3 Node.js scripts (search_actors.js, run_actor.js, fetch_actor_details.js) with CLI commands (apify actors call --json, actors search, actors info, datasets get-items)
• --json output as stable API contract - immune to upcoming CLI UI changes (Markdown default, colors)
• OAuth-first authentication (apify login) with env var fallback. Fixed security contradiction in actorization skill (was using apify login -t exposing tokens in shell history, aligned with PR fix: migrate security fixes to actorization skill #31)

Phase 2: Progressive disclosure restructure

• Replaced monolithic 400-line Actor index with hub-and-spoke architecture
• SKILL.md (~109 lines) routes to lean actor-index (206 lines) + 14 workflow guides + gotchas (108 lines)
• Simple task ("scrape Nike's Instagram") loads ~300 lines. Complex pipeline loads ~500. Neither loads the other 13 guides.

Phase 3: Research-driven workflow enrichment

• 4-workstream research: Notion internal use cases + AI research (Perplexity/Gemini/ChatGPT) + n8n template library scraping (85+ templates, 26 use Apify) + social media scraping
• 58 distinct workflow patterns mapped to Apify Actors, ranked by cross-source frequency
• Every workflow guide now has 4-6 pipelines with explicit Actor chaining, data piping (results[].website -> startUrls), PPE cost estimates, and gotchas

Phase 4: New content

• 4 new workflow categories: e-commerce price monitoring, contact enrichment, knowledge base/RAG, company research (covers 5,000+ Store Actors with previously zero workflow coverage)
• Enriched gotchas with anti-bot guidance (Cloudflare, SPA, fingerprinting), platform rate limits, cost estimation protocols

By the numbers

• 17 files, 1,597 lines (was 13 files, 782 lines)
• Token budget for simple tasks: ~300 lines (unchanged, progressive disclosure)
• 14 workflow guides with 4-6 pipelines each (was 10 with 1-4 each)
• Design principles: Anthropic's "Lessons from Building Skills" - skip the obvious, gotchas are highest-signal, hub-and-spoke progressive disclosure, don't railroad

Scope

• apify-ultimate-scraper skill only (full rewrite)
• apify-actorization auth fix (aligned with PR fix: migrate security fixes to actorization skill #31)
• apify-actor-development minor auth alignment (OAuth-first)
• commands/create-actor.md auth alignment
• Did NOT touch developer skill content (actor-development, actorization workflows) - Patrik's territory