Integrate Apify E-Commerce Scraper to replace mock data with live product search

[Copilot]

Replaces static mock data with real-time product scraping from Amazon using the Apify E-Commerce Scraping Tool. Users can now search for any product keyword and see live results.

Implementation

Core Integration

• Added apify-client package
• Created lib/apify-service.ts for Actor interaction and data mapping
• Converted app/page.tsx to client component with search state management

UI Enhancements

• Search triggers live Actor run with user-provided keyword
• Loading states with disabled form during execution
• Error handling with user-friendly messages
• Statistics cards now calculate average price from scraped data
• Data source indicator (Mock vs Apify)

Usage

// Service handles Actor calls and data mapping
export async function searchProducts(options: SearchProductsOptions): Promise<Product[]> {
  const run = await client.actor("apify/e-commerce-scraping-tool").call({
    keyword: options.keyword,
    marketplaces: [options.marketplace || "www.amazon.com"],
  });
  
  await client.run(run.id).waitForFinish();
  const dataset = client.dataset(run.defaultDatasetId);
  const { items } = await dataset.listItems({ limit: options.maxProducts || 20 });
  
  return items.map(mapApifyProductToProduct);
}

Configuration

Requires NEXT_PUBLIC_APIFY_TOKEN in .env.local. Added .env.example template and comprehensive README with setup instructions.

Screenshots

Initial State

Loading State

Original prompt

Integrate Apify E-Commerce Scraper for the webapp inside "ecomm-demo" folder to replace the mock product data with real e-commerce data.

When a user enters a search query in the SearchBar and clicks Submit, use that query to call the Apify E-Commerce Scraping Tool (apify/e-commerce-scraping-tool). Display the scraped results in the Statistics Cards, Product Table, and Product Cards instead of mock data.

How to do it

1. Examine the webapp structure and how mock data currently flows to the components
2. Check the Apify Actor details to understand its input/output schema
3. Set up the apify-client package and create a service to call the Actor with the search query
4. Parse the scraped data and map it to the Product type
5. Update the components to fetch from the Actor instead of mock data
6. Add error handling and loading states
7. Update the README with APIFY_TOKEN setup instructions

Key points

• The search query from SearchBar should be the input for the Actor
• Handle errors gracefully and show appropriate messages
• Keep APIFY_TOKEN in environment variables
• Test locally with a real search before submitting

Start by exploring the current webapp structure and the Actor's input/output.

Custom agent used: apify-integration-expert
Expert agent for integrating Apify Actors into codebases. Handles Actor selection, workflow design, implementation across JavaScript/TypeScript and Python, testing, and production-ready deployment.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.