Smart search doesn't detect volume variants for Durston Kakwa

[thomasbecker]

Description

When searching for "Durston Kakwa" in the smart search, the variant detection
correctly identifies size (S, M, L) and material (Ultra 200X, UltraGrid)
variants, but fails to detect the volume variants (40L and 55L).

Expected Behavior

The search should recognize that Durston Kakwa comes in two volume variants:

• Kakwa 40 (40L)
• Kakwa 55 (55L)

Each volume has its own product page on the manufacturer site:

• https://durstongear.com/products/kakwa-55
• https://durstongear.com/products/kakwa-40-backpack

Actual Behavior

Only 6 variants detected (size × material combinations):

• Small / Ultra 200X
• Small / UltraGrid
• Medium / Ultra 200X
• Medium / UltraGrid
• Large / Ultra 200X
• Large / UltraGrid

The 40L vs 55L volume distinction is missing.

Screenshots


Impact

Users adding a Durston Kakwa 40 will get incorrect weight data since the
system only scraped the Kakwa 55 page. The 40L and 55L have different weights
due to volume differences.

Technical Notes

The AI extraction appears to only scrape one product page rather than
recognizing that the search term maps to multiple distinct products with
different volumes.

[thomasbecker]

Additional Investigation

The Kakwa 40 appears to no longer be sold on durstongear.com. This explains why
the product family detection doesn't find it - there's no Kakwa 40 product page
in the search results for the detector to discover.

URLs checked:

• https://durstongear.com/products/kakwa-55 ✓ exists
• https://durstongear.com/products/kakwa-40-backpack - may be discontinued

This changes the nature of the issue:

• If Kakwa 40 is discontinued, the current behavior (showing only Kakwa 55
  variants) may be correct
• The system can only detect product families when multiple products appear in
  search results

Next steps:

1. Verify whether Kakwa 40 is actually discontinued
2. If discontinued, this may be expected behavior (can't detect products that
   don't exist)
3. If still available elsewhere, consider whether we should detect product
   families from other sources

[thomasbecker]

Correction: Kakwa 40 Sold via Partner

The Kakwa 40 is not discontinued - it's sold exclusively through Kaviso.com
(a Durston partner), not on durstongear.com directly.

This explains the detection issue:

• Kakwa 55: sold on durstongear.com (manufacturer site)
• Kakwa 40: sold on kaviso.com (retailer/partner site)

The product family detector currently only includes results flagged as
isManufacturer: true. Since Kaviso.com is a retailer, the Kakwa 40 results
get filtered out before family detection runs.

Potential fixes:

1. Expand product family detection to include high-confidence retailer results
   when they match the brand name
2. Recognize known brand partners (like Kaviso for Durston) as equivalent to
   manufacturer sites
3. Search for base product name without volume to cast a wider net

[thomasbecker]

Investigation Complete - Proposed Solution

Root Cause

The Kakwa 40 product exists on durstongear.com but is hidden from the main
product listing (/products.json). It's accessible via direct URL but won't
appear in search results or product feeds.

Source	Kakwa 40	Kakwa 55
/products.json (main listing)	❌ Not listed	✓ Listed
/products/[handle].json (direct)	✓ Works	✓ Works
Product page HTML	✓ Accessible	✓ Accessible
Navigation menu	✓ Shown	✓ Shown

Proposed Solution: AI + JSON Merge

A general-purpose approach that works for all manufacturers:

Enhanced extraction flow:

1. User searches "Durston Kakwa 55"
                ↓
2. Find Kakwa 55 page, extract Shopify JSON
   → 6 variants (size × material) with weights
                ↓
3. AI scans page HTML for product siblings
   → Finds "Kakwa 40" mentioned in navigation
                ↓
4. Probe sibling URLs for Shopify JSON
   → /products/kakwa-40-backpack.json exists
   → Extract 3 variants with weights
                ↓
5. Merge into ProductFamily response
   → Both products with accurate JSON-sourced weights
                ↓
6. Frontend shows product family picker
   → User selects which product + variant they own

Key design points:

1. AI identifies siblings - Scans page for related products mentioned in
   navigation, comparison tabs, "also available in" text, related products
   sections

2. JSON provides accuracy - Once AI identifies siblings by name, we probe
   for their Shopify JSON to get accurate weights (not AI-extracted)

3. General purpose - Works for any manufacturer, not Durston-specific

4. Graceful degradation - If no siblings found, current behavior unchanged

Implementation Tasks

• Extend AI extraction prompt to identify product siblings
• Add sibling URL probing (try common handle patterns)
• Merge sibling JSON data into ProductFamily response
• Update frontend to handle expanded product family data
• Add test cases for multi-volume products (Kakwa, tents with 1P/2P, etc.)