feat: unified smart search with exploration tracking and query program #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

aaronsb merged 23 commits into main from feature/unified-smart-search

Feb 3, 2026

.claude/todo-unified-query-exploration.md

-Original file line number
+Diff line change
@@ -0,0 +1,79 @@
+    # Unified Query Exploration System
+    ## Vision
+    The saved query is the universal unit of work across all explorer views. A query is an ordered list of Cypher statements with additive/subtractive operators, built interactively or by hand, that flows through every explorer identically.
+    ### Core Concept
+    ```
+    { name: string, statements: { op: '+' | '-', cypher: string }[] }
+    ```
+    Each statement represents one intentional action — a discrete thought in an exploration sequence. The order mirrors how the user actually explored. Replay executes them in order, merging (+) or removing (-) each result.
+    ### Entry Points (all produce the same saved artifact)
+    - **Smart Search** — interactive clicks, add-to-existing, add-adjacent → generates Cypher per action
+    - **Block Builder** — visual composition → compiles to Cypher
+    - **Cypher Editor** — hand-written or pasted from a friend
+    - **Saved query recall** — load a previously saved exploration
+    ### Explorer Views (all consume the same saved query)
+    | Explorer | What it shows |
+    |----------|---------------|
+    | 2D Graph | Force-directed visualization, primary interactive builder |
+    | 3D Graph | Same graph, spatial perspective |
+    | Cypher Editor | The statements as editable text, copy/pasteable |
+    | Vocabulary Analysis | Relationship type introspection on the query's result set |
+    | Document Explorer | Source documents contributing to the query's concepts |
+    | Polarity Explorer | Pick edges from the query's graph for polarity axis analysis |
+    ### Sidebar Consistency
+    Folder icon in the nav rail across all explorer views. Same saved queries list, different lens on the data. Switching views preserves the graph.
+    ## Implementation Phases
+    ### Phase 1: Exploration Tracking & Cypher Generation ✓
+    - [x] Add `ExplorationStep` type and `explorationSession` to graphStore
+    - [x] Add `addExplorationStep()`, `clearExploration()` store actions
+    - [x] Record steps at action points: handleLoadExplore, handleLoadPath, handleFollowConcept, handleAddToGraph
+    - [x] Create `cypherGenerator.ts` — convert steps to ordered Cypher statements with +/- operators
+    - [x] Persist `rawGraphData` + `explorationSession` to localStorage (survive refresh)
+    - [x] Add `subtractRawGraphData` and "Remove from Graph" context menu (op: '-')
+    ### Phase 2: Saved Queries Folder ✓
+    - [x] Unify sidebar folder icon across all explorer views (FolderOpen, consistent with report explorer)
+    - [x] Saved query data model: `{ name, statements: { op, cypher }[] }` via QueryDefinition
+    - [x] Save exploration → creates QueryDefinition with `definition_type: 'exploration'`
+    - [x] Load saved query → replays statements in order with +/- semantics via executeCypherQuery
+    - [x] Delete saved query (already worked via queryDefinitionStore)
+    ### Phase 3: Editor Integration
+    - [ ] "Export as Cypher" sends ordered statements to the Cypher editor
+    - [ ] Cypher editor displays +/- prefixed statements
+    - [ ] Execute from editor replays the statement sequence
+    - [x] Subtractive operator: context menu "Remove from Graph" option
+    ### Phase 4: Cross-Explorer Flow
+    - [ ] Vocabulary explorer reads same saved queries from folder
+    - [ ] Document explorer reads same saved queries
+    - [ ] Polarity explorer loads graph from saved query
+    - [ ] Verify all explorers share the same folder state
+    ### Phase 5: Documentation & Docstrings
+    - [ ] Add JSDoc docstrings to new exploration tracking code
+    - [ ] Add JSDoc docstrings to graphStore (actions, types)
+    - [ ] Add JSDoc docstrings to cypherGenerator
+    - [ ] Add JSDoc docstrings to SearchBar handlers
+    - [ ] Add JSDoc docstrings to useGraphContextMenu handlers
+    - [ ] Document the unified query exploration workflow in user manual
+    - [ ] Document the +/- operator algebra concept
+    ## Notes
+    - Block builder compiles TO Cypher but we don't decompose Cypher back to blocks (existing ADR)
+    - Graph accelerator makes this practical — path finding is now fast enough for interactive multi-step exploration
+    - The +/- algebra on statements is like set operations: union then difference, letting users sculpt their graph

api/app/models/query_definition.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -15,10 +15,11 @@ @@
         'cypher',
         'search',
         'polarity',
-        'connection'
+        'connection',
+        'exploration'
     ]
-    DefinitionType = Literal['block_diagram', 'cypher', 'search', 'polarity', 'connection']
+    DefinitionType = Literal['block_diagram', 'cypher', 'search', 'polarity', 'connection', 'exploration']
     class QueryDefinitionCreate(BaseModel):
@@ Expand Down @@

api/app/routes/queries.py

-Original file line number
+Diff line change
@@ Expand Up / @@ -1643,8 +1643,18 @@ async def execute_cypher_query( @@
                 # Extract nodes and relationships from each record
                 for key, value in record.items():
                     if isinstance(value, dict):
-                        # Check if it's a node (has 'id' and typically 'label')
-                        if 'id' in value:
+                        # Check for relationship first (AGE edges have start_id/end_id)
+                        start_id = value.get('start_id') or value.get('start')
+                        end_id = value.get('end_id') or value.get('end')
+                        if start_id and end_id:
+                            relationships.append(CypherRelationship(
+                                from_id=str(start_id),
+                                to_id=str(end_id),
+                                type=value.get('label', value.get('type', 'RELATED')),
+                                properties=value.get('properties', {})
+                            ))
+                        elif 'id' in value:
                             node_id = str(value['id'])
                             if node_id not in nodes_map:
                                 # Prefer properties.label (actual name) over AGE label (node type like "Concept")
@@ Expand Down @@

schema/migrations/050_exploration_definition_type.sql

-Original file line number
+Diff line change
@@ -0,0 +1,41 @@
+    -- Migration 050: Add 'exploration' to query_definitions definition_type
+    --
+    -- Supports saving graph explorations as ordered +/- Cypher statement sequences.
+    -- Each exploration is a replayable series of additive/subtractive graph operations.
+    -- ===========================================================================
+    -- Skip if already applied
+    DO $$
+    BEGIN
+        IF EXISTS (SELECT 1 FROM public.schema_migrations WHERE version = 50) THEN
+            RAISE NOTICE 'Migration 050 already applied, skipping';
+            RETURN;
+        END IF;
+        -- Drop and recreate CHECK constraint to include 'exploration'
+        ALTER TABLE kg_api.query_definitions
+            DROP CONSTRAINT IF EXISTS valid_definition_type;
+        ALTER TABLE kg_api.query_definitions
+            ADD CONSTRAINT valid_definition_type CHECK (definition_type IN (
+                'block_diagram',
+                'cypher',
+                'search',
+                'polarity',
+                'connection',
+                'exploration'
+            ));
+        COMMENT ON COLUMN kg_api.query_definitions.definition_type IS
+            'Type of query: block_diagram, cypher, search, polarity, connection, exploration';
+        RAISE NOTICE 'Migration 050: Added exploration definition type';
+    END $$;
+    -- ===========================================================================
+    -- Record Migration
+    -- ===========================================================================
+    INSERT INTO public.schema_migrations (version, name)
+    VALUES (50, 'exploration_definition_type')
+    ON CONFLICT (version) DO NOTHING;

web/src/api/client.ts

-Original file line number
+Diff line change
@@ Expand Up / @@ -98,7 +98,33 @@ class APIClient { @@
           }).then(r => r.data).catch(() => null)
         );
-        const allConceptDetails = (await Promise.all(conceptDetailsPromises)).filter(Boolean);
+        let allConceptDetails = (await Promise.all(conceptDetailsPromises)).filter(Boolean);
+        // Step 3b: Discover relationship targets missing from our set and fetch them.
+        // The /query/related traversal can miss neighbors (stale accelerator, etc.),
+        // but concept details include the actual relationships. Hydrate any targets
+        // we don't already have so the subgraph is complete.
+        const fetchedIds = new Set(allConceptIds);
+        const missingIds: string[] = [];
+        allConceptDetails.forEach((concept: any) => {
+          (concept.relationships || []).forEach((rel: any) => {
+            if (rel.to_id && !fetchedIds.has(rel.to_id)) {
+              fetchedIds.add(rel.to_id);
+              missingIds.push(rel.to_id);
+            }
+          });
+        });
+        if (missingIds.length > 0) {
+          const extraDetails = (await Promise.all(
+            missingIds.map(id =>
+              this.client.get(`/query/concept/${id}`, {
+                params: { include_grounding: false }
+              }).then(r => r.data).catch(() => null)
+            )
+          )).filter(Boolean);
+          allConceptDetails = [...allConceptDetails, ...extraDetails];
+        }
         // Step 4: Build nodes array (with grounding strength)
         const nodes = allConceptDetails.map((concept: any) => ({
@@ Expand All / @@ -111,7 +137,7 @@ class APIClient { @@
         // Step 5: Build links array from ALL concepts' relationships
         // Only include links where both source and target are in our node set
-        const nodeIdSet = new Set(allConceptIds);
+        const nodeIdSet = new Set(allConceptDetails.map((c: any) => c.concept_id));
         const links: any[] = [];
         const seenEdges = new Set<string>(); // Deduplicate edges
@@ Expand All / @@ -120,7 +146,9 @@ class APIClient { @@
             concept.relationships.forEach((rel: any) => {
               // Only include if target is in our subgraph
               if (nodeIdSet.has(rel.to_id)) {
-                const edgeKey = `${concept.concept_id}->${rel.to_id}-${rel.rel_type}`;
+                // Deduplicate: normalize edge key to treat A→B and B→A same-type as one edge
+                const [lo, hi] = [concept.concept_id, rel.to_id].sort();
+                const edgeKey = `${lo}<>${hi}-${rel.rel_type}`;
                 if (!seenEdges.has(edgeKey)) {
                   seenEdges.add(edgeKey);
                   links.push({
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: unified smart search with exploration tracking and query program #285

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!