Skip to content

LCORE-1446: Add support for metadata filters in Solr vector search #1581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
anik120 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

LCORE-1446: Add support for metadata filters in Solr vector search #1581

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions src/models/requests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,12 +282,34 @@ class QueryRequest(BaseModel):
solr: Optional[SolrVectorSearchRequest] = Field(
None,
description=(
"Solr inline RAG config: mode (semantic, hybrid, lexical) and filters; "
"a legacy filter-only object (e.g. fq) is still accepted."
"Solr inline RAG config: mode (semantic, hybrid, lexical) and filters. "
"Supports structured metadata filters (eq, ne, in, nin comparison operators). "
"Legacy filter-only objects (e.g. fq) are still accepted."
),
examples=[
{"mode": "hybrid", "filters": {"fq": ["product:*openshift*"]}},
{"filters": {"fq": ["product:*openshift*", "product_version:*4.16*"]}},
{
"mode": "hybrid",
"filters": {
"filters": {
"type": "eq",
"key": "platform",
"value": "openshift"
}
}
},
{
"mode": "semantic",
"filters": {
"filters": {
"type": "and",
"filters": [
{"type": "eq", "key": "platform", "value": "openshift"},
{"type": "in", "key": "version", "value": ["4.14", "4.15", "4.16"]}
]
}
}
},
{"filters": {"fq": ["product:*openshift*"]}},
Comment thread
coderabbitai[bot] marked this conversation as resolved.
],
)

Expand Down Expand Up @@ -752,7 +774,9 @@ class ResponsesRequest(BaseModel):
topic summary for new conversations. Defaults to True.
shield_ids: LCORE-specific list of safety shield IDs to apply. If None, all
configured shields are used.
solr: Optional Solr inline RAG options (mode, filters) or legacy filter-only dict.
solr: Optional Solr inline RAG options (mode, filters). Supports
structured metadata filters (eq, ne, in, nin comparison operators).
Legacy filter-only dicts are still accepted.
"""

input: ResponseInput
Expand Down
21 changes: 18 additions & 3 deletions src/utils/vector_search.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,11 @@ def _build_query_params(

Args:
solr: Optional structured Solr request (mode and filters from the API).
- mode: Solr search mode (semantic, hybrid, lexical)
- filters: Solr filter payload, may contain structured metadata filters

Returns:
Parameter dictionary for ``vector_io.query``.
Parameter dictionary for ``vector_io.query`` with extracted filters at top level.
"""
Comment thread
coderabbitai[bot] marked this conversation as resolved.
resolved_mode = (
solr.mode
Expand All @@ -78,8 +80,21 @@ def _build_query_params(
logger.debug("query_request.solr: %s", solr)

if solr is not None and solr.filters is not None:
params["solr"] = solr.filters
logger.debug("Final params with solr filters: %s", params)
# Extract structured metadata filters if present in solr.filters dict
# Filters need to be at top-level params for vector_io.query
if isinstance(solr.filters, dict) and "filters" in solr.filters:
params["filters"] = solr.filters["filters"]
logger.debug("Extracted filters from solr.filters: %s", params["filters"])

# Pass remaining solr.filters content (legacy fq, etc.) to params["solr"]
remaining_filters = {k: v for k, v in solr.filters.items() if k != "filters"}
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Format the comprehension with Black.

CI is already failing Black; this long single-line dict comprehension is one of the new spots that will be rewritten.

Black-style formatting 
-            remaining_filters = {k: v for k, v in solr.filters.items() if k != "filters"}
+            remaining_filters = {
+                k: v for k, v in solr.filters.items() if k != "filters"
+            }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@src/utils/vector_search.py` at line 90, Reformat the long dict comprehension
into Black-friendly multi-line form: break the expression that assigns
remaining_filters from solr.filters.items() across lines so Black won't reflow
it; locate the assignment to remaining_filters (the comprehension over
solr.filters.items()) and split the dict literal, the for clause, and the if
clause onto separate lines consistent with Black's preferred formatting.

if remaining_filters:
params["solr"] = remaining_filters
logger.debug("Remaining solr.filters: %s", remaining_filters)
else:
# Legacy format: entire solr.filters dict is passed as params["solr"]
params["solr"] = solr.filters
logger.debug("Legacy solr.filters format: %s", params["solr"])
else:
logger.debug("No solr filters provided")

Expand Down
90 changes: 84 additions & 6 deletions tests/unit/utils/test_vector_search.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,14 +67,89 @@ def test_default_params(self) -> None:
assert params["mode"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_MODE
assert "solr" not in params

def test_with_solr_filters(self) -> None:
"""Test parameters when solr filters are provided."""
solr = SolrVectorSearchRequest.model_validate({"filter": "value"})
def test_with_legacy_solr_filters(self) -> None:
"""Test parameters when legacy solr filters are provided."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"fq": ["platform:openshift"],
},
},
)
params = _build_query_params(solr=solr)

assert params["solr"] == {"fq": ["platform:openshift"]}
assert params["k"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_K
assert "filters" not in params

def test_with_structured_metadata_filters(self) -> None:
"""Test parameters with structured metadata filter format."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "eq",
"key": "platform",
"value": "openshift",
},
},
},
)
params = _build_query_params(solr=solr)

# Filters should be extracted to top-level
assert "filters" in params
assert params["filters"]["type"] == "eq"
assert params["filters"]["key"] == "platform"
assert params["filters"]["value"] == "openshift"
assert params["k"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_K
# No remaining solr params
assert "solr" not in params

def test_with_filters_and_other_solr_params(self) -> None:
"""Test parameters with both filters and other solr-specific params."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "in",
"key": "version",
"value": ["4.14", "4.15"],
},
"custom_param": "value",
},
},
)
params = _build_query_params(solr=solr)

assert params["solr"] == {"filter": "value"}
# Filters extracted to top-level
assert params["filters"]["type"] == "in"
assert params["filters"]["key"] == "version"
# Other params remain under solr key
assert params["solr"] == {"custom_param": "value"}
assert params["k"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_K

def test_with_compound_filter(self) -> None:
"""Test parameters with compound AND filter."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "and",
"filters": [
{"type": "eq", "key": "platform", "value": "openshift"},
{"type": "ne", "key": "status", "value": "archived"},
],
},
},
},
)
params = _build_query_params(solr=solr)

assert params["filters"]["type"] == "and"
assert len(params["filters"]["filters"]) == 2
assert "solr" not in params
Comment on lines +109 to +151
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Assert the full structured filter payload.

The mixed and compound tests should verify values/nested filters are preserved, not only type, key, and list length.

Strengthen assertions 
         # Filters extracted to top-level
         assert params["filters"]["type"] == "in"
         assert params["filters"]["key"] == "version"
+        assert params["filters"]["value"] == ["4.14", "4.15"]
         # Other params remain under solr key
         assert params["solr"] == {"custom_param": "value"}
@@
 
         assert params["filters"]["type"] == "and"
-        assert len(params["filters"]["filters"]) == 2
+        assert params["filters"]["filters"] == [
+            {"type": "eq", "key": "platform", "value": "openshift"},
+            {"type": "ne", "key": "status", "value": "archived"},
+        ]
         assert "solr" not in params
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def test_with_filters_and_other_solr_params(self) -> None:
"""Test parameters with both filters and other solr-specific params."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "in",
"key": "version",
"value": ["4.14", "4.15"],
},
"custom_param": "value",
},
},
)
params = _build_query_params(solr=solr)
assert params["solr"] == {"filter": "value"}
# Filters extracted to top-level
assert params["filters"]["type"] == "in"
assert params["filters"]["key"] == "version"
# Other params remain under solr key
assert params["solr"] == {"custom_param": "value"}
assert params["k"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_K
def test_with_compound_filter(self) -> None:
"""Test parameters with compound AND filter."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "and",
"filters": [
{"type": "eq", "key": "platform", "value": "openshift"},
{"type": "ne", "key": "status", "value": "archived"},
],
},
},
},
)
params = _build_query_params(solr=solr)
assert params["filters"]["type"] == "and"
assert len(params["filters"]["filters"]) == 2
assert "solr" not in params
def test_with_filters_and_other_solr_params(self) -> None:
"""Test parameters with both filters and other solr-specific params."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "in",
"key": "version",
"value": ["4.14", "4.15"],
},
"custom_param": "value",
},
},
)
params = _build_query_params(solr=solr)
# Filters extracted to top-level
assert params["filters"]["type"] == "in"
assert params["filters"]["key"] == "version"
assert params["filters"]["value"] == ["4.14", "4.15"]
# Other params remain under solr key
assert params["solr"] == {"custom_param": "value"}
assert params["k"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_K
def test_with_compound_filter(self) -> None:
"""Test parameters with compound AND filter."""
solr = SolrVectorSearchRequest.model_validate(
{
"filters": {
"filters": {
"type": "and",
"filters": [
{"type": "eq", "key": "platform", "value": "openshift"},
{"type": "ne", "key": "status", "value": "archived"},
],
},
},
},
)
params = _build_query_params(solr=solr)
assert params["filters"]["type"] == "and"
assert params["filters"]["filters"] == [
{"type": "eq", "key": "platform", "value": "openshift"},
{"type": "ne", "key": "status", "value": "archived"},
]
assert "solr" not in params
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@tests/unit/utils/test_vector_search.py` around lines 109 - 151, Update the
two tests test_with_filters_and_other_solr_params and test_with_compound_filter
to assert the full structured filter payload returned by _build_query_params
(not just type/key/length); for the first test, verify params["filters"] equals
the full dict {"type":"in","key":"version","value":["4.14","4.15"]} and that
params["solr"] equals {"custom_param":"value"} and k is default, and for the
second test assert params["filters"] equals the full nested compound dict (type
"and" with its "filters" list containing the exact eq and ne dicts) and that
"solr" is absent; use the SolrVectorSearchRequest-created expected structures to
compare deep equality.


def test_custom_mode(self) -> None:
"""Request mode overrides the default Solr vector_io mode."""
solr = SolrVectorSearchRequest(mode="lexical")
Expand All @@ -86,7 +161,8 @@ def test_custom_mode(self) -> None:
def test_mode_with_solr_filters(self) -> None:
"""Custom mode is combined with solr filter payload."""
solr = SolrVectorSearchRequest(
mode="semantic", filters={"fq": ["product:*openshift*"]}
mode="semantic",
filters={"fq": ["product:*openshift*"]},
)
params = _build_query_params(solr=solr)

Expand All @@ -95,7 +171,9 @@ def test_mode_with_solr_filters(self) -> None:

def test_mode_with_only_filters(self) -> None:
"""Mode is set to default value when only filters are provided."""
solr = SolrVectorSearchRequest(filters={"fq": ["product:*openshift*"]})
solr = SolrVectorSearchRequest(
filters={"fq": ["product:*openshift*"]},
)
params = _build_query_params(solr=solr)

assert params["mode"] == constants.SOLR_VECTOR_SEARCH_DEFAULT_MODE
Expand Down
Loading