Skip to content

Fix BQ Dataset Explorer returning empty results#372

Open
szmikler wants to merge 2 commits intoGoogleCloudDataproc:mainfrom
szmikler:fix-search-field-query
Open

Fix BQ Dataset Explorer returning empty results#372
szmikler wants to merge 2 commits intoGoogleCloudDataproc:mainfrom
szmikler:fix-search-field-query

Conversation

@szmikler
Copy link
Copy Markdown

Problem:
When using the search field in the BigQuery Dataset Explorer, searches for private datasets (user projects) were returning empty results locally, even when the datasets clearly existed. However, searches for public datasets were working as expected.

Cause:
The frontend was sending the search type parameter wrapped in parentheses: type=(table|dataset). When the backend transformed this into a Dataplex search query, it generated a malformed string like (type=TABLE OR type=DATASET). While the Dataplex API is lenient with public data, it is strictly sensitive to this syntax for private project entries, causing it to return zero results.

Solution:
Removed the redundant parentheses from the type parameter in the requestAPI call within bigQueryService.tsx. The parameter is now sent as type=table|dataset, allowing the backend to correctly construct the Dataplex search query.

Verification:

  • Verified via curl that removing parentheses from the search query returns the expected private datasets.
  • Confirmed that search functionality remains intact for public datasets.

To reproduce the issue and verify the fix, you can use the following curl commands. These simulate how the backend's search query behaves when it receives the parameter with or without parentheses.

Local Reproduction

Prerequisites

Ensure you have an active access token and your project ID set:

PROJECT_ID=$(gcloud config get-value project)
TOKEN=$(gcloud auth print-access-token)

To reproduce the issue exactly as it happens in the backend, we need to use the specific query string that the plugin's Python code generates when it receives the malformed input from the frontend.

The backend splits the input by | and prepends type=. When it receives (table|dataset), it produces this broken query: (type=(TABLE OR type=DATASET)).

Prerequisites

PROJECT_ID=$(gcloud config get-value project)
TOKEN=$(gcloud auth print-access-token)

1. Reproduce the Problem (Generation of Malformed Query)

This command uses the query string exactly as the current origin/main backend generates it. Notice the nested and mismatched parentheses (type=(TABLE ...)). This should return {} or no results for your private datasets.

curl -s -X POST \
  "https://dataplex.googleapis.com/v1/projects/$PROJECT_ID/locations/global:searchEntries" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: $PROJECT_ID" \
  -d "{
    \"query\": \"system=BIGQUERY (type=(TABLE OR type=DATASET)) projectid=$PROJECT_ID\",
    \"pageSize\": 10
  }"

2. Verify the Fix (Generation of Correct Query)

This command uses the query string generated when the frontend is fixed to send table|dataset. The backend then correctly generates (type=TABLE OR type=DATASET). This should successfully return your private datasets.

curl -s -X POST \
  "https://dataplex.googleapis.com/v1/projects/$PROJECT_ID/locations/global:searchEntries" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: $PROJECT_ID" \
  -d "{
    \"query\": \"system=BIGQUERY (type=TABLE OR type=DATASET) projectid=$PROJECT_ID\",
    \"pageSize\": 10
  }"

Why the difference?

In the first case, the backend's simple string manipulation turns (table|dataset) into (type=(TABLE OR type=DATASET)). Dataplex treats the literal ( at the start of (TABLE as part of the search token rather than a grouping operator, causing the match to fail for private datasets which require precise indexing. Removing the parentheses in the frontend ensures the backend constructs a valid logical OR group.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the BigQuery search API call by modifying the 'type' parameter in the query string. The reviewer suggests URL-encoding the 'searchTerm' to ensure special characters do not break the request structure.

Comment thread src/bigQuery/bigQueryService.tsx Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant