Skip to content

feat: Ignore temporary schemas ending with _next or _NEXT#11

Merged
lpillmann merged 2 commits intomainfrom
fix/ignore-next-schemas
Feb 27, 2026
Merged

feat: Ignore temporary schemas ending with _next or _NEXT#11
lpillmann merged 2 commits intomainfrom
fix/ignore-next-schemas

Conversation

@lpillmann
Copy link
Copy Markdown
Collaborator

@lpillmann lpillmann commented Feb 21, 2026

Summary

Fixes intermittent errors when temporary Snowflake schemas with _next suffix disappear during permifrost execution. Our ingestion framework creates temporary schemas with _next suffix (lower or upper case) that may be removed between when permifrost starts inspection and when it executes.

Problem

When permifrost starts, it sees these temporary schemas and includes them in its execution plan. If the schema disappears before permifrost finishes (e.g., the ingestion run completes), permifrost errors with object not exists or similar errors.

Solution

Filter out schemas ending with _next (case-insensitive) in show_schemas() — the central method called whenever permifrost needs to enumerate schemas in a database. This prevents temporary schemas from ever entering the execution plan.

Changes

  • src/permifrost/snowflake_connector.py — Added filtering in show_schemas() to skip schemas ending with _NEXT (case-insensitive check via .upper().endswith("_NEXT"))
  • tests/permifrost/test_snowflake_connector.py — Added test_show_schemas_filters_next_suffix() covering _next, _NEXT, and _Next variants; also fixed missing snowflake_connector_env fixture on test_show_schemas

Testing

  • 20 unit tests passing (all existing + 1 new)
  • New test verifies uppercase, lowercase, and mixed-case _next suffixes are all filtered
  • Verified with flake8, black, and mypy — all clean

Related

lpillmann and others added 2 commits February 21, 2026 03:14
- Filter out schemas with _next suffix (case-insensitive) in show_schemas()
- Add test to verify _NEXT schemas are properly filtered
- Fixes intermittent errors when schemas disappear during execution

Related to GEM-5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@lpillmann lpillmann marked this pull request as ready for review February 21, 2026 02:16
@lpillmann lpillmann merged commit 52eb945 into main Feb 27, 2026
2 checks passed
@lpillmann lpillmann deleted the fix/ignore-next-schemas branch February 27, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant