Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Sep 12, 2025

This PR adds a comprehensive GitHub Actions workflow to test the notebook utilities as requested in the issue. The workflow validates that the 00-notebookutils.py file can be successfully imported and executed in a containerized environment.

Implementation Details

The new .github/workflows/test.yml workflow includes a container test job:

Container Test Job

  • Builds the Docker image from the existing Dockerfile
  • Runs a dedicated test script (test_container_notebook_utils.py) inside the container environment
  • Tests 00-notebookutils.py execution and validates that all imports work correctly and expected client variables are created

Key Features

  • Dedicated Test Script: Uses test_container_notebook_utils.py stored in the repository instead of inline script creation
  • Container-Only Testing: Focuses exclusively on testing within the Docker container environment as requested
  • Proper Environment Setup: Configures all required environment variables (KBASE_AUTH_TOKEN, MINIO_ENDPOINT, SPARK_MASTER_URL, etc.)
  • External Service Mocking: Uses unittest.mock to mock MinIO, Spark, and other external services to prevent network calls
  • Comprehensive Validation: Verifies that all 4 expected client variables (governance, minio, task_service, spark_cluster) are successfully created
  • Robust Error Handling: Provides detailed logging and error reporting for debugging
  • CI Integration: Automatically triggers on pushes and pull requests to main/develop branches

Test Validation

The test script validates that:

  1. The 00-notebookutils.py file exists and has valid Python syntax
  2. All imports can be resolved without errors
  3. The notebook utilities can be executed successfully
  4. Expected client objects are created and available
  5. The execution completes without runtime errors

This ensures the notebook environment will work correctly when the container is deployed and users import the utilities in their Jupyter notebooks.

Fixes #15.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copy link
Contributor Author

Copilot AI commented Sep 12, 2025

@bio-boris 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs.

I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review.

Copilot AI and others added 2 commits September 12, 2025 22:45
Co-authored-by: bio-boris <1258634+bio-boris@users.noreply.github.com>
Co-authored-by: bio-boris <1258634+bio-boris@users.noreply.github.com>
Copilot AI changed the title [WIP] Create a github action test Add GitHub Action test for notebook utilities Sep 12, 2025
Copilot AI requested a review from bio-boris September 12, 2025 22:48
@bio-boris
Copy link
Contributor

@copilot Always test it inside the container and put the test into a file into the repo

Co-authored-by: bio-boris <1258634+bio-boris@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 13, 2025

@copilot Always test it inside the container and put the test into a file into the repo

I've moved the test script to a dedicated file in the repository (test_container_notebook_utils.py) and updated the workflow to only test inside the container as requested. The standalone test job has been removed, and the workflow now uses the repository file instead of creating the test script inline. Commit: bee4ccf

Co-authored-by: bio-boris <1258634+bio-boris@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create a github action test

2 participants