Skip to content

Pixify and fix plot issue#253

Merged
JeanBilheux merged 6 commits intonextfrom
pixify
Aug 26, 2025
Merged

Pixify and fix plot issue#253
JeanBilheux merged 6 commits intonextfrom
pixify

Conversation

@KedoKudo
Copy link
Copy Markdown
Contributor

This PR introduces the following changes:

  • switch to use pixi to manage the project
  • fix the issue where plots are missing when launching notebook to browser

EWM 12863

Test instructions

  • Check out feature branch
  • Run pixi install to install all the dependencies
  • For developers, make sure run at least once: pixi run -e dev pre-commit install
  • Run pixi run lab to start the juptyerlab instance
  • Run venus_display_nexus_tpx3.ipynb to verify that plot are visible from last cell.

Summary [AI]

This pull request modernizes and streamlines environment and dependency management for the project by fully migrating to Pixi, removing legacy Conda configuration files, and introducing robust automation for dependency updates and code quality checks. It also updates the documentation to reflect these changes and enhances pre-commit and CI workflows for better consistency and maintainability.

Migration to Pixi and Dependency Management:

  • Removed legacy Conda environment files (conda/neutron-imaging.yml, conda/neutron-imaging-dev.yml, environment.yml) and the manual setup script (config_conda_env.sh), consolidating all environment management under Pixi. (F4957d57R1, [1] [2] [3]
  • Added .gitattributes entry to treat pixi.lock as a generated YAML file and prevent 3-way merges.

Continuous Integration and Automation Improvements:

  • Rewrote the main CI workflow (test-and-deploy.yml) to use Pixi for environment setup, testing, linting, and dependency auditing, replacing Conda-based steps.
  • Added new workflows:
    • update-lockfiles.yml: Automates weekly updates of the pixi.lock file and creates pull requests for dependency updates.
    • pre-commit-autoupdate.yml: Automatically updates pre-commit hooks monthly and creates corresponding pull requests.
  • Introduced Dependabot configuration to automate updates for GitHub Actions and Python dependencies, grouping updates and limiting PR noise.

Pre-commit and Linting Enhancements:

  • Updated .pre-commit-config.yaml:
    • Upgraded hook versions and added new hooks for YAML, TOML, and XML checks.
    • Excluded pixi.lock from pre-commit checks and added a custom pixi-lock-check hook.
    • Improved exclusion patterns for notebook files.
  • Added .yamllint configuration for consistent YAML linting.

Documentation Updates:

  • Revised README.md to provide clear instructions for using Pixi, including installation, common commands, and updated testing and deployment sections. [1] [2]

Miscellaneous:

  • Minor update to use the latest version of the commit-comment GitHub Action in the deployment workflow.

Migration to Pixi and Dependency Management

  • Removed legacy Conda environment files and manual setup scripts, consolidating all environment and dependency management to Pixi (conda/neutron-imaging.yml, conda/neutron-imaging-dev.yml, environment.yml, config_conda_env.sh). (F4957d57R1, [1] [2] [3]
  • Added .gitattributes rule to treat pixi.lock as a generated file and prevent merge conflicts.

CI/CD and Automation

  • Rewrote the main CI workflow to use Pixi for all setup, testing, and linting, and added steps for dependency auditing.
  • Introduced workflows for automated Pixi lockfile updates and pre-commit hook updates, both creating PRs as needed. [1] [2]
  • Added Dependabot configuration for automated updates of GitHub Actions and Python dependencies, with grouped PRs.

Pre-commit and Linting

  • Upgraded and expanded pre-commit hooks, added exclusions for pixi.lock and notebooks, and included new YAML/TOML linters and a custom Pixi lockfile check.
  • Added .yamllint configuration for YAML style enforcement.

Documentation

  • Updated README.md to reflect Pixi usage, including installation, commands, and updated testing/deployment instructions. [1] [2]

Miscellaneous

  • Updated deployment workflow to use the latest version of the commit-comment action.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates the project from Conda to Pixi for environment and dependency management, while also fixing plot rendering issues in Jupyter notebooks. The migration consolidates all environment configuration into a single pyproject.toml file and introduces automated workflows for dependency updates and code quality checks.

  • Migration from Conda to Pixi with consolidated environment management in pyproject.toml
  • Implementation of automated CI/CD workflows for testing, dependency updates, and pre-commit maintenance
  • Fix for plot rendering issues in notebooks by properly capturing output in widgets

Reviewed Changes

Copilot reviewed 85 out of 522 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pyproject.toml New Pixi configuration with modular features and environments for scientific computing
notebooks/tests/test_imports.py Import validation test ensuring all required packages are available
notebooks/__code/venus_display_nexus_tpx3/main.py Fixed plot rendering by capturing output in widgets and improved code formatting
conda/* and environment.yml files Removed legacy Conda environment files
README.md Updated documentation with Pixi installation and usage instructions
.github/workflows/* New automated workflows for testing, dependency updates, and pre-commit maintenance
.pre-commit-config.yaml Enhanced pre-commit configuration with additional hooks and Pixi integration

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread pyproject.toml
ipython = "*"

[tool.pixi.feature.jupyter.tasks]
lab = "jupyter lab --NotebookApp.nbserver_extensions=\"{'jupyterlab':True}\""
Copy link

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --NotebookApp.nbserver_extensions flag is deprecated in newer JupyterLab versions. Consider using jupyter lab without this flag or use the modern configuration approach with jupyter_server_config.py if specific extensions need to be configured.

Suggested change
lab = "jupyter lab --NotebookApp.nbserver_extensions=\"{'jupyterlab':True}\""
lab = "jupyter lab"

Copilot uses AI. Check for mistakes.
ax1.set_xlabel("Event Time Offset (micros)")
ax1.set_ylabel("Counts")
ax1.set_title("Event Time Offset Histogram")
plt.tight_layout()
Copy link

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plt.tight_layout() call affects the global matplotlib state, but since plots are being captured within an output widget context, this should be applied to the specific figure instance. Use fig1.tight_layout() instead to avoid potential interference with other plots.

Suggested change
plt.tight_layout()
fig1.tight_layout()

Copilot uses AI. Check for mistakes.
im = ax2.imshow(full_image, cmap="viridis", interpolation="nearest")
plt.colorbar(im, ax=ax2)
ax2.set_title(os.path.basename(nexus_full_path))
plt.tight_layout()
Copy link

Copilot AI Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plt.tight_layout() call affects the global matplotlib state, but since plots are being captured within an output widget context, this should be applied to the specific figure instance. Use fig2.tight_layout() instead to avoid potential interference with other plots.

Suggested change
plt.tight_layout()
fig2.tight_layout()

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

@JeanBilheux JeanBilheux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested a couple of notebook. All the notebooks will be tested over a long period of time but the deployment and launch of the notebooks works perfectly. Plot not being displayed has been fixed as well.

@JeanBilheux JeanBilheux merged commit 6d534eb into next Aug 26, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants