Major Update: Simplified Data Structure for QUEENS

[sbrandstaeter]

🚨 Major Update: Simplified Data Structure for QUEENS

Overview

We’ve implemented a fundamental change to the directory structure used by QUEENS to simplify experiment management and streamline workflows. This update eliminates unnecessary nesting and redundant naming, resulting in a cleaner, more intuitive structure.

Related PRs:

• refactor: naming of simulation files #156
• Simplify base directory structure #157

What’s the New Structure?

Experiments and tests are now fully separated:

• User experiments are stored in: $HOME/queens-experiments/
• Cluster test outputs (remote only) are stored in: $HOME/queens-tests/

New Directory Layout

$HOME/queens-experiments
  ├── experiment_name_1
  └── experiment_name_2

$HOME/queens-tests  # only for remote cluster test runs
  ├── pytest-0
  │   ├── test_1
  │   └── test_2
  ├── pytest-1
      ├── test_1
      └── test_2

You can easily change the base directory by modifying the BASE_DATA_DIR variable in the queens.utils.config_directories module.

File Naming Cleanup

We’ve also revamped the file naming convention within each experiment folder to remove redundant details already implied by the directory path. The result is a layout that’s cleaner and more developer-friendly.

Example Layout

$HOME/queens-experiments/experiment_name_1
  ├── 0/
  │   ├── input_file.dat
  │   ├── metadata.yaml
  │   ├── jobscript.sh
  │   ├── geometry_file.exo
  │   └── output/
  │       ├── output.err
  │       ├── output.log
  │       └── output.vtu
  └── 1/
      ├── input_file.dat
      ├── metadata.yaml
      ├── jobscript.sh
      ├── geometry_file.exo
      └── output/
          ├── output.err
          ├── output.log
          └── output.vtu

Why This Matters

• Simplified file navigation
• Easier scripting and automation
• Eliminates legacy clutter
• Clear separation between user experiments and cluster tests

What You Need to Do

No immediate action is required. However, if you maintain a derivative project, you may want to update your workflows accordingly:

• Replace references to queens-simulation-data with queens-experiments.
• Adjust any scripts that depend on the old directory structure or file names.

If you encounter any issues during the transition, please open an issue or start a discussion.

Thank you for helping us keep QUEENS clean, modern, and user-friendly.

— The QUEENS Maintainer Team 👑