Skip to content

feat!: Refactor Voxel partitions - Simulation controllers #191

Merged
drodarie merged 9 commits intomainfrom
next
Nov 14, 2025
Merged

feat!: Refactor Voxel partitions - Simulation controllers #191
drodarie merged 9 commits intomainfrom
next

Conversation

@drodarie
Copy link
Contributor

@drodarie drodarie commented Nov 14, 2025 •
edited by github-actions bot
Loading

Describe the work done

List which issues this resolves:

Fix #145 #146 #179 #100 #46 #175 #174 #173 #172 #20 #189 #149 #44

Breaking Changes

  • NEST connection_models synapse attribute is now a list and is renamed synapses
  • NrrdVoxels source removed as a duplicate of sources. NrrdVoxels sources can now be either a string reference to a nrrd file in the config files or a castable dictionary or directly a casted NrrdFileDependency object.
  • NrrdVoxels keys is also removed. density_key used in placement will be either the reference to the file or the name of the file (similar to morphologies).

📚 Documentation preview 📚: https://bsb-hdf5--191.org.readthedocs.build/en/191/

📚 Documentation preview 📚: https://bsb--191.org.readthedocs.build/en/191/

📚 Documentation preview 📚: https://bsb-nest--191.org.readthedocs.build/en/191/

📚 Documentation preview 📚: https://bsb-json--191.org.readthedocs.build/en/191/

📚 Documentation preview 📚: https://bsb-core--191.org.readthedocs.build/en/191/

drodarie and others added 9 commits October 9, 2025 10:49
@drodarie @Helveg
feat!: multiple synapses per connections for NEST (#148)
0b58f80 
* feat: allow multiple synapse per NEST connection

fix #145

* test: add unittests for multi synapses

* fix: make delay not required for synapses that do not use it.

fix #146

* docs: fix guide_nest.rst

* chore: test nx vars for affected

* chore: cleanup

* fix: use r0manchak/nx-set-shas to get proper affected projects

* fix: test with manual fetching of BASE and HEAD

* fix: add missing quotes

* fix: pass NX var to command

* fix: gha for build and pull_request

* fix: second try with r0manchak/nx-set-shas

* fix: test without origin and with HEAD

* fix: test with origin and with HEAD

* fix: sending of coverage

* fix: missing square brackets

* fix: missing parenthesis Cylindrical targetting

* fix: move synaptic parameters tests to NestSynapseSettings

* Update packages/bsb-nest/bsb_nest/connection.py

Co-authored-by: Robin De Schepper <robin.deschepper93@gmail.com>

* fix: refactor NestConnectionSettings to have only one rule and set of constants

* fix: `insert_transmitter` unique under NEURON 9 (#177)

* feat(arborize): added a way to control the next id tag of schematics

* prevent issue 159

* removed test prints and addressed review comments

* fix: lint

---------

Co-authored-by: Robin De Schepper <robin@alexandria.sc>
Co-authored-by: drodarie <d.rodarie@gmail.com>

* fix: click default value test for None values

See click release notes for v8.3

* chore(release): 6.0.7 [skip ci]

* fix: use nest.CollocatedSynapses for connections built with NEST rule

* test: add comments to unit tests

---------

Co-authored-by: Robin De Schepper <robin.deschepper93@gmail.com>
Co-authored-by: Robin De Schepper <robin@alexandria.sc>
Co-authored-by: DBBS Laboratory <neurocomp_dbbs@unipv.it>
@drodarie
chore: merge branch 'main' into next
812a974
@filimarc
feat: Add simulation's controllers (#169)
6edfbb8 
* feat: start to add the possibility to register and use simulation's controllers

* test: Add test for async ending | remove AdapterProgress
* add get_components() in Simulation
* raise implement() to SimulationComponent level
* switch from prepare_samples to implement for Arbor devices
* create_device method is substitute by implement_components
* feat: give option to set a FixedStepProgressController at start

---------

Co-authored-by: filimarc <filippo.marchetti@unipv.it>
@filimarc
fix: correctly call synapse point process in arborize neuron builder (#…
d2556e8 
…181)

* fix: correct call for synapse point process in neuron builders

* test: add small test for insert_receiver

* fix: add a patch for test synapse to clear the syn added by test receiver

* chore: switch to NEURON 9

* test: refactor test

* test: refactor test

---------

Co-authored-by: filimarc <filippo.marchetti@unipv.it>
@Helveg @drodarie
feat: Added experimental OpenTelemetry integration (#184)
4966ecb 
* trace CLI context

* instrument node and command. add log events

* add otel api dep

* leave note on inf recurse in closure patching

* propagate otel ctx in job threads

* instrument node classes

* align option names to namespace names for easier inspection

* instrument commands

* removed meter code, moved otel code to profiling

* should auto classmap

* removed more meters, todo readd as traces

* docs

* fix: lint format and unit tests

* fixed otel under mpi. updated docs

* fix docs

* fix: Fix documentation and add missing python libs for running opentelemetry

* docs: Add explanations for opentelemetry while BSB is running

* updated docs

* fix: add mpi barrier in profiling test

---------

Co-authored-by: Robin De Schepper <robin@alexandria.sc>
Co-authored-by: drodarie <d.rodarie@gmail.com>
@drodarie @Helveg
feat!: Refactor voxel based partitions for atlases (#171)
846e446 
* feat: transfer atlas-tools branch from bsb-core

* test: add unittests for NRRDDependencyNodes

* chore: add pragma no cover

* fix: excluded lines for coverage.

* chore: add pragma: nocover to abstract functions.

* test: add tests for partitions

* fix: file cleaning during unittest in parallel

* fix: removes get_voxelset as duplicate function to_voxel in Voxels class

- deals with the case where a partition does not have the particular density key attached to its data.

* fix: move voxel utility functions to voxels.py

* fix: click default value test for None values

see release notes for v8.3

* fix: examples for allen partitions, add tests for voxel resolution

change atlas_datasets to voxel_datasets

* feat: generalize voxels utility functions to list of voxels

- simplify VolumetricRotations using voxels functions and NrrdDependencyNode
- get rid of space_origin attribute which can cause out of bound issues for partitions.

* fix: add MPI barriers in tearDownClass

* feat: refactor voxel_datasets so that they are at the root of the configuration.

* fix: docstrings

* feat: rename voxel_datasets into files, make FileDependencyNode dynamic

* feat: allow reference attributes to be directly cast

* docs: Start to rewrite documentation on reference attributes

* fix: in case of an attribute soft reference, cast values that are not ready or references in __ref__

2nd pass on documentation on reference attributes
rename hard_reference into reference_only

* fix: Make ConfigurationReferenceListAttribute values stored in cfglist

move set type to __init__.
Add tests for setting element of ConfigurationReferenceListAttribute

* docs: Add documentation about files linking

- Refactor configuration files documentation (merge parsers)
- Add docs about new config root component `files`
- Specify union type for RegionalReference

* fix: add cfgreflist to store both references and values.

* fix: missing return and improve links in docs

* use a property to lazily evaluate cfgattr.type

* fix: Apply suggestions from code review

Co-authored-by: Robin De Schepper <robin.deschepper93@gmail.com>

* docs: implement review feedback

* fix: type_handler name extraction for UnionType

* fix: implement feedback from review

* fix: implement feedback from review

* fix: limit logs during unittests #182

- disable tqdm if bsb log level < 2
- fix missing flag for bsb-arbor iso-docs

* fix: reimplement is_ref for reference, make VoxelDatasetReference return a Node

try setting bsb verbosity flag in env for gha

* fix: try setting env vars via GITHUB_ENV

* fix: use report instead of prints for NonTTYTerminalListener

Remove NX_DAEMON as it has no impact on command nx run or nx affected

* fix: removes ConfigurationReferenceAttribute.is_reference_value and Reference.is_ref

revert change on NonTTYTerminalListener

* fix: remove ConfigurationReferenceListAttribute.get_ref_key

make ConfigurationReferenceAttribute.get_ref_key protected

* feat: add option to quiet logs

* docs: Add documentation on BSB options

* fix: first pass implementing reviewer feedback

* removed NoRefAttrSignal, added ReferenceLambdaError

* added ref lambda enhancement, tests, and docs

* fix: test_options and lint

* fix: autoconfig list attribute parsing for sphinx

add missing python packages version

* fix: voxel references call when removing from cfgdict and use reflist in to_voxels

---------

Co-authored-by: Robin De Schepper <robin@alexandria.sc>
Co-authored-by: Robin De Schepper <robin.deschepper93@gmail.com>
@drodarie
feat: Add BSB examples (#186)
3f7fac1 
* docs: add yaml configurations snippets for the getting-started sections

* docs: add yaml configurations snippets for the neuron getting-started sections

* feat: the bsb simulate command will now create the output folder if provided. See #174

* fix: reference resolution when setting configuration

See #44, #175

* feat: isolate the getting-started examples in an independant folder.

* feat: add GHA for examples

* fix: examples.yml and re-add missing files.

* fix: re-add missing file.

* fix: examples.yml and getting-started_reconstruction.rst

* feat: isolate nest and neuron simulation examples.

* feat: isolate include_morphos examples and fix simulations

* feat: isolate writing_component examples

* feat: isolate atlas examples

* fix: add missing configuration and tests for writing_components

fix getting started unittests
fix bsb docs
move plot morpho to examples folder

* fix: unittests for examples, add noqa to avoid ruff fix

* fix: neuron simulation example

- add affinity parameter to speed up simulation.
- add warnings to VoxelIntersection in case of absence of matching labels #149
- add missing dependency to dbbs-catalogue
- add unittests
- exclude examples from pre-commit

* feat: add unittests for Allen example

- VoxelIntersection: add skips in case of absence of matching labels.
- add simple density file for tests
- add unittests
- move skip_tests for allen to bsb_test
- fix allen annotations download
- fix Readmes

* fix: add missing files

* feat: add unittests for include-morphologies examples

- fix atlas json config and readme

* fix: allen unittests, include_morphos docs

- revert skip of connectivity set creation for empty morphos

* fix: warning message

* feat: add examples as nx project

- move NEST installation bash script to devtools
- add unittests to NEST examples

* fix: examples gha and examples lint

* test: echo nest folder in GHA.

* fix: examples.yml

* fix: add nest folder variable in test_example script. add missing lib

* fix: install nest in a separate installation folder

Manually load the nest_vars before running the unittests
Remove manual run of examples GHA for each push

* feat: add example for cell labeling

* fix: change examples folders.

* feat: add morphology manipulation example.

- merge all morphology examples into one big script.
- change gha triggers

* feat: make examples bsb plugins + lint

* fix: labeling cells examples

- cleanup NEST build and installation folders in case of re-installation

* fix: neuromorpho url request

* feat: add nest repeated simulation example

* fix: remove nrnsub example

* fix: remove nrnsub example in docs

* fix: add mailing list in codemeta.json

* fix: remove unecessary unittest skips

* fix: dbbs-catalogue version

add store_active_config in nest-simulation example test

* fix: documentation for atlas and NEST repeated simulation

fix neuron example, bsb-core should be installed in parallel.

* fix: python neuron example

* fix: force store the active config during compile #189

pass lint and format
add missing lib for neuron examples
add flags for examples GHA.

* fix: case when NrrdVoxel.sources elem ref is not a string (dict or node)

* fix: make installation of libgsl global

* fix: skipIfOffline

* fix: add pragma: nocover for type checking

* fix: version of examples. Prevent examples to be released

* fix: bump gha versions, move NX_DAEMON to global env, use setup-node instead of direct install

* fix: re-add verify=False for NM selector
@drodarie
fix: add lockfile to prevent concurrent installation of NEST on GHA
4a1be6f
@drodarie
fix: add exit to installation script for NEST on GHA
aa5b560
@codecov
Copy link

codecov bot commented Nov 14, 2025

Codecov Report

❌ Patch coverage is 81.12927% with 127 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.17%. Comparing base (077e582) to head (aa5b560).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
packages/bsb-core/bsb/config/_attrs.py 79.34% 30 Missing and 8 partials ⚠️
packages/bsb-test/bsb_test/__init__.py 17.39% 19 Missing ⚠️
packages/bsb-core/bsb/simulation/adapter.py 76.92% 13 Missing and 2 partials ⚠️
packages/bsb-core/bsb/profiling.py 81.08% 11 Missing and 3 partials ⚠️
packages/bsb-core/bsb/config/refs.py 85.71% 5 Missing and 2 partials ⚠️
packages/bsb-core/bsb/topology/partition.py 89.65% 4 Missing and 2 partials ⚠️
...re/bsb/connectivity/detailed/voxel_intersection.py 66.66% 3 Missing and 1 partial ⚠️
packages/bsb-core/bsb/_options.py 78.57% 3 Missing ⚠️
packages/bsb-core/bsb/cli/commands/_commands.py 25.00% 3 Missing ⚠️
packages/bsb-core/bsb/config/_make.py 40.00% 1 Missing and 2 partials ⚠️
... and 9 more
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
+ Coverage   76.20%   78.17%   +1.96%     
==========================================
  Files         196      196              
  Lines       18476    17936     -540     
  Branches     2143     2096      -47     
==========================================
- Hits        14080    14021      -59     
+ Misses       3780     3362     -418     
+ Partials      616      553      -63

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This was linked to issues Nov 14, 2025
ref in config added to resolved scaffold is not casted. #44
Closed
Gap junction delay issue with bsb-nest #146
Closed
No error raised if "morphology_labels" are mispelled #149
Closed
Add missing yaml equivalents in the getting started #172
Closed
NEURON guide needs UX improvements and updates #173
Closed
Simulation argument -o does not create directory #174
Closed
issue attribute name _pp #179
Closed
Configuration may need to be saved after compile #189
Closed
@drodarie drodarie requested review from Helveg and filimarc November 14, 2025 17:17
filimarc
filimarc approved these changes Nov 14, 2025
View reviewed changes
Copy link
Contributor

@filimarc filimarc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@drodarie drodarie merged commit 0721515 into main Nov 14, 2025
37 checks passed
@Helveg Helveg deleted the next branch November 25, 2025 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment