Skip to content

Merge of porting locit and position mdses to use xarray datatrees. #352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vsmagalhaes merged 94 commits into from
Jan 16, 2026
Merged

Merge of porting locit and position mdses to use xarray datatrees. #352

vsmagalhaes merged 94 commits into from
Jan 16, 2026

Conversation

@vsmagalhaes
Copy link
Collaborator

@vsmagalhaes vsmagalhaes commented Jan 15, 2026

This is a merge of the Work in progress branch: 351 migration of astrohack data file classes to use xarray datatrees.

Here is merged the work that has been done to port locit mdses and position mds to use Xarray data trees as their format on disk and memory.
The new mds classes, AstrohackBeamcutFile, AstrohackLocitFile and AstrohackPosition file are now all kept in their own files are all descedent of a base class, AstrohackBaseFile, that implements IO, summary, data access and creation from scratch.

@vsmagalhaes
Created a base class for the new mds format using xarray data trees.
32461ed
@vsmagalhaes
started rewritting of beamcut_mds.py using the base_mds.py.
ac3d108
@vsmagalhaes
Adequating to changes in the base formats to use xarray datatrees
aec22dc
@vsmagalhaes
Added new version labelling routine that also adds that file was crea…
15a563f 
…ted with astrohack.
@vsmagalhaes
Factorized adding version to file in base_mds.py
29770da
@vsmagalhaes
work towards a factorized summary.
f8188b1
@vsmagalhaes
Factorized summary that gets depth and labels on its own.
d91964b
@vsmagalhaes
all methods in beamcut_mds.py reactivated.
26354fd
@vsmagalhaes
Added str to type tip for ddi.
55c984c
@vsmagalhaes
cleanup
2a5b5bf
@vsmagalhaes
Documentation revision.
5826796
@vsmagalhaes
Created function to construct a graph directly to a xdt inside a mds.
f6f922c
@vsmagalhaes
Beamcut now uses factorized machinery to build the datatree inside th…
752591d 
…e mds.
@vsmagalhaes
Made test_data_selection in test_beamcut.py robust to lists being in …
686b160 
…a different order.
@vsmagalhaes
Fixed error when previous test failed to download data.
36319cf
@vsmagalhaes
Skeleton of locit_mds.py
0bb7de5
@vsmagalhaes
Started work on porting extract_loct to use the new data format based…
aa93b8c 
… on datatrees.
@vsmagalhaes
Added items method to iterate through items just like with dictionaries.
8ba7048
@vsmagalhaes
Fully implemented change to new data format for extract_locit_2.py
d71931b
@vsmagalhaes
Debugged attribute keys that could not be saved to disk by xarray.
60f631c
@vsmagalhaes
Removed prints.
46a27f3
@vsmagalhaes
implemented print_source_table and print_array_configuration in locit…
7933512 
…_mds.py
@vsmagalhaes
implemented plotting routines in locit_mds.py
9b31e89
@vsmagalhaes
open_locit migrated to using the new AstrohackLocitFile class.
16c4db4
@vsmagalhaes
Moved utility function to print dictionary and its types to utils.text.
61ae9d4
@vsmagalhaes
Added values method to AstrohackBaseFile
f2ead5f
@vsmagalhaes
Created position_mds.py.
5daf099
@vsmagalhaes
Most of the migration work done in src/astrohack/locit_2.py
127b61a
@vsmagalhaes
DOcumentation updates to extract_locit_2.py
ddc90c2
@vsmagalhaes
Migrated locit_2.py's locit_separated_chunk to use the new positions_…
323af10 
…mds.
@vsmagalhaes vsmagalhaes self-assigned this Jan 15, 2026
@vsmagalhaes vsmagalhaes added the Area:Infrastructure Changes in infrastructure label Jan 15, 2026
@vsmagalhaes vsmagalhaes linked an issue Jan 15, 2026 that may be closed by this pull request
Migration of astrohack Data File classes to use Xarray datatrees #351
Closed
@vsmagalhaes
Removed problematic JetTransient property introduced by pycharm
456a9dc
@vsmagalhaes
Used nbstrip on locit_tutorial.ipynb
0bba7d1
@vsmagalhaes
Added skipping to test_dio.py for the moment.
f914deb
@vsmagalhaes
Fixed mds.param.json json validation file for mds.py
a8f6bb2
@vsmagalhaes
Added python3 shebang and executable flag to file_uploader.py
abc0730
@vsmagalhaes
Renamed position data folder.
6bc5910
@vsmagalhaes
Changed deprecated np.integer dtype for np.int64
f6d491a
@vsmagalhaes
tested changing tests to pull_request only
1e3d088
@vsmagalhaes
tested changing macos tests to pull_request only
5e4df9d
@vsmagalhaes
Changed test on pngs by adding a tolerance on the difference between …
368a246 
…pixels.
@vsmagalhaes
tested changing ipynb tests to pull_request only
14d4d10
@vsmagalhaes
Lowered tolerance on pngs to see if macos tests work
35259a8
@vsmagalhaes
Added more detailed png test to debug macos tests.
576a3e2
@vsmagalhaes
Now more detailed feedback is in all png tests in test_position_mds.py
b343ac1
@vsmagalhaes
PNG equality test now has a tolerance of 1e-5 on the mean difference …
e6624f8 
…between images, as macos seems to produce pngs that are ~5e-6 different from the ones produced in linux.
@vsmagalhaes vsmagalhaes changed the title Merge of complete porting to xarray datatrees of antenna position datasets. Merge of porting locit and position mdses to use xarray datatrees. Jan 16, 2026
@vsmagalhaes vsmagalhaes merged commit 4adeafa into astrohack-dev Jan 16, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@vsmagalhaes vsmagalhaes

Labels

Area:Infrastructure Changes in infrastructure

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Migration of astrohack Data File classes to use Xarray datatrees

2 participants

@vsmagalhaes