Add `.edf` and `.gb` (General Binary) adapters for ALS BL7.3.3 by Wiebke · Pull Request #4 · als-computing/splash_tiled

Wiebke · 2026-03-19T19:30:38Z

Builds on the initial adapter skeleton to add two working file adapters for the PILATUS 2M detector at beamline 7.3.3:

EDFAdapter: reads .edf image files via fabio, merging EDF header fields with metadata from a companion .txt file
GeneralBinaryPilatus2MAdapter: reads stitched .gb float32 images, combining metadata from paired _hi and _lo EDF companions; selects the later acquisition date
Both adapters subclass ArrayAdapter.

Includes round-trip integration tests (register, then read via tiled client) and unit tests for metadata parsing and merging logic. Test fixtures use synthetic data derived from real example files, with PI names anonymized.

Requires pip install ".[bl733]" for fabio.

To use the example config, populate a folder .data with .edf .gb and compainion .edf files and run:

tiled serve config ./example_configs/bl733/config.yaml

and

tiled register <tiled_uri> --verbose \
            --ext '.edf=application/x-edf' \
            --adapter 'application/x-edf=als_tiled.bl733.adapters.edf:EDFAdapter' \
            --ext '.gb=application/x-gb' \
            --adapter 'application/x-gb=als_tiled.bl733.adapters.gb:GeneralBinaryPilatus2MAdapter' \
            ./data

Some considerations:

Side-car .txt and .edf files are not registered as explicit data sources. This simplifies registration as a single file can serve as the "trigger" to register a scan and no custom detection of mimetime is needed, as the file ending directly translated to the mimetype). This does mean that built-in download capabilities in Tiled will not include these additional files. We have two option to make native download possible: A custom exporter that exports the full set of side-car, or switching to combined mimetypes (edf+txt), (gb+2txt+2edf). test_image_file_with_sidecar_metadata_file
Tests are inspired by round-trip registration tests for custom formats: bluesky/tiled/tests/test_custom_format.py
Current core Tiled adapters for images subclass either ArrayAdapter (e.g. bluesky/tiled/adapters/jpeg.py) or Adapter[ArrayStructure] (e.g. bluesky/tiled/adapters/npy.py. I choose ArrayAdapter as Adapter[ArrayStructure] versions seem to be earmarked for refactoring, see Adapter Rationalisation bluesky/tiled#1066

dylanmcreynolds · 2026-03-19T22:58:56Z

Can you please add an __init__.py in the bl733 folder?

>>> import als_tiled.bl733.edf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'als_tiled.bl733.edf'

Wiebke · 2026-03-20T16:43:11Z

__init__.py was indeed missing from als_tiled/bl733/ and als_tiled/bl733/adapters/. They have been added. The ModuleNotFoundError for als_tiled.bl733.edf is correct though: the adapter lives at als_tiled.bl733.adapters.edf.

The adapters/ subdirectory is intentional to leave room for als_tiled.bl733.exporters/ and similar additions later. Happy to flatten if a simpler structure is preferred.

dylanmcreynolds · 2026-03-20T17:24:39Z

Sorry, misread the structure in an IDE that flattens empty dirs. I like the adapters and exporters packages

src/als_tiled/bl733/adapters/edf.py

…ient]>=0.2.8` a core dependency in `pyproject.toml`

Otherwise test CI fails with missing module error

Wiebke · 2026-03-26T02:51:12Z

Since the initial review comments the following changes were made:

Updated tiled to the most recent release 0.2.8 (base image and core dependency)
Added .mypy.ini with per-module strictness (strict on adapters, error-checking on tests, loose elsewhere)
Updated pre-commit hook versions and added mypy. The precommit
Test dependencies now pull in bl733 and tiled-all via als_tiled[bl733,tiled-all] so pip install ".[test]" installs everything needed
Added test for GB registration when companion EDF files are missing

Wiebke · 2026-03-26T17:01:09Z

Some additional notes:

This builds on previous work in mlexchange/workflow-viz, in particular also mlexchange/workflow-viz/pull/2 by @rajasriramoju

fabio.open is an alias to fabio.openimage.openimage which in my current understanding eagerly reads data. This is in contrast to Pillow's [Image.open](https://pillow.readthedocs.io/en/stable/reference/Image.html#functions) or numpy.load for .npy files.
This lazy loading enables a different initialization scheme for ArrayAdapter that is not followed here.

dylanmcreynolds

I left a comment about logging that is pretty unimportant, happy to merge without.

dylanmcreynolds · 2026-03-26T18:37:54Z

src/als_tiled/bl733/adapters/edf.py

+        **kwargs: Optional[Any],
+    ) -> None:
+        """Adapter for `.edf` files (e.g. PILATUS3 2M) at ALS beamline 7.3.3."""
+        filepath = path_from_uri(data_uri)


I think I'd like to used the unused logger here to printout what file is being processed.

Added a logger.debug call.

dylanmcreynolds · 2026-03-26T18:38:28Z

src/als_tiled/bl733/adapters/gb.py

+        **kwargs: Optional[Any],
+    ) -> None:
+        """Adapter for a stitched detector image .gb produced at ALS beamline 7.3.3."""
+        filepath_gb = path_from_uri(data_uri)


I think I'd like to used the unused logger here to printout what file is being processed.

Added a logger.debug call.

dylanmcreynolds · 2026-03-26T18:44:06Z

tests/test_main.py

@@ -1,2 +1,2 @@
-def test_main_function():
+def test_main_function() -> None:


What an important Type hint :) . Perhaps we should remove this file.

Wiebke added 7 commits March 17, 2026 15:33

Add edf and gb adapters for tiled<0.1.0-b16

7505223

Add placeholders for adapter tests

d772743

Elevate edf function adapter to class

45a12dc

Elevate gb function adapter to class

7dfbfc3

Move txt parsing to separate metadata module

20f1e92

Add unit tests and example config for edf and gp adapters

6c41eef

Guard against spec duplication on adapter init

3b643a1

Add __init__.py to bl733 and bl733/adapters`

78d965a

dylanmcreynolds reviewed Mar 20, 2026

View reviewed changes

src/als_tiled/bl733/adapters/edf.py Show resolved Hide resolved

Wiebke added 8 commits March 25, 2026 14:29

Update base image to ghcr.io/bluesky/tiled:0.2.8 and make `tiled[cl…

b333712

…ient]>=0.2.8` a core dependency in `pyproject.toml`

Add mypy pre-commit and update others

94bbd01

Refactor tests to use fixtures and remove backend fixture

24e032b

Add __init__.py files to tests and tests/bl733 directories

260ce9d

Add mypy configuration .mypy.ini and update type hints in code

a790c9c

Add als_tiled[bl733] to test dependencies,

a8b4424

Otherwise test CI fails with missing module error

Update mypy configuration for tests to not ignore errors

b44c4f9

Tests need Tiled server dependencies

9a6dd60

Wiebke requested a review from dylanmcreynolds March 26, 2026 02:51

Expands mypy type checks to include tests

7531ae7

dylanmcreynolds approved these changes Mar 26, 2026

View reviewed changes

Wiebke added 5 commits March 26, 2026 12:00

Use already defined detector size constants for tests

32598f8

Only check header instead of full file for gb adapter

26de2ff

Normalize date format across edf and gb adapters

deb0f68

Install optional bl733 dependencies in docker image

96894d6

Debug logs for loading files

de65382

Wiebke added 2 commits March 26, 2026 17:23

No need to guard against date not being present in header

1036bae

Delete test_main.py placeholder

23723e3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `.edf` and `.gb` (General Binary) adapters for ALS BL7.3.3#4

Add `.edf` and `.gb` (General Binary) adapters for ALS BL7.3.3#4
Wiebke wants to merge 24 commits intomainfrom
bl733-adapters

Wiebke commented Mar 19, 2026 •

edited

Loading

Uh oh!

dylanmcreynolds commented Mar 19, 2026 •

edited

Loading

Uh oh!

Wiebke commented Mar 20, 2026 •

edited

Loading

Uh oh!

dylanmcreynolds commented Mar 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Wiebke commented Mar 26, 2026

Uh oh!

Wiebke commented Mar 26, 2026

Uh oh!

dylanmcreynolds left a comment

Uh oh!

dylanmcreynolds Mar 26, 2026

Uh oh!

Wiebke Mar 27, 2026 •

edited

Loading

Uh oh!

dylanmcreynolds Mar 26, 2026

Uh oh!

Wiebke Mar 27, 2026

Uh oh!

dylanmcreynolds Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -1,2 +1,2 @@
		def test_main_function():
		def test_main_function() -> None:

Conversation

Wiebke commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dylanmcreynolds commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Wiebke commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dylanmcreynolds commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Wiebke commented Mar 26, 2026

Uh oh!

Wiebke commented Mar 26, 2026

Uh oh!

dylanmcreynolds left a comment

Choose a reason for hiding this comment

Uh oh!

dylanmcreynolds Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Wiebke Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dylanmcreynolds Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Wiebke Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

dylanmcreynolds Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Wiebke commented Mar 19, 2026 •

edited

Loading

dylanmcreynolds commented Mar 19, 2026 •

edited

Loading

Wiebke commented Mar 20, 2026 •

edited

Loading

dylanmcreynolds commented Mar 20, 2026 •

edited

Loading

Wiebke Mar 27, 2026 •

edited

Loading