Updating a datalad-annex Git sibling does not work in adjusted mode

[mslw]

On Windows, running datalad update --how merge in a dataset previously cloned from WebDAV reports (ok) but the update is not applied (there are no new commits). I suppose, but can't confirm, that this is related to being on an adjusted branch.

Steps to reproduce (this assumes that the dataset is pushed from LINUX and cloned / updated from WINDOWS, not sure if the pushing OS matters):

LINUX:

(skipping creation steps)

create webdav sibling (git-only because that is the relevant part) and push:

datalad create-sibling-webdav -s sciebo --mode git-only https://fz-juelich.sciebo.de/remote.php/dav/files/m.szczepanik%40fz-juelich.de/2025-07-16/test_win_update
datalad push --to sciebo

WINDOWS:

clone the dataset:

datalad clone "datalad-annex::?type=webdav&encryption=none&exporttree=no&url=https%3A//fz-juelich.sciebo.de/remote.php/dav/files/m.szczepanik%2540fz-juelich.de/2025-07-16/test_win_update"

this retrieves the latest state (plus an adjusted branch), which is currently:

(test-update) C:\Users\mslw9\Documents\test-update\test_win_update>git log --pretty=oneline
fe3517e2608d6a14c67845a05f4703d227971042 (HEAD -> adjusted/main(unlocked)) git-annex adjusted branch
9da02e8e4ea922e9e301dce7af2fff943fd7e64e (origin/main, origin/HEAD, main) Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

LINUX:

(change content, datalad save, datalad push)

this is the state afterwards (f3f... commit is added):

❱ git log --pretty=oneline
f3f19e6fbbdba0af7768090a6430e4a64664d3cd (HEAD -> main, sciebo/main) Add content to README
9da02e8e4ea922e9e301dce7af2fff943fd7e64e Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

WINDOWS:

Update, this seemingly succeeds:

(test-update) C:\Users\mslw9\Documents\test-update\test_win_update>datalad update -s origin --how merge
[INFO] Fetching updates for Dataset("C:\Users\mslw9\Documents\test-update\test_win_update")
update.annex_sync(ok): . (dataset) [Ran git-annex-sync]
update(ok): . (dataset)
action summary:
  update (ok: 1)
  update.annex_sync (ok: 1)

However, the state is not changed (the f3f... "Add content to README" commit is missing):

(test-update) C:\Users\mslw9\Documents\test-update\test_win_update>git log --pretty=oneline
fe3517e2608d6a14c67845a05f4703d227971042 (HEAD -> adjusted/main(unlocked)) git-annex adjusted branch
9da02e8e4ea922e9e301dce7af2fff943fd7e64e (synced/main, main) Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

Software versions:
Linux: DataLad 1.2.1, datalad-next 1.5.0, cmd:annex: 10.20250630, cmd:git: 2.39.5
Windows: DataLad 1.2.1, datalad-next 1.5.0, cmd:annex: 10.20250606, cmd:git: 2.47.1.windows.2

[adswa]

I can reproduce this. What is interesting to me also is the difference in result reporting:

Linux:

(datalad) adina@muninn in /tmp/win-test on git:main
❱ datalad update --how merge
[INFO   ] Fetching updates for Dataset(/tmp/win-test) 
merge(ok): . (dataset) [Merged origin/main]   
update.annex_merge(ok): . (dataset) [Merged annex branch]
update(ok): . (dataset)
action summary:
  merge (ok: 1)
  update (ok: 1)
  update.annex_merge (ok: 1)

Windows:

(datalad) C:\Users\mih\win-test>datalad update --how merge [INFO] Fetching updates for Dataset("C:\Users\mih\win-test") update.annex_sync(ok): . (dataset) [Ran git-annex-sync] update(ok): . (dataset) action summary: update (ok: 1) update.annex_sync (ok: 1)

However, I was able to find the commit with gitk --all:

The upstream main branch seems fetched, but not merged? I can explicitly fetch the upstream branch:

(datalad) C:\Users\mih\win-test>git fetch origin main:main [INFO] Detected a filesystem without fifo support. [INFO] Disabling ssh connection caching. [INFO] Detected a crippled filesystem. From datalad-annex::?type=webdav&encryption=none&exporttree=no&url=https%3A//[fz-juelich.sciebo.de/remote.php/dav/files/a.wagner%2540fz-juelich.de/tmp/win-test](http://fz-juelich.sciebo.de/remote.php/dav/files/a.wagner%2540fz-juelich.de/tmp/win-test) 2a367b4..4697a64 main -> main

but this doesn't really help:

But I can git-annex merge:

git annex merge
merge git-annex ok
merge main
Updating 2a367b4..4697a64
Fast-forward 
  README | 1 +
 1 file changed, 1 insertion(+) 
(Merging into adjusted branch...)
Updating 489cc49..8b1c6ed 
Fast-forward
 README | 1 + 
 1 file changed, 1 insertion(+) ok

(need to stop exploring for a meeting - would be interesting to see why this doesn't work for webdav, AFAIK it isn't a problem in regular siblings)

[mih]

Can you confirm that this is really about windows and does not also apply to an adjusted mode repo on linux?

[mslw]

would be interesting to see why this doesn't work for webdav, AFAIK it isn't a problem in regular siblings

Although I haven't checked explicitly, I also think it isn't a problem with regular siblings. In this case, the WebDAV sibling uses a datalad-annex:: URL which uses the datalad-annex remote helper. I suppose (but, again, cannot confirm) that the problem would be there.

Can you confirm that this is really about windows and does not also apply to an adjusted mode repo on linux?

Good question, I will check that (must go AFK for a while first). It was observed on Windows, but I'd guess it is the adjusted mode,

[mslw]

I can confirm that this applies to adjusted mode, also on GNU/Linux, so it is not Windows-specific.

Keeping the original repo from the example above, I made another clone from the datalad-annex::?type=webdav URL (this time also on Linux), then comitted and pushed another change from the original repo. So the "pushing" repo (and sciebo) now has one extra commit (e86...):

(tmp-1b3ec553) mszczepanik@bnbnb64 in /tmp/foo on git:main
❱ git log --pretty=oneline
e860b276e0a6637d63361797429ed5bb96910d74 (HEAD -> main, sciebo/main) Another change to test adjusted mode
f3f19e6fbbdba0af7768090a6430e4a64664d3cd Add content to README
9da02e8e4ea922e9e301dce7af2fff943fd7e64e Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

While the clone is not able to apply the update (e86... missing):

(tmp-1b3ec553) mszczepanik@bnbnb64 in /tmp/foo_clone on git:main
❱ git annex adjust --unlock
adjust
Switched to branch 'adjusted/main(unlocked)'
ok

(tmp-1b3ec553) mszczepanik@bnbnb64 in /tmp/foo_clone on git:adjusted/main(unlocked)
❱ git log --pretty=oneline
4b9e260b0569794480c35062f6acedcb948c45d8 (HEAD -> adjusted/main(unlocked)) git-annex adjusted branch
f3f19e6fbbdba0af7768090a6430e4a64664d3cd (origin/main, origin/HEAD, main) Add content to README
9da02e8e4ea922e9e301dce7af2fff943fd7e64e Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

(tmp-1b3ec553) mszczepanik@bnbnb64 in /tmp/foo_clone on git:adjusted/main(unlocked)
❱ datalad update -s origin --how merge
[INFO   ] Fetching updates for Dataset(/tmp/foo_clone)
update.annex_sync(ok): . (dataset) [Ran git-annex-sync]
update(ok): . (dataset)
action summary:
  update (ok: 1)
  update.annex_sync (ok: 1)

(tmp-1b3ec553) mszczepanik@bnbnb64 in /tmp/foo_clone on git:adjusted/main(unlocked)
❱ git log --pretty=oneline
4b9e260b0569794480c35062f6acedcb948c45d8 (HEAD -> adjusted/main(unlocked)) git-annex adjusted branch
f3f19e6fbbdba0af7768090a6430e4a64664d3cd (synced/main, main) Add content to README
9da02e8e4ea922e9e301dce7af2fff943fd7e64e Add a readme
f71f97772b2380c00f90ca9fd7d1d1b7228ce9ae [DATALAD] new dataset

[adswa]

I looked into it for hours, and I thought I figured it out many times, and then not anymore again. And now I've ran out of time for the day. My last suspicion was that update does not get a target, but the _annex_sync update function that gets selected by update for annex repos in adjusted mode does not make use of it. grrr.

[mih]

This code should no longer be used. I should have removed it some time ago already. I will prioritize this now.