Support `TRANSFER-RETRIEVE-URL` extension

[mih]

This is a recent addition (https://git-annex.branchable.com/news/version_10.20251215/). See https://git-annex.branchable.com/todo/Special_remote_redirect_to_URL/ for more information.

This is a very useful feature, because it allows special remote implementations to leave all download handling to git-annex itself, which avoid redundant implementations and is likely to yield more secure and more coherent behavior.

At the moment only non-export retrieval is supported. In the TODO item listed above a few possible future directions are mentioned.

Looking at the code, it appears that support for this feature would need to be added to

AnnexRemote/annexremote/annexremote.py

Lines 602 to 619 in ad690db

	def do_TRANSFER(self, param):
	try:
	(method, key, file_) = param.split(" ", 2)
	except ValueError:
	raise SyntaxError("Expected Key File")
	if not (method == "STORE" or method == "RETRIEVE"):
	return self.do_UNKNOWN()
	func = getattr(self.remote, "transfer_{}".format(method.lower()), None)
	try:
	func(key, file_)
	except RemoteError as e:
	return "TRANSFER-FAILURE {method} {key} {e}".format(
	method=method, key=key, e=e
	)
	else:
	return "TRANSFER-SUCCESS {method} {key}".format(method=method, key=key)

One approach would be to support a return value for func(). If a non-None return value is observed, a

TRANSFER-RETRIEVE-URL Key <return-value>

response would be produced (rather than TRANSFER-SUCCESS).

However, this is a bit convoluted, because do_TRANSFER handles any transfer, not just retrieval.

An alternative approach would be to support a dedicated exception RetrievalRedirect(Exception), and have the above logic be triggered by this exception.

Personally, I am leaning towards implementing an approach with a conditional code path in the else branch at

AnnexRemote/annexremote/annexremote.py

Lines 618 to 619 in ad690db

	else:
	return "TRANSFER-SUCCESS {method} {key}".format(method=method, key=key)

, evaluating a return value. Something like:

else:
    if "TRANSFER-RETRIEVE-URL" in self.protocol.extensions and method == "RETRIEVE" and _ret:
        return "TRANSFER-RETRIEVE-URL {key} {url}".format(key=key, url=_ret)
    else:
        return "TRANSFER-SUCCESS {method} {key}".format(method=method, key=key)

[yarikoptic]

Although exception-way sounds "easier", I personally would prefer an explicit return type handling / if which would make it explicit.

Having not looked into this codebase recently, it raises a question on either it couldn't cause some silent artifact on already existing implementations using AnnexRemote to get a new behavior?

half-baked idea -- might be just noise, would need to check implementation

So may be we should introduce internal extensions_supported or alike where implementations could claim which extensions of annex they support and extensions_enabled which would pretty much be an overlap between what AnnexRemote instance and git-annex support - this way we can future proof it