Failure when trying to download a rucio-corrupted file #137
Description
See this for example: /scratch/yuanlq/workflows/runs/straxen_v2.2.1/xenonnt_offline/daniel_20240409/00/03/events_ID0000291.out.002. The failure happened here:
strax.storage.common.DataCorrupted: Cannot open metadata for xnt_051694:peak_positions_cnn-iongm54rho
The issue is that this rule xnt_051694:peak_positions_cnn is incomplete. See this
(XENONnT_development) yuanlq@ap23:/scratch/yuanlq/workflows/runs/straxen_v2.2.1/xenonnt_offline/daniel_20240409/00/03$ rucio list-rules xnt_051694:peak_positions_cnn-iongm54rho
/cvmfs/xenon.opensciencegrid.org/releases/nT/development/anaconda/envs/XENONnT_development/bin/rucio:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').run_script('rucio-clients==32.8.0', 'rucio')
ID ACCOUNT SCOPE:NAME STATE[OK/REPL/STUCK] RSE_EXPRESSION COPIES SIZE EXPIRES (UTC) CREATED (UTC)
-------------------------------- ---------- ---------------------------------------- ---------------------- ------------------ -------- ------ --------------- -------------------
e3667ddd171242b3a46121928bda4f09 production xnt_051694:peak_positions_cnn-iongm54rho OK[3/0/0] UC_MIDWAY_USERDISK 1 N/A 2024-04-07 06:54:55
(XENONnT_development) yuanlq@ap23:/scratch/yuanlq/workflows/runs/straxen_v2.2.1/xenonnt_offline/daniel_20240409/00/03$ rucio list-files xnt_051694:peak_positions_cnn-iongm54rho
/cvmfs/xenon.opensciencegrid.org/releases/nT/development/anaconda/envs/XENONnT_development/bin/rucio:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').run_script('rucio-clients==32.8.0', 'rucio')
+-------------------------------------------------+--------------------------------------+-------------+------------+----------+
| SCOPE:NAME | GUID | ADLER32 | FILESIZE | EVENTS |
|-------------------------------------------------+--------------------------------------+-------------+------------+----------|
| xnt_051694:peak_positions_cnn-iongm54rho-000000 | 7422C99D-721C-43EC-BE4D-1DB2E932C4D1 | ad:3eb8895b | 66.119 MB | |
| xnt_051694:peak_positions_cnn-iongm54rho-000002 | C3B3BC57-4F75-464F-BFF4-20D6DA9F119D | ad:cc34ae4d | 66.412 MB | |
| xnt_051694:peak_positions_cnn-iongm54rho-000004 | 04106F2F-674D-4042-B5F0-210582EDA52E | ad:ad6b2416 | 65.421 MB | |
+-------------------------------------------------+--------------------------------------+-------------+------------+----------+
Total files : 3
Total size : 197.952 MB
We want a mechanism that allow download to fail, but just modify the to process list.