diff --git a/.github/workflows/sphinx.yml b/.github/workflows/sphinx.yml new file mode 100644 index 0000000..e0afdfa --- /dev/null +++ b/.github/workflows/sphinx.yml @@ -0,0 +1,22 @@ +name: Sphinx build + +on: push + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - name: Build HTML + uses: ammaraskar/sphinx-action@master + - name: Upload artifacts + uses: actions/upload-artifact@v4 + with: + name: html-docs + path: docs/build/html/ + - name: Deploy + uses: peaceiris/actions-gh-pages@v3 + if: github.ref == 'refs/heads/develop' + with: + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_dir: docs/build/html diff --git a/LICENSE.md b/LICENSE.md new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE.md @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..05d026e --- /dev/null +++ b/README.rst @@ -0,0 +1,15 @@ +--------------- +DataCatalogDocs +=============== + +Describe DUNE Data Catalogs and the samweb->metacat/rucio transition + + https://dune.github.io/DataCatalogDocs/ + +Copyright © 2023 FERMI NATIONAL ACCELERATOR LABORATORY + +This repository, and all software contained within, is licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Copyright is granted to FERMI NATIONAL ACCELERATOR LABORATORY on behalf of the Deep Underground Neutrino Experiment (DUNE). Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. diff --git a/docs/DDInterface.jpg b/docs/DDInterface.jpg new file mode 100755 index 0000000..5a574e9 Binary files /dev/null and b/docs/DDInterface.jpg differ diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..d0c3cbf --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source +BUILDDIR = build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/build/html/google14b7a5ee057dfcfa.html b/docs/build/html/google14b7a5ee057dfcfa.html new file mode 100644 index 0000000..aeefc79 --- /dev/null +++ b/docs/build/html/google14b7a5ee057dfcfa.html @@ -0,0 +1 @@ +google-site-verification: google14b7a5ee057dfcfa.html \ No newline at end of file diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 0000000..747ffb7 --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=source +set BUILDDIR=build + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.https://www.sphinx-doc.org/ + exit /b 1 +) + +if "%1" == "" goto help + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/makeDiagram.sh b/docs/makeDiagram.sh new file mode 100755 index 0000000..29b7665 --- /dev/null +++ b/docs/makeDiagram.sh @@ -0,0 +1,8 @@ +export TEMP=$PWD +cd $HERE/python +pydeps DDInterface.py -T jpg --include-missing --cluster --rankdir BT +pydeps samtest.py -T jpg --include-missing --cluster --rankdir BT +mv samtest.jpg $HERE/docs +mv DDInterface.jpg $HERE/docs +chmod +x $HERE/docs/DDInterface.jpg +cd $TEMP diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 0000000..5b477df --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1 @@ +furo==2022.09.15 diff --git a/docs/samtest.jpg b/docs/samtest.jpg new file mode 100644 index 0000000..4dd3885 Binary files /dev/null and b/docs/samtest.jpg differ diff --git a/docs/source/Ack.rst b/docs/source/Ack.rst new file mode 100644 index 0000000..fdf2158 --- /dev/null +++ b/docs/source/Ack.rst @@ -0,0 +1,4 @@ +Acknowledgements +---------------- + +The creation of this documentation was supported by the National Science Foundation Awards 2112727 and 2410721, Dept. of Energy Award DE‐SC-0022271 and Fermi National Accelerator Laboratory. diff --git a/docs/source/Intro.rst b/docs/source/Intro.rst new file mode 100644 index 0000000..ee2e5db --- /dev/null +++ b/docs/source/Intro.rst @@ -0,0 +1,10 @@ +============ +Introduction +============ + +These pages provide documentation on DUNE data catalog systems. + +DUNE is transitioning away from using `samweb` to a system based on `metacat` and `rucio` + + + diff --git a/docs/source/MetacatQueryExamples.rst b/docs/source/MetacatQueryExamples.rst new file mode 100644 index 0000000..8650153 --- /dev/null +++ b/docs/source/MetacatQueryExamples.rst @@ -0,0 +1,121 @@ + +Metacat Query examples +====================== + + + This document includes examples of `metacat` queries + +Example: Get the raw data from given protodune-sp detector runs +--------------------------------------------------------------- + + +* metacat + + .. code-block:: bash + + metacat query "files from dune:all where core.file_type=detector \ + and core.run_type='protodune-sp' and core.data_tier=raw \ + and core.data_stream=physics and core.runs[any] in (5141,5143)" + + add `--summary` or `-s` after query if you want just the # of files + + *Notes:* + + + + - *things run faster if you ask for files from a known dataset like `dune:all`* + + - *core.runs[any] means check any of the runs associated with the file for being 5141* + + - *core.runs[any] in (5141, 5142, 5147) - any of these 3 runs* + + - *core.runs[any] = 5141- single run, equivalent: 5141 in core.runs* + + - *5141 in core.runs* also works + + - *you can ask for multiple runs by using the `in (X,Y)` syntax* + +Example: Save a dataset or definition query +------------------------------------------- + +If you are interested in everything physics from `protodune-sp`, you might want to save a generic dataset or query which you can then reuse in further filtered queries. Then as you narrow thing down you can build additional datasets. + + +* metacat + + To run a MQL query and create a new dataset with the query results: + + .. code-block:: bash + + metacat dataset create -f "files from dune:all where \ + ..." : + + .. code-block:: bash + + metacat dataset create -f @file_with_mql_query.txt \ + : + + You likely need to ask for your own namespace or use namespace `usertests`. + + To run a query and add matching files to an existing dataset: + + .. code-block:: bash + + metacat dataset add-files -q "files from dune:all where ..." : + + metacat dataset add-files -q @file_with_mql_query.txt : + + .. Note: this times out if all runs are included - I just did 5141 for this test. + + .. Note: Todo: a utility command that logs the query in the dataset metadata, possibly not in the "description" field + + check it by querying the files in the dataset + + .. code-block:: bash + + metacat query -s "files from schellma:protodune-sp-physics-generic" + + metacat dataset show schellma:protodune-sp-physics-generic + + children : + created_timestamp : 2022-10-08 11:41:54 + creator : schellma + description : files from dune:all where core.file_type=detector and core.run_type='protodune-sp' and core.data_stream=physics + file_count : 772631 + file_meta_requirements : {} + frozen : False + metadata : {} + monotonic : False + name : protodune-sp-physics-generic + namespace : schellma + parents : + + + .. :Note: I have not saved the query in the metacat dataset but just added it as an optional description. I have saved the list of files. In `metacat` datasets do not change (for example if another file passing the query requirements comes in from the DAQ) until you explicitly add the new file.* + + You can then ask for the subset from a particular data tier and run number. + + .. code-block:: bash + + metacat query "files from schellma:protodune-sp-physics-generic \ + where core.runs[all]=5141 and core.data_tier=raw" + +Find only the files not processed with a version of code +-------------------------------------------------------- + + +* metacat + + + .. code-block:: bash + + metacat query -s "files from schellma:protodune-sp-physics-generic \ + where core.data_tier=raw and 5141 in core.runs - parents(files \ + from schellma:protodune-sp-physics-generic where 5141 in core.runs \ + and core.data_tier='full-reconstructed' and core.application.version~'v08_27_.*')" + + 12 files + + .. :Note: TODO - get the file size as well? + + .. :Note: the syntax for a parameter matching is Regular Expressions, in particular '.\*' matches any string* diff --git a/docs/source/Notes.rst b/docs/source/Notes.rst new file mode 100644 index 0000000..75f9bd8 --- /dev/null +++ b/docs/source/Notes.rst @@ -0,0 +1,57 @@ +Notes on sphinx +--------------- + +This was my first sphinx project. + +I started with the https://www.sphinx-doc.org/en/master/tutorial/getting-started.html tutorial so I have a similar structure. + +I used the napoleon theme that Igor Mandrichenko used from https://metacat.readthedocs.io/en/latest/ + +rst format +********** + +https://docutils.sourceforge.io/docs/user/rst/quickstart.html + +Not my favorite - + + - the need to keep track of indentation and blank lines with poor error messages is painful + + - indentation differences when adding comments to code others wrote can be interesting + +Github actions +************** + +I set up github actions to build on my github page hschellman/DataChallengeWork-loginator which is based on Jake Calcutt's data dispatcher test system + +The github actions had some issues: + + - It assumed the code was in main - I had to move to develop + + - I had to add a gh-pages branch by hand + + - I adapted ``_ from Ammar Askar + + - The action can be found at `sphinx.yml `_ + + - I was not able to get the programoutput extension to work as adding a pip install for it did not work. So command line syntax is added by hand. + + - I had to add a + + .. code-block:: python + + autodoc_mock_imports = ["metacat","samweb_client","data_dispatcher"] + + to the `conf.py` to avoid errors due to imports of external code. I just got blanks otherwise. + +Google indexing Notes +********************* + +# Go to `Google Search Console `_ + +# choose URL prefix and enter your top level URL you will get back a + +# I github added build/html (just the directories) to github to be in the same directory as `index.html` and then placed the key file I got from google in that directory. + +# I then added that file to github, committed and pushed. + +# Once the action to build the docs go to the search console again and go to URL inspection. You should be able to check that the URL exists and request indexing. \ No newline at end of file diff --git a/docs/source/NotesOnSam_v4.fld/colorschememapping.xml b/docs/source/NotesOnSam_v4.fld/colorschememapping.xml new file mode 100644 index 0000000..b200daa --- /dev/null +++ b/docs/source/NotesOnSam_v4.fld/colorschememapping.xml @@ -0,0 +1,2 @@ + + \ No newline at end of file diff --git a/docs/source/NotesOnSam_v4.fld/filelist.xml b/docs/source/NotesOnSam_v4.fld/filelist.xml new file mode 100644 index 0000000..81b4f80 --- /dev/null +++ b/docs/source/NotesOnSam_v4.fld/filelist.xml @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/docs/source/NotesOnSam_v4.fld/themedata.thmx b/docs/source/NotesOnSam_v4.fld/themedata.thmx new file mode 100644 index 0000000..3d2a437 Binary files /dev/null and b/docs/source/NotesOnSam_v4.fld/themedata.thmx differ diff --git a/docs/source/NotesOnSam_v4.html b/docs/source/NotesOnSam_v4.html new file mode 100644 index 0000000..57f74b0 --- /dev/null +++ b/docs/source/NotesOnSam_v4.html @@ -0,0 +1,2589 @@ + + + + + + + + + + + + + + + + + + +
+ +

 

+ +

Notes on the SAM data catalog system

+ +

 

+ +

H. +Schellman 1-28-2020 - updated Dec 2021

+ +

 

+ +

SAM is a +data catalog originally designed for the D0 and CDF high energy physics +experiments at Fermilab.  It is now used +by most of the Intensity Frontier experiments at Fermilab.

+ +

 

+ +

The most +important objects cataloged in SAM are individual files and collections +of files called datasets.

+ +

 

+ +

Data +files themselves are not stored in SAM, their metadata is, and that metadata +allows you to search for and find the actual physical files.  Sam also provides mechanisms for initiating +and tracking file delivery through projects.

+ +

 

+ +

General considerations

+ +

 

+ +

SAM was +designed to ensure that large scale data-processing was done completely and +accurately which leads to some features not always present in a generic catalog +but very desirable if one wishes high standards of reproducibility and +documentation in data analysis.

+ +

 

+ +

For +example, at the time of the original design, the main storage medium was 8mm +tapes using consumer-grade drives.  Drive +and tape failure rates were > 1%.  +Several SAM design concepts, notably luminosity blocks and parentage +tracking, were introduced to allow accurate tracking of files and their +associated normalization in a high error-rate environment.

+ +

 

+ +

·      +Description of the +contents of data collections to allow later retrieval

+ +

 

+ +

·      +Tracking of object +and collection parentage and description of processing transformations to +document the full provenance of any data object and ensure accurate +normalization

+ +

 

+ +

·      +Grouping of objects +and collection into larger “datasets” based on their characteristics

+ +

 

+ +

·      +Storing physical +location of objects

+ +

 

+ +

·      +Tracking of the +processing of collections to allow reprocessing on failure and avoid +double processing.

+ +

 

+ +

·      +Methods (“projects”) +for delivering and tracking collections in multi-processing jobs

+ +

 

+ +

·      +Preservation of data +about processing/storage for debugging/reporting

+ +

 

+ +

The +first 3 goals relate to content and characteristics while the last 3 relate to +data storage and processing tools. 

+ +

 

+ +

Specifics

+ +

 

+ +

1.     +The current SAM +implementation uses the file as the basic unit of information.  Metadata is associated with the file +name.  Filenames must be unique in the +system. This prevents duplication of data in a sample, as a second copy cannot +be cataloged. This makes renaming a file very unwise. A very common practice is +to include some of the metadata in the filename, both to make it easier to +identify and to ensure uniqueness.   

+ +

 

+ +

2.     Metadata for a file can include file locations +but does not have to. A file can have no location at all, or many. When you +move or remove a file with an associated SAM location, you need to update the +location information. 

+ +

 

+ +

3.     +SAM does not move +files..  It provides location information +for a process to use in streaming or copying a file using its own methods. +Temporary locations (such as on a grid node) need not be reported to SAM. +Permanently storing or removing files requires both moving/removing the file itself +and updating its metadata to reflect that location and is generally left up to +special packages such as the Fermilab FTS (File Transfer Service) and SAM +projects.

+ +

 

+ +

4.     +Files which are +stored on disk or tape are expected to have appropriate file sizes and +checksums. One can have duplicate instances of a file in different locations, +but they must all be identical.  If one +reprocesses an input file, the output may well be subtly different (for example +dates stored in the file itself can change the checksum). SAM should force you +to choose which version, old or new, is acceptable.  It will not let you catalog both with the +same filename. As a result, if you get a named file out of SAM, you can be +reasonably certain you got the right copy.

+ +

 

+ +

5.     +files with duplicate +content but different names can be problematic. The reprocessed file mentioned +in part 4, if renamed, could cause significant problems if it were allowed into +the data sample along with the originals as a job processing all files might get +both copies. This is one of the major reasons for the checksums and unique +filenames. There is a temptation to put, for example, timestamps, in filenames +to generate unique names but that removes a protection against duplication.

+ +

 

+ +

6.     +Files can have parents +and children and, effectively, birth certificates that can tell you how +they were made.  An example would be a +set of raw data files RAWnnn processed with code X to produce a single +reconstructed file RECO.  One can tell +SAM that RAWnnn are the parents of RECO processed with version x of code X. If +one later finds another RAWnnn file that was missed in processing, SAM can tell +you it has not been processed yet with X (i.e., it has no children associated +with version x of X) and you can then choose to process that file. This use +case often occurs when a production job fails without reporting back or +succeeds but the copy back or catalog action fails.

+ +

 

+ +

The D0 +experiment required that all official processing going into SAM be done with +tagged releases and fixed parameter sets to increase reproducibility and the +tags for that information were included in the metadata. Calibration databases +were harder to timestamp so some variability was still possible if calibrations +were updated.

+ +

 

+ +

7.  SAM supports several types of data +description fields:

+ +

 

+ +

Values are standard across all implementations +like run_type, file_size …

+ +

 

+ +

Parameters are defined by the experiment for +example MC.Genieversion

+ +

 

+ +

Values +are common to almost all  HEP experiments +and are optimized for efficient queries.  +SAM also allows definition of “parameters” (by administrators) as they +are needed.  This allows the schema to be +modified easily as needs arrive.

+ +

 

+ +

8.  Metadata can also contain “spill” or +luminosity block information that allows a file to point to specific data +taking periods with smaller granularity than a run or subrun. When files are +merged, this spill information is also merged.

+ +

 

+ +

9. SAM +currently does not contain a means of easily determining which file a given +event is in.  If a daq system is writing +multiple streams, an event from a given subrun could be in any stream.  Adding an event database would be a useful +feature.

+ +

 

+ +

All of +these features are intended to assure that your data are well described and can +be found. As SAM stores full location information, this means any SAM-visible +location. In addition, if parentage information is provided, you can determine +and reproduce the full provenance of any file.

+ +

 

+ +

Datasets and projects

+ +

 

+ +

Datasets

+ +

 

+ +

In +addition to the files themselves, SAM allows you to define datasets.

+ +

 

+ +

A SAM dataset +is not a fixed list of files but a query against the SAM +database. An example would be “data_tier reconstructed and run_number 2001 and +version v10” which would be all files from run 2001 that are reconstructed data +produced by version v10. This dataset is dynamic. If one finds a missing file +from run 2001 and reconstructs it with v10, the dataset will grow. There are +also dataset snapshots that are derived from datasets and capture the +exact files in the dataset when the snapshot was made.

+ +

Note: +most other data catalogs assume a “dataset” is a fixed list of files.  This is a “snapshot” in SAM.

+ +

 

+ +

samweb +– samweb is the command +line and python API that allows queries of the SAM metadata, creation of +datasets and tools to track and deliver information to batch jobs.

+ +

 

+ +

samweb +can be acquired from ups via

+ +

setup samweb_client

+ +

 

+ +

Or +installed locally via

+ +

git clone http://cdcvs.fnal.gov/projects/sam-web-client

+ +

You then need to do something like:

+ +

export  +PATH=$HOME/sam-web-client/bin:${PATH}

+ +

export  +PYTHONPATH=$HOME/sam-web-client/python:${PYTHONPATH}

+ +

export SAM_EXPERIMENT=dune

+ +

 

+ +

 

+ +

Projects

+ +

 

+ +

 

+ +

SAM also +supports access tracking mechanisms called projects and consumers.  These are generally implemented for you +by grid processing scripts. Your job is to choose a dataset and then ask the +processing system to launch a project for that dataset.

+ +

 

+ +

A project +is effectively a processing campaign across a dataset which is owned by the SAM +system. At launch a snapshot is generated and then the files in the +snapshot are delivered to a set of consumers.  The project maintains an internal record of +the status of the files and consumers. Each grid process can instantiate a +consumer which is attached to the project.  +Those consumers then request “files” from the project and, when done +processing, tell the project of their status.  +

+ +

 

+ +

The +original SAM implementation actually delivered the files to local hard +drives.  Modern SAM delivers the location +information and expects the consumer to find the optimal delivery method. This +is a pull model, where the consuming process requests the next file rather than +having the file assigned to it.  This +makes the system more robust on distributed systems.

+ +

 

+ +

See +running projects at

+ +

 

+ +

http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects

+ +

 

+ +

Accessing the database in read mode

+ +

 

+ +

Checking +the database does not require special privileges but storing files and

+ +

running +projects modifies the database and requires authentication to the right +experimental group.   kx509 +authentication and membership in the experiment VO are needed.

+ +

 

+ +

Administrative +actions like adding new values are restricted to a small set of +superusers for each experiment.

+ +

 

+ +

Suggestions for configuring SAM (for admins)

+ +

 

+ +

First of +all, it really is nice to have filenames and dataset names that tell you what’s +in the box, although not required. The D0 and MINERvA conventions have been to +use “_” underscores between useful key strings. As a result, D0 and MINERvA +tried not to use “_” in metadata entries to allow cleaner parsing. “-“ is used +if needed in the metadata.

+ +

 

+ +

D0 also +appended processing information to filenames as they moved through the system +to assure that files run through different sequences had unique identifiers.

+ +

 

+ +

Example: +A Monte Carlo simulation file generated with version v3 and then reconstructed +with v5 might look like

+ +

 

+ +

SIM_MC_020000_0000_simv3.root +would be a parent of RECO_MC_020000_0000_simv3_recov5.root

+ +

Data +files are all children of the raw data while simulation files sometimes have +more

+ +

complicated +ancestry, with both unique generated events and overlay events from data as +parents.

+ +

 

+ +

Setting up SAM metadata (For admins)

+ +

 

+ +

This +needs to be done once, and very carefully, early in the experiment. It can grow +but thinking hard at the beginning saves a lot of pain later.

+ +

 

+ +

You need +to define data_tiers. These represent the different types of data that +you produced through your processing chain. Examples would be raw, pedsup, +calibrated, reconstructed, thumbnail, mc-generated, mc-geant, mc-overlaid,

+ +

 

+ +

run_type can be used to support multiple DAQ +instances.

+ +

 

+ +

data_stream is often used for trigger subsamples +that you may wish to split data into (for example pedestal vs data runs).

+ +

 

+ +

Generally, +you want to store data from a given data_tier with other data from that tier to +facilitate fast sequential access.

+ +

 

+ +

Applications

+ +

 

+ +

It is +useful, but not required to also define applications which are triads of +“appfamily”, “appname” and “version”. Those are used to figure out what changed +X to Y. There are also places to store the machine the application ran on and +the start and end time for the job.

+ +

 

+ +

The +query:

+ +

 

+ +

samweb list-files "data_tier raw and not isparentof: (data_tier +reconstructed and appname reco and version 7)"

+ +

 

+ +

Should, +in principle, list raw data files not yet processed by version 7 of reco to +produce files of tier reconstructed. You would use this to find lost files in +your reconstruction after a power outage.

+ +

 

+ +

It is +good practice to also store the name of the head application configuration file +for processing but this does not have a standard “value”

+ +

 

+ +

Example metadata from DUNE

+ +

 

+ +

Here are some examples of querying sam to get +file information

+ +

 

+ +

 

+ +

 

+ +

$ samweb get-metadata +np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root –json

+ +

 

+ +

{

+ +

 "file_name": +"np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root",

+ +

 "file_id": 7352771,

+ +

 "create_date": +"2018-10-29T14:59:42+00:00",

+ +

 "user": "dunepro",

+ +

 "update_date": +"2018-11-28T17:07:30+00:00",

+ +

 "update_user": "schellma",

+ +

 "file_size": 14264091111,

+ +

 "checksum": [

+ +

  +"enstore:1390300706",

+ +

  +"adler32:e8bf4e23"

+ +

 ],

+ +

 "content_status": "good",

+ +

 "file_type": "detector",

+ +

 "file_format": "artroot",

+ +

 "data_tier": +"full-reconstructed",

+ +

 "application": {

+ +

  +"family": "art",

+ +

  +"name": "reco",

+ +

  +"version": "v07_08_00_03"

+ +

 },

+ +

 "event_count": 108,

+ +

 "first_event": 21391,

+ +

 "last_event": 22802,

+ +

 "start_time": +"2018-10-28T17:34:58+00:00",

+ +

 "end_time": +"2018-10-29T14:55:42+00:00",

+ +

 "data_stream": "physics",

+ +

 "beam.momentum": 7.0,

+ +

 "data_quality.online_good_run_list": +1,

+ +

 "detector.hv_value": 180,

+ +

 "DUNE_data.acCouple": 0,

+ +

 "DUNE_data.calibpulsemode": 0,

+ +

 "DUNE_data.DAQConfigName": +"np04_WibsReal_Ssps_BeamTrig_00021",

+ +

 "DUNE_data.detector_config": +"cob2_rce01:cob2_rce02:cob2.. 4 more lines of text",

+ +

 "DUNE_data.febaselineHigh": 2,

+ +

 "DUNE_data.fegain": 2,

+ +

 "DUNE_data.feleak10x": 0,

+ +

 "DUNE_data.feleakHigh": 1,

+ +

 "DUNE_data.feshapingtime": 2,

+ +

 "DUNE_data.inconsistent_hw_config": +0,

+ +

 "DUNE_data.is_fake_data": 0,

+ +

 "runs": [

+ +

  [

+ +

   5141,

+ +

   1,

+ +

   +"protodune-sp"

+ +

  ]

+ +

 ],

+ +

 "parents": [

+ +

  {

+ +

   +"file_name": "np04_raw_run005141_0015_dl10.root",

+ +

   +"file_id": 6607417

+ +

  }

+ +

 ]

+ +

}

+ +

 

+ +

$ samweb +get-file-access-url +np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root

+ +

 

+ +

gsiftp://eospublicftp.cern.ch/eos/experiment/neutplatform/protodune/rawdata/np04/output/detector/full-reconstructed/07/35/27/71/np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root

+ +

gsiftp://fndca1.fnal.gov:2811/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/output/detector/full-reconstructed/07/35/27/71/np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root

+ +

 

+ +

$ samweb file-lineage +children np04_raw_run005141_0015_dl10.root

+ +

 

+ +

np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root

+ +

 

+ +

$ samweb file-lineage +parents\

+ +

np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root

+ +

 

+ +

np04_raw_run005141_0015_dl10.root

+ +

 

+ +

Merging and splitting (for experts)

+ +

 

+ +

Parentage +works pretty well if one is merging files but splitting them can become +problematic as it makes the parentage structure pretty complex.

+ +

Sam will let you merge files with different attributes if +you don’t check carefully. Generally, it is a good idea not to merge files from +different data tiers and certainly not from different data_types. Merging +across major processing versions should also be +avoided.

+ +

 

+ +

Example: Execute samweb Commands

+ +

 

+ +

There is +documentation at

+ +

https://cdcvs.fnal.gov/redmine/projects/sam/wiki/User_Guide_for_SAM

+ +

 

+ +

https://cdcvs.fnal.gov/redmine/projects/sam-main/wiki/Updated_dimensions_syntax

+ +

This +exercise will start you accessing data files that have been defined to the DUNE +Data Catalog. Execute the following commands after logging in to the DUNE +interactive node, creating the directories above -

+ +

Once per +session

+ +

setup sam_web_client #(or +set up your standalone version)

+ +

export +SAM_EXPERIMENT=dune

+ +

Then if +curious about a file:

+ +

samweb locate-file +np04_raw_run005141_0001_dl7.root

+ +

this +will give you output that looks like

+ +

 rucio:protodune-sp

+ +

enstore:/pnfs/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05(596@vr0072m8)

+ +

castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05

+ +

cern-eos:/eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05

+ +

which +are the locations of the file on disk and tape. We can use this to copy the +file from tape to our local disk.

+ +

Better +yet, you can use xrootd to access the file without copying it if it is staged +to disk.

+ +

Find the +xrootd uri via

+ +

 samweb get-file-access-url np04_raw_run005141_0001_dl7.root +--schema=root

+ +

 

+ +

root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05 +/np04_raw_run005141_0001_dl7.root

+ +

root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root

+ +

root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root

+ +

You can +localize your file with the --location argument (enstore, castor, cern-eos)

+ +

samweb +get-file-access-url np04_raw_run005141_0001_dl7.root --schema=root +--location=enstore

+ +

 

+ +

root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05 +/np04_raw_run005141_0001_dl7.root

+ +

 

+ +

samweb +get-file-access-url np04_raw_run005141_0001_dl7.root --schema=root +--location=cern-eos

+ +

 

+ +

root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root +

+ +

To get +SAM metadata for a file for which you know the name:

+ +

samweb  get-metadata np04_raw_run005141_0001_dl7.root +

+ +

# add the --json option +to get output in json format

+ +

To list +raw data files for a given run:

+ +

samweb list-files +"run_number 5141 and run_type protodune-sp and data_tier raw"

+ +

What +about a reconstructed version?  

+ +

samweb list-files +"run_number 5141 and run_type protodune-sp and data_tier +full-reconstructed and version (v07_08_00_03,v07_08_00_04)"

+ +

 

+ +

Gives a list of files  from the first production like

+ +

 

+ +

….

+ +

np04_raw_run005141_0001_dl7_reco_12736115_0_20181028T165152.root

+ +

….

+ +

 

+ +

We also group reconstruction versions into +Campaigns like PDSPProf4

+ +

 

+ +

samweb list-files +"run_number 5141 and run_type protodune-sp and data_tier +full-reconstructed and DUNE.campaign PDSPProd4"

+ +

 

+ +

Gives more recent files like:

+ +

 

+ +

np04_raw_run005141_0009_dl1_reco1_18126423_0_20210318T102429Z.root

+ +

samweb +allows you to select on a lot of parameters

+ +

Useful ProtoDUNE samweb parameters

+ +

https://dune-data.fnal.gov and https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets

+ +

 lists some official dataset definitions

+ +

You can make your own samweb dataset +definitions

+ +

First, make certain a definition does not +already exist that satisfies your needs by checking the official pages above.

+ +

First +check to see what you will get

+ +

samweb list-files +"data_tier full-reconstructed and DUNE.campaign PDSPProd4  and data_stream cosmics and run_type +protodune-sp and detector.hv_value 180" –summary

+ +

 

+ +

samweb create-definition +$USER-PDSPProd4_good_cosmics "data_tier full-reconstructed and +DUNE.campaign PDSPProd4  and data_stream +cosmics and run_type protodune-sp and detector.hv_value 180"

+ +

Note +that the username appears in the definition name - to prevent users from +getting confused with official samples, your user name is required in the +definition name.

+ +

prestaging

+ +

At CERN +files are either on eos or castor. At FNAL they can be on tape_backed dcache +which may mean they are on tape and may need to be prestaged to disk before +access.

+ +

setup fife_utils  # a new version we requested

+ +

 

+ +

# check to see if a file is on tape or disk

+ +

 

+ +

sam_validate_dataset  --locality  +--file np04_raw_run005141_0015_dl10.root --location=/pnfs/ +--stage_status

+ +

         Staging status for: file +np04_raw_run005141_0015_dl10.root

+ +

                Total Files: 1

+ +

              Tapes spanned: 1

+ +

      Percent files on disk: 0%

+ +

Percent bytes online +DCache: 0%

+ +

locality counts:

+ +

ONLINE: 0

+ +

NEARLINE: 1

+ +

NEARLINE_size: 8276312581

+ +

 

+ +

Oops - this one is not on +disk

+ +

returns +ONLINE_AND_NEARLINE: 1 if available on disk

+ +

sam_validate_dataset  --locality --name=schellma-1GeVMC-test +--stage_status --location=/pnfs/

+ +

         Staging status for: +defname:schellma-1GeVMC-test

+ +

                Total Files: 140

+ +

              Tapes spanned: 10

+ +

      Percent files on disk: 100%

+ +

Percent bytes online +DCache: 100%

+ +

locality counts:

+ +

ONLINE: 0

+ +

ONLINE_AND_NEARLINE: 140

+ +

ONLINE_AND_NEARLINE_size: +270720752891

+ +

 

+ +

No +ONLINE_NEARLINE means you need to prestage that file. Unfortunately, prestaging +requires a definition

+ +

Let's +find some for run 5141.  Your physics +group should already have some defined.

+ +

The +official Protodune dataset definitions are at:

+ +

https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets

+ +

 

+ +

samweb describe-definition +PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00

+ +

 

+ +

Is simulation for 10% of the +total sample

+ +

 

+ +

Gives this description:

+ +

samweb +describe-definition PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00

+ +

Definition Name: +PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00

+ +

  Definition Id: 635109

+ +

  Creation Date: 2021-08-02T16:57:20+00:00

+ +

       Username: dunepro

+ +

          Group: dune

+ +

     Dimensions: run_type 'protodune-sp' and +file_type mc and data_tier 'full-reconstructed' and dune.campaign PDSPProd4a +and dune_mc.beam_energy 1 and

+ +

dune_mc.space_charge yes +and dune_mc.generators beam_cosmics and version v09_17_01 and run_number in +18800650,.....

+ +

 

+ +

samweb list-files +"defname:PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00

+ +

> " --summary

+ +

File count:         5025

+ +

Total size:         9683195368818

+ +

Event count:     50250

+ +

 

+ +

samweb prestage-dataset +--def=PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00 --parallel=10

+ +

would +prestage all of the reconstructed data for run 5141

+ +

you can +check on the status by going to

+ +

http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects +and scrolling down to see your prestage link.

+ +

At CERN

+ +

You can +find local copies of files at CERN for interactive use.

+ +

samweb list-file-locations +--defname=runset-5141-raw-180kV-7GeV-v0 --schema=root --filter_path=castor

+ +

 

+ +

gives you:

+ +

root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/16/np04_raw_run005141_0015_dl3.root            castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/16       np04_raw_run005141_0015_dl3.root     8289321123

+ +

root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/17/np04_raw_run005141_0015_dl10.root            castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/17       np04_raw_run005141_0015_dl10.root    8276312581

+ +

 

+ +

 

+ +

 

+ +

 

+ +
+ + + + diff --git a/docs/source/NotesOnSam_v4.rtf b/docs/source/NotesOnSam_v4.rtf new file mode 100644 index 0000000..5b88fca --- /dev/null +++ b/docs/source/NotesOnSam_v4.rtf @@ -0,0 +1,713 @@ +{\rtf1\ansi\ansicpg1252\cocoartf2639 +\cocoatextscaling0\cocoaplatform0{\fonttbl\f0\fswiss\fcharset0 ArialMT;\f1\fswiss\fcharset0 Arial-BoldMT;\f2\froman\fcharset0 TimesNewRomanPSMT; +\f3\fswiss\fcharset0 Arial-ItalicMT;\f4\fswiss\fcharset0 Arial-BoldItalicMT;\f5\froman\fcharset0 TimesNewRomanPS-BoldMT; +\f6\fswiss\fcharset0 Helvetica;} +{\colortbl;\red255\green255\blue255;\red51\green51\blue51;\red17\green85\blue204;\red179\green36\blue37; +\red32\green33\blue34;} +{\*\expandedcolortbl;;\csgenericrgb\c20000\c20000\c20000;\csgenericrgb\c6667\c33333\c80000;\csgenericrgb\c70196\c14118\c14510; +\csgenericrgb\c12549\c12941\c13333;} +\margl1440\margr1440\vieww12240\viewh15100\viewkind1 +\deftab720 +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\fs40 \cf0 \sub \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f1\b\fs36 \cf0 \nosupersub Notes on the SAM data catalog system +\f0\b0\fs24 \ + \ +H. Schellman 1-28-2020 - updated Dec 2021\ + \ +SAM is a data catalog originally designed for the D0 and CDF high energy physics experiments at Fermilab. It is now used by most of the Intensity Frontier experiments at Fermilab. \ + \ +The most important objects cataloged in SAM are individual +\f1\b files +\f0\b0 and collections of files called +\f1\b datasets +\f0\b0 .\ + \ +Data files themselves are not stored in SAM, their metadata is, and that metadata allows you to search for and find the actual physical files. Sam also provides mechanisms for initiating and tracking file delivery through +\f1\b projects +\f0\b0 . \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 General considerations\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +SAM was designed to ensure that large scale data-processing was done completely and accurately which leads to some features not always present in a generic catalog but very desirable if one wishes high standards of reproducibility and documentation in data analysis.\ + \ +For example, at the time of the original design, the main storage medium was 8mm tapes using consumer-grade drives. Drive and tape failure rates were > 1%. Several SAM design concepts, notably luminosity blocks and parentage tracking, were introduced to allow accurate tracking of files and their associated normalization in a high error-rate environment. \ + \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \'b7 +\f2\fs14 +\f0\fs24 Description of the contents of data collections to allow later retrieval \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \'b7 +\f2\fs14 +\f0\fs24 Tracking of +\f3\i object and collection +\f0\i0 parentage and description of processing transformations to document the full provenance of any data +\f1\b object +\f0\b0 and ensure accurate normalization\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \'b7 +\f2\fs14 +\f0\fs24 Grouping of +\f3\i objects and collection +\f0\i0 into larger +\f1\b \'93datasets +\f0\b0 \'94 based on their characteristics\ + \ +\'b7 +\f2\fs14 +\f0\fs24 Storing +\f4\i\b physical +\f1\i0 location +\f0\b0 +\f3\i of objects\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\i0 \cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \'b7 +\f2\fs14 +\f0\fs24 Tracking of the processing of +\f3\i collections +\f0\i0 to allow reprocessing on failure and avoid double processing.\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \'b7 +\f2\fs14 +\f0\fs24 Methods (\'93 +\f1\b projects +\f0\b0 \'94) for delivering and tracking +\f3\i collections +\f0\i0 in multi-processing jobs\ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 + +\fs22 \cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 + +\fs24 \cf0 \'b7 +\f2\fs14 +\f0\fs24 Preservation of data about processing/storage for debugging/reporting\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +The first 3 goals relate to content and characteristics while the last 3 relate +\f3\i to data +\f0\i0 storage and processing tools. \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Specifics\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 1. +\f2\fs14 +\f0\fs24 The current SAM implementation uses the file as the basic unit of information. +\f1\b Metadata +\f0\b0 is associated with the file name. Filenames must be unique in the system. This prevents duplication of data in a sample, as a second copy cannot be cataloged. This makes renaming a file very unwise. A very common practice is to include some of the metadata in the filename, both to make it easier to identify and to ensure uniqueness. +\f3\i \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 + +\f0\i0 \cf0 2. +\f2\fs14 +\f5\b +\f1\fs24 Metadata +\f0\b0 for a file can include file locations but does not have to. A file can have no location at all, or many. When you move or remove a file with an associated SAM location, you need to update the location information. \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 + +\f3\i \cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 + +\f0\i0 \cf0 3. +\f2\fs14 +\f0\fs24 SAM does not move files.. It provides location information for a process to use in streaming or copying a file using its own methods. Temporary locations (such as on a grid node) need not be reported to SAM. Permanently storing or removing files requires both moving/removing the file itself and updating its metadata to reflect that location and is generally left up to special packages such as the Fermilab FTS (File Transfer Service) and SAM projects. \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 4. +\f2\fs14 +\f0\fs24 Files which are stored on disk or tape are expected to have appropriate file sizes and checksums. One can have duplicate instances of a file in different locations, but they must all be identical. If one reprocesses an input file, the output may well be subtly different (for example dates stored in the file itself can change the checksum). SAM should force you to choose which version, old or new, is acceptable. It will not let you catalog both with the same filename. As a result, if you get a named file out of SAM, you can be reasonably certain you got the right copy. \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 5. +\f2\fs14 +\f0\fs24 files with duplicate content but different names can be problematic. The reprocessed file mentioned in part 4, if renamed, could cause significant problems if it were allowed into the data sample along with the originals as a job processing all files might get both copies. This is one of the major reasons for the checksums and unique filenames. There is a temptation to put, for example, timestamps, in filenames to generate unique names but that removes a protection against duplication. \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +\pard\pardeftab720\li720\ri0\sl276\slmult1\partightenfactor0 +\cf0 6. +\f2\fs14 +\f0\fs24 Files can have +\f1\b parents +\f0\b0 and +\f1\b children +\f0\b0 and, effectively, birth certificates that can tell you how they were made. An example would be a set of raw data files RAWnnn processed with code X to produce a single reconstructed file RECO. One can tell SAM that RAWnnn are the parents of RECO processed with version x of code X. If one later finds another RAWnnn file that was missed in processing, SAM can tell you it has not been processed yet with X (i.e., it has no children associated with version x of X) and you can then choose to process that file. This use case often occurs when a production job fails without reporting back or succeeds but the copy back or catalog action fails. \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ +The D0 experiment required that all official processing going into SAM be done with tagged releases and fixed parameter sets to increase reproducibility and the tags for that information were included in the metadata. Calibration databases were harder to timestamp so some variability was still possible if calibrations were updated.\ + \ +7. SAM supports several types of data description fields:\ + \ + +\f1\b Values +\f0\b0 are standard across all implementations like run_type, file_size \'85\ + \ + +\f1\b Parameters +\f0\b0 are defined by the experiment for example MC.Genieversion\ + \ +Values are common to almost all HEP experiments and are optimized for efficient queries. SAM also allows definition of \'93parameters\'94 (by administrators) as they are needed. This allows the schema to be modified easily as needs arrive.\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f3\i \cf0 +\f0\i0 \ +8. Metadata can also contain \'93spill\'94 or luminosity block information that allows a file to point to specific data taking periods with smaller granularity than a run or subrun. When files are merged, this spill information is also merged.\ + \ +9. SAM currently does not contain a means of easily determining which file a given event is in. If a daq system is writing multiple streams, an event from a given subrun could be in any stream. Adding an event database would be a useful feature.\ + \ +All of these features are intended to assure that your data are well described and can be found. As SAM stores full location information, this means any SAM-visible location. In addition, if parentage information is provided, you can determine and reproduce the full provenance of any file.\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f1\b \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\fs34 \cf0 Datasets and projects\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ + +\f1\b Datasets\ + +\f0\b0 \ +In addition to the files themselves, SAM allows you to define datasets.\ + \ +A SAM +\f1\b dataset +\f0\b0 is \ul not a fixed list of files\ulnone but a +\f1\b query +\f0\b0 against the SAM database. An example would be \'93data_tier reconstructed and run_number 2001 and version v10\'94 which would be all files from run 2001 that are reconstructed data produced by version v10. This dataset is dynamic. If one finds a missing file from run 2001 and reconstructs it with v10, the dataset will grow. There are also dataset +\f1\b snapshots +\f0\b0 that are derived from datasets and capture the exact files in the dataset when the snapshot was made.\ +Note: most other data catalogs assume a \'93dataset\'94 is a fixed list of files. This is a \'93snapshot\'94 in SAM. \ + +\f1\b \ +samweb +\f0\b0 \'96 samweb is the command line and python API that allows queries of the SAM metadata, creation of datasets and tools to track and deliver information to batch jobs.\ +\ +samweb can be acquired from ups via \ +\pard\pardeftab720\fi720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 setup samweb_client\ +\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 Or installed locally via \ +\pard\pardeftab720\li360\ri220\sl276\slmult1\sb220\sa220\partightenfactor0 + +\f6\fs22 \cf2 git clone {\field{\*\fldinst{HYPERLINK "http://cdcvs.fnal.gov/projects/sam-web-client"}}{\fldrslt \cf3 \ul \ulc3 http://cdcvs.fnal.gov/projects/sam-web-client}}\ +\pard\pardeftab720\ri220\sl276\slmult1\sb220\sa220\partightenfactor0 + +\f0\fs24 \cf2 You then need to do something like:\ +\pard\pardeftab720\fi720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs20 \cf2 export PATH=$HOME/sam-web-client/bin:$\{PATH\}\ +export PYTHONPATH=$HOME/sam-web-client/python:$\{PYTHONPATH\}\ +export SAM_EXPERIMENT=dune\ +\pard\pardeftab720\li360\ri220\sl276\slmult1\sb220\sa220\partightenfactor0 + +\fs24 \cf2 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ + +\f1\b Projects\ + +\f4\i \ + +\f0\i0\b0 \ +SAM also supports access tracking mechanisms called +\f1\b projects +\f0\b0 and +\f1\b consumers. +\f0\b0 These are generally implemented for you by grid processing scripts. Your job is to choose a dataset and then ask the processing system to launch a project for that dataset.\ + \ +A +\f1\b project +\f0\b0 is effectively a processing campaign across a dataset which is owned by the SAM system. At launch a +\f1\b snapshot +\f0\b0 is generated and then the files in the snapshot are delivered to a set of +\f1\b consumers +\f0\b0 . The project maintains an internal record of the status of the files and consumers. Each grid process can instantiate a consumer which is attached to the project. Those consumers then request \'93files\'94 from the project and, when done processing, tell the project of their status. \ + \ +The original SAM implementation actually delivered the files to local hard drives. Modern SAM delivers the location information and expects the consumer to find the optimal delivery method. This is a pull model, where the consuming process requests the next file rather than having the file assigned to it. This makes the system more robust on distributed systems. \ + \ +See running projects at \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects"}}{\fldrslt \cf3 \ul \ulc3 http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects}}\cf3 \ul \ulc3 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ulnone \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Accessing the database in read mode\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +Checking the database does not require special privileges but storing files and\ +running projects modifies the database and requires authentication to the right experimental group. kx509 authentication and membership in the experiment VO are needed. \ + \ +Administrative actions like adding new +\f1\b values +\f0\b0 are restricted to a small set of superusers for each experiment. \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Suggestions for configuring SAM (for admins)\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +First of all, it really is nice to have filenames and dataset names that tell you what\'92s in the box, although not required. The D0 and MINERvA conventions have been to use \'93_\'94 underscores between useful key strings. As a result, D0 and MINERvA tried not to use \'93_\'94 in metadata entries to allow cleaner parsing. \'93-\'93 is used if needed in the metadata.\ + \ +D0 also appended processing information to filenames as they moved through the system to assure that files run through different sequences had unique identifiers.\ + \ +Example: A Monte Carlo simulation file generated with version v3 and then reconstructed with v5 might look like\ + \ +SIM_MC_020000_0000_simv3.root would be a parent of RECO_MC_020000_0000_simv3_recov5.root\ +Data files are all children of the raw data while simulation files sometimes have more\ +complicated ancestry, with both unique generated events and overlay events from data as parents.\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Setting up SAM metadata (For admins)\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +This needs to be done once, and very carefully, early in the experiment. It can grow but thinking hard at the beginning saves a lot of pain later.\ + \ +You need to define +\f1\b data_tiers +\f0\b0 . These represent the different types of data that you produced through your processing chain. Examples would be raw, pedsup, calibrated, reconstructed, thumbnail, mc-generated, mc-geant, mc-overlaid,\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f1\b \cf0 run_type +\f0\b0 can be used to support multiple DAQ instances. \ + \ + +\f1\b data_stream +\f0\b0 is often used for trigger subsamples that you may wish to split data into (for example pedestal vs data runs).\ + \ +Generally, you want to store data from a given data_tier with other data from that tier to facilitate fast sequential access. \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Applications\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +It is useful, but not required to also define +\f1\b applications +\f0\b0 which are triads of \'93appfamily\'94, \'93appname\'94 and \'93version\'94. Those are used to figure out what changed X to Y. There are also places to store the machine the application ran on and the start and end time for the job.\ +\ +The query:\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 samweb list-files "data_tier raw and not isparentof: (data_tier reconstructed and appname reco and version 7)"\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +Should, in principle, list raw data files not yet processed by version 7 of reco to produce files of tier reconstructed. You would use this to find lost files in your reconstruction after a power outage.\ + \ +It is good practice to also store the name of the head application configuration file for processing but this does not have a standard \'93value\'94\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 Example metadata from DUNE\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs22 \cf0 \ +Here are some examples of querying sam to get file information\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs24 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs20 \cf0 \ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 $ samweb get-metadata np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root \'96json\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +\{\ + "file_name": "np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root", \ + "file_id": 7352771, \ + "create_date": "2018-10-29T14:59:42+00:00", \ + "user": "dunepro", \ + "update_date": "2018-11-28T17:07:30+00:00", \ + "update_user": "schellma",\ + "file_size": 14264091111, \ + "checksum": [\ + "enstore:1390300706", \ + "adler32:e8bf4e23"\ + ], \ + "content_status": "good", \ + "file_type": "detector", \ + "file_format": "artroot", \ + "data_tier": "full-reconstructed", \ + "application": \{\ + "family": "art", \ + "name": "reco", \ + "version": "v07_08_00_03"\ + \}, \ + "event_count": 108, \ + "first_event": 21391, \ + "last_event": 22802, \ + "start_time": "2018-10-28T17:34:58+00:00", \ + "end_time": "2018-10-29T14:55:42+00:00", \ + "data_stream": "physics", \ + "beam.momentum": 7.0, \ + "data_quality.online_good_run_list": 1, \ + "detector.hv_value": 180, \ + "DUNE_data.acCouple": 0, \ + "DUNE_data.calibpulsemode": 0, \ + "DUNE_data.DAQConfigName": "np04_WibsReal_Ssps_BeamTrig_00021", \ + "DUNE_data.detector_config": "cob2_rce01:cob2_rce02:cob2.. 4 more lines of text", \ + "DUNE_data.febaselineHigh": 2, \ + "DUNE_data.fegain": 2, \ + "DUNE_data.feleak10x": 0, \ + "DUNE_data.feleakHigh": 1, \ + "DUNE_data.feshapingtime": 2, \ + "DUNE_data.inconsistent_hw_config": 0, \ + "DUNE_data.is_fake_data": 0, \ + "runs": [\ + [\ + 5141, \ + 1, \ + "protodune-sp"\ + ]\ + ], \ + "parents": [\ + \{\ + "file_name": "np04_raw_run005141_0015_dl10.root", \ + "file_id": 6607417\ + \}\ + ]\ +\}\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 $ samweb get-file-access-url np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\fs24 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs20 \cf0 gsiftp://eospublicftp.cern.ch/eos/experiment/neutplatform/protodune/rawdata/np04/output/detector/full-reconstructed/07/35/27/71/np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root\ +gsiftp://fndca1.fnal.gov:2811/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/output/detector/full-reconstructed/07/35/27/71/np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs24 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs20 \cf0 $ samweb file-lineage children np04_raw_run005141_0015_dl10.root \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 $ samweb file-lineage parents\\\ +np04_raw_run005141_0015_dl10_reco_12736632_0_20181028T182951.root\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +np04_raw_run005141_0015_dl10.root +\fs24 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf0 \ +Merging and splitting (for experts)\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +Parentage works pretty well if one is merging files but splitting them can become problematic as it makes the parentage structure pretty complex.\ +\pard\pardeftab720\ri0\sl276\slmult1\sb280\partightenfactor0 +\cf0 Sam will let you merge files with different attributes if you don\'92t check carefully. Generally, it is a good idea not to merge files from different data tiers and certainly not from different data_types. Merging across major processing version\cf2 s should also be avoided +\fs20 .\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs26 \cf4 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb360\sa80\partightenfactor0 + +\f1\b\fs34 \cf2 Example: Execute samweb Commands\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs24 \cf0 \ +There is documentation at \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "https://cdcvs.fnal.gov/redmine/projects/sam/wiki/User_Guide_for_SAM#Listing-Files-using-a-Variable-to-Build-the-Query"}}{\fldrslt \cf3 \ul https://cdcvs.fnal.gov/redmine/projects/sam/wiki/User_Guide_for_SAM}}\cf3 \ul \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 \ulnone \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "https://cdcvs.fnal.gov/redmine/projects/sam-main/wiki/Updated_dimensions_syntax"}}{\fldrslt \cf3 \ul https://cdcvs.fnal.gov/redmine/projects/sam-main/wiki/Updated_dimensions_syntax}}\cf3 \ul \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +\cf0 \ulnone This exercise will start you accessing data files that have been defined to the DUNE Data Catalog. Execute the following commands after logging in to the DUNE interactive node, creating the directories above - \ +Once per session\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 setup sam_web_client #(or set up your standalone version)\ +export SAM_EXPERIMENT=dune\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 Then if curious about a file: \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb locate-file np04_raw_run005141_0001_dl7.root\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 this will give you output that looks like \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs22 \cf0 rucio:protodune-sp\ +enstore:/pnfs/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05(596@vr0072m8)\ +castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05\ +cern-eos:/eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\fs24 \cf0 which are the locations of the file on disk and tape. We can use this to copy the file from tape to our local disk. \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f1\b \cf0 Better yet, you can use xrootd to access the file without copying it if it is staged to disk. +\f0\b0 \ +Find the xrootd uri via \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs22 \cf0 +\f6 samweb get-file-access-url np04_raw_run005141_0001_dl7.root --schema=root\ + +\f0 \ +root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05 /np04_raw_run005141_0001_dl7.root\ +root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root\ +root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\fs24 \cf0 You can localize your file with the --location argument (enstore, castor, cern-eos) \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb get-file-access-url np04_raw_run005141_0001_dl7.root --schema=root --location=enstore\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/tape_backed/dunepro/protodune/np04/beam/detector/None/raw/06/60/59/05 /np04_raw_run005141_0001_dl7.root\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 samweb get-file-access-url np04_raw_run005141_0001_dl7.root --schema=root --location=cern-eos\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/59/05/np04_raw_run005141_0001_dl7.root +\fs24 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +\cf0 To get SAM metadata for a file for which you know the name: \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb get-metadata np04_raw_run005141_0001_dl7.root \ +# add the --json option to get output in json format\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 To list raw data files for a given run: \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb list-files "run_number 5141 and run_type protodune-sp and data_tier raw"\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 What about a reconstructed version? +\fs22 +\f6 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 samweb list-files "run_number 5141 and run_type protodune-sp and data_tier full-reconstructed and version (v07_08_00_03,v07_08_00_04)"\ + \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 Gives a list of files from the first production like \ + \ +\'85.\ +np04_raw_run005141_0001_dl7_reco_12736115_0_20181028T165152.root\ +\'85.\ +\ +We also group reconstruction versions into Campaigns like PDSPProf4\ +\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6 \cf0 samweb list-files "run_number 5141 and run_type protodune-sp and data_tier full-reconstructed and DUNE.campaign PDSPProd4"\ +\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 Gives more recent files like:\ +\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs20 \cf0 np04_raw_run005141_0009_dl1_reco1_18126423_0_20210318T102429Z.root +\fs22 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\fs24 \cf0 samweb allows you to select on a lot of parameters \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "https://wiki.dunescience.org/wiki/Useful_ProtoDUNE_samweb_parameters"}}{\fldrslt \cf3 \ul Useful ProtoDUNE samweb parameters}}\cf3 \ul \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "https://dune-data.fnal.gov/"}}{\fldrslt \cf3 https://dune-data.fnal.gov}}\cf0 \ulnone and {\field{\*\fldinst{HYPERLINK "https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets"}}{\fldrslt +\fs22 \cf3 https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets}}\ + lists some official dataset definitions \ +\pard\pardeftab720\ri0\sl276\slmult1\sb280\sa80\partightenfactor0 + +\f1\b\fs26 \cf0 You can make your own samweb dataset definitions\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0\b0\fs22 \cf0 First, make certain a definition does not already exist that satisfies your needs by checking the official pages above. \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\fs24 \cf0 First check to see what you will get \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb list-files "data_tier full-reconstructed and DUNE.campaign PDSPProd4 and data_stream cosmics and run_type protodune-sp and detector.hv_value 180" \'96summary\ + \ +samweb create-definition $USER-PDSPProd4_good_cosmics "data_tier full-reconstructed and DUNE.campaign PDSPProd4 and data_stream cosmics and run_type protodune-sp and detector.hv_value 180"\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 Note that the username appears in the definition name - to prevent users from getting confused with official samples, your user name is required in the definition name. \ +\pard\pardeftab720\ri0\sl276\slmult1\sb280\sa80\partightenfactor0 + +\f1\b\fs26 \cf0 prestaging\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\b0\fs24 \cf0 At CERN files are either on eos or castor. At FNAL they can be on tape_backed dcache which may mean they are on tape and may need to be prestaged to disk before access.\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 setup fife_utils # a new version we requested\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +# check to see if a file is on tape or disk\ +\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs20 \cf0 sam_validate_dataset --locality --file np04_raw_run005141_0015_dl10.root --location=/pnfs/ --stage_status\ + Staging status for: file np04_raw_run005141_0015_dl10.root\ + Total Files: 1\ + Tapes spanned: 1\ + Percent files on disk: 0%\ +Percent bytes online DCache: 0%\ +locality counts:\ +ONLINE: 0\ +NEARLINE: 1\ +NEARLINE_size: 8276312581\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs22 \cf0 \ +Oops - this one is not on disk \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 returns ONLINE_AND_NEARLINE: 1 if available on disk \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs20 \cf0 sam_validate_dataset --locality --name=schellma-1GeVMC-test --stage_status --location=/pnfs/\ + Staging status for: defname:schellma-1GeVMC-test\ + Total Files: 140\ + Tapes spanned: 10\ + Percent files on disk: 100%\ +Percent bytes online DCache: 100%\ +locality counts:\ +ONLINE: 0\ +ONLINE_AND_NEARLINE: 140\ +ONLINE_AND_NEARLINE_size: 270720752891\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs22 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 No ONLINE_NEARLINE means you need to prestage that file. Unfortunately, prestaging requires a definition \ +Let's find some for run 5141. Your physics group should already have some defined.\ +The official Protodune dataset definitions are at: \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets"}}{\fldrslt +\fs22 \cf3 https://wiki.dunescience.org/wiki/ProtoDUNE-SP_datasets}}\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\fs20 \cf0 samweb describe-definition +\f0 \cf5 PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00\ +\ +Is simulation for 10% of the total sample\ + +\f6 \cf0 \ +Gives this description:\ +samweb describe-definition PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00\ +Definition Name: PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00\ + Definition Id: 635109\ + Creation Date: 2021-08-02T16:57:20+00:00\ + Username: dunepro\ + Group: dune\ + Dimensions: run_type 'protodune-sp' and file_type mc and data_tier 'full-reconstructed' and dune.campaign PDSPProd4a and dune_mc.beam_energy 1 and\ +dune_mc.space_charge yes and dune_mc.generators beam_cosmics and version v09_17_01 and run_number in 18800650,.....\ +\ +samweb list-files "defname:PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00\ +> " --summary\ +File count: 5025\ +Total size: 9683195368818\ +Event count: 50250\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs22 \cf0 samweb prestage-dataset --def= +\fs20 PDSPProd4a_MC_1GeV_reco1_sce_datadriven_v1_00 +\fs22 --parallel=10\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\f0\fs24 \cf0 would prestage all of the reconstructed data for run 5141 \ +you can check on the status by going to \ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +{\field{\*\fldinst{HYPERLINK "http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects"}}{\fldrslt \cf3 \ul http://samweb.fnal.gov:8480/station_monitor/dune/stations/dune/projects}} and scrolling down to see your prestage link. \ +At CERN\ +You can find local copies of files at CERN for interactive use. \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f6\fs20 \cf0 samweb list-file-locations --defname=runset-5141-raw-180kV-7GeV-v0 --schema=root --filter_path=castor \ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 + +\f0 \cf0 \ +gives you:\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 +\cf0 root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/16/np04_raw_run005141_0015_dl3.root castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/16 np04_raw_run005141_0015_dl3.root 8289321123\ +\pard\pardeftab720\ri0\sl276\slmult1\partightenfactor0 +\cf0 root://castorpublic.cern.ch//castor/cern.ch/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/17/np04_raw_run005141_0015_dl10.root castor:/neutplatform/protodune/rawdata/np04/detector/None/raw/06/60/74/17 np04_raw_run005141_0015_dl10.root 8276312581\ +\pard\pardeftab720\ri0\sl276\slmult1\sb240\sa240\partightenfactor0 + +\fs24 \cf0 \ +\ +\ +\ +} \ No newline at end of file diff --git a/docs/source/OfficialDataCatalog.rst b/docs/source/OfficialDataCatalog.rst new file mode 100644 index 0000000..0e3ef92 --- /dev/null +++ b/docs/source/OfficialDataCatalog.rst @@ -0,0 +1,13 @@ +The official DUNE data catalog supported by the production team +=============================================================== + +`Data catalog documentation `_ is maintained in the `computing basics tutorial `_. + +This is often all you need. The syntax for queries using an official dataset is: + +"files from : limit " + + + +.. .. include:: OfficialDatasets_include.md +.. :parser: myst_parser.sphinx_ \ No newline at end of file diff --git a/docs/source/OlderSam.rst b/docs/source/OlderSam.rst new file mode 100644 index 0000000..bfe7279 --- /dev/null +++ b/docs/source/OlderSam.rst @@ -0,0 +1,15 @@ +Older sam documentation +----------------------- + +Official samweb documentiton from redmine +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + `samweb users guide `_ + + +DUNE usage NotesOnSam_v4 +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. raw:: html + + :file: NotesOnSam_v4.html diff --git a/docs/source/__pycache__/lumache.cpython-39.pyc b/docs/source/__pycache__/lumache.cpython-39.pyc new file mode 100644 index 0000000..787f790 Binary files /dev/null and b/docs/source/__pycache__/lumache.cpython-39.pyc differ diff --git a/docs/source/commands.csv b/docs/source/commands.csv new file mode 100644 index 0000000..13f3d28 --- /dev/null +++ b/docs/source/commands.csv @@ -0,0 +1,5 @@ +metacat,samweb,comment +metacat query 'files from dnamespace:dataset where x=y',samweb list-files ' x y ', +??,samweb list-files --summary 'x y', +metacat file show -m :,samweb get-metadata , +metacat query 'files from dune:all with name=',,finds the namespace \ No newline at end of file diff --git a/docs/source/conf.py b/docs/source/conf.py new file mode 100644 index 0000000..41c1181 --- /dev/null +++ b/docs/source/conf.py @@ -0,0 +1,88 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# some additions from ivmfnal's metacat +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os +import sys +sys.path.insert(0, os.path.abspath('.')) + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. +import pathlib +#top = pathlib.Path(__file__).parents[2].resolve().as_posix() +#print("top",top) +#py = os.path.join(top,"python") +#sys.path.insert(0,top) +#sys.path.insert(0,py) +#sys.path.insert(0,os.path.join(top,"tests")) + + + +print ("PATH",sys.path) + +# -- Project information ----------------------------------------------------- + +project = 'DataCatalogDocs' +copyright = '2023, Fermi National Accelerator Laboratory' +author = 'Heidi Schellman and Jacob Calcutt' + +# The full version, including alpha/beta/rc tags +release = '0.2' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.duration', + 'sphinx.ext.doctest', + 'sphinx.ext.autodoc', + 'sphinx.ext.autosummary', + 'sphinx.ext.napoleon', + 'sphinx.ext.autosectionlabel', + 'sphinx.ext.viewcode', + 'sphinx.ext.inheritance_diagram', + #'sphinx_sitemap', had to back it out as not available + #'sphinxcontrib.programoutput', # not available to github +] + +autoclass_content = "both" # from ivmfnal + +html_baseurl = 'https://dune.github.io/DataCatalogDocs/' + +autosectionlabel_prefix_document = True + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + +autodoc_mock_imports = ["metacat","samweb_client","data_dispatcher"] +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'furo' +html_theme = 'nature' + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +#html_static_path = ['_static'] + +html_static_path = [] diff --git a/docs/source/conversion.csv b/docs/source/conversion.csv new file mode 100644 index 0000000..37db0db --- /dev/null +++ b/docs/source/conversion.csv @@ -0,0 +1,48 @@ +Metacat,type,Samweb +,, +Basics,, +,, +fid,int,file_id +namespace,string, +name,string,file_name +creator,string,user +created_timestamp,timestamp,create_date +size,int,file_size +checksums,dictionary,check_sum (dict) +retired,bool, +retired_by,string, +retired_timestamp,timestamp, +updated_by,string,update_user +updated_timestamp,timestamp,update_date +update_comment,blob, +,, +Core attributes,, +,, +core.application.version,string,app_family +core.application.family,string,app_version +core.application.name,string,app_name +core.event_count,int,event_count +core.first_event_number,int,first_event +core.last_event_number,int,last_event +core.start_time,timestamp,start_time +core.end_time,timestamp,end_time +core.file_content_status,string,content_status +core.data_stream,string,data_stream +core.data_tier,string,data_tier +core.events,array, +core.file_type,text,file_type +core.file_format,text,file_format +core.run_type,text,run_type +core.runs,array,run_number (integer part) +core.runs_subruns,array,run_number (part past the decimal point) +core.raw_timestamp,timestamp, +,, +dune_mc.*, all types, DUNE_MC.* +Retention/access keys,, +,, +retention.status,string, +retention.class,string, +,, +Additional attributes,, +,, +.. … ,"string,int, bool, ….",. diff --git a/docs/source/fardet-hd.json b/docs/source/fardet-hd.json new file mode 100644 index 0000000..bcb963c --- /dev/null +++ b/docs/source/fardet-hd.json @@ -0,0 +1,78 @@ +{ + "checksums": { + "adler32": "2ecd0e98" + }, + "children": [ + { + "fid": "n8OUCNB8Rz6Gb9uS", + "name": "nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco__20240405T212128Z_reco2_ana.root", + "namespace": "fardet-hd" + } + ], + "created_timestamp": 1712356547.182858, + "creator": "dunepro", + "fid": "4kDVdRLgTuSssgkD", + "metadata": { + "art.file_format_era": "ART_2011a", + "art.file_format_version": 15, + "art.first_event": 76801, + "art.last_event": 76900, + "art.process_name": "Reco2", + "core.application": "art.reco2", + "core.application.family": "art", + "core.application.name": "reco2", + "core.application.version": "v09_81_00d02", + "core.data_stream": "out1", + "core.data_tier": "full-reconstructed", + "core.end_time": "2024-04-05T22:31:22", + "core.event_count": 100, + "core.file_format": "artroot", + "core.file_type": "mc", + "core.first_event_number": 76801, + "core.group": "dune", + "core.last_event_number": 76900, + "core.run_type": "fardet-hd", + "core.runs": [ + 1427 + ], + "core.runs_subruns": [ + 142700001 + ], + "core.start_time": "2024-04-05T21:22:54", + "dune.campaign": "fd_mc_2023a_reco2", + "dune.config_file": "standard_reco2_dune10kt_nu_1x2x6.fcl", + "dune.output_status": "confirmed", + "dune.requestid": "ritm1780305", + "dune.workflow": { + "cpuinfo": "AMD EPYC 7763 64-Core Processor", + "hostname": "heplnc164.pp.rl.ac.uk", + "job_id": "189271.98@justin-prod-sched01.dune.hep.ac.uk", + "jobscript_cpu_seconds": 3743, + "jobscript_finish": 1712356520, + "jobscript_real_seconds": 4439, + "jobscript_start": 1712352080, + "os_release": "Scientific Linux release 7.9 (Nitrogen)", + "site_name": "UK_RAL-PPD", + "stage_id": 1, + "user": "benjamin@fnal.gov", + "workflow_id": 1769 + }, + "dune_mc.detector_type": "fardet-hd", + "dune_mc.gen_fcl_filename": "prodgenie_nu_dune10kt_1x2x6.fcl" + }, + "name": "nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco__20240405T212128Z_reco2.root", + "namespace": "fardet-hd", + "parents": [ + { + "fid": "80286097", + "name": "nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco.root", + "namespace": "fardet-hd" + } + ], + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 1763366388, + "updated_by": "dunepro", + "updated_timestamp": 1712510605.16971 +} diff --git a/docs/source/glossary.rst b/docs/source/glossary.rst new file mode 100644 index 0000000..f4b6123 --- /dev/null +++ b/docs/source/glossary.rst @@ -0,0 +1,87 @@ +.. _glossary: + +Glossary +-------- + +Incomplete list of keys used + +**Keys must be lower case** values can be either. + +Minimal terms for raw and processed data +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +`[]` denotes fields automatically filled in by the system + +:name: File name +:namespace: Metacat namespace for file +:checksums: dictionary of checksums - adler32 is the default +:`[created_timestamp]`: Unix timestamp for when file was cataloged +:creator: account that created the file +:size: size in bytes +:`[fid]`: hash-name for the file - equivalent to namespace:name +:`[retired]`: has this file been retired? +:`[retired_by]`: who did it? +:`[retired_timestamp]`: when was it retired +:`[updated_by]`: who has updated this catalog entry? +:`[updated_timestamp]`: when did they do it? +:parents: [The files that this file was produced from, you need to declare parents when declaring child files] +:`[children]`: [list of files that are derived from this file - autogenerated when you declare the child as having parents] + +:metadata: description of file contents with this content + +generally required +"""""""""""""""""" + +:core.data_stream: type of data taking (commissioning, calibration, test, physics, cosmics) +:core.data_tier: type of data, generally indicates steps in a processing sequence (raw, g4, ) +:core.event_count: number of events in the file +:core.file_content_status: status of the file - default is "good" +:core.file_format: format of the data (hdf5, root) +:core.file_type: flag to tell mc from data, (detector or mc) +:core.run_type: which detector took the data "protodune-sp, hd-fardet ..." +:core.runs: [list of runs] +:core.runs_subruns: [list of subruns in run*100000+subrun format] +:retention.status: should be "active" flag to tell if the file is being used and should be retained +:retention.class: flag used to determine retention status (physics, test, ...) + +optional but useful +""""""""""""""""""" +:*core.start_time*: unix UTC time at which the process that created the file started +:*core.end_time*: unix UTC time at which the process that created the file ended - +:*core.events*: [list of events in the file], not useful for multiple runs +:*core.first_event_number*: first event number +:*core.last_event_number*: last event number + + +Additional terms used for reconstruction and simulation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + +:dune.campaign: A big scale activity used for production - examples are PDSPProd4a and fd_mc_2023a_reco2 +:dune.requestid: The formal request id for the campaign in the system +:dune.config_file: The top level configuration used to produce this file +:dune.workflow: a description of the workflow that produced this file - produced by the JustIn system +:dune.output_status: the value should be "confirmed" - this tells you that the output exists +:core.application.family: broad description of the application (art/edepsim) +:core.application.name: the specific application, reco1/reco2/detsim... +:core.application.version: the DUNESW version +:origin.applications.config_files: dictionary describing the fcl files for a processing chain. + + +Minimal Monte Carlo terms +^^^^^^^^^^^^^^^^^^^^^^^^^^ +:core.group: Should be `dune` or a physics group +:dune_mc.gen_fcl_filename: tells you the generator fcl file so you know what kind of mc it is. +:dune_mc.geometry_version: the geometry version used +:dune_mc.generators: + +MC terms specific to particular detectors +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:dune_mc.liquid_flow: +:dune_mc.electron_lifetime: PD/FD - electron lifetime +:dune_mc.space_charge: PD/FD - space charge +:dune_mc.with_cosmics: PD/FD - cosmics as well as beam +:Beam.momentum: PD/FD - beam momentum in GeV/c +:Beam.polarity: PD/FD - polarity of beam + + +there may be others in future. diff --git a/docs/source/h4.json b/docs/source/h4.json new file mode 100644 index 0000000..55e1f35 --- /dev/null +++ b/docs/source/h4.json @@ -0,0 +1,75 @@ + { + "checksums": { + "adler32": "c27f09b5" + }, + "created_timestamp": 1691937037.698784, + "creator": "dunepro", + "fid": "0eUkJLB3SQOmlA9N", + "metadata": { + "art.file_format_era": "ART_2011a", + "art.file_format_version": 15, + "art.first_event": 9301, + "art.last_event": 9400, + "art.process_name": "Reco1", + "art.run_type": "fardet-vd", + "core.application": "art.reco", + "core.application.family": "art", + "core.application.name": "reco", + "core.application.version": "v09_75_03d00", + "core.data_stream": "out1", + "core.data_tier": "hit-reconstructed", + "core.end_time": 1691937015.0, + "core.event_count": 100, + "code.file_content_status":"good", + "core.file_format": "artroot", + "core.file_type": "mc", + "core.first_event_number": 9301, + "core.last_event_number": 9400, + "core.run_type": "fardet-vd", + "core.runs": [ + 27279 + ], + "core.runs_subruns": [ + 2727900001 + ], + "core.start_time": 1691936982.0, + "dune.campaign": "fd_mc_2023a", + "dune.requestid": "ritm1780305", + "dune.workflow": { + "cpuinfo": "AMD EPYC 7551P 32-Core Processor", + "hostname": "hd80.dice.priv", + "job_id": "27279.93@justin-prod-sched01.dune.hep.ac.uk", + "jobscript_cpu_seconds": 37173, + "jobscript_finish": 1691937022, + "jobscript_real_seconds": 37212, + "jobscript_start": 1691899809, + "os_release": "Scientific Linux release 7.9 (Nitrogen)", + "request_id": 1019, + "site_name": "UK_Bristol", + "stage_id": 1, + "user": "amcnab@fnal.gov" + }, + "dune_mc.beam_flux_id": "1", + "dune_mc.beam_polarity": "rhc", + "dune_mc.detector_type": "fardet-vd", + "dune_mc.electron_lifetime": "10.4", + "dune_mc.gen_fcl_filename": "prodgenie_anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg.fcl", + "dune_mc.generators": "genie", + "dune_mc.generators_version": "3.04_00c", + "dune_mc.geometry_version": "dunevd10kt_3view_30deg_v3_refactored_1x8x6ref.gdml", + "dune_mc.liquid_flow": "no", + "dune_mc.mixerconfig": "numu2nutau_nue2numu", + "dune_mc.space_charge": "no", + "dune_mc.with_cosmics": 0 + “retention.class”: “simulation”, + “retention.status”: “active” + }, + "name": "anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg_27279_93_20230813T041011Z_gen_g4_detsim_hitreco.root", + "namespace": "fardet-vd", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 1495285749, + "updated_by": null, + "updated_timestamp": 1691937037.698784 + } diff --git a/docs/source/index.rst b/docs/source/index.rst new file mode 100644 index 0000000..ba7bc55 --- /dev/null +++ b/docs/source/index.rst @@ -0,0 +1,48 @@ +.. DataCatalogDocs documentation master file, created by + sphinx-quickstart on Sat Dec 24 19:16:47 2022. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +DUNE Data Catalog Documentation +=============================== + +.. toctree:: + :maxdepth: 2 + :caption: Contents: + + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` + +Contents +-------- + +.. toctree:: + + Intro + OfficialDataCatalog + setup + MetacatQueryExamples + glossary + template_input + reco + mcreco + runInfo + related + Notes + Ack + +Old Sam Documentation +--------------------- + +.. toctree:: + + sam2metacat + metadatameaning + OlderSam + diff --git a/docs/source/mcreco.rst b/docs/source/mcreco.rst new file mode 100644 index 0000000..216b2a4 --- /dev/null +++ b/docs/source/mcreco.rst @@ -0,0 +1,52 @@ +Reconstructed MC File Example +----------------------------- + +This is from recent simulation of the FD - here are some additional fields for derived files + + +.. code-block:: javascript + + "parents": [ + { + "fid": "80286097", + "name": "nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco.root", + "namespace": "fardet-hd" + } + ] + + + "children": [ + { + "fid": "n8OUCNB8Rz6Gb9uS", + "name": "nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco__20240405T212128Z_reco2_ana.root", + "namespace": "fardet-hd" + } + ] + + + "dune.workflow": { + "cpuinfo": "AMD EPYC 7763 64-Core Processor", + "hostname": "heplnc164.pp.rl.ac.uk", + "job_id": "189271.98@justin-prod-sched01.dune.hep.ac.uk", + "jobscript_cpu_seconds": 3743, + "jobscript_finish": 1712356520, + "jobscript_real_seconds": 4439, + "jobscript_start": 1712352080, + "os_release": "Scientific Linux release 7.9 (Nitrogen)", + "site_name": "UK_RAL-PPD", + "stage_id": 1, + "user": "benjamin@fnal.gov", + "workflow_id": 1769 + } + +metacat +======= + +`metacat file show -l -m -j fardet-hd:nu_dune10kt_1x2x6_1427_768_20230827T213411Z_gen_g4_detsim_hitreco__20240405T212128Z_reco2.root` + +.. include:: fardet-hd.json + :literal: + + + + diff --git a/docs/source/mctemplate.rst b/docs/source/mctemplate.rst new file mode 100644 index 0000000..fb547cc --- /dev/null +++ b/docs/source/mctemplate.rst @@ -0,0 +1,141 @@ +Template for minimal metadata for a Monte Carlo file +---------------------------------------------------- + +Here is an example of minimal metadata for a raw data file. + +You can add additional information to the metadata object. + +Metacat version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "checksums": { + "adler32": "c27f09b5" + }, + "created_timestamp": 1691937037.698784, + "creator": "dunepro", + "fid": "0eUkJLB3SQOmlA9N", + "metadata": { + "art.file_format_era": "ART_2011a", + "art.file_format_version": 15, + "art.first_event": 9301, + "art.last_event": 9400, + "art.process_name": "Reco1", + "art.run_type": "fardet-vd", + "core.application": "art.reco", + "core.application.family": "art", + "core.application.name": "reco", + "core.application.version": "v09_75_03d00", + "core.data_stream": "out1", + "core.data_tier": "hit-reconstructed", + "core.end_time": 1691937015.0, + "core.event_count": 100, + "code.file_content_status":"good", + "core.file_format": "artroot", + "core.file_type": "mc", + "core.first_event_number": 9301, + "core.last_event_number": 9400, + "core.run_type": "fardet-vd", + "core.runs": [ + 27279 + ], + "core.runs_subruns": [ + 2727900001 + ], + "core.start_time": 1691936982.0, + "dune.campaign": "fd_mc_2023a", + "dune.requestid": "ritm1780305", + "dune.workflow": { + "cpuinfo": "AMD EPYC 7551P 32-Core Processor", + "hostname": "hd80.dice.priv", + "job_id": "27279.93@justin-prod-sched01.dune.hep.ac.uk", + "jobscript_cpu_seconds": 37173, + "jobscript_finish": 1691937022, + "jobscript_real_seconds": 37212, + "jobscript_start": 1691899809, + "os_release": "Scientific Linux release 7.9 (Nitrogen)", + "request_id": 1019, + "site_name": "UK_Bristol", + "stage_id": 1, + "user": "amcnab@fnal.gov" + }, + "dune_mc.beam_flux_id": "1", + "dune_mc.beam_polarity": "rhc", + "dune_mc.detector_type": "fardet-vd", + "dune_mc.electron_lifetime": "10.4", + "dune_mc.gen_fcl_filename": "prodgenie_anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg.fcl", + "dune_mc.generators": "genie", + "dune_mc.generators_version": "3.04_00c", + "dune_mc.geometry_version": "dunevd10kt_3view_30deg_v3_refactored_1x8x6ref.gdml", + "dune_mc.liquid_flow": "no", + "dune_mc.mixerconfig": "numu2nutau_nue2numu", + "dune_mc.space_charge": "no", + "dune_mc.with_cosmics": 0, + "retention.class": "simulation", + "retention.status": "active" + }, + "name": "anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg_27279_93_20230813T041011Z_gen_g4_detsim_hitreco.root", + "namespace": "fardet-vd", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 1495285749, + "updated_by": null, + "updated_timestamp": 1691937037.698784 + } + + +samweb version +^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "file_name": "anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg_27279_93_20230813T041011Z_gen_g4_detsim_hitreco.root", + "file_id": 80035348, + "create_date": "2023-08-14T12:26:26+00:00", + "user": "dunepro", + "file_size": 1495285749, + "checksum": [ + "adler32:c27f09b5" + ], + "content_status": "good", + "file_type": "mc", + "file_format": "artroot", + "group": "dune", + "data_tier": "hit-reconstructed", + "application": { + "family": "art", + "name": "reco", + "version": "v09_75_03d00" + }, + "event_count": 100, + "first_event": 9301, + "last_event": 9400, + "start_time": "2023-08-13T14:29:42+00:00", + "end_time": "2023-08-13T14:30:15+00:00", + "data_stream": "out1", + "DUNE.campaign": "fd_mc_2023a", + "DUNE.requestid": "ritm1780305", + "DUNE_MC.beam_flux_ID": 1, + "DUNE_MC.beam_polarity": "rhc", + "DUNE_MC.detector_type": "fardet-vd", + "DUNE_MC.electron_lifetime": "10.4", + "DUNE_MC.gen_fcl_filename": "prodgenie_anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg.fcl", + "DUNE_MC.generators": "genie", + "DUNE_MC.generators_version": "3.04_00c", + "DUNE_MC.geometry_version": "dunevd10kt_3view_30deg_v3_refactored_1x8x6ref.gdml", + "DUNE_MC.liquid_flow": "no", + "DUNE_MC.mixerconfig": "numu2nutau_nue2numu", + "DUNE_MC.space_charge": "no", + "DUNE_MC.with_cosmics": 0, + "runs": [ + [ + 27279, + 1, + "fardet-vd" + ] + ] + } diff --git a/docs/source/meta_reco.json b/docs/source/meta_reco.json new file mode 100644 index 0000000..deb52d3 --- /dev/null +++ b/docs/source/meta_reco.json @@ -0,0 +1,63 @@ +{ + "checksums": { + "adler32": "03d6cb63", + "enstore": "1952172898", + "md5": "9c9d59cc33f3b0e2167c2acb3228241e" + }, + "created_timestamp": 1616170094.9023, + "creator": "dunepro", + "fid": "52500057", + "metadata": { + "art.file_format_era": "ART_2011a", + "art.process_name": "Reco1", + "art.run_type": "protodune-sp", + "core.application": "art.reco", + "core.application.family": "art", + "core.application.name": "reco", + "core.application.version": "v09_09_01", + "core.data_stream": "physics", + "core.data_tier": "full-reconstructed", + "core.end_time": 1616168817, + "core.end_time_utc_text": "2021-03-19 15:46:57", + "core.event_count": 108, + "core.file_content_status": "good", + "core.file_format": "artroot", + "core.file_type": "detector", + "core.first_event_number": 1584, + "core.last_event_number": 3028, + "core.process_id": 15256421, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": 1616065002, + "core.start_time_utc_text": "2021-03-18 10:56:42", + "data_quality.online_good_run_list": 1, + "detector.hv_value": 180, + "dune.campaign": "PDSPProd4", + "dune_data.accouple": 0, + "dune_data.calibpulsemode": 0, + "dune_data.daqconfigname": "np04_WibsReal_Ssps_BeamTrig_00021", + "dune_data.febaselinehigh": 2, + "dune_data.fegain": 2, + "dune_data.feleak10x": 0, + "dune_data.feleakhigh": 1, + "dune_data.feshapingtime": 2, + "dune_data.inconsistent_hw_config": 0, + "dune_data.is_fake_data": 0, + "origin.worker_id": "15256421", + "retention.class": "physics", + "retention.status": "active" + }, + "name": "np04_raw_run005141_0002_dl2_reco1_18126921_0_20210318T103516Z.root", + "namespace": "pdsp_det_reco", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 3690769125, + "updated_by": "dunepro", + "updated_timestamp": 1633446346.80565 +} diff --git a/docs/source/metadatameaning.rst b/docs/source/metadatameaning.rst new file mode 100644 index 0000000..2afe2c3 --- /dev/null +++ b/docs/source/metadatameaning.rst @@ -0,0 +1,34 @@ +Metadata categories and samweb->metacat conversion +-------------------------------------------------- + +Samweb and metacat turn out to need quite a lot of information to describe a file. + +There is + +* basic information about the file as a file - size, creation, creator, checksum +* core information about the file contents such as run_number, data tier, and trigger stream that are needed to define data samples. Most of these are now in the core category in metacat. If it is a core item, you probably should be filling it. +* data that is useful to know - provenance (what was it created from, how, who did it) + +* additional information that may be specific to a given physics group or extra details on reconstruction/simulation parameters. + +:doc:`template` shows a metacat example for a raw data file - sam fields are similar with `core.` removed + +:doc:`mctemplate` shows a metacat example for a monte carlo file + +Both sam and metacat put the more detailed information in objects with . format. Metacat extends this to ... + +Here is a table to convert fields +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. csv-table:: metacat equivalents for sam fields + :file: conversion.csv + :widths: 30,30,40 + :header-rows: 1 + +Here is a table of common command translations +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. csv-table:: metacat equivalents for sam commands + :file: commands.csv + :widths: 40,40,20 + :header-rows: 1 diff --git a/docs/source/old/mctemplate.rst b/docs/source/old/mctemplate.rst new file mode 100644 index 0000000..1ebf2e7 --- /dev/null +++ b/docs/source/old/mctemplate.rst @@ -0,0 +1,141 @@ +Template for minimal metadata for a Monte Carlo file +---------------------------------------------------- + +Here is an example of minimal metadata for a an MC reco1 data file. + +You can add additional information to the metadata object. + +Metacat version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "checksums": { + "adler32": "c27f09b5" + }, + "created_timestamp": 1691937037.698784, + "creator": "dunepro", + "fid": "0eUkJLB3SQOmlA9N", + "metadata": { + "art.file_format_era": "ART_2011a", + "art.file_format_version": 15, + "art.first_event": 9301, + "art.last_event": 9400, + "art.process_name": "Reco1", + "art.run_type": "fardet-vd", + "core.application": "art.reco", + "core.application.family": "art", + "core.application.name": "reco", + "core.application.version": "v09_75_03d00", + "core.data_stream": "out1", + "core.data_tier": "hit-reconstructed", + "core.end_time": 1691937015.0, + "core.event_count": 100, + "code.file_content_status":"good", + "core.file_format": "artroot", + "core.file_type": "mc", + "core.first_event_number": 9301, + "core.last_event_number": 9400, + "core.run_type": "fardet-vd", + "core.runs": [ + 27279 + ], + "core.runs_subruns": [ + 2727900001 + ], + "core.start_time": 1691936982.0, + "dune.campaign": "fd_mc_2023a", + "dune.requestid": "ritm1780305", + "dune.workflow": { + "cpuinfo": "AMD EPYC 7551P 32-Core Processor", + "hostname": "hd80.dice.priv", + "job_id": "27279.93@justin-prod-sched01.dune.hep.ac.uk", + "jobscript_cpu_seconds": 37173, + "jobscript_finish": 1691937022, + "jobscript_real_seconds": 37212, + "jobscript_start": 1691899809, + "os_release": "Scientific Linux release 7.9 (Nitrogen)", + "request_id": 1019, + "site_name": "UK_Bristol", + "stage_id": 1, + "user": "amcnab@fnal.gov" + }, + "dune_mc.beam_flux_id": "1", + "dune_mc.beam_polarity": "rhc", + "dune_mc.detector_type": "fardet-vd", + "dune_mc.electron_lifetime": "10.4", + "dune_mc.gen_fcl_filename": "prodgenie_anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg.fcl", + "dune_mc.generators": "genie", + "dune_mc.generators_version": "3.04_00c", + "dune_mc.geometry_version": "dunevd10kt_3view_30deg_v3_refactored_1x8x6ref.gdml", + "dune_mc.liquid_flow": "no", + "dune_mc.mixerconfig": "numu2nutau_nue2numu", + "dune_mc.space_charge": "no", + "dune_mc.with_cosmics": 0, + "retention.class": "simulation", + "retention.status": "active" + }, + "name": "anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg_27279_93_20230813T041011Z_gen_g4_detsim_hitreco.root", + "namespace": "fardet-vd", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 1495285749, + "updated_by": null, + "updated_timestamp": 1691937037.698784 + } + + +samweb version +^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "file_name": "anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg_27279_93_20230813T041011Z_gen_g4_detsim_hitreco.root", + "file_id": 80035348, + "create_date": "2023-08-14T12:26:26+00:00", + "user": "dunepro", + "file_size": 1495285749, + "checksum": [ + "adler32:c27f09b5" + ], + "content_status": "good", + "file_type": "mc", + "file_format": "artroot", + "group": "dune", + "data_tier": "hit-reconstructed", + "application": { + "family": "art", + "name": "reco", + "version": "v09_75_03d00" + }, + "event_count": 100, + "first_event": 9301, + "last_event": 9400, + "start_time": "2023-08-13T14:29:42+00:00", + "end_time": "2023-08-13T14:30:15+00:00", + "data_stream": "out1", + "DUNE.campaign": "fd_mc_2023a", + "DUNE.requestid": "ritm1780305", + "DUNE_MC.beam_flux_ID": 1, + "DUNE_MC.beam_polarity": "rhc", + "DUNE_MC.detector_type": "fardet-vd", + "DUNE_MC.electron_lifetime": "10.4", + "DUNE_MC.gen_fcl_filename": "prodgenie_anu_numu2nutau_nue2numu_dunevd10kt_1x8x6_3view_30deg.fcl", + "DUNE_MC.generators": "genie", + "DUNE_MC.generators_version": "3.04_00c", + "DUNE_MC.geometry_version": "dunevd10kt_3view_30deg_v3_refactored_1x8x6ref.gdml", + "DUNE_MC.liquid_flow": "no", + "DUNE_MC.mixerconfig": "numu2nutau_nue2numu", + "DUNE_MC.space_charge": "no", + "DUNE_MC.with_cosmics": 0, + "runs": [ + [ + 27279, + 1, + "fardet-vd" + ] + ] + } diff --git a/docs/source/old/template.rst b/docs/source/old/template.rst new file mode 100644 index 0000000..a400dea --- /dev/null +++ b/docs/source/old/template.rst @@ -0,0 +1,106 @@ +Template for minimal metadata for a file +---------------------------------------- + +Here is an example of minimal metadata for a raw data file. + +Note that all keys must now be lower case. + +Times need to be UTC. + +You can add additional information to the metadata object. + +Metacat version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "name": "np04_raw_run005141_0015_dl3.root", + "namespace": "protodune-sp", + "checksums": { + "adler32": "0e94618d" + }, + "created_timestamp": 2018-10-10 17:38:27.681780+00:00, + "creator": "dunepro", + "size": 8289321123, + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "updated_by": null, + "updated_timestamp": null + "metadata": { + "core.application.family": "art", + "core.application.name": "dune-artdaq", + "core.application.version": "v1_16_00b", + "core.data_stream": "physics", + "core.data_tier": "raw", + "core.end_time": "2018-10-10 17:38:27.681780+00:00", + "core.event_count": 108, + "core.events": [1,2,3,4], + "core.file_content_status": "good", + "core.file_format": "hdf5", + "core.file_type": "detector", + "core.first_event_number": 20921, + "core.last_event_number": 22349, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": "2018-10-10 17:38:27.681780+00:00", + "retention.status": "active", + "retention.class": "physics" + }, + + } + +Additional information +^^^^^^^^^^^^^^^^^^^^^^ + + This information is added later to track changes + + .. code-block:: javascript + + {"retired": false, + "retired_by": null, + "retired_timestamp": null, + "updated_by": null, + "updated_timestamp": null + } + + + +Samweb version +^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "file_name": "filename", + "file_id": 6607416, + "create_date": "2018-10-10T17:38:27+00:00", + "user": "dunepro", + "file_size": 8289321123, + "checksum": [ "adler32:0e94618d"], + "content_status": "good", + "file_type": "detector", + "file_format": "root", + "data_tier": "raw", + "data_stream": "physics", + "application": { + "family": "art", + "name": "dune-artdaq", + "version": "v1_16_00b" + }, + "event_count": 108, + "first_event": 20921, + "last_event": 22349, + "start_time": "2018-10-10T17:25:01+00:00", + "end_time": "2018-10-10T17:32:16+00:00", + "runs": [ [ 5141, 1, "protodune-sp" ] ] + } + + + diff --git a/docs/source/parameters.csv b/docs/source/parameters.csv new file mode 100644 index 0000000..3852573 --- /dev/null +++ b/docs/source/parameters.csv @@ -0,0 +1,395 @@ +user,file_type,file_format,data_tier,application,data_stream,run_type,file_count,total_file_size_GB,total_event_count,detector.hv_value,artdaq-core.timestamp,artdaq-core.version,artdaq.timestamp,artdaq.version,dune-artdaq.timestamp,dune-artdaq.version,dune-raw-data.timestamp,dune-raw-data.version,dataset.tag,DUNE.campaign,DUNE.poms_campaign_id,DUNE.production_status,DUNE_MC.detector_type,DUNE_MC.electron_lifetime,DUNE_MC.with_cosmics,MC.with_cosmics,detector.hv_status,DUNE.requestid,DUNE.generators,DUNE_MC.beam_flux_ID,DUNE_MC.geometry_version,DUNE_MC.name,DUNE_MC.neutrino_flavors,DUNE_MC.TopVolume,LBNF_MC.HornCurrent,NearDetector_MC.OffAxisPosition,beam.momentum,beam.polarity,DUNE_MC.beam_energy,DUNE_MC.liquid_flow,DUNE_MC.space_charge,MC.liquid_flow,MC.space_charge,data_quality.online_good_run_list,DUNE_MC.generators,art.variation,DUNE.fcl_name,DUNE.fcl_path,DUNE.fcl_version_tag,info.cpusec,info.memory,info.physicsgroup,info.wallsec +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",cosmics,protodune-sp,23312,217207,2674177,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",test,protodune-sp,10039,129956,2859655,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",test,protodune-sp,4205,77154,885392,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",cosmics,protodune-sp,4352,76056,590438,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",noise,protodune-sp,563,10004,75456,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",out1,physics,5,11,50,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_04'}",physics,protodune-sp,43744,763992,5764708,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",test,protodune-sp,80785,359456,13054022,,12-Nov-2018-13:38:24-UTC,v3_04_02,12-Nov-2018-13:38:28-UTC,v3_03_00_gamma,12-Nov-2018-13:38:15-UTC,v1_16_00b,12-Nov-2018-13:38:13-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",test,protodune-sp,701,5257,108435,0,12-Nov-2018-13:38:24-UTC,v3_04_02,12-Nov-2018-13:38:28-UTC,v3_03_00_gamma,12-Nov-2018-13:38:15-UTC,v1_16_00b,12-Nov-2018-13:38:13-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",physics,protodune-sp,66544,531313,8257050,,12-Nov-2018-13:38:24-UTC,v3_04_02,12-Nov-2018-13:38:28-UTC,v3_03_00_gamma,12-Nov-2018-13:38:15-UTC,v1_16_00b,12-Nov-2018-13:38:13-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",test,protodune-sp,6957,49673,1289879,180,12-Nov-2018-13:38:24-UTC,v3_04_02,12-Nov-2018-13:38:28-UTC,v3_03_00_gamma,12-Nov-2018-13:38:15-UTC,v1_16_00b,12-Nov-2018-13:38:13-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_03_00'}",noise,protodune-sp,1068,3634,142901,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_03_00'}",physics,protodune-sp,928,3019,124735,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,simulated,"{'family': 'art', 'name': 'g4', 'version': 'v08_03_00'}",out1,physics,73705,26938,3687320,,,,,,,,,,poms_depends_435657_1,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,162457,931719,15044924,,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,116373,634148,10766088,180,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,2198,18067,128326,144,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,2628,18719,164484,130,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",physics,protodune-sp,6021,17936,646056,140,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_03_00'}",out1,physics,66444,21843,2729800,,,,,,,,,,poms_depends_438864_1,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'development'}",out1,physics,28218,330959,9309210,,,,,,,,,,,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,1681,13852,102837,38,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,1978,16297,119397,77,18-Jan-2019-15:04:13-UTC,v3_04_05-12-g07c4dc2,18-Jan-2019-15:04:16-UTC,merge_snapshot_15Nov18-64-gbd7a816,18-Jan-2019-15:04:06-UTC,v1_16_00b,18-Jan-2019-15:04:04-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,40503,82620,30250879,,31-Jan-2019-09:35:58-UTC,pdune_11Jan2019,31-Jan-2019-09:36:01-UTC,pdune_11Jan2019,31-Jan-2019-09:35:51-UTC,v1_16_00b,31-Jan-2019-09:35:49-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",physics,protodune-sp,920,2552,97289,150,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",physics,protodune-sp,1639,4198,174810,160,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",physics,protodune-sp,836,2250,88060,130,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",physics,protodune-sp,201,503,20397,175,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_03_00'}",out1,physics,373729,43512,5679970,,,,,,,,,,,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,125132,210647,11314833,180,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,84,13,3318,140,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,4337,14306,2497388,180,12-Feb-2019-09:12:57-UTC,pdune_11Jan2019,12-Feb-2019-09:13:00-UTC,pdune_11Jan2019,12-Feb-2019-09:12:50-UTC,v1_16_00b,12-Feb-2019-09:12:48-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,325,4865,16141,126,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,627,9401,30780,162,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,317,4705,15742,108,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,405,4541,19281,90,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,unknown,artroot,full-reconstructed,"{'family': 'art', 'name': 'larsoft', 'version': 'v07_11_00'}",,,1999,20274,199900,,,,,,,,,,,mcc11,2724,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,317,4519,15803,72,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,4299,5912,160664,130,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,314,4220,15422,54,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,3827,1763,101915,38,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,3406,9357,126061,144,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,simulated,"{'family': 'art', 'name': 'g4', 'version': 'v07_04_00'}",,physics,26765,37957,364590,,,,,,,,,,,MCC11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v07_04_00'}",,physics,26506,53430,264790,,,,,,,,,,,MCC11,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,701,5164,83715,0,05-Mar-2019-18:15:10-UTC,pdune_20190222,05-Mar-2019-18:15:14-UTC,pdune_20190222,05-Mar-2019-18:15:01-UTC,v1_16_00b,05-Mar-2019-18:14:58-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,699,996,178860,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,317,3535,15633,18,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_07_00_01'}",,physics,65040,155818,650130,,,,,,,,,,,MCC11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,307,3863,15275,36,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_07_00'}",out1,physics,3066,75,306600,,,,,,,,,,poms_depends_457089_1,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",test,protodune-sp,830,3373,66359,120,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,2756,15144,449057,0,12-Nov-2018-13:38:24-UTC,v3_04_02,12-Nov-2018-13:38:28-UTC,v3_03_00_gamma,12-Nov-2018-13:38:15-UTC,v1_16_00b,12-Nov-2018-13:38:13-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,407,471,401101,170,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,433,1763,31372,160,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,510,3295,25987,90,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,1,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_07_00'}",out1,physics,3032,182,303200,,,,,,,,,,,mcc11,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_07_00'}",out1,physics,1026,92,102600,,,,,,,,,,,mcc11,,,,,,,,RITM0720289,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,531,2171,35168,135,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,152,595,9669,90,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,380,1486,24155,45,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",calibration,protodune-sp,252,1003,15534,18,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v1_16_00b'}",cosmics,protodune-sp,519,3646,28345,135,14-Mar-2019-20:29:08-UTC,pdune_20190312,14-Mar-2019-20:29:12-UTC,pdune_20190313b,14-Mar-2019-20:29:00-UTC,v1_16_00b,14-Mar-2019-20:28:57-UTC,v1_17_06a,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,340,220,21687,45,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v07_08_00_05'}",cosmics,protodune-sp,483,679,34171,160,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",test,protodune-sp,36705,144780,2976321,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'garsoft', 'version': 'v02_04_00'}",,neardet,32,39,9600,,,,,,,,,,,trj_mpd_dec2019,,,,,,,,,GENIE,1,MPD_ECalOctagon_60l_UniformMagnet.gdml,nue_dec2019_mpd_tpc_g4lbne_v3r2p4b_FHC_ND_RIK,"12, -12",volGArTPC,300.0,0.0,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",cosmics,protodune-sp,204977,831925,22411160,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",cosmics,protodune-sp,62804,260111,2087409,180,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_35_00'}",out1,fardet-sp,126054,19329,3151350,,,,,,,,,,,NERSCtest,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_27_01_01'}",cosmics,protodune-sp,54248,39539,1897400,180,,,,,,,,,,keepup_v08_27_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",test,protodune-sp,4373,17951,139249,180,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,hit-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_33_00'}",cosmics,protodune-dp,12664,8595,378099,,,,,,,,,,,NP02_run2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",calibration,protodune-sp,5497,18705,1188522,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_38_01'}",physics,protodune-sp,4096,249851,549069,180,,,,,,,,,,keep_recob_v08_38_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_10_31_pdune_e17_s83'}",test,protodune-sp,1944,7916,288171,,05-Feb-2020-18:59:10-UTC,artdaq-v3_07_00-14-gf006583,05-Feb-2020-18:59:12-UTC,v3_07_01-78-g22a1bd5e,05-Feb-2020-18:59:03-UTC,v2019_10_31_pdune_e17_s83,05-Feb-2020-18:58:58-UTC,v2019_10_31_pdune_e17_s83,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_08_29_develop_self_trigger_and_felix'}",test,protodune-sp,46384,185628,3530723,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,28-Aug-2019-19:38:17-UTC,v2019_08_28_protoDFO_pduneHacks,13-Dec-2019-11:47:47-UTC,v2019_08_29_develop_self_trigger_and_felix,25-Nov-2019-16:59:08-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_37_00'}",out1,fardet-sp,17358,14236,6843300,,,,,,,,,,,MC_Fall2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,13756,28819,137560,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,1.0,1,1.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,13780,28887,137800,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,528,1944,55716,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,1377,6048,189242,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,0.3,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,9766,39784,1256803,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,1405,6355,188237,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,6.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,13780,23666,137800,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,13756,23692,137560,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,1.0,1,1.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,19212,40681,192120,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,2.0,1,2.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,19217,40709,192170,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,2.0,1,2.0,no,yes,no,yes,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,1034,4567,138451,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,0.5,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,66,209,6505,130,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,5866,19734,629059,140,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,1142,3826,121577,160,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,4235,7501,42350,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,3.0,1,3.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,4277,9143,42770,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,3.0,1,3.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_48_01'}",out1,protodune-sp,4299,9193,42990,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,3.0,1,3.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,4234,7474,42340,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,3.0,1,3.0,no,yes,no,yes,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,580,2487,78306,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,2.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,1590,7065,213203,180,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,3.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,190,619,19217,175,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,mc,root,raw,"{'family': 'genie', 'name': 'ghep', 'version': 'v2_12_10c'}",,neardet,245,2,2450000,,,,,,,,,,,mtanaz_mpdactive_nov2019,,,,,,,,,GENIE,1,nd_hall_only_mpd.gdml,nu_nov2019_mpdactive_g4lbne_v3r2p4b_FHC_ND,"12, -12, 14, -14",,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,10963,19200,109630,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,2.0,1,2.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",out1,protodune-sp,10963,19139,109630,,,,,,,,,,,PDSPProd3,,final,protoDUNE SP,3ms,1,1,,,,,,,,,,,2.0,1,2.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,root,raw,"{'family': 'genie', 'name': 'ghep', 'version': 'v2_12_10c'}",,neardet,489,5,4890000,,,,,,,,,,,mtanaz_mpd_ecal_nonactive_nov2019,,,,,,,,,GENIE,1,nd_hall_only_mpd_antifid.gdml,nu_nov2019_mpd_ecal_nonactive_g4lbne_v3r2p4b_FHC_ND,"12, -12, 14, -14",,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",physics,protodune-sp,6091,20501,653421,140,,,,,,,,,,PDSPProd3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_48_01'}",cosmics,protodune-sp,908,728,24654,180,,,,,,,,,,keepup_v08_48_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_01'}",cosmics,protodune-sp,28526,26266,1009634,180,,,,,,,,,,keepup_v08_55_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,hdf5,raw,,test,protodune-sp,180917,779847,21890766,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_55_02'}",out1,protodune-sp,12853,30068,128530,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_55_02'}",out1,protodune-sp,12865,30117,128650,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,12740,33834,127400,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,no,no,no,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,12761,33870,127610,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_55_02'}",out1,protodune-sp,25994,60838,259940,,,,,,,,,,,MC_Summer2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,25775,68441,257750,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0986948,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,12731,905,127310,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,12727,921,127270,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM0986948,,,,,,,,,1.0,1,1.0,no,no,no,no,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,49103,2894,2595320,,,,,,,,,,,MC_Summer2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",noise,protodune-sp,1980,7935,120997,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_59_00'}",cosmics,protodune-dp,1005,689,29991,,,,,,,,,,,NP02_keepup_Summer2020,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,23381,1048,2338100,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0998939,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",out1,protodune-sp,28252,29342,2825102,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0998939,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_57_00'}",out1,protodune-sp,297,24,2970,,,,,,,,,,,MC_Summer2020,,test,protoDUNE SP,35ms,1,1,,RITM0993763,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v08_57_00'}",out1,protodune-sp,301,761,3010,,,,,,,,,,,MC_Summer2020,,test,protoDUNE SP,35ms,1,1,,RITM0993763,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_57_00'}",out1,protodune-sp,297,857,2970,,,,,,,,,,,MC_Summer2020,,test,protoDUNE SP,35ms,1,1,,RITM0993763,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",physics,protodune-sp,1412,6636,187000,180,,,,,,,,,,GPUaaS_RITM0997258,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_62_00'}",out1,fardet-sp,3285,2791,1311000,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0892689,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",physics,protodune-sp,1605,1494,215216,180,,,,,,,,,,diffusion_RITM0986948,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",physics,protodune-sp,1604,98594,215082,180,,,,,,,,,,diffusion_RITM0986948,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_03_00'}",out1,physics,127,249,1270,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM1020527,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_03_00'}",out1,physics,963,2195,9630,,,,,,,,,,,MC_Summer2020,,,protoDUNE SP,35ms,1,1,,RITM1020527,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,4545,18445,503130,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,545,422,57535,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,29610,66766,296100,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,29723,67056,297230,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27678,52988,276780,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27830,53369,278300,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,583,527,78710,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,2.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,10535,48365,1419127,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,2.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,10045,44580,1302085,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,46582,41892,6134087,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,1431,1331,191825,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,3.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,6986,33028,932564,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,3.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,9724,46800,1301722,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,6.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,9102,8583,1218087,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,6.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,4211,19688,560979,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.5,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,4173,3773,555878,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.5,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,3418,16049,460349,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.3,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,1305,1198,179254,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.3,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8356,15987,83560,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8296,18650,82960,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8374,16037,83740,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8298,18666,82980,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,6098,22199,654158,140,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,455,1603,47638,160,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,76,262,7526,130,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,120266,294819,1202660,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,124080,267909,1240800,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,120367,295102,1203670,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,123660,267653,1236600,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,158,549,15787,175,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,116256,9465,1162560,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,116128,9680,1161280,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8019,559,80190,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8019,568,80190,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27351,1912,273510,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27596,1966,275960,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,778,551,83540,140,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_10_02'}",out1,fardet-sp,56940,39687,17808300,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0892689,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",test,protodune-sp,36705,144780,2976321,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'garsoft', 'version': 'v02_04_00'}",,neardet,32,39,9600,,,,,,,,,,,trj_mpd_dec2019,,,,,,,,,GENIE,1,MPD_ECalOctagon_60l_UniformMagnet.gdml,nue_dec2019_mpd_tpc_g4lbne_v3r2p4b_FHC_ND_RIK,"12, -12",volGArTPC,300.0,0.0,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",cosmics,protodune-sp,204977,831925,22411160,,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",cosmics,protodune-sp,62804,260111,2087409,180,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_35_00'}",out1,fardet-sp,126054,19329,3151350,,,,,,,,,,,NERSCtest,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_27_01_01'}",cosmics,protodune-sp,54248,39539,1897400,180,,,,,,,,,,keepup_v08_27_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,unknown,raw,"{'family': 'art', 'name': 'dune-artdaq', 'version': 'v2019_11_25_develop_fixed_crtgen'}",test,protodune-sp,4373,17951,139249,180,18-Jul-2019-01:37:59-UTC,v2019_07_17_protoDFO_testing,13-Nov-2019-20:58:24-UTC,v2019_11_13_dune_artdaq_October2019,25-Nov-2019-17:09:41-UTC,v2019_11_25_develop_fixed_crtgen,25-Nov-2019-17:09:37-UTC,v2019_08_29_for_dune-artdaq_self_trigger_and_felix,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,hit-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_33_00'}",cosmics,protodune-dp,12664,8595,378099,,,,,,,,,,,NP02_run2019,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,28952,65455,289520,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,28877,65319,288770,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27924,1999,279240,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27821,53511,278210,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27905,53759,279050,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,27817,1955,278170,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_10_02'}",out1,fardet-sp,56940,39687,17808300,,,,,,,,,,,MC_Summer2020,,,,,,,,RITM0892689,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8450,19101,84500,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8181,573,81810,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8438,19061,84380,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8246,15889,82460,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8240,15859,82400,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8196,584,81960,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,10045,44580,1302085,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,46582,41892,6134087,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,124591,305380,1245910,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,124425,304949,1244250,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,122362,263683,1223620,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,122447,264617,1224470,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,122225,9933,1222250,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,122438,10192,1224380,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_27_01_01'}",cosmics,protodune-sp,54248,39539,1897400,180,,,,,,,,,,keepup_v08_27_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,3418,16049,460349,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.3,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,4581,10340,45810,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,4583,320,45830,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,4592,10372,45920,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,4583,8825,45830,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,4593,8837,45930,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,4583,326,45830,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.3,1,0.3,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",physics,protodune-sp,1412,6636,187000,180,,,,,,,,,,GPUaaS_RITM0997258,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",physics,protodune-sp,516,471,68044,180,,,,,,,,,,GPUaaS_RITM0997258,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,10535,48365,1419127,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,2.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,766,1717,7660,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,764,1454,7640,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,762,1453,7620,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,767,1720,7670,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",cosmics,protodune-dp,1015,700,30291,,,,,,,,,,,NP02_keepup_March2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,6986,33028,932564,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,3.0,,,,,,,1,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,501,34,5010,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,501,35,5010,,,,,,,,,,,MC_Winter2021,,,protoDUNE SP,35ms,1,1,,RITM1086087,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,4545,18445,503130,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_27_01_01'}",cosmics,protodune-sp,55388,41888,1951474,,,,,,,,,,,keepup_v08_27_01,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,338,254,36140,160,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,298,1170,31828,160,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,920,3705,97288,150,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,917,717,96967,150,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,778,3209,82459,130,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,776,622,82245,130,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,230,1051,31064,180,,,,,,,,,,RITM1103733,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",physics,protodune-sp,229,202,30927,180,,,,,,,,,,RITM1103733,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,30679,2183,306790,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,30773,2147,307730,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,68945,156048,689450,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,69030,156274,690300,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,69120,133354,691200,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,69003,133401,690030,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1115963,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,72625,179914,726250,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1129142,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,70527,154448,705270,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1129142,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,72657,179936,726570,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1129142,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,70569,154000,705690,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1129142,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v08_55_02'}",cosmics,protodune-sp,35668,507536,1214212,180,,,,,,,,,,diffusion_RITM1131173,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,18361,36029,183610,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1146449,,,,,,,,,2.0,1,2.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,17904,35210,179040,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1146449,,,,,,,,,2.0,1,2.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,18025,41285,180250,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1146449,,,,,,,,,2.0,1,2.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,18493,42362,184930,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1146449,,,,,,,,,2.0,1,2.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,27345,69147,273450,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1150297,,,,,,,,,7.0,1,7.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,27391,69237,273910,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1150297,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,27285,61206,272850,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1150297,,,,,,,,,7.0,1,7.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,27258,61374,272580,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1150297,,,,,,,,,7.0,1,7.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_10_02'}",out1,fardet-sp,4710,2314,888125,,,,,,,,,,,MC_Summer2021,,,,,,,,RITM1149890,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_22_03_01'}",physics,protodune-sp,16066,78780,2098445,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,5390,11243,53900,,,,,,,,,,,RITM1147082,,,protoDUNE SP,35ms,1,1,,RITM1147082,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,5128,9064,51280,,,,,,,,,,,RITM1147082,,,protoDUNE SP,35ms,1,1,,RITM1147082,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,4567,10315,45670,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.3,1,0.3,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,4570,10325,45700,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.3,1,0.3,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,4553,8759,45530,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.3,1,0.3,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,4546,8759,45460,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.3,1,0.3,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,8304,18801,83040,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,8258,15965,82580,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,8318,18839,83180,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,8276,15973,82760,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1187486,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,31107,61532,311070,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1188410,,,,,,,,,3.0,1,3.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,31074,61611,310740,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1188410,,,,,,,,,3.0,1,3.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,31326,72143,313260,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1188410,,,,,,,,,3.0,1,3.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,31365,72244,313650,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1188410,,,,,,,,,3.0,1,3.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,124577,308609,1245770,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1202507,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,124636,308661,1246360,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1202507,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,123986,270575,1239860,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1202507,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,123885,271295,1238850,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1202507,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,3405,15539,424979,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,9535,45832,1232766,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,2803,14139,371330,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.5,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,10495,52361,1413742,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,2.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,3187,16245,428823,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,0.3,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,6911,35497,922499,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,3.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,9255,48378,1238611,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,6.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,760,3481,86533,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,5894,23315,632189,140,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,447,1708,46778,160,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,76,284,7526,130,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,154,580,15359,175,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,1.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,673,2954,71140,150,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,1129,5052,119572,130,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,7.0,,,,,,,1,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_32_00'}",out1,fardet-sp,249,320,99600,,,,,,,,,,,MC_Fall2021,,,,,,,,RITM1251140,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_32_00'}",out1,fardet-sp,249,16,99600,,,,,,,,,,,MC_Fall2021,,,,,,,,RITM1251140,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,920,4016,97288,150,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,49115,244499,6430371,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_35_00_01'}",out1,protodune-sp,19976,38799,199760,,,,,,,,,,,PDSPProd4a_SCEsyst,,,protoDUNE SP,35ms,1,1,,RITM1268625,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,103598,214619,1035980,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu_numu2nutau_nue2numu,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,96466,196913,964660,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,98955,209046,989550,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu_numu2nue_nue2nutau,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",physics,protodune-sp,49115,244499,6430371,180,,,,,,,,,,PDSPProd4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,99052,204386,990520,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_anu,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,88755,187301,887550,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_anu_numu2nutau_nue2numu,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,97520,211986,975200,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_anu_numu2nue_nue2nutau,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,96466,196913,964660,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,98955,209046,989550,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu_numu2nue_nue2nutau,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,103598,214619,1035980,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,genie_nu_numu2nutau_nue2numu,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,80562,112120,5192760,180,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,643482,1340656,6434820,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_37_00'}",out1,fardet-vd,49985,97103,499850,,,,,,,,,,,FDVDProd1,,,,,,,,RITM1276306,,,,,,,,,,,,,,,,,marley,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,65894,98868,4492443,180,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,30,73,21499,0,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,850,1816,68106,120,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,200,123,12243,18,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,312,667,18468,90,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_42_04_00'}",out1,fardet-vd,103,14,102010,,,,,,,,,,,RITM1331650,,,,,,,,RITM1331650,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,218,475,13489,45,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,984,2185,65164,135,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,386,837,27902,160,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,807,1865,64584,120,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,675,1554,46778,130,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,30,73,21499,0,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,424,1016,28046,135,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,152,354,9388,45,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,171,106,10431,18,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_30_00'}",cosmics,protodune-sp,170,369,9594,90,,,,,,,,,,RITM1312299,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_44_00_02'}",test,vd-coldbox-top,842,1187,92490,,,,,,,,,,,VD_coldbox_TDE_2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_44_00_02'}",out1,physics,363,1278,3630,,,,,,,,,,,RITM1331140,,,protoDUNE SP,35ms,1,1,,RITM1331140,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_44_00_02'}",out1,physics,363,1272,3630,,,,,,,,,,,RITM1331140,,,protoDUNE SP,35ms,1,1,,RITM1331140,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_44_00_02'}",out1,protodune-sp,2810,9904,28100,,,,,,,,,,,RITM1331140,,,protoDUNE SP,35ms,1,1,,RITM1331140,,,,,,,,,1.0,1,1.0,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,reco-recalibrated,"{'family': 'art', 'name': 'reco', 'version': 'v09_44_00_02'}",out1,protodune-sp,2785,9773,27850,,,,,,,,,,,RITM1331140,,,protoDUNE SP,35ms,1,1,,RITM1331140,,,,,,,,,1.0,1,1.0,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_46_00_00'}",out1,fardet-sp,19817,28785,4954250,,,,,,,,,,,RITM1355274,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hA_BR,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,9950,11369,1990050,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hA_LFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,9923,11818,1984650,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hA_ESF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,8050,9350,1610050,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hN_BR,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,9949,11430,1989850,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hN_LFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,9942,12238,1988450,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,NNBarAtm_hN_ESF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,208,99610,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hA_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,209,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hA_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,217,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hN_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,217,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hN_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,214,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hA_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,248,224,99200,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,protondecay_hN_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,671,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hA_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,246,705,98400,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hA_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,712,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hN_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,249,703,99600,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hN_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,249,807,99600,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hN_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-sp,250,676,100000,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,legacynnbar_hA_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_47_00_01'}",out1,fardet-hd,35609,36152,7064425,,,,,,,,,,,MC_Winter2022,,,,,,,,RITM1355274,,,,,,,,,,,,,,,,,scalar_Ar_m05_b10_G19_00a_20220302T020738,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_49_00d00'}",test,vd-coldbox-top,17921,25259,1967719,,,,,,,,,,,VD_coldbox_TDE_2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",cosmics,protodune-sp,72580,64866,2296178,,,,,,,,,,,RITM1382619,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_49_00d00'}",study,vd-coldbox-top,5460,7212,598960,,,,,,,,,,,VD_coldbox_TDE_2021,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,hdf5,raw,,test,dc4-vd-coldbox-bottom,49327,211358,3056585,,,,,,,,,,,dc4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_52_00d00'}",out1,physics,6400,61485,1280000,,,,,,,,,,,FDHDSNProd1,,,,,,,,RITM1464893,,,,,,,,,,,,,,,,,,es,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,332685,10884,22722840,,,,,,,,,,,FDVDProd2,,,,,,,,RITM1457487,,,,,,,,,,,,,,,,,,NoRawDigits,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,332578,485523,22722850,,,,,,,,,,,FDVDProd2,,,,,,,,RITM1457487,,,,,,,,,,,,,,,,,,NoRawDigits,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,1,0,1,,,,,,,,,,,FDVDProd2_test,,,,,,,,RITM1457487,,,,,,,,,,,,,,,,,,WithRawDigits,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,1,0,1,,,,,,,,,,,FDVDProd2_test,,,,,,,,RITM1457487,,,,,,,,,,,,,,,,,,WithRawDigits,,,,,,, +dunepro,mc,binary,pandora_info,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,100,2,5000,,,,,,,,,,,PerlmutterTest_RITM1479848,,,,,,,,RITM1479848,,,,,,,,,,,,,,,,,,NoRawDigits,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_53_00d02'}",out1,fardet-vd,200,93,10000,,,,,,,,,,,PerlmutterTest_RITM1479848,,,,,,,,RITM1479848,,,,,,,,,,,,,,,,,,NoRawDigits,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_52_00d00'}",out1,physics,5900,56272,1180000,,,,,,,,,,,FDHDSNProd1v2,,,,,,,,RITM1464893,,,,,,,,,,,,,,,,,,es,,,,,,, +dunepro,mc,root,root-tuple-virtual,"{'family': 'protoduneana', 'name': 'pdspana', 'version': 'v09_52_00d00'}",physics,protodune-sp,32737,113,1537293,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,,,,,,,,,,6.0,1,6.0,no,yes,no,yes,,beam_cosmics_datadriven,,pduneana_Prod4a_MC_sce_6GeV.fcl,/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v09_52_00d00/job,v09_52_00d00,5404.20357,4202,dunepdhd,8188.032621 +dunepro,mc,root,root-tuple-virtual,"{'family': 'protoduneana', 'name': 'pdspana', 'version': 'v09_52_00d00'}",physics,protodune-sp,35766,142,1762443,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,,,,,,,,,,6.0,1,6.0,no,no,no,no,,beam_cosmics_datadriven,,pduneana_Prod4a_MC_sce_6GeV.fcl,/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v09_52_00d00/job,v09_52_00d00,3033.892011,3787,dunepdhd,7641.22312 +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_02d00'}",physics,protodune-sp,81,290,8110,130,,,,,,,,,,AlternateSCE_RITM1506913,,,,,,,,,,,,,,,,,1.0,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_01d00'}",cosmics,protodune-dp,2375,771,71215,,,,,,,,,,,NP02_keepup_March2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_02d00'}",physics,protodune-sp,1346,4999,143512,160,,,,,,,,,,AlternateSCE_RITM1506913,,,,,,,,,,,,,,,,,1.0,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_02d00'}",physics,protodune-sp,6139,22935,658685,140,,,,,,,,,,AlternateSCE_RITM1506913,,,,,,,,,,,,,,,,,1.0,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_02d00'}",physics,protodune-sp,11336,51285,1462677,180,,,,,,,,,,AlternateSCE_RITM1506913,,,,,,,,,,,,,,,,,1.0,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_58_02d00'}",physics,protodune-sp,201,728,20396,175,,,,,,,,,,AlternateSCE_RITM1506913,,,,,,,,,,,,,,,,,1.0,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_63_00d00'}",study,vd-coldbox-top,510,668,30520,,,,,,,,,,,VD_coldbox_CRP2_CRP3_2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_36_00'}",out1,protodune-sp,1621,1011,404085,,,,,,,,,,,RITM1557373,,,,,,,,RITM1557373,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_63_01d00'}",cosmics,protodune-dp,28247,9910,846449,,,,,,,,,,,NP02_keepup_March2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,18753,27293,4688250,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hA_BR,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,249,209,99210,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hA_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,9669,11078,1933850,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hA_LFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,248,209,99200,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hA_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,249,218,99600,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hN_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,247,226,98800,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hN_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,249,219,99600,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hN_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,124886,130729,26452310,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,scalar_Ar_m05_b1p1_G19_01a_20220302T043327,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,248,671,99200,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hA_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,9581,11804,1916250,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hN_ESF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,248,706,99200,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hN_LocalFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,243,702,97200,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hA_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,249,813,99600,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hN_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,7827,9110,1565450,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hN_BR,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,249,715,99600,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hN_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,9644,11102,1928850,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hN_LFG,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,247,213,98800,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,protondecay_hA_EffSF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,9633,11507,1926650,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,NNBarAtm_hA_ESF,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_64_01d00'}",out1,physics,247,673,98800,,,,,,,,,,,MC_Winter2023_RITM1592444_reReco,,,,,,,,RITM1592444,,,,,,,,,,,,,,,,,legacynnbar_hA_BodekRitchie,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_36_00'}",out1,protodune-sp,3,0,15,,,,,,,,,,,RITM1632937_test,,,,,,,,RITM1632937,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_36_00'}",out1,protodune-sp,5376,3361,1344000,,,,,,,,,,,RITM1632937,,,,,,,,RITM1632937,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_67_00d00'}",study,vd-coldbox-top,508,666,30400,,,,,,,,,,,VD_coldbox_CRP2_CRP3_2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,20420,46383,204200,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,20408,39544,204080,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,20406,39610,204060,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,20418,46363,204180,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_69_01d00'}",study,vd-coldbox-top,511,668,30580,,,,,,,,,,,VD_coldbox_CRP2_CRP3_2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,detector,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_72_00d00'}",study,vd-coldbox-top,512,668,30640,,,,,,,,,,,VD_coldbox_CRP2_CRP3_2022,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8246,15889,82460,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8438,19061,84380,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_09_01'}",out1,protodune-sp,8450,19101,84500,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_09_01'}",out1,protodune-sp,8240,15859,82400,,,,,,,,,,,PDSPProd4,,,protoDUNE SP,35ms,1,1,,RITM1036677,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,15963,36052,159630,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,detector-simulated,"{'family': 'art', 'name': 'detsim', 'version': 'v09_17_01'}",out1,protodune-sp,15961,36059,159610,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,15959,30699,159590,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,yes,no,yes,,beam_cosmics_datadriven,,,,,,,, +dunepro,mc,artroot,full-reconstructed,"{'family': 'art', 'name': 'reco', 'version': 'v09_17_01'}",out1,protodune-sp,15953,30740,159530,,,,,,,,,,,PDSPProd4a,,,protoDUNE SP,35ms,1,1,,RITM1648093,,,,,,,,,0.5,1,0.5,no,no,no,no,,beam_cosmics_datadriven,,,,,,,, diff --git a/docs/source/pdspmc.json b/docs/source/pdspmc.json new file mode 100644 index 0000000..51e283b --- /dev/null +++ b/docs/source/pdspmc.json @@ -0,0 +1,48 @@ +{ + "checksums": { + "adler32": "d8cc5791", + }, + "created_timestamp": 1618046947.65281, + "creator": "dunepro", + "fid": "52681720", + "metadata": { + "beam.polarity": 1, + "core.application": "g4", + "core.application.family": "g4", + "core.application.name": "g4beamline", + "core.application.version": "v34b", + "core.data_stream": "beam", + "core.data_tier": "g4-beam", + "core.config_file_name": "????", + "core.end_time": 1618046252, + "core.event_count": 10, + "core.file_content_status": "good", + "core.file_format": "root", + "core.file_type": "mc", + "core.first_event_number": 1041, + "core.group": "dune", + "core.last_event_number": 1050, + "core.run_type": "pdsp-beam", + "core.runs": [ + 43390924 + ], + "core.runs_subruns": [ + 4339092400002 + ], + "core.start_time": 1618042577, + "core.start_time_utc_text": "2021-04-10 08:16:17", + "dune.campaign": "PDSPProd4a", + "dune.requestid": "", + "dune_mc.generators": "g4beamline", + "retention.class": "simulation", + "retention.status": "active" + }, + "name": "", + "namespace": "pdsp_mc_reco", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 1795295560, + "updated_by": null, + "updated_timestamp": null +} diff --git a/docs/source/rawfile.json b/docs/source/rawfile.json new file mode 100644 index 0000000..bce8d27 --- /dev/null +++ b/docs/source/rawfile.json @@ -0,0 +1,291 @@ +{ + "checksums": { + "adler32": "8e5309cd" + }, + "created_timestamp": 1539193167.20406, + "creator": "dunepro", + "fid": "6607266", + "metadata": { + "DUNE_data.detector_config.object": [ + "cob2_rce01", + "cob2_rce02", + "cob2_rce03", + "cob2_rce04", + "cob2_rce05", + "cob2_rce06", + "cob2_rce07", + "cob2_rce08", + "cob3_rce01", + "cob3_rce02", + "cob3_rce03", + "cob3_rce04", + "cob3_rce05", + "cob3_rce06", + "cob3_rce07", + "cob3_rce08", + "cob4_rce01", + "cob4_rce02", + "cob4_rce03", + "cob4_rce04", + "cob4_rce05", + "cob4_rce06", + "cob4_rce07", + "cob4_rce08", + "cob5_rce01", + "cob5_rce02", + "cob5_rce03", + "cob5_rce04", + "cob5_rce05", + "cob5_rce06", + "cob5_rce07", + "cob5_rce08", + "cob6_rce01", + "cob6_rce02", + "cob6_rce03", + "cob6_rce04", + "cob6_rce05", + "cob6_rce06", + "cob6_rce07", + "cob6_rce08", + "cob7_rce01", + "cob7_rce02", + "cob8_rce01", + "cob8_rce02", + "cob8_rce03", + "cob8_rce04", + "cob8_rce05", + "cob8_rce06", + "cob8_rce07", + "cob8_rce08", + "felix00", + "felix01", + "felix02", + "felix03", + "felix04", + "felix14", + "felix15", + "felix16", + "felix17", + "felix18", + "ssp101", + "ssp102", + "ssp103", + "ssp104", + "ssp201", + "ssp202", + "ssp203", + "ssp204", + "ssp301", + "ssp302", + "ssp303", + "ssp304", + "ssp401", + "ssp402", + "ssp403", + "ssp404", + "ssp501", + "ssp502", + "ssp504", + "ssp601", + "ssp602", + "ssp603", + "ssp604", + "timing_0", + "trigger_0", + "wib101", + "wib102", + "wib103", + "wib104", + "wib105", + "wib201", + "wib202", + "wib203", + "wib204", + "wib205", + "wib301", + "wib302", + "wib303", + "wib304", + "wib305", + "wib401", + "wib402", + "wib403", + "wib404", + "wib405", + "wib501", + "wib502", + "wib503", + "wib504", + "wib505", + "wib601", + "wib602", + "wib603", + "wib604", + "wib605" + ], + "artdaq-core.timestamp": "10-ott-2018-09:22:14-UTC", + "artdaq-core.version": "v3_04_02", + "artdaq.timestamp": "10-ott-2018-09:22:18-UTC", + "artdaq.version": "v3_03_00_beta", + "core.application": "art.dune-artdaq", + "core.application.family": "art", + "core.application.name": "dune-artdaq", + "core.application.version": "v1_16_00b", + "core.data_stream": "physics", + "core.data_tier": "raw", + "core.end_time": 1539192738, + "core.end_time_utc_text": "2018-10-10 17:32:18", + "core.event_count": 108, + "core.events": [ + 20912, + 20919, + 20924, + 20940, + 20943, + 20952, + 21002, + 21011, + 21020, + 21022, + 21034, + 21036, + 21038, + 21048, + 21050, + 21056, + 21066, + 21144, + 21175, + 21176, + 21192, + 21199, + 21204, + 21220, + 21223, + 21231, + 21282, + 21291, + 21300, + 21302, + 21314, + 21316, + 21318, + 21328, + 21330, + 21336, + 21346, + 21424, + 21455, + 21456, + 21473, + 21479, + 21484, + 21500, + 21503, + 21511, + 21562, + 21571, + 21580, + 21582, + 21594, + 21596, + 21598, + 21608, + 21610, + 21615, + 21626, + 21704, + 21735, + 21736, + 21753, + 21759, + 21764, + 21780, + 21782, + 21791, + 21842, + 21851, + 21865, + 21872, + 21874, + 21875, + 21877, + 21883, + 21888, + 21901, + 21903, + 22014, + 22016, + 22020, + 22037, + 22040, + 22042, + 22047, + 22056, + 22062, + 22133, + 22136, + 22137, + 22141, + 22147, + 22156, + 22163, + 22164, + 22170, + 22175, + 22180, + 22298, + 22299, + 22304, + 22307, + 22308, + 22327, + 22328, + 22335, + 22339, + 22411, + 22413 + ], + "core.file_content_status": "good", + "core.file_format": "unknown", + "core.file_type": "detector", + "core.first_event_number": 20912, + "core.last_event_number": 22413, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": 1539192299, + "core.start_time_utc_text": "2018-10-10 17:24:59", + "data_quality.online_good_run_list": 1, + "detector.hv_value": 180, + "dune-artdaq.timestamp": "10-ott-2018-09:22:06-UTC", + "dune-artdaq.version": "v1_16_00b", + "dune-raw-data.timestamp": "10-ott-2018-09:22:04-UTC", + "dune-raw-data.version": "v1_17_06a", + "dune_data.accouple": 0, + "dune_data.calibpulsemode": 0, + "dune_data.daqconfigname": "np04_WibsReal_Ssps_BeamTrig_00021", + "dune_data.detector_config": "cob2_rce01:cob2_rce02:cob2_rce03:cob2_rce04:cob2_rce05:cob2_rce06:cob2_rce07:cob2_rce08:cob3_rce01:cob3_rce02:cob3_rce03:cob3_rce04:cob3_rce05:cob3_rce06:cob3_rce07:cob3_rce08:cob4_rce01:cob4_rce02:cob4_rce03:cob4_rce04:cob4_rce05:cob4_rce06:cob4_rce07:cob4_rce08:cob5_rce01:cob5_rce02:cob5_rce03:cob5_rce04:cob5_rce05:cob5_rce06:cob5_rce07:cob5_rce08:cob6_rce01:cob6_rce02:cob6_rce03:cob6_rce04:cob6_rce05:cob6_rce06:cob6_rce07:cob6_rce08:cob7_rce01:cob7_rce02:cob8_rce01:cob8_rce02:cob8_rce03:cob8_rce04:cob8_rce05:cob8_rce06:cob8_rce07:cob8_rce08:felix00:felix01:felix02:felix03:felix04:felix14:felix15:felix16:felix17:felix18:ssp101:ssp102:ssp103:ssp104:ssp201:ssp202:ssp203:ssp204:ssp301:ssp302:ssp303:ssp304:ssp401:ssp402:ssp403:ssp404:ssp501:ssp502:ssp504:ssp601:ssp602:ssp603:ssp604:timing_0:trigger_0:wib101:wib102:wib103:wib104:wib105:wib201:wib202:wib203:wib204:wib205:wib301:wib302:wib303:wib304:wib305:wib401:wib402:wib403:wib404:wib405:wib501:wib502:wib503:wib504:wib505:wib601:wib602:wib603:wib604:wib605", + "dune_data.febaselinehigh": 2, + "dune_data.fegain": 2, + "dune_data.feleak10x": 0, + "dune_data.feleakhigh": 1, + "dune_data.feshapingtime": 2, + "dune_data.inconsistent_hw_config": 0, + "dune_data.is_fake_data": 0, + "retention.class": "rawdata", + "retention.status": "active" + }, + "name": "np04_raw_run005141_0015_dl2.root", + "namespace": "protodune-sp", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 8280799211, + "updated_by": "schellma", + "updated_timestamp": 1542579196.2592 +} diff --git a/docs/source/reco.rst b/docs/source/reco.rst new file mode 100644 index 0000000..e80e5f5 --- /dev/null +++ b/docs/source/reco.rst @@ -0,0 +1,22 @@ +Reconstructed File Example +-------------------------- + +metacat +======= + +`metacat file show -m -j pdsp_det_reco:np04_raw_run005141_0002_dl2_reco1_18126921_0_20210318T103516Z.root` + +.. include:: meta_reco.json + :literal: + + + + + +samweb +====== + +`samweb get-metadata np04_raw_run005141_0002_dl2_reco1_18126921_0_20210318T103516Z.root --json` + +.. include:: sam_reco.json + :literal: \ No newline at end of file diff --git a/docs/source/related.rst b/docs/source/related.rst new file mode 100644 index 0000000..02da361 --- /dev/null +++ b/docs/source/related.rst @@ -0,0 +1,31 @@ +Related documentation +===================== + +metacat +------- + + `metacat documentation `_ + + `metacat gui `_ + + `DUNE metacat server status `_ + + +Data Dispatcher (dd) +-------------------- + + `Data Dispatcher documentation `_ + + `Data Dispatcher monitor `_ + +samweb +------ + + `official samweb `_ + + `samweb project monitor `_ + +JustIn +------ + + `JustIn documentation `_ \ No newline at end of file diff --git a/docs/source/runInfo.rst b/docs/source/runInfo.rst new file mode 100644 index 0000000..56131be --- /dev/null +++ b/docs/source/runInfo.rst @@ -0,0 +1,44 @@ +Information that should probably go into a runs database +-------------------------------------------------------- + +February 27, 2023 HMS + +This is data found in the file data for protoDUNE run 1 that would be good candidates to go into a runs condition database for Run II + +.. code-block:: json + + { + "run_number": 5141, + "file_type": "detector", + "run_type": "protodune-sp", + "event_count": 100, + "first_event": 1, + "last_event": 100, + "start_time": "2018-10-17T19:45:47+00:00", + "end_time": "2018-10-17T19:56:50+00:00", + "purpose": "cosmics", + "artdaq-core.timestamp": "11-Oct-2018-22:06:58-UTC", + "artdaq-core.version": "v3_04_02", + "artdaq.timestamp": "11-Oct-2018-22:07:02-UTC", + "artdaq.version": "v3_03_00_beta", + "beam.momentum": 1, + "data_quality.online_good_run_list": 1, + "detector.hv_value": 180, + "dune-artdaq.timestamp": "11-Oct-2018-22:06:51-UTC", + "dune-artdaq.version": "v1_16_00b", + "dune-raw-data.timestamp": "11-Oct-2018-22:06:48-UTC", + "dune-raw-data.version": "v1_17_06a", + "DUNE_data.acCouple": 0, + "DUNE_data.calibpulsemode": 0, + "DUNE_data.DAQConfigName": "np04_WibsReal_Ssps_BeamTrig1GeV_00011", + "DUNE_data.detector_config": ".. long list of configured readout boards ...", + "DUNE_data.febaselineHigh": 1, + "DUNE_data.fegain": 2, + "DUNE_data.feleak10x": 0, + "DUNE_data.feleakHigh": 1, + "DUNE_data.feshapingtime": 2, + "DUNE_data.inconsistent_hw_config": 0, + "DUNE_data.is_fake_data": 0, + "subruns": [], + "beam spills": [] + } \ No newline at end of file diff --git a/docs/source/sam2metacat.rst b/docs/source/sam2metacat.rst new file mode 100644 index 0000000..e41271f --- /dev/null +++ b/docs/source/sam2metacat.rst @@ -0,0 +1,231 @@ + +Sam to Metacat Conversion guide +=============================== + + + This document includes examples of `sam` queries gathered from DUNE Dataset definitions and their `metacat` translations + +Get metacat started +------------------- + +First find the documentation: + +https://metacat.readthedocs.io/en/latest/index.html + +metacat is a `ups` product so you can get it by + +.. code-block:: bash + + source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh + setup python v3_9_2 # this avoids system python which may be very old + setup metacat + +but you can also do a local install using: + +https://metacat.readthedocs.io/en/latest/ui.html#installation + +Make certain you can point to the metacat server: + +.. code-block:: bash + + export METACAT_AUTH_SERVER_URL=https://metacat.fnal.gov:8143/auth/dune + export METACAT_SERVER_URL=https://metacat.fnal.gov:9443/dune_meta_prod/app + + +Then authenticate to metacat: + + + +.. code-block:: bash + + metacat auth login -m password $USER + Password: + User: schellma + Expires: Thu Oct 13 16:27:29 2022 + +*Note: you can also authenticate via other methods, for example* + +.. code-block:: bash + + kx509 + export X509_USER_PROXY=/tmp/x509up_u$(id -u) + export X509_USER_KEY=$X509_USER_PROXY + metacat auth login -m x509 $USER + +.. note:: + + If you are not on a Fermilab machine you may need to add your local credentials to the list of DN's and explicitly tell metacat your FNAL user id. + + do this + + .. code-block:: bash + + metacat auth mydn + + + 1. Log in to MetaCat GUI using services password + 2. Go to your user profile https://metacat.fnal.gov:9443/dune_prod/app/gui/user?username= + 3. Copy-paste the output from "metacat auth mydn" into blank text box in front of Add button + 4. Click Add + + then + + .. code-block:: bash + + metacat auth login -m x509 + + +Example: Get the raw data from given protodune-sp detector runs +--------------------------------------------------------------- + +* samweb + + .. code-block:: bash + + samweb list-files "file_type detector and run_type 'protodune-sp'\ + and data_tier raw and data_stream physics and run_number 5141,5143" + + .. Note: Note that you need to specify the `file_type` (detector/mc), which experiment it was (`protodune-sp`), what tier of data it was (`raw`) and what kind of running (`physics`) it was.* + + add `--summary` if you wish to know how many files there are. + +* metacat + + .. code-block:: bash + + metacat query "files from dune:all where core.file_type=detector \ + and core.run_type='protodune-sp' and core.data_tier=raw \ + and core.data_stream=physics and core.runs[any] in (5141,5143)" + + add `--summary` after query if you want just the # of files + + *Notes:* + + - *many of the metadata values are now in categories like `core`* + + - *things run faster if you ask for files from a known dataset like `dune:all`* + + - *core.runs[any] means check any of the runs associated with the file for being 5141* + + - *core.runs[any] in (5141, 5142, 5147) - any of these 3 runs* + + - *core.runs[any] = 5141- single run, equivalent: 5141 in core.runs* + + - *you can ask for multiple runs by using the `in (X,Y)` syntax* + +Example: Save a dataset or definition query +------------------------------------------- + +If you are interested in everything physics from `protodune-sp`, you might want to save a generic dataset or query which you can then reuse in further filtered queries. Then as you narrow thing down you can build additional datasets. + +* *samweb* + + in sam you save a definition, which is the query + + .. code-block:: bash + + samweb create-definition schellma-protodune-sp-physics-generic \ + "file_type detector and run_type 'protodune-sp' and data_stream physics" ` + + You can then ask for: + + .. code-block:: bash + + samweb list-files "defname:schellma-protodune-sp-physics-generic \ + and data_tier raw and run_number 5141" --summary + + *Note: a sam definition is a query, not a list of files and can change, for example if more data are added. You need to make a `snapshot` to make a list that does not change.* + + *Another note: sam also prepends the user name to the definition so that you can't mess up official queries. This is handled in metacat by the introduction of namespaces.* + +* metacat + + To run a MQL query and create a new dataset with the query results: + + .. code-block:: bash + + metacat dataset create -f "files from dune:all where \ + ..." : + + .. code-block:: bash + + metacat dataset create -f @file_with_mql_query.txt \ + : + + + To run a query and add matching files to an existing dataset: + + .. code-block:: bash + + metacat dataset add-files -q "files from dune:all where ..." : + + metacat dataset add-files -q @file_with_mql_query.txt : + + .. Note: this times out if all runs are included - I just did 5141 for this test. + + .. Note: Todo: a utility command that logs the query in the dataset metadata, possibly not in the "description" field + + check it by querying the files in the dataset + + .. code-block:: bash + + metacat query -s "files from schellma:protodune-sp-physics-generic" + + metacat dataset show schellma:protodune-sp-physics-generic + + children : + created_timestamp : 2022-10-08 11:41:54 + creator : schellma + description : files from dune:all where core.file_type=detector and core.run_type='protodune-sp' and core.data_stream=physics + file_count : 772631 + file_meta_requirements : {} + frozen : False + metadata : {} + monotonic : False + name : protodune-sp-physics-generic + namespace : schellma + parents : + + + .. :Note: I have not saved the query in the metacat dataset but just added it as an optional description. I have saved the list of files. In `metacat` datasets do not change (for example if another file passing the query requirements comes in from the DAQ) until you explicitly add the new file.* + + You can then ask for the subset from a particular data tier and run number. + + .. code-block:: bash + + metacat query "files from schellma:protodune-sp-physics-generic \ + where core.runs[all]=5141 and core.data_tier=raw" + +Find only the files not processed with a version of code +-------------------------------------------------------- + +* samweb + + + .. code-block:: bash + + samweb list-files "defname:schellma-protodune-sp-physics-generic \ + and data_tier raw and run_number 5141 minus \ + isparentof:(defname:schellma-protodune-sp-physics-generic\ + and data_tier 'full-reconstructed' and run_number 5141 and version v08_27_% )" --summary + + File count: 12 + Total size: 95354212618 + Event count: 1241 + + +* metacat + + + .. code-block:: bash + + metacat query -s "files from schellma:protodune-sp-physics-generic \ + where core.data_tier=raw and 5141 in core.runs - parents(files \ + from schellma:protodune-sp-physics-generic where 5141 in core.runs \ + and core.data_tier='full-reconstructed' and core.application.version~'v08_27_.*')" + + 12 files + + .. :Note: TODO - get the file size as well? + + .. :Note: the syntax for a parameter matching is Regular Expressions, in particular '.\*' matches any string* diff --git a/docs/source/sam_reco.json b/docs/source/sam_reco.json new file mode 100644 index 0000000..76fcbd2 --- /dev/null +++ b/docs/source/sam_reco.json @@ -0,0 +1,63 @@ +{ + "file_name": "np04_raw_run005141_0002_dl2_reco1_18126921_0_20210318T103516Z.root", + "file_id": 52500057, + "create_date": "2021-03-19T16:08:14+00:00", + "user": "dunepro", + "update_date": "2021-10-05T15:05:46+00:00", + "update_user": "dunepro", + "file_size": 3690769125, + "checksum": [ + "enstore:1952172898", + "adler32:03d6cb63", + "md5:9c9d59cc33f3b0e2167c2acb3228241e" + ], + "content_status": "good", + "file_type": "detector", + "file_format": "artroot", + "data_tier": "full-reconstructed", + "application": { + "family": "art", + "name": "reco", + "version": "v09_09_01" + }, + "process_id": 15256421, + "event_count": 108, + "first_event": 1584, + "last_event": 3028, + "start_time": "2021-03-18T10:56:42+00:00", + "end_time": "2021-03-19T15:46:57+00:00", + "data_stream": "physics", + "art.file_format_era": "ART_2011a", + "art.file_format_version": 13.0, + "art.first_event": 1584.0, + "art.last_event": 3028.0, + "art.process_name": "Reco1", + "art.run_type": "protodune-sp", + "beam.momentum": 7.0, + "data_quality.online_good_run_list": 1, + "detector.hv_value": 180, + "DUNE.campaign": "PDSPProd4", + "DUNE_data.acCouple": 0, + "DUNE_data.calibpulsemode": 0, + "DUNE_data.DAQConfigName": "np04_WibsReal_Ssps_BeamTrig_00021", + "DUNE_data.febaselineHigh": 2, + "DUNE_data.fegain": 2, + "DUNE_data.feleak10x": 0, + "DUNE_data.feleakHigh": 1, + "DUNE_data.feshapingtime": 2, + "DUNE_data.inconsistent_hw_config": 0, + "DUNE_data.is_fake_data": 0, + "runs": [ + [ + 5141, + 1, + "protodune-sp" + ] + ], + "parents": [ + { + "file_name": "np04_raw_run005141_0002_dl2.root", + "file_id": 6605772 + } + ] + } \ No newline at end of file diff --git a/docs/source/samweb_raw.json b/docs/source/samweb_raw.json new file mode 100644 index 0000000..3a3e718 --- /dev/null +++ b/docs/source/samweb_raw.json @@ -0,0 +1,32 @@ + { + "file_name": "filename", + "file_id": 6607416, + "create_date": "2018-10-10T17:38:27+00:00", + "user": "dunepro", + "file_size": 8289321123, + "checksum": [ + "adler32:0e94618d" + ], + "content_status": "good", + "file_type": "detector", + "file_format": "root", + "data_tier": "raw", + "data_stream": "physics", + "application": { + "family": "art", + "name": "dune-artdaq", + "version": "v1_16_00b" + }, + "event_count": 108, + "first_event": 20921, + "last_event": 22349, + "start_time": "2018-10-10T17:25:01+00:00", + "end_time": "2018-10-10T17:32:16+00:00", + "runs": [ + [ + 5141, + 1, + "protodune-sp" + ] + ] + } diff --git a/docs/source/setup.rst b/docs/source/setup.rst new file mode 100644 index 0000000..803e7e2 --- /dev/null +++ b/docs/source/setup.rst @@ -0,0 +1,50 @@ + +Setting up metacat +================== + + +Get metacat started +------------------- + +First find the documentation: + +https://metacat.readthedocs.io/en/latest/index.html + +metacat is a `ups` (SL7) and `spack` (AL9) product so you can get it by + +(SL7) + +.. code-block:: bash + + setup metacat + +or (AL9) + +.. code-block:: bash + + spack load metacat + +but you can also do a local install using: + +https://metacat.readthedocs.io/en/latest/ui.html#installation + +Make certain you can point to the metacat server: + +.. code-block:: bash + + export METACAT_AUTH_SERVER_URL=https://metacat.fnal.gov:8143/auth/dune + export METACAT_SERVER_URL=https://metacat.fnal.gov:9443/dune_meta_prod/app + + +Then authenticate to metacat using your FNAL username and services password: + +.. code-block:: bash + + metacat auth login -m password $USER + Password: + User: schellma + Expires: Thu Oct 13 16:27:29 2022 + +See the `metacat documentation `_ for other authentication methods such as tokens. + + diff --git a/docs/source/template.json b/docs/source/template.json new file mode 100644 index 0000000..09449fa --- /dev/null +++ b/docs/source/template.json @@ -0,0 +1,43 @@ +{ + # these are autofilled + "checksums": { + "adler32": "0e94618d" + }, + "created_timestamp": 2018-10-10 17:38:27.681780+00:00, + "creator": "dunepro", + "fid": "6607416", + # + "metadata": { + "core.application": "art.dune-artdaq", + "core.application.family": "art", + "core.application.name": "dune-artdaq", + "core.application.version": "v1_16_00b", + "core.data_stream": "physics", + "core.data_tier": "raw", + "core.end_time": "2018-10-10 17:38:27.681780+00:00", + "core.event_count": 108, + "core.events": [1,2,3,4], + "core.file_content_status": "good", + "core.file_format": "hdf5", + "core.file_type": "detector", + "core.first_event_number": 20921, + "core.last_event_number": 22349, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": "2018-10-10 17:38:27.681780+00:00", + "core.raw_timestamp": "2018-10-10 17:38:27.681780+00:00", + }, + "name": "np04_raw_run005141_0015_dl3.root", + "namespace": "protodune-sp", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "size": 8289321123, + "updated_by": null, + "updated_timestamp": null +} diff --git a/docs/source/template.rst b/docs/source/template.rst new file mode 100644 index 0000000..a400dea --- /dev/null +++ b/docs/source/template.rst @@ -0,0 +1,106 @@ +Template for minimal metadata for a file +---------------------------------------- + +Here is an example of minimal metadata for a raw data file. + +Note that all keys must now be lower case. + +Times need to be UTC. + +You can add additional information to the metadata object. + +Metacat version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "name": "np04_raw_run005141_0015_dl3.root", + "namespace": "protodune-sp", + "checksums": { + "adler32": "0e94618d" + }, + "created_timestamp": 2018-10-10 17:38:27.681780+00:00, + "creator": "dunepro", + "size": 8289321123, + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "updated_by": null, + "updated_timestamp": null + "metadata": { + "core.application.family": "art", + "core.application.name": "dune-artdaq", + "core.application.version": "v1_16_00b", + "core.data_stream": "physics", + "core.data_tier": "raw", + "core.end_time": "2018-10-10 17:38:27.681780+00:00", + "core.event_count": 108, + "core.events": [1,2,3,4], + "core.file_content_status": "good", + "core.file_format": "hdf5", + "core.file_type": "detector", + "core.first_event_number": 20921, + "core.last_event_number": 22349, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": "2018-10-10 17:38:27.681780+00:00", + "retention.status": "active", + "retention.class": "physics" + }, + + } + +Additional information +^^^^^^^^^^^^^^^^^^^^^^ + + This information is added later to track changes + + .. code-block:: javascript + + {"retired": false, + "retired_by": null, + "retired_timestamp": null, + "updated_by": null, + "updated_timestamp": null + } + + + +Samweb version +^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "file_name": "filename", + "file_id": 6607416, + "create_date": "2018-10-10T17:38:27+00:00", + "user": "dunepro", + "file_size": 8289321123, + "checksum": [ "adler32:0e94618d"], + "content_status": "good", + "file_type": "detector", + "file_format": "root", + "data_tier": "raw", + "data_stream": "physics", + "application": { + "family": "art", + "name": "dune-artdaq", + "version": "v1_16_00b" + }, + "event_count": 108, + "first_event": 20921, + "last_event": 22349, + "start_time": "2018-10-10T17:25:01+00:00", + "end_time": "2018-10-10T17:32:16+00:00", + "runs": [ [ 5141, 1, "protodune-sp" ] ] + } + + + diff --git a/docs/source/template_input.rst b/docs/source/template_input.rst new file mode 100644 index 0000000..11049b6 --- /dev/null +++ b/docs/source/template_input.rst @@ -0,0 +1,165 @@ +Template for minimal metadata for a raw data file +------------------------------------------------- + +Here is an example of minimal metadata for a raw data file. +This has been revised to show what an input metacat-native +file should look like. + +Some fields are auto-genenerated and documented under additional informatiob below + +See the :ref:`glossary` for the definition of the keys. + +Note that all keys must now be lower case. + +Times need to be Float seconds.milliseconds past the epoch + +You can add additional information to the metadata object. +Any additional core.* attributes that are added require code changes to +the Declaration Daemon. +Any additional attributes added to other existing categories require notification +so we can back-declare them to SAM. +Any new categories require approval from admin. +ProtoDUNE NP04 can't generate core.application.* they are excused from doing that. +Minerva is allowed to add core.group, core.lum_block_ranges, misc.file_partition, online.triggerconfig, online.triggertype, core.dqc_quality, dune.mx2x2_global_run_numbers + +Metacat version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "name": "np04_raw_run005141_0015_dl3.root", + "namespace": "protodune-sp", + "checksums": { + "adler32": "0e94618d" + }, + "size": 8289321123, + "metadata": { + "core.application.family": "art", + "core.application.name": "dune-artdaq", + "core.application.version": "v1_16_00b", + "core.data_stream": "physics", + "core.data_tier": "raw", + "core.end_time": 1539193107.0, + "core.event_count": 108, + "core.events": [1,2,3,4], + "core.file_content_status": "good", + "core.file_format": "hdf5", + "core.file_type": "detector", + "core.first_event_number": 20921, + "core.last_event_number": 22349, + "core.run_type": "protodune-sp", + "core.runs": [ + 5141 + ], + "core.runs_subruns": [ + 514100001 + ], + "core.start_time": 1539193107.0, + "dune.daq_test": "True", + "retention.status": "active", + "retention.class": "physics" + }, + + } + +Minerva version +^^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "checksums": { + "adler32": "e854e667" + }, + "created_timestamp": 1691170423.961402, + "metadata": { + "core.data_stream": "pdstl", + "core.data_tier": "binary-raw", + "core.end_time": 1690992518, + "core.event_count": 100, + "core.file_format": "binary", + "core.file_type": "importedDetector", + "core.first_event_number": 4592941, + "core.group": "minerva", + "core.last_event_number": 4593040, + "core.lum_block_ranges": [ + [ + 4592941, + 4593040 + ] + ], + "core.run_type": "neardet-2x2-minerva", + "core.runs": [ + 40074 + ], + "core.runs_subruns": [ + 4007400001 + ], + "core.start_time": 1690992515, + "dune.daq_test": "True", + "misc.file_partition": 1, + "online.triggerconfig": "NOFILE", + "online.triggertype": "oneshot", + "retention.status": "active", + "retention.class": "commissioning" + }, + "name": "TS1_00040074_0001_pdstl_v09_2308021608_RawData.dat", + "namespace": "neardet-2x2-minerva", + "size": 10543808 + + } + +Additional information +^^^^^^^^^^^^^^^^^^^^^^ + + This information is added later to track changes + Values for all of these are added automatically at time of creation and + they can not be added at file creation time or modified via a json file. + + + .. code-block:: javascript + + {"created_timestamp": 1539193107.681780, + "creator": "dunepro", + "retired": false, + "retired_by": null, + "retired_timestamp": null, + "updated_by": null, + "updated_timestamp": null + } + + + +Samweb version +^^^^^^^^^^^^^^ + +.. code-block:: javascript + + { + "file_name": "filename", + "file_id": 6607416, + "create_date": "2018-10-10T17:38:27+00:00", + "user": "dunepro", + "file_size": 8289321123, + "checksum": [ "adler32:0e94618d"], + "content_status": "good", + "file_type": "detector", + "file_format": "root", + "data_tier": "raw", + "data_stream": "physics", + "application": { + "family": "art", + "name": "dune-artdaq", + "version": "v1_16_00b" + }, + "event_count": 108, + "first_event": 20921, + "last_event": 22349, + "start_time": "2018-10-10T17:25:01+00:00", + "end_time": "2018-10-10T17:32:16+00:00", + "runs": [ [ 5141, 1, "protodune-sp" ] ] + } + + + diff --git a/doxygen.cfg b/doxygen.cfg new file mode 100644 index 0000000..10ccaef --- /dev/null +++ b/doxygen.cfg @@ -0,0 +1,2639 @@ +# Doxyfile 1.9.2 + +# This file describes the settings to be used by the documentation system +# doxygen (www.doxygen.org) for a project. +# +# All text after a double hash (##) is considered a comment and is placed in +# front of the TAG it is preceding. +# +# All text after a single hash (#) is considered a comment and will be ignored. +# The format is: +# TAG = value [value, ...] +# For lists, items can also be appended using: +# TAG += value [value, ...] +# Values that contain spaces should be placed between quotes (\" \"). + +#--------------------------------------------------------------------------- +# Project related configuration options +#--------------------------------------------------------------------------- + +# This tag specifies the encoding used for all characters in the configuration +# file that follow. The default is UTF-8 which is also the encoding used for all +# text before the first occurrence of this tag. Doxygen uses libiconv (or the +# iconv built into libc) for the transcoding. See +# https://www.gnu.org/software/libiconv/ for the list of possible encodings. +# The default value is: UTF-8. + +DOXYFILE_ENCODING = UTF-8 + +# The PROJECT_NAME tag is a single word (or a sequence of words surrounded by +# double-quotes, unless you are using Doxywizard) that should identify the +# project for which the documentation is generated. This name is used in the +# title of most generated pages and in a few other places. +# The default value is: My Project. + +PROJECT_NAME = "data delivery logging" + +# The PROJECT_NUMBER tag can be used to enter a project or revision number. This +# could be handy for archiving the generated documentation or if some version +# control system is used. + +PROJECT_NUMBER = + +# Using the PROJECT_BRIEF tag one can provide an optional one line description +# for a project that appears at the top of each page and should give viewer a +# quick idea about the purpose of the project. Keep the description short. + +PROJECT_BRIEF = + +# With the PROJECT_LOGO tag one can specify a logo or an icon that is included +# in the documentation. The maximum height of the logo should not exceed 55 +# pixels and the maximum width should not exceed 200 pixels. Doxygen will copy +# the logo to the output directory. + +PROJECT_LOGO = + +# The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path +# into which the generated documentation will be written. If a relative path is +# entered, it will be relative to the location where doxygen was started. If +# left blank the current directory will be used. + +OUTPUT_DIRECTORY = docs + +# If the CREATE_SUBDIRS tag is set to YES then doxygen will create 4096 sub- +# directories (in 2 levels) under the output directory of each output format and +# will distribute the generated files over these directories. Enabling this +# option can be useful when feeding doxygen a huge amount of source files, where +# putting all generated files in the same directory would otherwise causes +# performance problems for the file system. +# The default value is: NO. + +CREATE_SUBDIRS = YES + +# If the ALLOW_UNICODE_NAMES tag is set to YES, doxygen will allow non-ASCII +# characters to appear in the names of generated files. If set to NO, non-ASCII +# characters will be escaped, for example _xE3_x81_x84 will be used for Unicode +# U+3044. +# The default value is: NO. + +ALLOW_UNICODE_NAMES = NO + +# The OUTPUT_LANGUAGE tag is used to specify the language in which all +# documentation generated by doxygen is written. Doxygen will use this +# information to generate all constant output in the proper language. +# Possible values are: Afrikaans, Arabic, Armenian, Brazilian, Catalan, Chinese, +# Chinese-Traditional, Croatian, Czech, Danish, Dutch, English (United States), +# Esperanto, Farsi (Persian), Finnish, French, German, Greek, Hungarian, +# Indonesian, Italian, Japanese, Japanese-en (Japanese with English messages), +# Korean, Korean-en (Korean with English messages), Latvian, Lithuanian, +# Macedonian, Norwegian, Persian (Farsi), Polish, Portuguese, Romanian, Russian, +# Serbian, Serbian-Cyrillic, Slovak, Slovene, Spanish, Swedish, Turkish, +# Ukrainian and Vietnamese. +# The default value is: English. + +OUTPUT_LANGUAGE = English + +# If the BRIEF_MEMBER_DESC tag is set to YES, doxygen will include brief member +# descriptions after the members that are listed in the file and class +# documentation (similar to Javadoc). Set to NO to disable this. +# The default value is: YES. + +BRIEF_MEMBER_DESC = YES + +# If the REPEAT_BRIEF tag is set to YES, doxygen will prepend the brief +# description of a member or function before the detailed description +# +# Note: If both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the +# brief descriptions will be completely suppressed. +# The default value is: YES. + +REPEAT_BRIEF = YES + +# This tag implements a quasi-intelligent brief description abbreviator that is +# used to form the text in various listings. Each string in this list, if found +# as the leading text of the brief description, will be stripped from the text +# and the result, after processing the whole list, is used as the annotated +# text. Otherwise, the brief description is used as-is. If left blank, the +# following values are used ($name is automatically replaced with the name of +# the entity):The $name class, The $name widget, The $name file, is, provides, +# specifies, contains, represents, a, an and the. + +ABBREVIATE_BRIEF = "The $name class" \ + "The $name widget" \ + "The $name file" \ + is \ + provides \ + specifies \ + contains \ + represents \ + a \ + an \ + the + +# If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then +# doxygen will generate a detailed section even if there is only a brief +# description. +# The default value is: NO. + +ALWAYS_DETAILED_SEC = NO + +# If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all +# inherited members of a class in the documentation of that class as if those +# members were ordinary class members. Constructors, destructors and assignment +# operators of the base classes will not be shown. +# The default value is: NO. + +INLINE_INHERITED_MEMB = NO + +# If the FULL_PATH_NAMES tag is set to YES, doxygen will prepend the full path +# before files name in the file list and in the header files. If set to NO the +# shortest path that makes the file name unique will be used +# The default value is: YES. + +FULL_PATH_NAMES = YES + +# The STRIP_FROM_PATH tag can be used to strip a user-defined part of the path. +# Stripping is only done if one of the specified strings matches the left-hand +# part of the path. The tag can be used to show relative paths in the file list. +# If left blank the directory from which doxygen is run is used as the path to +# strip. +# +# Note that you can specify absolute paths here, but also relative paths, which +# will be relative from the directory where doxygen is started. +# This tag requires that the tag FULL_PATH_NAMES is set to YES. + +STRIP_FROM_PATH = + +# The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the +# path mentioned in the documentation of a class, which tells the reader which +# header file to include in order to use a class. If left blank only the name of +# the header file containing the class definition is used. Otherwise one should +# specify the list of include paths that are normally passed to the compiler +# using the -I flag. + +STRIP_FROM_INC_PATH = + +# If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but +# less readable) file names. This can be useful is your file systems doesn't +# support long names like on DOS, Mac, or CD-ROM. +# The default value is: NO. + +SHORT_NAMES = NO + +# If the JAVADOC_AUTOBRIEF tag is set to YES then doxygen will interpret the +# first line (until the first dot) of a Javadoc-style comment as the brief +# description. If set to NO, the Javadoc-style will behave just like regular Qt- +# style comments (thus requiring an explicit @brief command for a brief +# description.) +# The default value is: NO. + +JAVADOC_AUTOBRIEF = NO + +# If the JAVADOC_BANNER tag is set to YES then doxygen will interpret a line +# such as +# /*************** +# as being the beginning of a Javadoc-style comment "banner". If set to NO, the +# Javadoc-style will behave just like regular comments and it will not be +# interpreted by doxygen. +# The default value is: NO. + +JAVADOC_BANNER = NO + +# If the QT_AUTOBRIEF tag is set to YES then doxygen will interpret the first +# line (until the first dot) of a Qt-style comment as the brief description. If +# set to NO, the Qt-style will behave just like regular Qt-style comments (thus +# requiring an explicit \brief command for a brief description.) +# The default value is: NO. + +QT_AUTOBRIEF = NO + +# The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make doxygen treat a +# multi-line C++ special comment block (i.e. a block of //! or /// comments) as +# a brief description. This used to be the default behavior. The new default is +# to treat a multi-line C++ comment block as a detailed description. Set this +# tag to YES if you prefer the old behavior instead. +# +# Note that setting this tag to YES also means that rational rose comments are +# not recognized any more. +# The default value is: NO. + +MULTILINE_CPP_IS_BRIEF = NO + +# By default Python docstrings are displayed as preformatted text and doxygen's +# special commands cannot be used. By setting PYTHON_DOCSTRING to NO the +# doxygen's special commands can be used and the contents of the docstring +# documentation blocks is shown as doxygen documentation. +# The default value is: YES. + +PYTHON_DOCSTRING = YES + +# If the INHERIT_DOCS tag is set to YES then an undocumented member inherits the +# documentation from any documented member that it re-implements. +# The default value is: YES. + +INHERIT_DOCS = YES + +# If the SEPARATE_MEMBER_PAGES tag is set to YES then doxygen will produce a new +# page for each member. If set to NO, the documentation of a member will be part +# of the file/class/namespace that contains it. +# The default value is: NO. + +SEPARATE_MEMBER_PAGES = NO + +# The TAB_SIZE tag can be used to set the number of spaces in a tab. Doxygen +# uses this value to replace tabs by spaces in code fragments. +# Minimum value: 1, maximum value: 16, default value: 4. + +TAB_SIZE = 4 + +# This tag can be used to specify a number of aliases that act as commands in +# the documentation. An alias has the form: +# name=value +# For example adding +# "sideeffect=@par Side Effects:^^" +# will allow you to put the command \sideeffect (or @sideeffect) in the +# documentation, which will result in a user-defined paragraph with heading +# "Side Effects:". Note that you cannot put \n's in the value part of an alias +# to insert newlines (in the resulting output). You can put ^^ in the value part +# of an alias to insert a newline as if a physical newline was in the original +# file. When you need a literal { or } or , in the value part of an alias you +# have to escape them by means of a backslash (\), this can lead to conflicts +# with the commands \{ and \} for these it is advised to use the version @{ and +# @} or use a double escape (\\{ and \\}) + +ALIASES = + +# Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources +# only. Doxygen will then generate output that is more tailored for C. For +# instance, some of the names that are used will be different. The list of all +# members will be omitted, etc. +# The default value is: NO. + +OPTIMIZE_OUTPUT_FOR_C = NO + +# Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or +# Python sources only. Doxygen will then generate output that is more tailored +# for that language. For instance, namespaces will be presented as packages, +# qualified scopes will look different, etc. +# The default value is: NO. + +OPTIMIZE_OUTPUT_JAVA = NO + +# Set the OPTIMIZE_FOR_FORTRAN tag to YES if your project consists of Fortran +# sources. Doxygen will then generate output that is tailored for Fortran. +# The default value is: NO. + +OPTIMIZE_FOR_FORTRAN = NO + +# Set the OPTIMIZE_OUTPUT_VHDL tag to YES if your project consists of VHDL +# sources. Doxygen will then generate output that is tailored for VHDL. +# The default value is: NO. + +OPTIMIZE_OUTPUT_VHDL = NO + +# Set the OPTIMIZE_OUTPUT_SLICE tag to YES if your project consists of Slice +# sources only. Doxygen will then generate output that is more tailored for that +# language. For instance, namespaces will be presented as modules, types will be +# separated into more groups, etc. +# The default value is: NO. + +OPTIMIZE_OUTPUT_SLICE = NO + +# Doxygen selects the parser to use depending on the extension of the files it +# parses. With this tag you can assign which parser to use for a given +# extension. Doxygen has a built-in mapping, but you can override or extend it +# using this tag. The format is ext=language, where ext is a file extension, and +# language is one of the parsers supported by doxygen: IDL, Java, JavaScript, +# Csharp (C#), C, C++, Lex, D, PHP, md (Markdown), Objective-C, Python, Slice, +# VHDL, Fortran (fixed format Fortran: FortranFixed, free formatted Fortran: +# FortranFree, unknown formatted Fortran: Fortran. In the later case the parser +# tries to guess whether the code is fixed or free formatted code, this is the +# default for Fortran type files). For instance to make doxygen treat .inc files +# as Fortran files (default is PHP), and .f files as C (default is Fortran), +# use: inc=Fortran f=C. +# +# Note: For files without extension you can use no_extension as a placeholder. +# +# Note that for custom extensions you also need to set FILE_PATTERNS otherwise +# the files are not read by doxygen. When specifying no_extension you should add +# * to the FILE_PATTERNS. +# +# Note see also the list of default file extension mappings. + +EXTENSION_MAPPING = + +# If the MARKDOWN_SUPPORT tag is enabled then doxygen pre-processes all comments +# according to the Markdown format, which allows for more readable +# documentation. See https://daringfireball.net/projects/markdown/ for details. +# The output of markdown processing is further processed by doxygen, so you can +# mix doxygen, HTML, and XML commands with Markdown formatting. Disable only in +# case of backward compatibilities issues. +# The default value is: YES. + +MARKDOWN_SUPPORT = YES + +# When the TOC_INCLUDE_HEADINGS tag is set to a non-zero value, all headings up +# to that level are automatically included in the table of contents, even if +# they do not have an id attribute. +# Note: This feature currently applies only to Markdown headings. +# Minimum value: 0, maximum value: 99, default value: 5. +# This tag requires that the tag MARKDOWN_SUPPORT is set to YES. + +TOC_INCLUDE_HEADINGS = 5 + +# When enabled doxygen tries to link words that correspond to documented +# classes, or namespaces to their corresponding documentation. Such a link can +# be prevented in individual cases by putting a % sign in front of the word or +# globally by setting AUTOLINK_SUPPORT to NO. +# The default value is: YES. + +AUTOLINK_SUPPORT = YES + +# If you use STL classes (i.e. std::string, std::vector, etc.) but do not want +# to include (a tag file for) the STL sources as input, then you should set this +# tag to YES in order to let doxygen match functions declarations and +# definitions whose arguments contain STL classes (e.g. func(std::string); +# versus func(std::string) {}). This also make the inheritance and collaboration +# diagrams that involve STL classes more complete and accurate. +# The default value is: NO. + +BUILTIN_STL_SUPPORT = NO + +# If you use Microsoft's C++/CLI language, you should set this option to YES to +# enable parsing support. +# The default value is: NO. + +CPP_CLI_SUPPORT = NO + +# Set the SIP_SUPPORT tag to YES if your project consists of sip (see: +# https://www.riverbankcomputing.com/software/sip/intro) sources only. Doxygen +# will parse them like normal C++ but will assume all classes use public instead +# of private inheritance when no explicit protection keyword is present. +# The default value is: NO. + +SIP_SUPPORT = NO + +# For Microsoft's IDL there are propget and propput attributes to indicate +# getter and setter methods for a property. Setting this option to YES will make +# doxygen to replace the get and set methods by a property in the documentation. +# This will only work if the methods are indeed getting or setting a simple +# type. If this is not the case, or you want to show the methods anyway, you +# should set this option to NO. +# The default value is: YES. + +IDL_PROPERTY_SUPPORT = YES + +# If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC +# tag is set to YES then doxygen will reuse the documentation of the first +# member in the group (if any) for the other members of the group. By default +# all members of a group must be documented explicitly. +# The default value is: NO. + +DISTRIBUTE_GROUP_DOC = NO + +# If one adds a struct or class to a group and this option is enabled, then also +# any nested class or struct is added to the same group. By default this option +# is disabled and one has to add nested compounds explicitly via \ingroup. +# The default value is: NO. + +GROUP_NESTED_COMPOUNDS = NO + +# Set the SUBGROUPING tag to YES to allow class member groups of the same type +# (for instance a group of public functions) to be put as a subgroup of that +# type (e.g. under the Public Functions section). Set it to NO to prevent +# subgrouping. Alternatively, this can be done per class using the +# \nosubgrouping command. +# The default value is: YES. + +SUBGROUPING = YES + +# When the INLINE_GROUPED_CLASSES tag is set to YES, classes, structs and unions +# are shown inside the group in which they are included (e.g. using \ingroup) +# instead of on a separate page (for HTML and Man pages) or section (for LaTeX +# and RTF). +# +# Note that this feature does not work in combination with +# SEPARATE_MEMBER_PAGES. +# The default value is: NO. + +INLINE_GROUPED_CLASSES = NO + +# When the INLINE_SIMPLE_STRUCTS tag is set to YES, structs, classes, and unions +# with only public data fields or simple typedef fields will be shown inline in +# the documentation of the scope in which they are defined (i.e. file, +# namespace, or group documentation), provided this scope is documented. If set +# to NO, structs, classes, and unions are shown on a separate page (for HTML and +# Man pages) or section (for LaTeX and RTF). +# The default value is: NO. + +INLINE_SIMPLE_STRUCTS = NO + +# When TYPEDEF_HIDES_STRUCT tag is enabled, a typedef of a struct, union, or +# enum is documented as struct, union, or enum with the name of the typedef. So +# typedef struct TypeS {} TypeT, will appear in the documentation as a struct +# with name TypeT. When disabled the typedef will appear as a member of a file, +# namespace, or class. And the struct will be named TypeS. This can typically be +# useful for C code in case the coding convention dictates that all compound +# types are typedef'ed and only the typedef is referenced, never the tag name. +# The default value is: NO. + +TYPEDEF_HIDES_STRUCT = NO + +# The size of the symbol lookup cache can be set using LOOKUP_CACHE_SIZE. This +# cache is used to resolve symbols given their name and scope. Since this can be +# an expensive process and often the same symbol appears multiple times in the +# code, doxygen keeps a cache of pre-resolved symbols. If the cache is too small +# doxygen will become slower. If the cache is too large, memory is wasted. The +# cache size is given by this formula: 2^(16+LOOKUP_CACHE_SIZE). The valid range +# is 0..9, the default is 0, corresponding to a cache size of 2^16=65536 +# symbols. At the end of a run doxygen will report the cache usage and suggest +# the optimal cache size from a speed point of view. +# Minimum value: 0, maximum value: 9, default value: 0. + +LOOKUP_CACHE_SIZE = 0 + +# The NUM_PROC_THREADS specifies the number threads doxygen is allowed to use +# during processing. When set to 0 doxygen will based this on the number of +# cores available in the system. You can set it explicitly to a value larger +# than 0 to get more control over the balance between CPU load and processing +# speed. At this moment only the input processing can be done using multiple +# threads. Since this is still an experimental feature the default is set to 1, +# which effectively disables parallel processing. Please report any issues you +# encounter. Generating dot graphs in parallel is controlled by the +# DOT_NUM_THREADS setting. +# Minimum value: 0, maximum value: 32, default value: 1. + +NUM_PROC_THREADS = 1 + +#--------------------------------------------------------------------------- +# Build related configuration options +#--------------------------------------------------------------------------- + +# If the EXTRACT_ALL tag is set to YES, doxygen will assume all entities in +# documentation are documented, even if no documentation was available. Private +# class members and static file members will be hidden unless the +# EXTRACT_PRIVATE respectively EXTRACT_STATIC tags are set to YES. +# Note: This will also disable the warnings about undocumented members that are +# normally produced when WARNINGS is set to YES. +# The default value is: NO. + +#HMS +EXTRACT_ALL = YES + +# If the EXTRACT_PRIVATE tag is set to YES, all private members of a class will +# be included in the documentation. +# The default value is: NO. + +EXTRACT_PRIVATE = NO + +# If the EXTRACT_PRIV_VIRTUAL tag is set to YES, documented private virtual +# methods of a class will be included in the documentation. +# The default value is: NO. + +EXTRACT_PRIV_VIRTUAL = NO + +# If the EXTRACT_PACKAGE tag is set to YES, all members with package or internal +# scope will be included in the documentation. +# The default value is: NO. + +EXTRACT_PACKAGE = NO + +# If the EXTRACT_STATIC tag is set to YES, all static members of a file will be +# included in the documentation. +# The default value is: NO. + +EXTRACT_STATIC = NO + +# If the EXTRACT_LOCAL_CLASSES tag is set to YES, classes (and structs) defined +# locally in source files will be included in the documentation. If set to NO, +# only classes defined in header files are included. Does not have any effect +# for Java sources. +# The default value is: YES. + +EXTRACT_LOCAL_CLASSES = YES + +# This flag is only useful for Objective-C code. If set to YES, local methods, +# which are defined in the implementation section but not in the interface are +# included in the documentation. If set to NO, only methods in the interface are +# included. +# The default value is: NO. + +EXTRACT_LOCAL_METHODS = NO + +# If this flag is set to YES, the members of anonymous namespaces will be +# extracted and appear in the documentation as a namespace called +# 'anonymous_namespace{file}', where file will be replaced with the base name of +# the file that contains the anonymous namespace. By default anonymous namespace +# are hidden. +# The default value is: NO. + +EXTRACT_ANON_NSPACES = NO + +# If this flag is set to YES, the name of an unnamed parameter in a declaration +# will be determined by the corresponding definition. By default unnamed +# parameters remain unnamed in the output. +# The default value is: YES. + +RESOLVE_UNNAMED_PARAMS = YES + +# If the HIDE_UNDOC_MEMBERS tag is set to YES, doxygen will hide all +# undocumented members inside documented classes or files. If set to NO these +# members will be included in the various overviews, but no documentation +# section is generated. This option has no effect if EXTRACT_ALL is enabled. +# The default value is: NO. + +HIDE_UNDOC_MEMBERS = NO + +# If the HIDE_UNDOC_CLASSES tag is set to YES, doxygen will hide all +# undocumented classes that are normally visible in the class hierarchy. If set +# to NO, these classes will be included in the various overviews. This option +# has no effect if EXTRACT_ALL is enabled. +# The default value is: NO. + +HIDE_UNDOC_CLASSES = NO + +# If the HIDE_FRIEND_COMPOUNDS tag is set to YES, doxygen will hide all friend +# declarations. If set to NO, these declarations will be included in the +# documentation. +# The default value is: NO. + +HIDE_FRIEND_COMPOUNDS = NO + +# If the HIDE_IN_BODY_DOCS tag is set to YES, doxygen will hide any +# documentation blocks found inside the body of a function. If set to NO, these +# blocks will be appended to the function's detailed documentation block. +# The default value is: NO. + +HIDE_IN_BODY_DOCS = NO + +# The INTERNAL_DOCS tag determines if documentation that is typed after a +# \internal command is included. If the tag is set to NO then the documentation +# will be excluded. Set it to YES to include the internal documentation. +# The default value is: NO. + +INTERNAL_DOCS = NO + +# With the correct setting of option CASE_SENSE_NAMES doxygen will better be +# able to match the capabilities of the underlying filesystem. In case the +# filesystem is case sensitive (i.e. it supports files in the same directory +# whose names only differ in casing), the option must be set to YES to properly +# deal with such files in case they appear in the input. For filesystems that +# are not case sensitive the option should be be set to NO to properly deal with +# output files written for symbols that only differ in casing, such as for two +# classes, one named CLASS and the other named Class, and to also support +# references to files without having to specify the exact matching casing. On +# Windows (including Cygwin) and MacOS, users should typically set this option +# to NO, whereas on Linux or other Unix flavors it should typically be set to +# YES. +# The default value is: system dependent. + +CASE_SENSE_NAMES = NO + +# If the HIDE_SCOPE_NAMES tag is set to NO then doxygen will show members with +# their full class and namespace scopes in the documentation. If set to YES, the +# scope will be hidden. +# The default value is: NO. + +HIDE_SCOPE_NAMES = NO + +# If the HIDE_COMPOUND_REFERENCE tag is set to NO (default) then doxygen will +# append additional text to a page's title, such as Class Reference. If set to +# YES the compound reference will be hidden. +# The default value is: NO. + +HIDE_COMPOUND_REFERENCE= NO + +# If the SHOW_HEADERFILE tag is set to YES then the documentation for a class +# will show which file needs to be included to use the class. +# The default value is: YES. + +SHOW_HEADERFILE = YES + +# If the SHOW_INCLUDE_FILES tag is set to YES then doxygen will put a list of +# the files that are included by a file in the documentation of that file. +# The default value is: YES. + +SHOW_INCLUDE_FILES = YES + +# If the SHOW_GROUPED_MEMB_INC tag is set to YES then Doxygen will add for each +# grouped member an include statement to the documentation, telling the reader +# which file to include in order to use the member. +# The default value is: NO. + +SHOW_GROUPED_MEMB_INC = NO + +# If the FORCE_LOCAL_INCLUDES tag is set to YES then doxygen will list include +# files with double quotes in the documentation rather than with sharp brackets. +# The default value is: NO. + +FORCE_LOCAL_INCLUDES = NO + +# If the INLINE_INFO tag is set to YES then a tag [inline] is inserted in the +# documentation for inline members. +# The default value is: YES. + +INLINE_INFO = YES + +# If the SORT_MEMBER_DOCS tag is set to YES then doxygen will sort the +# (detailed) documentation of file and class members alphabetically by member +# name. If set to NO, the members will appear in declaration order. +# The default value is: YES. + +SORT_MEMBER_DOCS = YES + +# If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the brief +# descriptions of file, namespace and class members alphabetically by member +# name. If set to NO, the members will appear in declaration order. Note that +# this will also influence the order of the classes in the class list. +# The default value is: NO. + +SORT_BRIEF_DOCS = NO + +# If the SORT_MEMBERS_CTORS_1ST tag is set to YES then doxygen will sort the +# (brief and detailed) documentation of class members so that constructors and +# destructors are listed first. If set to NO the constructors will appear in the +# respective orders defined by SORT_BRIEF_DOCS and SORT_MEMBER_DOCS. +# Note: If SORT_BRIEF_DOCS is set to NO this option is ignored for sorting brief +# member documentation. +# Note: If SORT_MEMBER_DOCS is set to NO this option is ignored for sorting +# detailed member documentation. +# The default value is: NO. + +SORT_MEMBERS_CTORS_1ST = NO + +# If the SORT_GROUP_NAMES tag is set to YES then doxygen will sort the hierarchy +# of group names into alphabetical order. If set to NO the group names will +# appear in their defined order. +# The default value is: NO. + +SORT_GROUP_NAMES = NO + +# If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be sorted by +# fully-qualified names, including namespaces. If set to NO, the class list will +# be sorted only by class name, not including the namespace part. +# Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES. +# Note: This option applies only to the class list, not to the alphabetical +# list. +# The default value is: NO. + +SORT_BY_SCOPE_NAME = NO + +# If the STRICT_PROTO_MATCHING option is enabled and doxygen fails to do proper +# type resolution of all parameters of a function it will reject a match between +# the prototype and the implementation of a member function even if there is +# only one candidate or it is obvious which candidate to choose by doing a +# simple string match. By disabling STRICT_PROTO_MATCHING doxygen will still +# accept a match between prototype and implementation in such cases. +# The default value is: NO. + +STRICT_PROTO_MATCHING = NO + +# The GENERATE_TODOLIST tag can be used to enable (YES) or disable (NO) the todo +# list. This list is created by putting \todo commands in the documentation. +# The default value is: YES. + +GENERATE_TODOLIST = YES + +# The GENERATE_TESTLIST tag can be used to enable (YES) or disable (NO) the test +# list. This list is created by putting \test commands in the documentation. +# The default value is: YES. + +GENERATE_TESTLIST = YES + +# The GENERATE_BUGLIST tag can be used to enable (YES) or disable (NO) the bug +# list. This list is created by putting \bug commands in the documentation. +# The default value is: YES. + +GENERATE_BUGLIST = YES + +# The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or disable (NO) +# the deprecated list. This list is created by putting \deprecated commands in +# the documentation. +# The default value is: YES. + +GENERATE_DEPRECATEDLIST= YES + +# The ENABLED_SECTIONS tag can be used to enable conditional documentation +# sections, marked by \if ... \endif and \cond +# ... \endcond blocks. + +ENABLED_SECTIONS = + +# The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the +# initial value of a variable or macro / define can have for it to appear in the +# documentation. If the initializer consists of more lines than specified here +# it will be hidden. Use a value of 0 to hide initializers completely. The +# appearance of the value of individual variables and macros / defines can be +# controlled using \showinitializer or \hideinitializer command in the +# documentation regardless of this setting. +# Minimum value: 0, maximum value: 10000, default value: 30. + +MAX_INITIALIZER_LINES = 30 + +# Set the SHOW_USED_FILES tag to NO to disable the list of files generated at +# the bottom of the documentation of classes and structs. If set to YES, the +# list will mention the files that were used to generate the documentation. +# The default value is: YES. + +SHOW_USED_FILES = YES + +# Set the SHOW_FILES tag to NO to disable the generation of the Files page. This +# will remove the Files entry from the Quick Index and from the Folder Tree View +# (if specified). +# The default value is: YES. + +SHOW_FILES = YES + +# Set the SHOW_NAMESPACES tag to NO to disable the generation of the Namespaces +# page. This will remove the Namespaces entry from the Quick Index and from the +# Folder Tree View (if specified). +# The default value is: YES. + +SHOW_NAMESPACES = YES + +# The FILE_VERSION_FILTER tag can be used to specify a program or script that +# doxygen should invoke to get the current version for each file (typically from +# the version control system). Doxygen will invoke the program by executing (via +# popen()) the command command input-file, where command is the value of the +# FILE_VERSION_FILTER tag, and input-file is the name of an input file provided +# by doxygen. Whatever the program writes to standard output is used as the file +# version. For an example see the documentation. + +FILE_VERSION_FILTER = + +# The LAYOUT_FILE tag can be used to specify a layout file which will be parsed +# by doxygen. The layout file controls the global structure of the generated +# output files in an output format independent way. To create the layout file +# that represents doxygen's defaults, run doxygen with the -l option. You can +# optionally specify a file name after the option, if omitted DoxygenLayout.xml +# will be used as the name of the layout file. See also section "Changing the +# layout of pages" for information. +# +# Note that if you run doxygen from a directory containing a file called +# DoxygenLayout.xml, doxygen will parse it automatically even if the LAYOUT_FILE +# tag is left empty. + +LAYOUT_FILE = + +# The CITE_BIB_FILES tag can be used to specify one or more bib files containing +# the reference definitions. This must be a list of .bib files. The .bib +# extension is automatically appended if omitted. This requires the bibtex tool +# to be installed. See also https://en.wikipedia.org/wiki/BibTeX for more info. +# For LaTeX the style of the bibliography can be controlled using +# LATEX_BIB_STYLE. To use this feature you need bibtex and perl available in the +# search path. See also \cite for info how to create references. + +CITE_BIB_FILES = + +#--------------------------------------------------------------------------- +# Configuration options related to warning and progress messages +#--------------------------------------------------------------------------- + +# The QUIET tag can be used to turn on/off the messages that are generated to +# standard output by doxygen. If QUIET is set to YES this implies that the +# messages are off. +# The default value is: NO. + +QUIET = NO + +# The WARNINGS tag can be used to turn on/off the warning messages that are +# generated to standard error (stderr) by doxygen. If WARNINGS is set to YES +# this implies that the warnings are on. +# +# Tip: Turn warnings on while writing the documentation. +# The default value is: YES. + +WARNINGS = YES + +# If the WARN_IF_UNDOCUMENTED tag is set to YES then doxygen will generate +# warnings for undocumented members. If EXTRACT_ALL is set to YES then this flag +# will automatically be disabled. +# The default value is: YES. + +WARN_IF_UNDOCUMENTED = YES + +# If the WARN_IF_DOC_ERROR tag is set to YES, doxygen will generate warnings for +# potential errors in the documentation, such as documenting some parameters in +# a documented function twice, or documenting parameters that don't exist or +# using markup commands wrongly. +# The default value is: YES. + +WARN_IF_DOC_ERROR = YES + +# If WARN_IF_INCOMPLETE_DOC is set to YES, doxygen will warn about incomplete +# function parameter documentation. If set to NO, doxygen will accept that some +# parameters have no documentation without warning. +# The default value is: YES. + +WARN_IF_INCOMPLETE_DOC = YES + +# This WARN_NO_PARAMDOC option can be enabled to get warnings for functions that +# are documented, but have no documentation for their parameters or return +# value. If set to NO, doxygen will only warn about wrong parameter +# documentation, but not about the absence of documentation. If EXTRACT_ALL is +# set to YES then this flag will automatically be disabled. See also +# WARN_IF_INCOMPLETE_DOC +# The default value is: NO. + +WARN_NO_PARAMDOC = NO + +# If the WARN_AS_ERROR tag is set to YES then doxygen will immediately stop when +# a warning is encountered. If the WARN_AS_ERROR tag is set to FAIL_ON_WARNINGS +# then doxygen will continue running as if WARN_AS_ERROR tag is set to NO, but +# at the end of the doxygen process doxygen will return with a non-zero status. +# Possible values are: NO, YES and FAIL_ON_WARNINGS. +# The default value is: NO. + +WARN_AS_ERROR = NO + +# The WARN_FORMAT tag determines the format of the warning messages that doxygen +# can produce. The string should contain the $file, $line, and $text tags, which +# will be replaced by the file and line number from which the warning originated +# and the warning text. Optionally the format may contain $version, which will +# be replaced by the version of the file (if it could be obtained via +# FILE_VERSION_FILTER) +# The default value is: $file:$line: $text. + +WARN_FORMAT = "$file:$line: $text" + +# The WARN_LOGFILE tag can be used to specify a file to which warning and error +# messages should be written. If left blank the output is written to standard +# error (stderr). + +WARN_LOGFILE = + +#--------------------------------------------------------------------------- +# Configuration options related to the input files +#--------------------------------------------------------------------------- + +# The INPUT tag is used to specify the files and/or directories that contain +# documented source files. You may enter file names like myfile.cpp or +# directories like /usr/src/myproject. Separate the files or directories with +# spaces. See also FILE_PATTERNS and EXTENSION_MAPPING +# Note: If this tag is empty the current directory is searched. + +INPUT = . + +# This tag can be used to specify the character encoding of the source files +# that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses +# libiconv (or the iconv built into libc) for the transcoding. See the libiconv +# documentation (see: +# https://www.gnu.org/software/libiconv/) for the list of possible encodings. +# The default value is: UTF-8. + +INPUT_ENCODING = UTF-8 + +# If the value of the INPUT tag contains directories, you can use the +# FILE_PATTERNS tag to specify one or more wildcard patterns (like *.cpp and +# *.h) to filter out the source-files in the directories. +# +# Note that for custom extensions or not directly supported extensions you also +# need to set EXTENSION_MAPPING for the extension otherwise the files are not +# read by doxygen. +# +# Note the list of default checked file patterns might differ from the list of +# default file extension mappings. +# +# If left blank the following patterns are tested:*.c, *.cc, *.cxx, *.cpp, +# *.c++, *.java, *.ii, *.ixx, *.ipp, *.i++, *.inl, *.idl, *.ddl, *.odl, *.h, +# *.hh, *.hxx, *.hpp, *.h++, *.l, *.cs, *.d, *.php, *.php4, *.php5, *.phtml, +# *.inc, *.m, *.markdown, *.md, *.mm, *.dox (to be provided as doxygen C +# comment), *.py, *.pyw, *.f90, *.f95, *.f03, *.f08, *.f18, *.f, *.for, *.vhd, +# *.vhdl, *.ucf, *.qsf and *.ice. + +FILE_PATTERNS = *.c \ + *.cc \ + *.cxx \ + *.cpp \ + *.c++ \ + *.java \ + *.ii \ + *.ixx \ + *.ipp \ + *.i++ \ + *.inl \ + *.idl \ + *.ddl \ + *.odl \ + *.h \ + *.hh \ + *.hxx \ + *.hpp \ + *.h++ \ + *.l \ + *.cs \ + *.d \ + *.php \ + *.php4 \ + *.php5 \ + *.phtml \ + *.inc \ + *.m \ + *.markdown \ + *.md \ + *.mm \ + *.dox \ + *.py \ + *.pyw \ + *.f90 \ + *.f95 \ + *.f03 \ + *.f08 \ + *.f18 \ + *.f \ + *.for \ + *.vhd \ + *.vhdl \ + *.ucf \ + *.qsf \ + *.ice + +# The RECURSIVE tag can be used to specify whether or not subdirectories should +# be searched for input files as well. +# The default value is: NO. + +RECURSIVE = YES + +# The EXCLUDE tag can be used to specify files and/or directories that should be +# excluded from the INPUT source files. This way you can easily exclude a +# subdirectory from a directory tree whose root is specified with the INPUT tag. +# +# Note that relative paths are relative to the directory from which doxygen is +# run. + +EXCLUDE = + +# The EXCLUDE_SYMLINKS tag can be used to select whether or not files or +# directories that are symbolic links (a Unix file system feature) are excluded +# from the input. +# The default value is: NO. + +EXCLUDE_SYMLINKS = NO + +# If the value of the INPUT tag contains directories, you can use the +# EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude +# certain files from those directories. +# +# Note that the wildcards are matched against the file with absolute path, so to +# exclude all test directories for example use the pattern */test/* + +EXCLUDE_PATTERNS = + +# The EXCLUDE_SYMBOLS tag can be used to specify one or more symbol names +# (namespaces, classes, functions, etc.) that should be excluded from the +# output. The symbol name can be a fully qualified name, a word, or if the +# wildcard * is used, a substring. Examples: ANamespace, AClass, +# AClass::ANamespace, ANamespace::*Test +# +# Note that the wildcards are matched against the file with absolute path, so to +# exclude all test directories use the pattern */test/* + +EXCLUDE_SYMBOLS = + +# The EXAMPLE_PATH tag can be used to specify one or more files or directories +# that contain example code fragments that are included (see the \include +# command). + +EXAMPLE_PATH = + +# If the value of the EXAMPLE_PATH tag contains directories, you can use the +# EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp and +# *.h) to filter out the source-files in the directories. If left blank all +# files are included. + +EXAMPLE_PATTERNS = * + +# If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be +# searched for input files to be used with the \include or \dontinclude commands +# irrespective of the value of the RECURSIVE tag. +# The default value is: NO. + +EXAMPLE_RECURSIVE = NO + +# The IMAGE_PATH tag can be used to specify one or more files or directories +# that contain images that are to be included in the documentation (see the +# \image command). + +IMAGE_PATH = + +# The INPUT_FILTER tag can be used to specify a program that doxygen should +# invoke to filter for each input file. Doxygen will invoke the filter program +# by executing (via popen()) the command: +# +# +# +# where is the value of the INPUT_FILTER tag, and is the +# name of an input file. Doxygen will then use the output that the filter +# program writes to standard output. If FILTER_PATTERNS is specified, this tag +# will be ignored. +# +# Note that the filter must not add or remove lines; it is applied before the +# code is scanned, but not when the output code is generated. If lines are added +# or removed, the anchors will not be placed correctly. +# +# Note that for custom extensions or not directly supported extensions you also +# need to set EXTENSION_MAPPING for the extension otherwise the files are not +# properly processed by doxygen. + +INPUT_FILTER = + +# The FILTER_PATTERNS tag can be used to specify filters on a per file pattern +# basis. Doxygen will compare the file name with each pattern and apply the +# filter if there is a match. The filters are a list of the form: pattern=filter +# (like *.cpp=my_cpp_filter). See INPUT_FILTER for further information on how +# filters are used. If the FILTER_PATTERNS tag is empty or if none of the +# patterns match the file name, INPUT_FILTER is applied. +# +# Note that for custom extensions or not directly supported extensions you also +# need to set EXTENSION_MAPPING for the extension otherwise the files are not +# properly processed by doxygen. + +FILTER_PATTERNS = + +# If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using +# INPUT_FILTER) will also be used to filter the input files that are used for +# producing the source files to browse (i.e. when SOURCE_BROWSER is set to YES). +# The default value is: NO. + +FILTER_SOURCE_FILES = NO + +# The FILTER_SOURCE_PATTERNS tag can be used to specify source filters per file +# pattern. A pattern will override the setting for FILTER_PATTERN (if any) and +# it is also possible to disable source filtering for a specific pattern using +# *.ext= (so without naming a filter). +# This tag requires that the tag FILTER_SOURCE_FILES is set to YES. + +FILTER_SOURCE_PATTERNS = + +# If the USE_MDFILE_AS_MAINPAGE tag refers to the name of a markdown file that +# is part of the input, its contents will be placed on the main page +# (index.html). This can be useful if you have a project on for instance GitHub +# and want to reuse the introduction page also for the doxygen output. + +USE_MDFILE_AS_MAINPAGE = + +#--------------------------------------------------------------------------- +# Configuration options related to source browsing +#--------------------------------------------------------------------------- + +# If the SOURCE_BROWSER tag is set to YES then a list of source files will be +# generated. Documented entities will be cross-referenced with these sources. +# +# Note: To get rid of all source code in the generated output, make sure that +# also VERBATIM_HEADERS is set to NO. +# The default value is: NO. + +SOURCE_BROWSER = NO + +# Setting the INLINE_SOURCES tag to YES will include the body of functions, +# classes and enums directly into the documentation. +# The default value is: NO. + +INLINE_SOURCES = NO + +# Setting the STRIP_CODE_COMMENTS tag to YES will instruct doxygen to hide any +# special comment blocks from generated source code fragments. Normal C, C++ and +# Fortran comments will always remain visible. +# The default value is: YES. + +STRIP_CODE_COMMENTS = YES + +# If the REFERENCED_BY_RELATION tag is set to YES then for each documented +# entity all documented functions referencing it will be listed. +# The default value is: NO. + +REFERENCED_BY_RELATION = NO + +# If the REFERENCES_RELATION tag is set to YES then for each documented function +# all documented entities called/used by that function will be listed. +# The default value is: NO. + +REFERENCES_RELATION = NO + +# If the REFERENCES_LINK_SOURCE tag is set to YES and SOURCE_BROWSER tag is set +# to YES then the hyperlinks from functions in REFERENCES_RELATION and +# REFERENCED_BY_RELATION lists will link to the source code. Otherwise they will +# link to the documentation. +# The default value is: YES. + +REFERENCES_LINK_SOURCE = YES + +# If SOURCE_TOOLTIPS is enabled (the default) then hovering a hyperlink in the +# source code will show a tooltip with additional information such as prototype, +# brief description and links to the definition and documentation. Since this +# will make the HTML file larger and loading of large files a bit slower, you +# can opt to disable this feature. +# The default value is: YES. +# This tag requires that the tag SOURCE_BROWSER is set to YES. + +SOURCE_TOOLTIPS = YES + +# If the USE_HTAGS tag is set to YES then the references to source code will +# point to the HTML generated by the htags(1) tool instead of doxygen built-in +# source browser. The htags tool is part of GNU's global source tagging system +# (see https://www.gnu.org/software/global/global.html). You will need version +# 4.8.6 or higher. +# +# To use it do the following: +# - Install the latest version of global +# - Enable SOURCE_BROWSER and USE_HTAGS in the configuration file +# - Make sure the INPUT points to the root of the source tree +# - Run doxygen as normal +# +# Doxygen will invoke htags (and that will in turn invoke gtags), so these +# tools must be available from the command line (i.e. in the search path). +# +# The result: instead of the source browser generated by doxygen, the links to +# source code will now point to the output of htags. +# The default value is: NO. +# This tag requires that the tag SOURCE_BROWSER is set to YES. + +USE_HTAGS = NO + +# If the VERBATIM_HEADERS tag is set the YES then doxygen will generate a +# verbatim copy of the header file for each class for which an include is +# specified. Set to NO to disable this. +# See also: Section \class. +# The default value is: YES. + +VERBATIM_HEADERS = YES + +#--------------------------------------------------------------------------- +# Configuration options related to the alphabetical class index +#--------------------------------------------------------------------------- + +# If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index of all +# compounds will be generated. Enable this if the project contains a lot of +# classes, structs, unions or interfaces. +# The default value is: YES. + +ALPHABETICAL_INDEX = YES + +# In case all classes in a project start with a common prefix, all classes will +# be put under the same header in the alphabetical index. The IGNORE_PREFIX tag +# can be used to specify a prefix (or a list of prefixes) that should be ignored +# while generating the index headers. +# This tag requires that the tag ALPHABETICAL_INDEX is set to YES. + +IGNORE_PREFIX = + +#--------------------------------------------------------------------------- +# Configuration options related to the HTML output +#--------------------------------------------------------------------------- + +# If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output +# The default value is: YES. + +GENERATE_HTML = YES + +# The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a +# relative path is entered the value of OUTPUT_DIRECTORY will be put in front of +# it. +# The default directory is: html. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_OUTPUT = html + +# The HTML_FILE_EXTENSION tag can be used to specify the file extension for each +# generated HTML page (for example: .htm, .php, .asp). +# The default value is: .html. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_FILE_EXTENSION = .html + +# The HTML_HEADER tag can be used to specify a user-defined HTML header file for +# each generated HTML page. If the tag is left blank doxygen will generate a +# standard header. +# +# To get valid HTML the header file that includes any scripts and style sheets +# that doxygen needs, which is dependent on the configuration options used (e.g. +# the setting GENERATE_TREEVIEW). It is highly recommended to start with a +# default header using +# doxygen -w html new_header.html new_footer.html new_stylesheet.css +# YourConfigFile +# and then modify the file new_header.html. See also section "Doxygen usage" +# for information on how to generate the default header that doxygen normally +# uses. +# Note: The header is subject to change so you typically have to regenerate the +# default header when upgrading to a newer version of doxygen. For a description +# of the possible markers and block names see the documentation. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_HEADER = + +# The HTML_FOOTER tag can be used to specify a user-defined HTML footer for each +# generated HTML page. If the tag is left blank doxygen will generate a standard +# footer. See HTML_HEADER for more information on how to generate a default +# footer and what special commands can be used inside the footer. See also +# section "Doxygen usage" for information on how to generate the default footer +# that doxygen normally uses. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_FOOTER = + +# The HTML_STYLESHEET tag can be used to specify a user-defined cascading style +# sheet that is used by each HTML page. It can be used to fine-tune the look of +# the HTML output. If left blank doxygen will generate a default style sheet. +# See also section "Doxygen usage" for information on how to generate the style +# sheet that doxygen normally uses. +# Note: It is recommended to use HTML_EXTRA_STYLESHEET instead of this tag, as +# it is more robust and this tag (HTML_STYLESHEET) will in the future become +# obsolete. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_STYLESHEET = + +# The HTML_EXTRA_STYLESHEET tag can be used to specify additional user-defined +# cascading style sheets that are included after the standard style sheets +# created by doxygen. Using this option one can overrule certain style aspects. +# This is preferred over using HTML_STYLESHEET since it does not replace the +# standard style sheet and is therefore more robust against future updates. +# Doxygen will copy the style sheet files to the output directory. +# Note: The order of the extra style sheet files is of importance (e.g. the last +# style sheet in the list overrules the setting of the previous ones in the +# list). For an example see the documentation. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_EXTRA_STYLESHEET = + +# The HTML_EXTRA_FILES tag can be used to specify one or more extra images or +# other source files which should be copied to the HTML output directory. Note +# that these files will be copied to the base HTML output directory. Use the +# $relpath^ marker in the HTML_HEADER and/or HTML_FOOTER files to load these +# files. In the HTML_STYLESHEET file, use the file name only. Also note that the +# files will be copied as-is; there are no commands or markers available. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_EXTRA_FILES = + +# The HTML_COLORSTYLE_HUE tag controls the color of the HTML output. Doxygen +# will adjust the colors in the style sheet and background images according to +# this color. Hue is specified as an angle on a color-wheel, see +# https://en.wikipedia.org/wiki/Hue for more information. For instance the value +# 0 represents red, 60 is yellow, 120 is green, 180 is cyan, 240 is blue, 300 +# purple, and 360 is red again. +# Minimum value: 0, maximum value: 359, default value: 220. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_COLORSTYLE_HUE = 220 + +# The HTML_COLORSTYLE_SAT tag controls the purity (or saturation) of the colors +# in the HTML output. For a value of 0 the output will use gray-scales only. A +# value of 255 will produce the most vivid colors. +# Minimum value: 0, maximum value: 255, default value: 100. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_COLORSTYLE_SAT = 100 + +# The HTML_COLORSTYLE_GAMMA tag controls the gamma correction applied to the +# luminance component of the colors in the HTML output. Values below 100 +# gradually make the output lighter, whereas values above 100 make the output +# darker. The value divided by 100 is the actual gamma applied, so 80 represents +# a gamma of 0.8, The value 220 represents a gamma of 2.2, and 100 does not +# change the gamma. +# Minimum value: 40, maximum value: 240, default value: 80. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_COLORSTYLE_GAMMA = 80 + +# If the HTML_TIMESTAMP tag is set to YES then the footer of each generated HTML +# page will contain the date and time when the page was generated. Setting this +# to YES can help to show when doxygen was last run and thus if the +# documentation is up to date. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_TIMESTAMP = NO + +# If the HTML_DYNAMIC_MENUS tag is set to YES then the generated HTML +# documentation will contain a main index with vertical navigation menus that +# are dynamically created via JavaScript. If disabled, the navigation index will +# consists of multiple levels of tabs that are statically embedded in every HTML +# page. Disable this option to support browsers that do not have JavaScript, +# like the Qt help browser. +# The default value is: YES. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_DYNAMIC_MENUS = YES + +# If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML +# documentation will contain sections that can be hidden and shown after the +# page has loaded. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_DYNAMIC_SECTIONS = NO + +# With HTML_INDEX_NUM_ENTRIES one can control the preferred number of entries +# shown in the various tree structured indices initially; the user can expand +# and collapse entries dynamically later on. Doxygen will expand the tree to +# such a level that at most the specified number of entries are visible (unless +# a fully collapsed tree already exceeds this amount). So setting the number of +# entries 1 will produce a full collapsed tree by default. 0 is a special value +# representing an infinite number of entries and will result in a full expanded +# tree by default. +# Minimum value: 0, maximum value: 9999, default value: 100. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_INDEX_NUM_ENTRIES = 100 + +# If the GENERATE_DOCSET tag is set to YES, additional index files will be +# generated that can be used as input for Apple's Xcode 3 integrated development +# environment (see: +# https://developer.apple.com/xcode/), introduced with OSX 10.5 (Leopard). To +# create a documentation set, doxygen will generate a Makefile in the HTML +# output directory. Running make will produce the docset in that directory and +# running make install will install the docset in +# ~/Library/Developer/Shared/Documentation/DocSets so that Xcode will find it at +# startup. See https://developer.apple.com/library/archive/featuredarticles/Doxy +# genXcode/_index.html for more information. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +GENERATE_DOCSET = NO + +# This tag determines the name of the docset feed. A documentation feed provides +# an umbrella under which multiple documentation sets from a single provider +# (such as a company or product suite) can be grouped. +# The default value is: Doxygen generated docs. +# This tag requires that the tag GENERATE_DOCSET is set to YES. + +DOCSET_FEEDNAME = "Doxygen generated docs" + +# This tag specifies a string that should uniquely identify the documentation +# set bundle. This should be a reverse domain-name style string, e.g. +# com.mycompany.MyDocSet. Doxygen will append .docset to the name. +# The default value is: org.doxygen.Project. +# This tag requires that the tag GENERATE_DOCSET is set to YES. + +DOCSET_BUNDLE_ID = org.doxygen.Project + +# The DOCSET_PUBLISHER_ID tag specifies a string that should uniquely identify +# the documentation publisher. This should be a reverse domain-name style +# string, e.g. com.mycompany.MyDocSet.documentation. +# The default value is: org.doxygen.Publisher. +# This tag requires that the tag GENERATE_DOCSET is set to YES. + +DOCSET_PUBLISHER_ID = org.doxygen.Publisher + +# The DOCSET_PUBLISHER_NAME tag identifies the documentation publisher. +# The default value is: Publisher. +# This tag requires that the tag GENERATE_DOCSET is set to YES. + +DOCSET_PUBLISHER_NAME = Publisher + +# If the GENERATE_HTMLHELP tag is set to YES then doxygen generates three +# additional HTML index files: index.hhp, index.hhc, and index.hhk. The +# index.hhp is a project file that can be read by Microsoft's HTML Help Workshop +# on Windows. In the beginning of 2021 Microsoft took the original page, with +# a.o. the download links, offline the HTML help workshop was already many years +# in maintenance mode). You can download the HTML help workshop from the web +# archives at Installation executable (see: +# http://web.archive.org/web/20160201063255/http://download.microsoft.com/downlo +# ad/0/A/9/0A939EF6-E31C-430F-A3DF-DFAE7960D564/htmlhelp.exe). +# +# The HTML Help Workshop contains a compiler that can convert all HTML output +# generated by doxygen into a single compiled HTML file (.chm). Compiled HTML +# files are now used as the Windows 98 help format, and will replace the old +# Windows help format (.hlp) on all Windows platforms in the future. Compressed +# HTML files also contain an index, a table of contents, and you can search for +# words in the documentation. The HTML workshop also contains a viewer for +# compressed HTML files. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +GENERATE_HTMLHELP = NO + +# The CHM_FILE tag can be used to specify the file name of the resulting .chm +# file. You can add a path in front of the file if the result should not be +# written to the html output directory. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +CHM_FILE = + +# The HHC_LOCATION tag can be used to specify the location (absolute path +# including file name) of the HTML help compiler (hhc.exe). If non-empty, +# doxygen will try to run the HTML help compiler on the generated index.hhp. +# The file has to be specified with full path. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +HHC_LOCATION = + +# The GENERATE_CHI flag controls if a separate .chi index file is generated +# (YES) or that it should be included in the main .chm file (NO). +# The default value is: NO. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +GENERATE_CHI = NO + +# The CHM_INDEX_ENCODING is used to encode HtmlHelp index (hhk), content (hhc) +# and project file content. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +CHM_INDEX_ENCODING = + +# The BINARY_TOC flag controls whether a binary table of contents is generated +# (YES) or a normal table of contents (NO) in the .chm file. Furthermore it +# enables the Previous and Next buttons. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +BINARY_TOC = NO + +# The TOC_EXPAND flag can be set to YES to add extra items for group members to +# the table of contents of the HTML help documentation and to the tree view. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTMLHELP is set to YES. + +TOC_EXPAND = NO + +# If the GENERATE_QHP tag is set to YES and both QHP_NAMESPACE and +# QHP_VIRTUAL_FOLDER are set, an additional index file will be generated that +# can be used as input for Qt's qhelpgenerator to generate a Qt Compressed Help +# (.qch) of the generated HTML documentation. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +GENERATE_QHP = NO + +# If the QHG_LOCATION tag is specified, the QCH_FILE tag can be used to specify +# the file name of the resulting .qch file. The path specified is relative to +# the HTML output folder. +# This tag requires that the tag GENERATE_QHP is set to YES. + +QCH_FILE = + +# The QHP_NAMESPACE tag specifies the namespace to use when generating Qt Help +# Project output. For more information please see Qt Help Project / Namespace +# (see: +# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#namespace). +# The default value is: org.doxygen.Project. +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHP_NAMESPACE = org.doxygen.Project + +# The QHP_VIRTUAL_FOLDER tag specifies the namespace to use when generating Qt +# Help Project output. For more information please see Qt Help Project / Virtual +# Folders (see: +# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#virtual-folders). +# The default value is: doc. +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHP_VIRTUAL_FOLDER = doc + +# If the QHP_CUST_FILTER_NAME tag is set, it specifies the name of a custom +# filter to add. For more information please see Qt Help Project / Custom +# Filters (see: +# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters). +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHP_CUST_FILTER_NAME = + +# The QHP_CUST_FILTER_ATTRS tag specifies the list of the attributes of the +# custom filter to add. For more information please see Qt Help Project / Custom +# Filters (see: +# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#custom-filters). +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHP_CUST_FILTER_ATTRS = + +# The QHP_SECT_FILTER_ATTRS tag specifies the list of the attributes this +# project's filter section matches. Qt Help Project / Filter Attributes (see: +# https://doc.qt.io/archives/qt-4.8/qthelpproject.html#filter-attributes). +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHP_SECT_FILTER_ATTRS = + +# The QHG_LOCATION tag can be used to specify the location (absolute path +# including file name) of Qt's qhelpgenerator. If non-empty doxygen will try to +# run qhelpgenerator on the generated .qhp file. +# This tag requires that the tag GENERATE_QHP is set to YES. + +QHG_LOCATION = + +# If the GENERATE_ECLIPSEHELP tag is set to YES, additional index files will be +# generated, together with the HTML files, they form an Eclipse help plugin. To +# install this plugin and make it available under the help contents menu in +# Eclipse, the contents of the directory containing the HTML and XML files needs +# to be copied into the plugins directory of eclipse. The name of the directory +# within the plugins directory should be the same as the ECLIPSE_DOC_ID value. +# After copying Eclipse needs to be restarted before the help appears. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +GENERATE_ECLIPSEHELP = NO + +# A unique identifier for the Eclipse help plugin. When installing the plugin +# the directory name containing the HTML and XML files should also have this +# name. Each documentation set should have its own identifier. +# The default value is: org.doxygen.Project. +# This tag requires that the tag GENERATE_ECLIPSEHELP is set to YES. + +ECLIPSE_DOC_ID = org.doxygen.Project + +# If you want full control over the layout of the generated HTML pages it might +# be necessary to disable the index and replace it with your own. The +# DISABLE_INDEX tag can be used to turn on/off the condensed index (tabs) at top +# of each HTML page. A value of NO enables the index and the value YES disables +# it. Since the tabs in the index contain the same information as the navigation +# tree, you can set this option to YES if you also set GENERATE_TREEVIEW to YES. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +DISABLE_INDEX = NO + +# The GENERATE_TREEVIEW tag is used to specify whether a tree-like index +# structure should be generated to display hierarchical information. If the tag +# value is set to YES, a side panel will be generated containing a tree-like +# index structure (just like the one that is generated for HTML Help). For this +# to work a browser that supports JavaScript, DHTML, CSS and frames is required +# (i.e. any modern browser). Windows users are probably better off using the +# HTML help feature. Via custom style sheets (see HTML_EXTRA_STYLESHEET) one can +# further fine tune the look of the index (see "Fine-tuning the output"). As an +# example, the default style sheet generated by doxygen has an example that +# shows how to put an image at the root of the tree instead of the PROJECT_NAME. +# Since the tree basically has the same information as the tab index, you could +# consider setting DISABLE_INDEX to YES when enabling this option. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +GENERATE_TREEVIEW = NO + +# When both GENERATE_TREEVIEW and DISABLE_INDEX are set to YES, then the +# FULL_SIDEBAR option determines if the side bar is limited to only the treeview +# area (value NO) or if it should extend to the full height of the window (value +# YES). Setting this to YES gives a layout similar to +# https://docs.readthedocs.io with more room for contents, but less room for the +# project logo, title, and description. If either GENERATOR_TREEVIEW or +# DISABLE_INDEX is set to NO, this option has no effect. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +FULL_SIDEBAR = NO + +# The ENUM_VALUES_PER_LINE tag can be used to set the number of enum values that +# doxygen will group on one line in the generated HTML documentation. +# +# Note that a value of 0 will completely suppress the enum values from appearing +# in the overview section. +# Minimum value: 0, maximum value: 20, default value: 4. +# This tag requires that the tag GENERATE_HTML is set to YES. + +ENUM_VALUES_PER_LINE = 4 + +# If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be used +# to set the initial width (in pixels) of the frame in which the tree is shown. +# Minimum value: 0, maximum value: 1500, default value: 250. +# This tag requires that the tag GENERATE_HTML is set to YES. + +TREEVIEW_WIDTH = 250 + +# If the EXT_LINKS_IN_WINDOW option is set to YES, doxygen will open links to +# external symbols imported via tag files in a separate window. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +EXT_LINKS_IN_WINDOW = NO + +# If the HTML_FORMULA_FORMAT option is set to svg, doxygen will use the pdf2svg +# tool (see https://github.com/dawbarton/pdf2svg) or inkscape (see +# https://inkscape.org) to generate formulas as SVG images instead of PNGs for +# the HTML output. These images will generally look nicer at scaled resolutions. +# Possible values are: png (the default) and svg (looks nicer but requires the +# pdf2svg or inkscape tool). +# The default value is: png. +# This tag requires that the tag GENERATE_HTML is set to YES. + +HTML_FORMULA_FORMAT = png + +# Use this tag to change the font size of LaTeX formulas included as images in +# the HTML documentation. When you change the font size after a successful +# doxygen run you need to manually remove any form_*.png images from the HTML +# output directory to force them to be regenerated. +# Minimum value: 8, maximum value: 50, default value: 10. +# This tag requires that the tag GENERATE_HTML is set to YES. + +FORMULA_FONTSIZE = 10 + +# Use the FORMULA_TRANSPARENT tag to determine whether or not the images +# generated for formulas are transparent PNGs. Transparent PNGs are not +# supported properly for IE 6.0, but are supported on all modern browsers. +# +# Note that when changing this option you need to delete any form_*.png files in +# the HTML output directory before the changes have effect. +# The default value is: YES. +# This tag requires that the tag GENERATE_HTML is set to YES. + +FORMULA_TRANSPARENT = YES + +# The FORMULA_MACROFILE can contain LaTeX \newcommand and \renewcommand commands +# to create new LaTeX commands to be used in formulas as building blocks. See +# the section "Including formulas" for details. + +FORMULA_MACROFILE = + +# Enable the USE_MATHJAX option to render LaTeX formulas using MathJax (see +# https://www.mathjax.org) which uses client side JavaScript for the rendering +# instead of using pre-rendered bitmaps. Use this if you do not have LaTeX +# installed or if you want to formulas look prettier in the HTML output. When +# enabled you may also need to install MathJax separately and configure the path +# to it using the MATHJAX_RELPATH option. +# The default value is: NO. +# This tag requires that the tag GENERATE_HTML is set to YES. + +USE_MATHJAX = NO + +# With MATHJAX_VERSION it is possible to specify the MathJax version to be used. +# Note that the different versions of MathJax have different requirements with +# regards to the different settings, so it is possible that also other MathJax +# settings have to be changed when switching between the different MathJax +# versions. +# Possible values are: MathJax_2 and MathJax_3. +# The default value is: MathJax_2. +# This tag requires that the tag USE_MATHJAX is set to YES. + +MATHJAX_VERSION = MathJax_2 + +# When MathJax is enabled you can set the default output format to be used for +# the MathJax output. For more details about the output format see MathJax +# version 2 (see: +# http://docs.mathjax.org/en/v2.7-latest/output.html) and MathJax version 3 +# (see: +# http://docs.mathjax.org/en/latest/web/components/output.html). +# Possible values are: HTML-CSS (which is slower, but has the best +# compatibility. This is the name for Mathjax version 2, for MathJax version 3 +# this will be translated into chtml), NativeMML (i.e. MathML. Only supported +# for NathJax 2. For MathJax version 3 chtml will be used instead.), chtml (This +# is the name for Mathjax version 3, for MathJax version 2 this will be +# translated into HTML-CSS) and SVG. +# The default value is: HTML-CSS. +# This tag requires that the tag USE_MATHJAX is set to YES. + +MATHJAX_FORMAT = HTML-CSS + +# When MathJax is enabled you need to specify the location relative to the HTML +# output directory using the MATHJAX_RELPATH option. The destination directory +# should contain the MathJax.js script. For instance, if the mathjax directory +# is located at the same level as the HTML output directory, then +# MATHJAX_RELPATH should be ../mathjax. The default value points to the MathJax +# Content Delivery Network so you can quickly see the result without installing +# MathJax. However, it is strongly recommended to install a local copy of +# MathJax from https://www.mathjax.org before deployment. The default value is: +# - in case of MathJax version 2: https://cdn.jsdelivr.net/npm/mathjax@2 +# - in case of MathJax version 3: https://cdn.jsdelivr.net/npm/mathjax@3 +# This tag requires that the tag USE_MATHJAX is set to YES. + +MATHJAX_RELPATH = + +# The MATHJAX_EXTENSIONS tag can be used to specify one or more MathJax +# extension names that should be enabled during MathJax rendering. For example +# for MathJax version 2 (see +# https://docs.mathjax.org/en/v2.7-latest/tex.html#tex-and-latex-extensions): +# MATHJAX_EXTENSIONS = TeX/AMSmath TeX/AMSsymbols +# For example for MathJax version 3 (see +# http://docs.mathjax.org/en/latest/input/tex/extensions/index.html): +# MATHJAX_EXTENSIONS = ams +# This tag requires that the tag USE_MATHJAX is set to YES. + +MATHJAX_EXTENSIONS = + +# The MATHJAX_CODEFILE tag can be used to specify a file with javascript pieces +# of code that will be used on startup of the MathJax code. See the MathJax site +# (see: +# http://docs.mathjax.org/en/v2.7-latest/output.html) for more details. For an +# example see the documentation. +# This tag requires that the tag USE_MATHJAX is set to YES. + +MATHJAX_CODEFILE = + +# When the SEARCHENGINE tag is enabled doxygen will generate a search box for +# the HTML output. The underlying search engine uses javascript and DHTML and +# should work on any modern browser. Note that when using HTML help +# (GENERATE_HTMLHELP), Qt help (GENERATE_QHP), or docsets (GENERATE_DOCSET) +# there is already a search function so this one should typically be disabled. +# For large projects the javascript based search engine can be slow, then +# enabling SERVER_BASED_SEARCH may provide a better solution. It is possible to +# search using the keyboard; to jump to the search box use + S +# (what the is depends on the OS and browser, but it is typically +# , /