Errors with OOI side data storage

[John-Ragland]

Handling how data is stored on OOI side is a moving target: A quick summary of different time regions is given below to document what I know.

• before June 7th, 2023
  • When data is fragmented, there is a separate mseed file for each fragment
• after June 7th, but before December (unknown exact date) 2023:
  • multiple fragments are saved in 5 min mseed files, but the actual start and end times are deleted.
  • IF there is exactly the right number of points for the 5 min segment, it might be possible to concatenate these
    • This is being explored with gapless merge feature (see Experimenting with gapless merge #173)
  • If the 5 min data segment is not fully complete (i.e. there is some gap in data), the data segment is completely unusable.
• after December 2023:
  • multiple fragments are saved in 5 min mseed files, and the correct start and end times are stored in the mseed file. This means that the data can be reconstructed, but at a non-zero computational cost. Spreading this computation to multiple cores is being trying in merging traces in get_acoustic_data takes forever #158
  • It might also be possible to write a custom merge function that can work much faster