GSF issues/limitations

[ejheffron]

Starting this issue thread to document some of the issues, or maybe more correctly limitations, we've found with GSF (Generic Sensor Format) files. As background, we've used GSF as our preferred processed point data export and have used it as an internal/requestable archive format as well as submitted it to public archive (though I don't think it is made available on NCEI) for work with OET and others. We were and are using this format because it is an open format, is supported by many mapping software programs, supports processing flags so that all of the processing work we do isn't lost, and can, theoretically at least, be re-processed or have additional processing done to it. But we have hit some limitations and I'm sure other have as well. Documenting here for comment and tracking.

[ejheffron]

General note: When evaluating or attempting to reprocess GSF files in QPS Qimera (all versions through to the current version, 2.5.3), GSF files can be imported as either Raw Sonar Files or Processed Point Files. Importing as Raw Sonar Files allows you to adjust sound velocity (applying a new sound speed profile or overall sound velocity strategy) and make other changes that require re-ray tracing and core reprocessing to generate corrected sounding locations, allows you to see previously rejected soundings and un-reject if necessary, do further rejection, run filters, use blocking filters, etc. It also comes with the
"do this at your own risk" warning in the attached image. Importing as Processed Point Files allows you to do point editing only -- rejecting/unrejecting, running filters -- but no changes to SV strategy or the like. Would be good to dig out and lay out all of the functionality differences but that is a task for another day.

[ejheffron]

Qimera Sound Velocity: When you import a GSF files into Qimera (2.5.1 and earlier, assume same for current version) that has more than one sound speed profile associated with it (Real-Time Scheduling), the GSF will fail to load properly, with an SVP lookup failure. The sound speed profile for that line can be manaully set to a specific profile and the line will load. This doesn't seem to happen for every file with real-time scheduling. This has been reported to QPS (CQM-1046).

[ejheffron]

GSFs for KMALL files with extra detections, when loaded in QPS Qimera: We had a few files where we think extra detections were turned on in SIS inadvertantly. The KMALL files load and process in Qimera fine, though we did note the reported beam number in the 3D Editor was really high -- that's one of the reasons we suspect extra detections. When the GSF files exported from these files are loaded into Qimera as raw point files, the results are a crazy mess. The files can be loaded as processed point files and seem to be ok. This has been reported to QPS (CQM-6130) and we've asked them to confirm that it is the extra detections causing this issue.

[ejheffron]

GSF files exported out of recent versions of Qimera won't load in Fledermaus 7.8.12. They will load in Fledermaus 8. I assume this is because updates to Fledermaus 7.8.12 stopped in 2021 and there had been an update to the GSF library or to QPS handling of GSF files since then, which is understandable -- but still a gotcha for those of us clinging desperately to Fledermaus 7 for any variety of reasons. Not sure exactly which version of Qimera is associated with the change. Currently using 2.5.4 but several versions prior were producing GSF files that wouldn't load in Fledermaus 7.

[ggdp]

[heart] Syxinh Le reacted to your message:

…

________________________________ From: ejheffron ***@***.***> Sent: Wednesday, August 30, 2023 2:36:55 PM To: oceanmapping/community ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [oceanmapping/community] GSF issues/limitations (Issue #37) GSF files exported out of recent versions of Qimera won't load in Fledermaus 7.8.12. They will load in Fledermaus 8. I assume this is because updates to Fledermaus 7.8.12 stopped in 2021 and there had been an update to the GSF library or to QPS handling of GSF files since then, which is understandable -- but still a gotcha for those of us clinging desperately to Fledermaus 7 for any variety of reasons. Not sure exactly which version of Qimera is associated with the change. Currently using 2.5.4 but several versions prior were producing GSF files that wouldn't load in Fledermaus 7. — Reply to this email directly, view it on GitHub<#37 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AVHXZFZA47UGKYYFPTFXKJ3XX5FYPANCNFSM6AAAAAAW4CCKGQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[lindsaymbc]

Qimera Sound Velocity: When you import a GSF files into Qimera (2.5.1 and earlier, assume same for current version) that has more than one sound speed profile associated with it (Real-Time Scheduling), the GSF will fail to load properly, with an SVP lookup failure. The sound speed profile for that line can be manaully set to a specific profile and the line will load. This doesn't seem to happen for every file with real-time scheduling. This has been reported to QPS (CQM-1046).

Good news, this has been fixed in Qimera ver 2.6.2

[lindsaymbc]

General note: When evaluating or attempting to reprocess GSF files in QPS Qimera (all versions through to the current version, 2.5.3), GSF files can be imported as either Raw Sonar Files or Processed Point Files. Importing as Raw Sonar Files allows you to adjust sound velocity (applying a new sound speed profile or overall sound velocity strategy) and make other changes that require re-ray tracing and core reprocessing to generate corrected sounding locations, allows you to see previously rejected soundings and un-reject if necessary, do further rejection, run filters, use blocking filters, etc. It also comes with the "do this at your own risk" warning in the attached image. Importing as Processed Point Files allows you to do point editing only -- rejecting/unrejecting, running filters -- but no changes to SV strategy or the like. Would be good to dig out and lay out all of the functionality differences but that is a task for another day.

Chasing another issue and QPS gave us some good information about using GSF in Qimera:

• There is a free GSF viewer as part of the Qinsy install, and you can just run the exe.
• If you import GSF as raw sonar files, don't run automatically to do full ray travel to generate QPD points in the project. After import select them all and run Tools → Manual Processing → Direct to QPD. This will create QPDs filled with the processed points from the GSF file, while preserving the processing settings and original vessel editors settings if you need to reprocess some lines later. This also avoids any failures if missing or incorrect data in packets in GSF.