New sensor-related features

[gmaze]

Will close #529
Following the last Euro-Argo Meeting, it appears some easy-to-use sensor-related features could be useful to some operators or experts.

This PR provides a new ArgoSensor and OEMSensorMetaData classes to address this.

• ArgoSensor
  • Reference tables
    • Access reference tables for SENSOR_MODEL (R27), SENSOR (R25) and SENSOR_MAKER (R26)
    • Mapping between SENSOR_MODEL (R27) and SENSOR (R25)
    • Search in SENSOR_MODEL (R27) reference table
  • Search for all Argo floats equipped with one or more exact sensor model(s)
  • Easily loop through ArgoFloat instances for each float equipped with a sensor model
  • Directly work with a known and exact sensor model
  • From the command line (CLI)
• OEM sensor meta-data
• PR to-do list

ArgoSensor

Reference tables

Access reference tables for SENSOR_MODEL (R27), SENSOR (R25) and SENSOR_MAKER (R26)

from argopy import ArgoSensor
sensor = ArgoSensor()

sensor.ref.model.to_dataframe() # Return reference table R27 with the list of sensor models as a DataFrame
sensor.ref.model.hint()      # Return list of sensor model names (possible values for 'SENSOR_MODEL' parameter)

sensor.ref.type.to_dataframe()  # Return reference table R25 with the list of sensor types as a DataFrame
sensor.ref.type.hint()       # Return list of sensor types (possible values for 'SENSOR' parameter)

sensor.ref.maker.to_dataframe()  # Return reference table R26 with the list of manufacturer as a DataFrame
sensor.ref.maker.hint()       # Return list of manufacturer names (possible values for 'SENSOR_MAKER' parameter)

Mapping between SENSOR_MODEL (R27) and SENSOR (R25)

from argopy import ArgoSensor
sensor = ArgoSensor()

sensor.ref.model.to_type('SBE61') # Return sensor type (R25) of a given model (R27)

sensor.ref.type.to_model('FLUOROMETER_CDOM') # Return all possible model names (R27) for a given sensor type (R25)

Search in SENSOR_MODEL (R27) reference table

from argopy import ArgoSensor
sensor = ArgoSensor()

sensor.ref.model.search('RBR')  # Search and return a DataFrame
sensor.ref.model.search('RBR', output='name') # Search and return a list of names instead
sensor.ref.model.search('SBE61*')  # Use of wildcards
sensor.ref.model.search('*Deep*')  # Search is case-insensitive

Search for all Argo floats equipped with one or more exact sensor model(s)

from argopy import ArgoSensor

sensors = ArgoSensor()

# Search and return a list of WMOs equipped
sensors.search('SBE61_V5.0.2')

# Search and return a list of sensor serial numbers in Argo
sensors.search('ECO_FLBBCD_AP2', output='sn')

# Search and return a list of tuples with WMOs and sensors serial number
sensors.search('SBE', output='wmo_sn')

# Search and return a DataFrame with full sensor information from floats equipped
sensors.search('RBR', output='df')

# Search multiple models at once
sensors.search(['ECO_FLBBCD_AP2', 'ECO_FLBBCD'])

If the sensor model name is not found in the meta-data base, the error message will provide some hints to help you find the exact name:

ArgoSensor().search('SB')  # 'SB' is not an exact sensor model name

Throw a OptionValueError error message like this:

OptionValueError: The model 'SB' is not exact. Possible hints are: 'DSB301-10-C85', 'QUARTZDYNE_DSB301-10-C85', 'SBE', 'SBE37', 'SBE41', 'SBE41CP', 'SBE41CP_IDO', 'SBE41CP_IDO_V2.0b', 'SBE41CP_V1', 'SBE41CP_V1.1', 'SBE41CP_V1.2', 'SBE41CP_V1.2a', 'SBE41CP_V1.3', 'SBE41CP_V1.3b', 'SBE41CP_V1.4', 'SBE41CP_V1.5', 'SBE41CP_V1.7', 'SBE41CP_V1.8', 'SBE41CP_V1.9', 'SBE41CP_V1.9a' or more ...

Easily loop through ArgoFloat instances for each float equipped with a sensor model

from argopy import ArgoSensor

sensors = ArgoSensor()

# Trivial example:
model = "RAFOS"
for af in sensors.iterfloats_with(model):
    print(af.WMO)

# Example to gather all platform types for all WMOs equipped with a list of sensor models
models = ['ECO_FLBBCD_AP2', 'ECO_FLBBCD']
results = {}
for af in sensors.iterfloats_with(models):
    if 'meta' in af.ls_dataset():
        platform_type = af.metadata['platform']['type']  # e.g. 'PROVOR_V_JUMBO'
        if platform_type in results.keys():
            results[platform_type].extend([af.WMO])
        else:
            results.update({platform_type: [af.WMO]})
    else:
        print(f"No meta file for float {af.WMO}")
[f"{r:15s}: {len(results[r])} floats" for r in results.keys()]

Directly work with a known and exact sensor model

Use an exact sensor model name to create an instance:

from argopy import ArgoSensor
sensor = ArgoSensor('RBR_ARGO3_DEEP6K')

which return some information:

<argosensor.RBR_ARGO3_DEEP6K>
TYPE➤ 'Conductivity Temperature Depth (CTD) sensors package measuring pressure', 'Conductivity Temperature Depth (CTD) sensors package measuring temperature' and 'Conductivity Temperature Depth (CTD) sensors package measuring conductivity'
MODEL➤ RBRargo3 CTD deep6k
✅ This model is not deprecated.
🔗 http://vocab.nerc.ac.uk/collection/R27/current/RBR_ARGO3_DEEP6K/
❝A system comprising temperature, conductivity and pressure sensors with Computational Fluid Dynamic (CFD) optimised flow path, manufactured by RBR. Salinity is measured by induction of water flushing freely through the conductivity cell, and is accurate to within 10 cm of the air-ocean interface. The unit is built with a compact titanium housing rated to 6000 dbar, and is specifically designed to be integrated with glass sphere profiling floats part of the Deep Argo program. The system supports both spot and continuous sampling modes, with sampling speeds configurable to up to 8 Hz.❞

also available in the attributes:

sensor.vocabulary  # The "SENSOR_MODEL" vocabulary for this sensor model
sensor.type  # The "SENSOR" vocabulary for this sensor model

and can be used to list float or/and serial number equipped with this model:

sensor.search(output='wmo')
sensor.search(output='sn')
sensor.search(output='wmo_sn')
sensor.search(output='df')  # A pandas DataFrame will all information 

From the command line (CLI)

Get serialized search results from the command-line with ArgoSensor.cli_search

python -c "from argopy import ArgoSensor; ArgoSensor().cli_search('RBR', output='wmo')"
python -c "from argopy import ArgoSensor; ArgoSensor().cli_search('RBR', output='sn')"
python -c "from argopy import ArgoSensor; ArgoSensor().cli_search('RBR', output='wmo_sn')"
python -c "from argopy import ArgoSensor; ArgoSensor().cli_search('RBR', output='df')"

OEM sensor meta-data

This PR also introduces the OEMSensorMetaData class to work with sensor meta-data from manufacturer and complying to sensor schema from https://github.com/euroargodev/sensor_metadata_json.

from argopy import OEMSensorMetaData

# Use this option to run or avoid json schema validation compliance:
OEMSensorMetaData(validate=True) 
OEMSensorMetaData(validate=True, validation_error='raise')  # Default is to 'warn'  

Currently argopy supports 2 manufacturers API from RBR and Seabird.

# Direct call to the RBR api with a sensor serial number:
OEMSensorMetaData().from_rbr(208380)  

# Direct call to Seabird api with a sensor serial number and model name"
OEMSensorMetaData().from_seabird(2444, 'SATLANTIC_OCR504_ICSW')  

Note that the RBR api requires an authentication key (you can contact RBR at argo@rbr-global.com if you do not have an such a key). Argopy will try to get the key from the environment variableRBR_API_KEY or from the option rbr_api_key. You can set the key temporarily in your code with:

argopy.set_options(rbr_api_key="********")

The OEMSensorMetaData instance return a object that has all available metadata properties, notably about pre-deployment calibration.

Since this remains in dev. from the manufacturer side, we also provide examples:

OEMSensorMetaData().list_examples

OEMSensorMetaData().from_examples('WETLABS-ECO_FLBBAP2-8589')

OEMSensorMetaData().from_dict(jsdata)  # Use any compliant json data

PR to-do list

• Add new features
• Add/update documentation and docstrings
• Add CI tests covering new features
• CI tests pinned/upstream passed

[gmaze]

Hi @SBS-EREHM

As suggested, I implemented the possibility to search for more than one model:

from argopy import ArgoSensor
sensors = ArgoSensor()
sensors.search(['ECO_FLBBCD_AP2', 'ECO_FLBBCD'])

When meta-data are required for the output with serial numbers or as a dataframe, the metadata fetching is done in parallel with multithreading, which makes this quite rapid.

[gmaze]

This PR won't be merged
New features are implemented in #572