add costing to server response

[mathleur]

Description

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

🌈🌦️📖🚧 Documentation 🚧📖🌦️🌈
https://sites.ecmwf.int/docs/dev-section/qubed/pull-requests/PR-27

[peshence]

shouldn't we use the .n_nodes and .n_leaves instead?

[j08lue]

I do not have access to the Confluence page referenced above https://sites.ecmwf.int/docs/dev-section/qubed/pull-requests/PR-27 - can you give a brief description of this feature, please?

Costing in terms of data volume spanned by a given Qubed query?

[mathleur]

Hi @j08lue,

We don't yet have an associated Confluence page for this, but we just wanted to create an estimate response size of the Qube that we return. We're not entirely sure yet on the metric to use to measure this, but we'd be happy to hear about your use case if this is also useful to you or implement/think of a different metric!

[j08lue]

we'd be happy to hear about your use case if this is also useful to you or implement/think of a different metric

We are developing a web application to automatically generate Polytope / MARS data requests for Digital Twin data from user prompts (https://github.com/ecmwf/digital-twin-assistant), using Qubed under the hood to find available fields of interest.

I think it is generally very useful information for a user to know how large the data volume is, before they hit send.

As application developers, we could use this info to double-check that a request is small enough to execute live and synchronously as a data preview. If such a metric was available, we would also always surface it alongside the Polytope JSON our tool generates.

[mathleur]

Hi @j08lue,

Ah yes this is an interesting use case!
Unfortunately we weren't yet planning to return the expected data sizes, as this involves a bit more work and information that isn't yet available in the Qubes we use. However we can definitely open an issue and investigate how to implement this efficiently!