Skip to content

Improve API of clientarch#1231

Open
fernandohds564 wants to merge 60 commits intomasterfrom
dev/enh/clientarch-improve-api
Open

Improve API of clientarch#1231
fernandohds564 wants to merge 60 commits intomasterfrom
dev/enh/clientarch-improve-api

Conversation

@fernandohds564
Copy link
Copy Markdown
Contributor

@fernandohds564 fernandohds564 commented Mar 27, 2026

To be merged after #1230.

This PR changes the interface of the classes PVData, PVDataSet and ClientArchiver from siriuspy.clientarch package.

The main modifications are:

  • renaming of all methods of ClientArchiver to snake_case, the python standard;
  • addition of new funcionalities in ClientArchiver:
    • login now takes the password as an optional argument. If not provided, it will prompt the user for the password, in a discrete way. The old interface required the user to write its password in scripts and or ipython terminals, which would result in sensitive data being saved in files and logged in histories, etc...
    • get_data now supports all types of processing the archiver interface provides, not just the 'mean'. It takes two parameters, namely proc_type_param1 and proc_type_param2 to control the parameters of the chosen statistics. Some statistics may use one or none of them;
    • get_report was created and supports all types of reports provided by the archiver interface;
    • get_detailed_appliance_metrics was created and provides the results of the 'getApplianceMetricsForAppliance' call to archiver;
    • get_process_metrics_for_appliance was created and provides the results of the 'getProcessMetricsDataForAppliance';
    • the property timeout was renamed to query_timeout to better reflect its meaning: the timeout of each query transaction, instead of the total timeout of all queries.
    • the property parallel_query_bin_interval was renamed to query_split_interval. Now it will always be taken into consideration when creating queries urls, for splitting large time intervals into smaller ones;
    • the option parallel of get_data was removed. we can achieve similar results by setting query_split_interval to 0;
    • the property query_max_concurrency was created to limit the maximum number of concurrent queries that will be requested from the server. It is an additional control for optimizing the retrieval time.
  • changes/improvements of PVData API:
    • implementation of __str__ method to show a nice visualization of the class properties
    • removal of timestamp_start and timestamp_stop. Now only time_start and time_stop exists. Their getter return a clientarch.Time but their setters accept anything that can be converted to Time object, including timestamps;
    • the properties processing_type, processing_type_param1, processing_type_param2 and query_split_interval were added to control how self.connector.get_data will be called inside the self.update method. This means the self.update method does not take any arguments beside a query_timeout, that overwrites self.connector.query_timeout if passed;
    • the properties query_max_concurrency and query_timeout were added to ease setting of the self.connector respective properties;
  • Changes/improvements of PVDataSet API:
    • implementation of __str__ method to show a nice visualization of the class properties
    • removal of timestamp_start and timestamp_stop. Now only time_start and time_stop exists. Their getter return a clientarch.Time if all internal PVData objects have equal start_time|stop_time or a list of them, one for each PVData object. Their setters accept anything that can be converted to Time object, including timestamps. It also accepts a list of Time objects, referring to the desired start_time|stop_time of each PVData.
    • the properties processing_type, processing_type_param1, processing_type_param2 and query_split_interval were added to control how self.connector.get_data will be called inside the self.update method. Their getters return a single value or a list of values depending if the associated PVData's are identical or not, and the setter also accepts a list of values, one for each PVData. This means the self.update method does not take any arguments beside a query_timeout, that overwrites self.connector.query_timeout if passed;
    • the properties query_max_concurrency and query_timeout were added to ease setting of the self.connector respective properties;

This PR was testes in the control room computers, using the script gensumm_machreport.py and other manual tests.

this code was developed with the help of Github Copilot.
This code was done with the help of Google AI Mode.

Since the nest_asyncio package was archived,
we implemented again a solution based on
running a different event loop in a new thread
however, this thread is long lived and all tasks
are submitted using asyncio.run_coroutine_threadsafe
function. Which guarantees that exceptions will
be passed to the end user and that there will be
no infinite hangs.
previous implementation was obscure
and had a bug. last interval had duration
larger than specified by `interval` input.
previous implementation of constructor was obscure
and didn't accept some standard inputs, such as the result
from calling Time.get_iso8601().
… created by getData, not only a simplified one.
now the properties in PVData and PVDataSet are called `query_bin_interval` instead of `parallel_query_bin_interval` because all interaction with server will be parallel. If a single request is desired, than query_bin_interval should be modified accordingly.
Please, check change log at description of #1231
Adds configurable concurrency limit for async queries

Introduces a maximum concurrency parameter for managing the number of simultaneous asynchronous queries to the server, improving resource control and preventing overload. Allows dynamic adjustment of concurrency at runtime via a property, and ensures requests are correctly throttled using an asyncio semaphore. Also fixes an issue with tuple assignment in value extraction.
…hecks

fix name of input variable of
get_request_url_for_get_data and method
to initialize pvdatas in PVDataSet.
Introduces methods to fetch detailed and process metrics for the archiver appliance, improving observability and diagnostics. Also adds debugging output for URL requests to aid troubleshooting.
Introduces a `use_async` switch to allow requests via threads
or async methods, improving flexibility and compatibility.
Implements threaded login, logout, and request handling using
the `requests` library and thread pools. Enables concurrent
requests for environments where async is not preferred or
supported.
Corrects indentation in a docstring for clarity and adds missing trailing commas in argument lists to prevent potential syntax issues. Improves code readability and consistency.
Introduces a unified interface to choose between async, threaded, and serial query methods for network operations, improving flexibility and control over concurrency.

Replaces the simple async/threaded flag with a more robust approach, updates session handling, and adds serial (synchronous) support. Improves error handling and logs additional debug information.

Helps address cases where async may not be suitable, enabling users to switch strategies as needed.
Simplifies query handling by eliminating threaded and serial methods,
retaining only async-based requests. Reduces code complexity,
removes unnecessary dependencies, and unifies session management.
Focuses maintenance and feature development on the async workflow.
…ohttp.

Only return data in json format. Fall back to text when error occurs.
Removes the option to return raw responses, ensuring all requests
consistently parse and return JSON or text. Simplifies method
signatures and internal logic, improving maintainability and
reducing ambiguity around response formats.
Introduces a method to retrieve process variable type information,
enhancing the client's capability to introspect PV details, such as archiving
policyName.
…iate

Moves the debug log statement to occur after acquiring the lock.
Eliminates the explicit timeout when retrieving the result of an async future.
Timeout is controlled at the individual request level.
Updates all references, method signatures, and documentation to consistently use 'query_timeout' instead of 'timeout' for query-related timeouts.
Improves code clarity by distinguishing per-query timeouts from other potential timeout types and avoids ambiguity.
@fernandohds564 fernandohds564 marked this pull request as ready for review April 2, 2026 17:40
Moves semaphore creation and cleanup into the asynchronous request handler to ensure concurrency control is scoped to each request, simplifying event loop and concurrency management. This avoids potential issues with event loop association and ensures correct concurrency behavior.
Ensures query bin interval values are consistently cast to integer
when set or used, improving type safety and preventing potential
errors from passing float values where integers are expected.

Updates documentation to reflect the expected integer type.
…ClientArchiver.

Sets the query bin interval for all remaining PVs and datasets to the total time
window, reducing the number of queries and improving update performance.
… in python <=3.9.

Improves subclassing compatibility by ensuring that construction
methods always return an instance of the correct subclass, even
on Python versions prior to 3.9 where tz-aware methods may return
a base datetime object. Enhances compatibility and consistency
across Python versions.
Adds a guard clause to avoid attempting logout when there is no active session.
…quests

Wraps client sessions with a connector configured with ssl=False to allow requests to URLs with IP addresses without triggering SSL certificate errors. This change improves reliability when accessing resources over plain IP connections.
Improves user experience by securely prompting for a password via the console when one is not supplied to the login method, avoiding the need to include sensitive information directly in code or scripts.
… with IP address.

Configures session to accept unsafe cookies when accessing servers via IP address, preventing SSL errors in environments like control rooms where domain names may not be used.
… working.

It is required to define the field `"Host"` in the header so that
the server accepts the request. Otherwise an 404 error is launched.
Thanks to Google AI for the help with this bug.

Refactors session creation to set required headers internally,
removes unused imports, and ensures previous sessions are
properly logged out before new logins. Enhances compatibility
with IP-based server access and prevents authentication issues
by explicitly setting necessary headers.
…wing user to set query_bin_interval to 0 or None.
…ize _time_start and _time_stop with Time.now()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant