Open
Conversation
this code was developed with the help of Github Copilot.
This code was done with the help of Google AI Mode. Since the nest_asyncio package was archived, we implemented again a solution based on running a different event loop in a new thread however, this thread is long lived and all tasks are submitted using asyncio.run_coroutine_threadsafe function. Which guarantees that exceptions will be passed to the end user and that there will be no infinite hangs.
previous implementation was obscure and had a bug. last interval had duration larger than specified by `interval` input.
previous implementation of constructor was obscure and didn't accept some standard inputs, such as the result from calling Time.get_iso8601().
… created by getData, not only a simplified one.
now the properties in PVData and PVDataSet are called `query_bin_interval` instead of `parallel_query_bin_interval` because all interaction with server will be parallel. If a single request is desired, than query_bin_interval should be modified accordingly.
Please, check change log at description of #1231
Adds configurable concurrency limit for async queries Introduces a maximum concurrency parameter for managing the number of simultaneous asynchronous queries to the server, improving resource control and preventing overload. Allows dynamic adjustment of concurrency at runtime via a property, and ensures requests are correctly throttled using an asyncio semaphore. Also fixes an issue with tuple assignment in value extraction.
…hecks fix name of input variable of get_request_url_for_get_data and method to initialize pvdatas in PVDataSet.
Introduces methods to fetch detailed and process metrics for the archiver appliance, improving observability and diagnostics. Also adds debugging output for URL requests to aid troubleshooting.
Introduces a `use_async` switch to allow requests via threads or async methods, improving flexibility and compatibility. Implements threaded login, logout, and request handling using the `requests` library and thread pools. Enables concurrent requests for environments where async is not preferred or supported.
Corrects indentation in a docstring for clarity and adds missing trailing commas in argument lists to prevent potential syntax issues. Improves code readability and consistency.
Introduces a unified interface to choose between async, threaded, and serial query methods for network operations, improving flexibility and control over concurrency. Replaces the simple async/threaded flag with a more robust approach, updates session handling, and adds serial (synchronous) support. Improves error handling and logs additional debug information. Helps address cases where async may not be suitable, enabling users to switch strategies as needed.
Simplifies query handling by eliminating threaded and serial methods, retaining only async-based requests. Reduces code complexity, removes unnecessary dependencies, and unifies session management. Focuses maintenance and feature development on the async workflow.
…ohttp. Only return data in json format. Fall back to text when error occurs. Removes the option to return raw responses, ensuring all requests consistently parse and return JSON or text. Simplifies method signatures and internal logic, improving maintainability and reducing ambiguity around response formats.
Introduces a method to retrieve process variable type information, enhancing the client's capability to introspect PV details, such as archiving policyName.
…iate Moves the debug log statement to occur after acquiring the lock.
Eliminates the explicit timeout when retrieving the result of an async future. Timeout is controlled at the individual request level.
Updates all references, method signatures, and documentation to consistently use 'query_timeout' instead of 'timeout' for query-related timeouts. Improves code clarity by distinguishing per-query timeouts from other potential timeout types and avoids ambiguity.
Moves semaphore creation and cleanup into the asynchronous request handler to ensure concurrency control is scoped to each request, simplifying event loop and concurrency management. This avoids potential issues with event loop association and ensures correct concurrency behavior.
Ensures query bin interval values are consistently cast to integer when set or used, improving type safety and preventing potential errors from passing float values where integers are expected. Updates documentation to reflect the expected integer type.
…ClientArchiver. Sets the query bin interval for all remaining PVs and datasets to the total time window, reducing the number of queries and improving update performance.
… in python <=3.9. Improves subclassing compatibility by ensuring that construction methods always return an instance of the correct subclass, even on Python versions prior to 3.9 where tz-aware methods may return a base datetime object. Enhances compatibility and consistency across Python versions.
Adds a guard clause to avoid attempting logout when there is no active session.
…quests Wraps client sessions with a connector configured with ssl=False to allow requests to URLs with IP addresses without triggering SSL certificate errors. This change improves reliability when accessing resources over plain IP connections.
Improves user experience by securely prompting for a password via the console when one is not supplied to the login method, avoiding the need to include sensitive information directly in code or scripts.
… with IP address. Configures session to accept unsafe cookies when accessing servers via IP address, preventing SSL errors in environments like control rooms where domain names may not be used.
… working. It is required to define the field `"Host"` in the header so that the server accepts the request. Otherwise an 404 error is launched. Thanks to Google AI for the help with this bug. Refactors session creation to set required headers internally, removes unused imports, and ensures previous sessions are properly logged out before new logins. Enhances compatibility with IP-based server access and prevents authentication issues by explicitly setting necessary headers.
…wing user to set query_bin_interval to 0 or None.
…re intuitive reading.
…one aware object.
…ize _time_start and _time_stop with Time.now()
Also, remove deprecated hidden attributes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
To be merged after #1230.
This PR changes the interface of the classes
PVData,PVDataSetandClientArchiverfromsiriuspy.clientarchpackage.The main modifications are:
ClientArchiverto snake_case, the python standard;ClientArchiver:loginnow takes the password as an optional argument. If not provided, it will prompt the user for the password, in a discrete way. The old interface required the user to write its password in scripts and or ipython terminals, which would result in sensitive data being saved in files and logged in histories, etc...get_datanow supports all types of processing the archiver interface provides, not just the'mean'. It takes two parameters, namelyproc_type_param1andproc_type_param2to control the parameters of the chosen statistics. Some statistics may use one or none of them;get_reportwas created and supports all types of reports provided by the archiver interface;get_detailed_appliance_metricswas created and provides the results of the'getApplianceMetricsForAppliance'call to archiver;get_process_metrics_for_appliancewas created and provides the results of the'getProcessMetricsDataForAppliance';timeoutwas renamed toquery_timeoutto better reflect its meaning: the timeout of each query transaction, instead of the total timeout of all queries.parallel_query_bin_intervalwas renamed toquery_split_interval. Now it will always be taken into consideration when creating queries urls, for splitting large time intervals into smaller ones;parallelofget_datawas removed. we can achieve similar results by settingquery_split_intervalto0;query_max_concurrencywas created to limit the maximum number of concurrent queries that will be requested from the server. It is an additional control for optimizing the retrieval time.PVDataAPI:__str__method to show a nice visualization of the class propertiestimestamp_startandtimestamp_stop. Now onlytime_startandtime_stopexists. Their getter return aclientarch.Timebut their setters accept anything that can be converted toTimeobject, including timestamps;processing_type,processing_type_param1,processing_type_param2andquery_split_intervalwere added to control howself.connector.get_datawill be called inside theself.updatemethod. This means theself.updatemethod does not take any arguments beside aquery_timeout, that overwritesself.connector.query_timeoutif passed;query_max_concurrencyandquery_timeoutwere added to ease setting of theself.connectorrespective properties;PVDataSetAPI:__str__method to show a nice visualization of the class propertiestimestamp_startandtimestamp_stop. Now onlytime_startandtime_stopexists. Their getter return aclientarch.Timeif all internalPVDataobjects have equalstart_time|stop_timeor a list of them, one for eachPVDataobject. Their setters accept anything that can be converted toTimeobject, including timestamps. It also accepts a list ofTimeobjects, referring to the desiredstart_time|stop_timeof eachPVData.processing_type,processing_type_param1,processing_type_param2andquery_split_intervalwere added to control howself.connector.get_datawill be called inside theself.updatemethod. Their getters return a single value or a list of values depending if the associatedPVData's are identical or not, and the setter also accepts a list of values, one for eachPVData. This means theself.updatemethod does not take any arguments beside aquery_timeout, that overwritesself.connector.query_timeoutif passed;query_max_concurrencyandquery_timeoutwere added to ease setting of theself.connectorrespective properties;This PR was testes in the control room computers, using the script
gensumm_machreport.pyand other manual tests.