Support parallel execution of test steps/cases/modules inside an ETS #47
Description
Background and Motivation:
Performance optimisation is required to reduce startup time and validation time; reduction of startup time will simplify cloud deployment horizontal scaling, while reduction in validation time will be helpful while integrating ETF with INSPIRE Geoportal, or any other Metadata related workflow/pipeline.
One identified performance issue is that test inside each Executable Test Suite are executed sequentially, not in parallel.
Proposed change
Include parallelisation inside the ETS, at the level of Test Modules, Test Cases, Test Steps or Assertions. The responsibility of the threads remains in the ETS itself, i.e. it is up to the test developer and the capabilities of the test engine whether and how items are parallelised.
There must be a communication between the ETS and the Framework:
- the ETS must inform ETF if it supports parallel execution with other ETSs.
- ETF then has to run a limited number of ETS in parallel. The number of parallel executed ETS depends on the total system resources and the current utilisation by a number of users.
- The ETS must indicate whether it can execute test items (Test Cases, Assertions, etc) in parallel.
- The ETF then has to decide how many threads are allowed to run concurrently based on system load and inform the ETS.
Alternatives
n/a
Funding
JRC will be ready to fund within its current development contract.
Additional information
n/a