Split E2E Test tasks into multiple subtasks#1811
Open
Mythicaeda wants to merge 10 commits intodevelopfrom
Open
Split E2E Test tasks into multiple subtasks#1811Mythicaeda wants to merge 10 commits intodevelopfrom
Mythicaeda wants to merge 10 commits intodevelopfrom
Conversation
Mythicaeda
added
ci
Mythicaeda
force-pushed
the
refactor/speed-up-github-e2eTests
branch
from
ed75cca to
600aa1f
Compare
Upon further examination, all Action Server tests are currently unit tests that can be run with the entire system down, not E2E tests
Java currently takes multiple minutes to compile. This is more than sufficient time for Hasura to complete its start up tasks, instead of needing to delay for 30s after starting up.
- using `false` for this test run, but it should be configured to compare to develop (defaults to only caching `main` and `master`, which aren't branches in the repo)
…r" step - Setup Gradle includes the `validate wrapper` step - activate gradle caching during `setup gradle`
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Currently, the E2E Test task takes 15 minutes to execute, which is absurdly high. This PR aims to improve that number without making the tests flaky.
Breakdown of E2E Suite test speed locally (with services already setup):
:e2eTest:e2eTest:: ~6 mins:dbTest:e2eTest:: ~2 mins:sequencing-server:e2eTest:: ~1 min:action-server:e2eTest:: ~5 secThe first approach is to split the tests by suite into multiple parallel jobs. The suites should already be independent (especially as they're a mixture of Java and TS tests), but by running them in separate jobs we avoid any issues where two suites are trying to alter the DB at the same time. Additionally, by having them as separate jobs, it will be clearer which suite in particular is failing, as opposed to needing to go through the collected folder.
Breakdown of tests after this approach:
unitTests: 6 min (2 min running tests, 3.5 min assembling):e2eTest:e2eTest:: 11 min (5 min running tests, 3.5 min assembling):dbTest:e2eTest:: 4.5 min (4 min running tests):sequencing-server:e2eTest:: 7 min (1.5 min running tests, 3.5 min assembling the Java backend):action-server:e2eTest:: 5 min (10s running tests, 3 min assembling the Java backend)After that, I added the following improvements
e2e-testenvironment, it doesn't actually require the backend to be runningdevelopnode_modulesforaction-serverandsequencing-server. These seemed to not be cached by the gradle caching task, so they're now being manually cachedBreakdown of tests after all of this:
unitTests: 4.5 min (2 min running tests, 2 min assembling):e2eTest:e2eTest:: 9 min (5 min running tests, 2 min assembling):dbTest:e2eTest:: 4.5 min (4 min running tests):sequencing-server:e2eTest:: 5 min (1.5 min running tests, 2 min assembling):action-server:e2eTest:: 30s min (10s running tests)