Conversation
1e2fb3d to
86ad16e
Compare
317a0d1 to
a141abb
Compare
This reverts commit 71d7136.
Turn `PostgresDatabase` into a hook too.
Include overall Data Center plan with hook timings. Plans should be reusable with different ways of getting a `TcpHost`: * new via Docker * new via AWS * existing machines This commit injects an `Infrastructure` SPI to reuse a plan with different infra, but to prove the concept, it has to fit the current CloudFormation stack in aws-infrastructure / 2-nodes-dc*.yaml The API smells a little, because a big one-shot CFN stack is split into multiple fine-grained methods. Alternative API could be: `fun plan(host: TcpHost) : JiraNodePlan` Other matters to resolve: * loadbalancer (plan?) API * getting results out of DC instance and node hooks
Download reports per host.
2b0843a to
681c0df
Compare
|
Locally failing like this: https://scans.gradle.com/s/375dwevxnzzse |
To read: * https://danwalsh.livejournal.com/74095.html * bmuschko/gradle-docker-plugin#890 Maybe it's because of multiple port exports?
2cd6743 to
3aacf87
Compare
| hooks | ||
| .poll() | ||
| ?.call(ssh, jira, this, reports) | ||
| ?: break |
There was a problem hiding this comment.
Maybe this is not important, however I'm gonna note it:
If I'm not wrong it's possible to create a PostInstallHook that'd create an infinite loop here.
| private fun installInParallel(nodes: Collection<PlannedHttpNode>): List<PlannedInstalledJira> = nodes | ||
| .asSequence() | ||
| .asStream() | ||
| .parallel() |
There was a problem hiding this comment.
Are we fine with this parallel stream here?
Avoid errors in Jira logs like: ``` Caused by: java.lang.NullPointerException at sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1264) [?:1.8.0-adoptopenjdk] at sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:219) [?:1.8.0-adoptopenjdk] at sun.awt.FontConfiguration.init(FontConfiguration.java:107) [?:1.8.0-adoptopenjdk] at sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:774) [?:1.8.0-adoptopenjdk] at sun.font.SunFontManager$2.run(SunFontManager.java:431) [?:1.8.0-adoptopenjdk] at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0-adoptopenjdk] at sun.font.SunFontManager.<init>(SunFontManager.java:376) [?:1.8.0-adoptopenjdk] at sun.awt.FcFontManager.<init>(FcFontManager.java:35) [?:1.8.0-adoptopenjdk] at sun.awt.X11FontManager.<init>(X11FontManager.java:57) [?:1.8.0-adoptopenjdk] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [?:1.8.0-adoptopenjdk] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0-adoptopenjdk] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0-adoptopenjdk] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0-adoptopenjdk] at java.lang.Class.newInstance(Class.java:442) [?:1.8.0-adoptopenjdk] at sun.font.FontManagerFactory$1.run(FontManagerFactory.java:83) [?:1.8.0-adoptopenjdk] at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0-adoptopenjdk] at sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74) [?:1.8.0-adoptopenjdk] at java.awt.Font.getFont2D(Font.java:491) [?:1.8.0-adoptopenjdk] at java.awt.Font.canDisplay(Font.java:1980) [?:1.8.0-adoptopenjdk] at com.octo.captcha.component.image.fontgenerator.RandomFontGenerator.checkFontCanDisplayCharacters(RandomFontGenerator.java:223) [jcaptcha-2.0-alpha-1.jar:?] at com.octo.captcha.component.image.fontgenerator.RandomFontGenerator.cleanFontList(RandomFontGenerator.java:179) [jcaptcha-2.0-alpha-1.jar:?] at com.octo.captcha.component.image.fontgenerator.RandomFontGenerator.initializeFonts(RandomFontGenerator.java:94) [jcaptcha-2.0-alpha-1.jar:?] at com.octo.captcha.component.image.fontgenerator.RandomFontGenerator.<init>(RandomFontGenerator.java:81) [jcaptcha-2.0-alpha-1.jar:?] at com.atlassian.jira.servlet.CensoredCaptchaEngine.createFontGenerator(CensoredCaptchaEngine.java:86) [classes/:?] at com.atlassian.jira.servlet.CensoredCaptchaEngine.createWord2Image(CensoredCaptchaEngine.java:65) [classes/:?] at com.atlassian.jira.servlet.CensoredCaptchaEngine.buildInitialFactories(CensoredCaptchaEngine.java:48) [classes/:?] at com.octo.captcha.engine.image.ListImageCaptchaEngine.<init>(ListImageCaptchaEngine.java:24) [jcaptcha-2.0-alpha-1.jar:?] at com.atlassian.jira.servlet.CensoredCaptchaEngine.<init>(CensoredCaptchaEngine.java:39) [classes/:?] at com.atlassian.jira.servlet.JiraImageCaptchaServiceImpl.<clinit>(JiraImageCaptchaServiceImpl.java:13) [classes/:?] ... 137 more ```
Fix `DockerIT`.
Perhaps it will consume less disk space in peaks.
Try to avoid error on GitHub Actions:
```
com.atlassian.performance.tools.infrastructure.api.distribution.PublicJiraServiceDeskDistributionIT > shouldDownloadJiraServiceDesk FAILED
java.lang.Exception at PublicJiraServiceDeskDistributionIT.kt:17
Caused by: net.schmizz.sshj.connection.ConnectionException at PublicJiraServiceDeskDistributionIT.kt:17
Caused by: java.util.concurrent.TimeoutException at PublicJiraServiceDeskDistributionIT.kt:17
java.io.IOException: No space left on device
com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device
at com.esotericsoftware.kryo.io.Output.flush(Output.java:165)
```
3aacf87 to
48bb603
Compare
|
|
||
| class Builder( | ||
| private var plan: JiraNodePlan | ||
|
|
|
|
||
| internal fun call(nodes: List<PreInstallHooks>, reports: Reports) { | ||
| hooks | ||
| .parallelStream() |
There was a problem hiding this comment.
Another usage of parallel stream. Isn't this risky?
| private fun download(ssh: SshConnection) { | ||
| ssh.execute("sudo mkdir -p $localHome") | ||
| val jiraHome = jiraHomeSource.download(ssh) | ||
| ssh.execute("sudo mv $jiraHome/{data,plugins,import,export} $localHome") |
There was a problem hiding this comment.
Note: Today I noticed that this line will fail when jirahome is empty. I'm writing a test for this in aws-infrastructure with the old API.
In case you'd try to use empty DC instance this may be a problem.
There was a problem hiding this comment.
Fixing it in old API: atlassian/aws-infrastructure#176
Possibly same change should be applied here.
7722512 to
59bceaf
Compare
Tests were consuming too much space in CI, up to ~15 GB in spikes
59bceaf to
c46b1f6
Compare
| import java.time.Duration | ||
| import java.time.Instant | ||
|
|
||
| class JiraPlanIT { |
There was a problem hiding this comment.
Would be good to have also a test for starting empty Jira (no dataset) without setup.
Consumers should provide a working path
Fix errors like:
```
java.lang.Exception: Failed to Installing and configuring apache load balancer despite 2 attempts
at com.atlassian.performance.tools.jvmtasks.api.IdempotentAction.retry(IdempotentAction.kt:42)
at com.atlassian.performance.tools.infrastructure.api.loadbalancer.ApacheProxyPlan.materialize(ApacheProxyPlan.kt:29)
at com.atlassian.performance.tools.infrastructure.api.jira.instance.JiraDataCenterPlan.materialize(JiraDataCenterPlan.kt:30)
at com.atlassian.performance.tools.awsinfrastructure.api.jira.HooksDataCenterFormulaIT$shouldProvisionDc$1.invoke(HooksDataCenterFormulaIT.kt:121)
at com.atlassian.performance.tools.awsinfrastructure.api.jira.HooksDataCenterFormulaIT$shouldProvisionDc$1.invoke(HooksDataCenterFormulaIT.kt:41)
at com.atlassian.performance.tools.awsinfrastructure.api.jira.HooksDataCenterFormulaIT.makeFailureObservable(HooksDataCenterFormulaIT.kt:70)
at com.atlassian.performance.tools.awsinfrastructure.api.jira.HooksDataCenterFormulaIT.shouldProvisionDc(HooksDataCenterFormulaIT.kt:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.lang.Thread.run(Thread.java:750)
Suppressed: java.lang.RuntimeException: Potential Jira problems found in workspace: {}
at com.atlassian.performance.tools.awsinfrastructure.api.jira.HooksDataCenterFormulaIT.makeFailureObservable(HooksDataCenterFormulaIT.kt:74)
... 48 more
Caused by: java.lang.Exception: Error while executing sed -i -r 's/Listen 80/Listen 80/g' /etc/apache2/ports.conf. Exit status code SshResult(exitStatus=4, output=, errorOutput=sed: couldn't open temporary file /etc/apache2/sed3Zlw2Y: Permission denied
)
at com.atlassian.performance.tools.ssh.SshjConnection.execute(SshjConnection.kt:40)
at com.atlassian.performance.tools.ssh.api.SshConnection.execute(SshConnection.kt:27)
at com.atlassian.performance.tools.infrastructure.api.Sed.replace(Sed.kt:14)
at com.atlassian.performance.tools.infrastructure.api.loadbalancer.ApacheProxyPlan.tryToProvision(ApacheProxyPlan.kt:37)
at com.atlassian.performance.tools.infrastructure.api.loadbalancer.ApacheProxyPlan.access$tryToProvision(ApacheProxyPlan.kt:17)
at com.atlassian.performance.tools.infrastructure.api.loadbalancer.ApacheProxyPlan$materialize$1.invoke(ApacheProxyPlan.kt:27)
at com.atlassian.performance.tools.infrastructure.api.loadbalancer.ApacheProxyPlan$materialize$1.invoke(ApacheProxyPlan.kt:17)
at com.atlassian.performance.tools.jvmtasks.api.IdempotentAction.retry(IdempotentAction.kt:28)
... 53 more
```
906087f to
0b3c72c
Compare
1634a16 to
90b1e0e
Compare
90b1e0e to
cacb5f0
Compare

This is the 2023 version of #103
Rebased, resolved a lot of conflicts and cleaned up a little. Turns out many changes were done independently on
masteralready. Still gotta squash them commits...