SwiftOtter-SOP-348 Enable Compiled Configuration Cache for Test execution

[lbajsarowicz]

Description (*)

The cache named compiled_config hosts all the di.xml compilation outcomes, including information about plugins, virtual classes etc.

With Magento 2.4.8 upgrade, the performance degradation of tests execution for multiple projects became significant. Mostly, due to the CPU usage.

Turning on compiled_config mitigates the issue, slightly improving the overall Integration Tests performance, without affecting areas such as Configured Object Manager.

2.4.6-p12

Wall Time     20.5s
I/O Wait      3.24s
CPU Time      17.2s
Memory        269MB
Network       1.27s  21.7KB    19rq
SQL           793ms  1871rq

2.4.8-p3

Wall Time  1min 32s
I/O Wait      1.69s
CPU Time   1min 30s
Memory        383MB
Network         n/a     n/a     n/a
SQL           501ms  1746rq

Related Pull Requests

Fixed Issues (if relevant)

1. Integration Tests performance degradation after 2.4.8 upgrade

Manual testing scenarios (*)

1. ...
2. ...

Questions or comments

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] SwiftOtter-SOP-348 Enable Compiled Configuration Cache for Test execution #40253: SwiftOtter-SOP-348 Enable Compiled Configuration Cache for Test execution

[m2-assistant]

Hi @lbajsarowicz. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.
❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

---

For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

[Den4ik]

@magento run all tests

[engcom-Hotel]

@magento create issue

[lbajsarowicz]

@magento run Integration Tests, Functional Tests B2B

[engcom-Hotel]

@magento run all tests

[engcom-Hotel]

@magento run all tests

[engcom-Hotel]

@magento run Integration Tests

[engcom-Hotel]

@magento run all tests

[engcom-Hotel]

Hello @lbajsarowicz,

Thank you for your contribution!

Please have a look into the integration test failure, it seems it is failing due to the PR changes.

[lbajsarowicz]

@engcom-Hotel I don't have access to Adobe Commerce (B2B) to verify

1) Magento\PurchaseOrder\Model\Notification\NotificationTest::testNotificationApprovalRequired with data set "offline_payment_method" ('checkmo', 'Check / Money order')
Failed asserting that null is not null.

For me it looks like the root cause of the issue is invalid test isolation for that specific test, as the failure happens in

        $transportBuilderMock = $this->objectManager->get(TransportBuilderMock::class);
        $sentMessage = $transportBuilderMock->getSentMessage();
        $this->assertNotNull($sentMessage);

[lbajsarowicz]

@magento run Integration Tests

[engcom-Hotel]

I don't have access to Adobe Commerce (B2B) to verify

1) Magento\PurchaseOrder\Model\Notification\NotificationTest::testNotificationApprovalRequired with data set "offline_payment_method" ('checkmo', 'Check / Money order')
Failed asserting that null is not null.

For me it looks like the root cause of the issue is invalid test isolation for that specific test, as the failure happens in

        $transportBuilderMock = $this->objectManager->get(TransportBuilderMock::class);
        $sentMessage = $transportBuilderMock->getSentMessage();
        $this->assertNotNull($sentMessage);

Okay @lbajsarowicz, let me ask someone from the internal team to check this issue.

@engcom-Dash can you please look into the B2B integration failure.

[lbajsarowicz]

@engcom-Hotel and @engcom-Dash I've checked there are 17 tests in \Magento\PurchaseOrder\Model\Notification\NotificationTest, only one test fails. That is additional proof that the issue is single-test-specific.

Knowing B2B from the past, that's very likely that the Queue hasn't been cleared correctly after previous test, resulting in

$consumer = $this->consumerFactory->get(self::CONSUMER_NAME);
$consumer->process(1);

Consumer processing the message published by previous task, as a result the Queue isn't sending email message, so the Transport Mock isn't reporting existence of a message.