Redesign test utilities #1234
Description
While some test abstractions and utilities in Spring Shell 3 are quite useful (like ShellScreen, ShellAssertions and @ShellTest), some other APIs are actually designed to test Spring Shell's code instead of the user's code (which essentially should be command execution).
APIs like InteractiveShellSession and NonInteractiveSession provide no added value from a command execution perspective (the outcome of a command should be the same regardless of the execution mode). These APIs actually test that the implementations of Spring Shell runners are correct, and not really the execution of the user's command. For instance, here is an example from the reference docs of v3:
@ShellTest
@DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
class InteractiveTestSample {
@Autowired
ShellTestClient client;
@Test
void test() {
InteractiveShellSession session = client
.interactive()
.run();
await().atMost(2, TimeUnit.SECONDS).untilAsserted(() -> {
ShellAssertions.assertThat(session.screen())
.containsText("shell");
});
}
}In this test code, it is not obvious which commands are tested and where they are defined or registered. This test actually shows that Spring Shell is doing its job correctly (and therefore has no added value to be written by the end user). Moreover, this test requires asynchronous test utilities to wait before being able to assert on the result. I also don't understand why we need to reload the test context after each test method (usage of @DirtiesContext).
The ShellTestClient should rather provide an API to send a command (as if the user typed it in the shell) and return a shell screen (as if the shell printed it) ready for assertions, something like:
@ExtendWith(SpringExtension.class)
class ShellTestClientTests {
@Test
void testCommandExecution(@Autowired ShellTestClient shellTestClient) throws Exception {
// when
ShellScreen shellScreen = shellTestClient.sendCommand("test");
// then
ShellAssertions.assertThat(shellScreen).containsText("Test command executed");
}
@Configuration
static class TestCommands {
@Bean
public Command test() {
return new AbstractCommand("test", "A test command") {
@Override
public ExitStatus doExecute(CommandContext commandContext) {
commandContext.outputWriter().println("Test command executed");
return ExitStatus.OK;
}
};
}
}
}From a user's perspective, a shell command test should be as simple as that.
In a Spring Boot world, the user should be able to define a Spring Shell Boot application and test commands as follows:
@SpringBootApplication
public class ExampleShellApplication {
@Command(name = "hi", description = "Says hello")
public String hello() {
return "hello";
}
}
@ShellTest
@ContextConfiguration(classes = ExampleShellApplication.class)
class ShellTestIntegrationTests {
@Test
void testCommandExecution(@Autowired ShellTestClient client) throws Exception {
// when
ShellScreen shellScreen = client.sendCommand("hi");
// then
ShellAssertions.assertThat(shellScreen).containsText("hello");
}
}The code currently in the testing modules is overly complex for no real added value (asynchronous executions with blocking queues, the need for asynchronous test utilities, an entire (!) shaded jediterm fork just for tests, etc). Moreover, these utilities do not seem to be quite stable (the interactive session tests hang from time to time) and they are explicitly documented as such (see #1225).
Spring Shell 4 is a good opportunity to redesign and simplify test utilities.