[Help Wanted] Want projects to test drive TTNT

[Genki-S]

I am having trouble testing out TTNT in order to check if this tool is applicable to big projects like rails/rails. I am trying to build my own toy app and use TTNT with it, but I realized this is making my development really slow.

If you have some projects you can use TTNT on, please let me know. I am searching for projects which meet these criteria:

• Uses Ruby
• Uses Minitest or RSpec for testing
• Have good amount of testing code (hopefully over 10 test files)
• In active development and code changes on a daily basis
• Project is not THAT big: Making hundreds of developers confused when TTNT messes things up is not desirable

What I want to test is:

• Whether or not there are cases TTNT cannot select failing test
  • for now this is expected when code change is introduced on native extension C code (because coverage cannot reach to native code)
• How much impact TTNT can have in terms of cutting off the time of test run

Regarding this, I want to run 2 kinds of tests (test without TTNT and with TTNT) each time (e.g. each time CI runs). And I want to collect data on those runs. So with this experiment some delay of CI is expected.

Thank you for your interest!

[GolfyMcG]

@Genki-S - hey there! My team (github.com/healthify) but had a few questions:

1. We use RSpec - when will that be supported?
2. We are on Ruby 2.1.5 - do you think that will be a problem?
3. I'm slightly confused by how this works. The line sounds great:

   Given base commit and target commit, output test files you should run
   But I want to understand this a little better:
   Generate test-to-code mapping for a given commit
   Any more specifics on how you envision this actually working out on a project with a few hundred tests that's constantly running on CI would be great. The README explains some of the underlying concepts but I wasn't sure what's automated and what's manual.

Any clarity would be great - thanks!

[Genki-S]

Hello @GolfyMcG, thank you for your interest! Let me answer your questions:

1. I will implement RSpec adapter soon. I believe it does not take me more than 2 weeks.

2. Because all tests pass with my Ruby 2.1.6, I don't think it should be a problem. I am happy to fix problems along the way if those happen, though.

3. I'm sorry the README is confusing (and actually it's a little obsolete). Thank you for your feedback, I will definitely update it. As a quick answer, the statement

   Generate test-to-code mapping for a given commit

   is kind of underlying process. All you need to do when using TTNT is (1) "anchoring" and (2) "run". With anchoring, you run $ rake ttnt:test:anchor command and then the commit your HEAD points to (the commit you are on now) will be marked as "base" commit. When you make new commits, you can run $ rake ttnt:test:run to select the tests you should run (regarding changes between "base" commit and current HEAD commit). This task automatically runs selected tests in current implementation.

   So, in a nutshell, you have to execute $ rake ttnt:test:anchor manually from time to time to create "base" commits. Otherwise, you can basically use $ rake ttnt:test:run instead of your usual $ rake test thing (if you use rake for testing).

As for how I envision TTNT to run on CIs -- I hope one day all test run is replaced by TTNT (since its job is to filter out tests which you don't have to run. So basically TTNT can (1) make test run faster and (2) catch all test failures introduced by changes you make).

While testing phase, I want subjects to run 2 test suites on CIs. One is whole suite test run, and the other is TTNT test run (in order to confirm TTNT does not filter out failing tests). The downside is that this approach makes CI queue fill up by 2X speed.

I will update the README but if you have any further questions please let me know.
Thanks! 😄

[GolfyMcG]

This was really useful! We're definitely still interested but I can't make any promises. Feel free to post here when you've got an update on RSpec and I'll have the rest of the team check it out!

[Genki-S]

Got it! I'll follow up when RSpec adapter is implemented 👍

[dleve123]

@Genki-S

@GolfyMcG's colleague here!

What's the story for the RSpec adapter looking like? Is it still on this project's roadmap?

Have a good holiday!

[arturopie]

@Genki-S I'm interested in adding RSpec support to this gem. What's the status on that? How can I help?

[arturopie]

@Genki-S are you still maintaining this library?

[Genki-S]

I'm sorry being so late to reply. I'm having a long vacation now, which ends shortly. I'm willing to continue developing TTNT starting again from next April (I can't code until I go back home).

@arturopie RSpec support is in progress but far from complete (and it's a shame I don't have the code with me now). It would be great if you could help me! You can implement it and open a pull request if you want.