Enable automated testing on Travis CI

[mbrukman]

Right now, our Travis config only runs a build, but no tests, because the full suite times out: Travis CI build/test jobs time out after 50 minutes, and the test suite requires far longer than that.

Note that a Travis CI has a concept of a Build which consists of one or more Jobs: each Job has a 50 minute timeout, but there's no limit (to my knowledge) on the number of Jobs that can be included in a single Build, so in theory, we can split up the test suite as needed to fit into the 50 minute timeout.

The goal of this issue is to enable automated test runs for each PR and on the master branch on Travis CI. To accomplish this, we may also need to address one or more of the following issues:

• #5: Categorize tests by runtime
• #18: Add timeouts to test classes and methods

[sjudeng]

@mbrukman See comment at end of #5. I think there might be a non-timeout issue running tests under Travis. But on the timeout issue note if you break tests out by module all except hbase and cassandra pass under 50 minutes.

[sjudeng]

Regarding running hbase and cassandra tests under Travis, again assuming no non-timeout issues crop up, how about grouping tests as part of Travis build matrix?

env:
  matrix:
    - MODULE='cassandra' ARGS='-Dtest=ThriftPartitionGraphTest'
    - MODULE='cassandra' ARGS='-Dtest=ThriftDistributedStoreManagerTest'

The largest run time for a test class I see is about 1500 seconds, well under 50 minute timeout. Could group tests as well based on timing (e.g. -Dtest=ThriftPartitionGraphTest,ThriftDistributedStoreManagerTest). I'd write a script to generate the matrix based on test output from a successful run. It would be a huge ugly test matrix but given the choice I'd prefer this in Travis configuration rather than modifying JanusGraph test classes with category annotations. For one it is really all just to accomodate a (not unreasonable) Travis limitation. Also there would be only one file to maintain.

[mbrukman]

Partial progress made: PR #85 adds testing of all modules via Travis CI, even though some of them are marked as optional, because they are either known to fail, or fail non-deterministically.

The following changes have been merged to fix tests:

• #76: fix Cassandra keyspace regex
• #77: ensure Log4j is used for testing

[sjudeng]

@mbrukman With the merge of #384 we now have all default tests in all modules running under Travis. All tests pass consistently and sporadic job failures can (and should) be restarted individually by reviewers/maintainers.

https://travis-ci.org/JanusGraph/janusgraph/builds/251247896

Given this I think we can close out this issue and also #5 and #18, which are no longer needed. The only other thing I can think of on this would be to add support for running the TinkerPop tests through Travis. But I think that can be another issue, and actually I'd probably advocate we not pursue that in order to keep our Travis usage at a reasonable level.