New Cassandra backend using CQL w/ prepared statements

[rustyrazorblade]

Thrift is deprecated and will be removed in Cassandra 4.0.

For perf reasons, avoid IN() clauses for selects, prefer concurrent async queries.

[sharpau]

A request for any new Cassandra backend work: please consider better handling for vertices with large numbers of edges. In particular, although one can partition adjacency lists in Titan (http://s3.thinkaurelius.com/docs/titan/1.0.0/graph-partitioning.html), the results of traversals are still subject to hitting the Cassandra frame size limit, even though each individual partition should not.

Given how well the Tinkerpop API adheres to a lazy, streaming data model, it's frustrating when Titan's backend implementation does not.

[dylanht]

@sharpau thank you for the advice. I'm going to self-assign this one, and intend to deprecate thrift support.

[sjudeng]

The thread below mentions using thrift to enable support for ScyllaDB. Would this capability be affected by the proposed updates?

scylladb/scylladb#693
#53

[rustyrazorblade]

@dylanht I can help with the data modeling on this. I'm one of the first people to put titan + cassandra in production, and seen where it fails (miserably).

[dylanht]

@sjudeng I just had a look over ScyllaDB's docs on Cassandra compatibility and it's my understanding that we can use CQL Binary Protocol with ScyllaDB just as well as Thrift (the REGISTER and AUTH_RESPONSE requests, and the AUTHENTICATE response are not supported by Scylla in case that alarms anyone), and reading through the comments in the issue you linked it seems Thrift support in ScyllaDB was actually the initial barrier to integration. Now that they have implemented support for Thrift scylladb/scylladb#399 and given the patches @duartAn linked recently that apparently fix the OLAP queries @pluradj was running, my impression is that things are "working" on ScyllaDB master via Thrift. Correct me if I'm wrong, but it looks like support for CQL in Titan especially with @rustyrazorblade offering to help us out (thank you Jon!) might immediately provide a more stable/performant integration with ScyllaDB.

I agree with the assessment of Netflix on moving away from Astyanax/Thrift, and I think deprecation or "maintenance mode" for the analogous components in JanusGraph is reasonable. To keep things clean for this work, I won't deprecate anything and we can just create a new backend configurable via storage.backend=cassandracql or just cql, and post a separate issue for deprecation of Astyanax/Thrift support. WDYT?

[rustyrazorblade]

+1 to backend = cql.

While I think it would be nice if ScyllaDB worked correctly, I don't know if that should be the main goal of a CQL implementation. Fortunately I think a reasonable design for a graph doesn't really require a lot of CQL that ScyllaDB doesn't implement, so it should work out.

Generally, it's a good idea to avoid the following Cassandra featuers:

• Materialized Views. They are still unstable. Currently lead to issues with bootstrap & repair.
• LWT (IF NOT EXISTS). These rely on Paxos and perform extremely poorly under contention, resulting in node failure.
• Secondary indexes. Performance wise, they're pretty terrible. Implemented as hidden tables under the hood.

[sjudeng]

@dylanht Thanks for the great info! Sounds promising it might just work out. As a separate effort from this we should probably add some tests to validate ScyllaDB support in legacy cassandra and/or cql backend.

[dylanht]

@rustyrazorblade +1 that the goal here should be integration with Cassandra and not ScyllaDB if I'm reading into your statement correctly. While ScyllaDB's blurb on licensing is reassuring, as they say some entities will not approve AGPLv3.0 regardless. It seems the pragmatic thing to do is to focus on Apache 2.0 licensed Cassandra which likely (speculation on my part, but if I had to guess) accounts for the majority of deployments and expertise out there among teams interested in JanusGraph anyways. Thank you for the guidance on features to avoid up-front. I'm planning to push initial commits to this branch over the weekend https://github.com/dylanht/janusgraph/tree/cql-support

@sjudeng +1 on an issue for adding some tests to support ScyllaDB running under both Thrift and CQL, it looks like we have existing deployments and a lot of interest carried over from those involved in the issue you linked to above. ScyllaDB team seems to like the idea of a reliable integration point, so IMO there's a ton of value added here for little effort and we can work things out with them if it comes to it.

[rustyrazorblade]

that the goal here should be integration with Cassandra and not ScyllaDB if I'm reading into your statement correctly.

Exactly. If ScyllaDB claims they're a drop in replacement for Cassandra, it's on them to provide the compatibility. I also haven't talked to a single person using it in production, so compromising on the data model of the graph would be a tradeoff with very little gain.

[fppt]

+1 on the general sentiments of dropping thrift regardless of ScyllaDB. The objective is to maintain the data model on a Cassandra backend. If Scylla claims to be a drop in replacement for Cassandra then it is up to them to provide the CQL interface.

[dylanht]

@fppt @rustyrazorblade agreed, I appropriated #53 for exploring/testing the "drop-in" of ScyllaDB, with the idea that we just test it according to what's existing for Cassandra and if it lives up to the claims great, we'll provide some basic documentation amounting to little more than changing the storage.backend value. If we hit snags and as we iterate on janusgraph-cassandra, at least we'll have it under test. I noted that #53 depends on this issue and will follow after, and I'm thinking we deal with Thrift/Astyanax backend deprecation/removal respectively in separate issues.