Stop JSON-encoding Messages

[paddycarver]

This doesn't really make a whole lot of sense. We should use a binary-safe encoding (perhaps msgpack?) instead.

Source: http://www.reddit.com/r/programming/comments/11sgc7/pastry_a_distributed_hash_table_in_go/c6pagbn

[paddycarver]

A quick test shows that msgpack works as expected. Some more thought and testing should be done on this, though.

[bketelsen]

msgpack hasn't been very well received in the programming world though. I suggest bson encoding. It's standard and more widely used. Just my 0.02

[paddycarver]

@bketelsen a fair point. bson seems like a good contender, especially as Gustavo already wrote an excellent package for it.

Is there a reason it's better than Protocol Buffers, though?

[bketelsen]

protocol buffers are often hard to use outside of Go. Very difficult from Ruby for example. Internally to Pastry it may not be an issue. We did this dance at Clarity and settled on BSON because of wide acceptance, wide availability of client libs, efficient(mostly) protocol that is still self-describing. Skynet now uses BSON RPC internally too. Standard Go RPC with bson encoding.

[paddycarver]

This seems like a good argument for BSON. Hm. I'll have to think on this.

[alecthomas]

Here's a naive benchmark of various native Go serializers.

BSON is fastest, but only negligibly. However for pure structs with basic types (no maps or lists), BSON was significantly faster.

[paddycarver]

@alecthomas I was just looking at that :) I'm going to take a stab at swapping in vitess' bson serialiser. Vitess looks more desirable than mgo's simply because of the mgo data-loss problem you noted.