Protocol Buffers for Swish

This repository provides a Swish implementation of Google's Protocol Buffers.

Message definitions

(define-message type (field-name field-type field-number) ...)

The following field types are supported:

• Integer: int32, int64, uint32, uint64, sint32, sint64, fixed32, fixed64, sfixed32, and sfixed64
• Enumeration: (enum enum-type)
• Floating-point: float and double
• Boolean: bool
• String: string
• Bytevector: bytes
• Message: (message message-type)
• List of any of the above: (list type)
• Map: (map key-type value-type)
• Flonum-only vector: (flvector float) and (flvector double)

Each message definition produces a corresponding tuple definition:

(define-tuple type (field-name ...))

Each message definition uses define-property to associate the procedures that read, size, merge, and write it with type.

Enumeration definitions

(define-enum type (name value) ...)

Each enumeration definition produces a macro that converts the member name to its int32 value. Each enum field stores the int32 value, not a symbol. The field may store an int32 value not specified in the enumeration. Use (type name) to express the int32 value of member name in enumeration type. We don't use symbols because there is not a guaranteed one-to-one mapping from values to symbols.

Limitations

• Field presence is not tracked, and default values are not serialized.
• One-of fields are treated like regular fields.
• Groups and extensions are not supported.
• The writer does not honor the [packed=false] option, but the reader supports both packed and unpacked repeated scalar numeric fields.
• Definitions are not validated.
• The reader does not eliminate duplicate map keys.

Representation

• Integer fields are represented by exact integers (fixnums and bignums). The default value is 0.
• Enum fields are represented by exact integers. The default value is 0.
• Floating-point fields are represented by flonums and can contain any real number. The default value is 0.0.
• Boolean fields are represented by booleans. The default value is #f.
• String fields are represented by strings with the UTF-8 encoding. The default value is the empty string.
• Bytes fields are represented by bytevectors. The default value is the empty bytevector.
• Message fields are represented by a tuple or #f for missing. The default value is #f.
• Repeated fields are represented by either a list of the underlying type or an flvector. The default value is the empty list or the empty flvector.
• Map fields types are represented by a list of pairs. For each pair, the car is the key, and the cdr is the value. The default value is the empty list.

Maker

(make-message type (field value) ...)

This macro makes a message of type with each specified field set to its corresponding value. Unspecified fields are set to their default values.

Reader

(read-message type ip [limit])

This macro reads a message of type from binary input port or bytevector ip. When limit is present, it specifies the maximum number of bytes to read.

Writer

(write-message type e)

This macro writes a message e of type to a bytevector.

(write-message type e op)

This macros writes a message e of type to binary output port op.

(write-message type e op size)

This macro calls procedure size with the number of bytes required to write message e of type and then writes message e to binary output port op.

Because the protobuf format uses length prefixes for nested fields, the size of a nested object must be known before it can be streamed. The simple approach of buffering the nested object results in quadratic allocation for deeply nested messages. To eliminate this inefficiency, the writer first traverses the message and caches in an eq-hash-table the sizes of nested messages, packed scalars, and pairs in maps. It can then generate the output without any intermediate buffers.

Scheme Protocol Buffer Compiler Plugin

generator.ss is the source code for the protoc-gen-scheme protoc plugin. It generates S-expression definitions of messages and enumerations from protobuf text definitions.

Use protoc-gen-scheme

protoc --plugin=path-to-protoc-gen-scheme --scheme_out=OUT_DIR PROTO_FILES

Extensions

This plugin supports an extension to represent repeated floating-point numbers as an flvector. At the command line include the following option: --proto_path=path-to-protoc-gen-scheme

In your protobuf configuration file, import swish_options.proto, and add the option [(swish.protobuf.swishtype)="flvector"] to each field. An example is shown below.

import "swish_options.proto";

message ExampleMessage {
  repeated double numbers = 1 [(swish.protobuf.swishtype)="flvector"];
}