Cannot export large RRGs with Capnp

[AlexPoupakis]

Trying to write a large RRG as a binary file fails with the message tried to allocate list with too many elements. The RRG in question has ~250M nodes and ~750M edges. It seems that Capnp has a size limit of 2^29-1 (~536M) elements on individual Lists.

Expected Behaviour

The serializer should be able to write the RRG as a binary file regardless of the number of nodes/edges. Moreover, the serializer is especially useful for large RRGs, since their xml version can be >50 GB, making reading/writing in xml format unnecessarily time-consuming.

Current Behaviour

Executing VPR with the flag --write_rr_graph rr_graph.bin for large RRGs yields the error capnp/layout.c++:1227: failed: tried to allocate list with too many elements

Possible Solution

The bug exists because in the /libs/libvtrcapnproto/gen/rr_graph_uxsdcxx.capnp, the RrNodes and RrEdges structs define the nodes and edges respectively as Lists.

A potential fix, with backward compatibility in mind, is implementing a chunked schema and adding extra fields to struct RrGraph. Depending on the number of elements, the edges can be either stored as struct RrEdges or as struct RrEdgesChunks by modifying the reader & writer.

The existing /libs/libvtrcapnproto/gen/rr_graph_uxsdcxx.capnp looks like:

struct RrNodes {
	nodes @0 :List(Node);
}

struct RrEdges {
	edges @0 :List(Edge);
}

struct RrGraph {
	toolComment @0 :Text;
	toolName @1 :Text;
	toolVersion @2 :Text;
	channels @3 :Channels;
	switches @4 :Switches;
	segments @5 :Segments;
	blockTypes @6 :BlockTypes;
	grid @7 :GridLocs;
	rrNodes @8 :RrNodes;
	rrEdges @9 :RrEdges;
}

With the proposed solution it would look like:

struct RrNodes {
	nodes @0 :List(Node);
}

struct RrNodesChunks {
    chunks @0 :List(RrNodes);
}

struct RrEdges {
	edges @0 :List(Edge);
}

struct RrEdgesChunks {
    chunks @0 :List(RrEdges);
}

struct RrGraph {
	toolComment @0 :Text;
	toolName @1 :Text;
	toolVersion @2 :Text;
	channels @3 :Channels;
	switches @4 :Switches;
	segments @5 :Segments;
	blockTypes @6 :BlockTypes;
	grid @7 :GridLocs;
	rrNodes @8 :RrNodes;
	rrEdges @9 :RrEdges;
	rrNodesChunks @10 :RrNodesChunks;
	rrEdgesChunks @11 :RrEdgesChunks;
}

Context

Working with large, externally loaded RRGs is cumbersome via the xml format due to the large sizes and high read/write times. Supporting arbitrarily-sized RRGs via Capnp is both the expected behavior and the only way to reduce VPR's overhead.