Streaming API

[synthfi]

One of MessagePack's design ideas is that it should be easy to implement a streaming API, such as in an server-client situation.

How can the internal design be worked so that the unpacking API can work in this fashion? So that something like:

mp = msgpack()
call mp%streaming_mode(1000) ! create an internal buffer of 1000 elements
do
  ! get data from somewhere, store in the `buf` array
  call mp%load_data(buf)
  if (mp%failed()) then
    ! error handling
  end if
  if (mp%value_available()) then
      call mp%next_value(mp_value)
      ! mp_value will be the unpacked value
  end if
end do

would work?

[synthfi]

Useful procedures:

• streaming_mode(num). Sets up an internal buffer of num elements, turns on streaming mode
• normal_mode(). Turns off streaming mode

Streaming only:

• load_data(buf). copies data into internal buffer
• value_available(). peers into buffer and returns if a full value can be unpacked.
• next_value(mpv). Unpack a value from the buffer. If something goes wrong, returns mp_nil_type
• buffer_size(). Returns size of internal buffer
• buffer_used(). Returns number of bytes used in the internal buffer
• clear_buffer(). Clears the internal buffer of all data.

Would most likely want to use a circular buffer to prevent unnecessary copies and allow for easy clearing. The value_available procedure would trigger the most refactoring, as full unpacking is not necessary, only calculation of header size and checking against the full size of the buffer.

[synthfi]

basic implementation added in #37, though more still needs to be done