bug: stream buffer for output rails not able to handle metadata

[andompesta]

Did you check docs and existing issues?

• I have read all the NeMo-Guardrails docs
• I have updated the package to the latest version before submitting this issue
• (optional) I have used the develop branch
• I have searched the existing issues of NeMo-Guardrails

Python version (python --version)

3.12.3

Operating system/version

Linux

NeMo-Guardrails version (if you must use a specific version and not the latest

0.14.0

Describe the bug

When using streaming-output rails with the include_generation_metadata flag some of the chunks generated by the streamhandler are not pure str, but rather dictionaries containing a string token and some additional metadata.

This creates some problem with the buffer consuming the streamingHandler queue: the buffer assumes all chunks in the queue to be strings. Thus the when concatenating the different chunks to obtain a partial prompt an error is raised.

Steps To Reproduce

if __name__ == "__main__":
    config = RailsConfig.from_path(
        "/nemoguardrails/nemoguardrails-configs/config/amdocs"
    )
    rails = LLMRails(config, verbose=True)
    # amdocs_framework = AmdocsFramework()

    # tell guardrail to call your framework for generation
    # rails.llm_generation_actions.passthrough_fn = amdocs_framework.entrypoint

    history = [
        {
            "role": "user",
            "content": "hi, how are you",
        }
    ]

    async def stream_chat():
        async for chunk in rails.stream_async(messages=history, include_generation_metadata=True):
            print(f"CHUNK: {chunk}")

    asyncio.run(stream_chat())

Expected Behavior

Generate chunks containing additional metadata in the async for while running streaming output rails

Actual Behavior

LLM Prompt (3cf9f..) - general
                                                                          
User                                                                      
hi, how are you                                                           

Traceback (most recent call last):
  File "/nemoguardrails/nemoguardrails-configs/src/amdocs/amdocs_run.py", line 114, in <module>
    asyncio.run(stream_chat())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/nemoguardrails/nemoguardrails-configs/src/amdocs/amdocs_run.py", line 111, in stream_chat
    async for chunk in rails.stream_async(messages=history, include_generation_metadata=True):
  File "/nemoguardrails/nemoguardrails/rails/llm/llmrails.py", line 1296, in _run_output_rails_in_streaming
    async for chunk_list, chunk_str_rep in buffer_strategy(streaming_handler):
  File "/nemoguardrails/nemoguardrails/rails/llm/buffer.py", line 76, in __call__
    self.generate_chunk_str(
  File "/nemoguardrails/nemoguardrails/rails/llm/buffer.py", line 102, in generate_chunk_str
    return "".join(new_chunks)
           ^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, dict found