Skip to content

feature: GenerationLog option support in nemoguardrails with colang 2 #1108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task done
nirmittumeryk opened this issue Apr 10, 2025 · 1 comment
Open
1 task done
Labels
duplicate This issue or pull request already exists enhancement New feature or request

Comments

@nirmittumeryk
Copy link

nirmittumeryk commented Apr 10, 2025

Did you check the docs?

  • I have read all the NeMo-Guardrails docs

Is your feature request related to a problem? Please describe.

Currently, the GenerationOptions class includes a log option with fields like activated_rails, llm_calls, internal_events, and colang_history. These work in colang 1.x, but using them with Colang 2.0 raises a ValueError due to lack of support. This limits the or analyze the system when using newer Colang flows.

We rely on this logging capability to capture product-level metrics (e.g., which rails were activated, which certain actions were triggered, LLM calls, etc.) that are essential to our observability stack. Losing this in Colang 2.0 limits our ability to track user interactions, rail usage, and model behavior.

Describe the solution you'd like

We’d like support for the log option in GenerationOptions to be added for Colang 2.0.

Describe alternatives you've considered

Describe alternatives you've considered
Reverting to Colang 1.x.

Additional context

activated rails:

if (
options.log.activated_rails
or options.log.llm_calls
or options.log.internal_events
or options.log.colang_history
):
raise ValueError(
"The log option is not supported for Colang 2.0 configurations."
)

@nirmittumeryk nirmittumeryk added enhancement New feature or request status: needs triage New issues that have not yet been reviewed or categorized. labels Apr 10, 2025
@Pouyanpi Pouyanpi changed the title feature: feature: GenerationLog option support in nemoguardrails with colang 2 Apr 10, 2025
@Pouyanpi
Copy link
Collaborator

Thank you @nirmittumeryk for opening this issue. We are aware of this limitation and we are tracking it at #795 .

@Pouyanpi Pouyanpi added duplicate This issue or pull request already exists and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants