Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions .github/workflows/pr_qc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,16 @@ jobs:
steps:
- name: Check out repository code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: false
python-version: 3.11
- name: Run qa
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
pip install ".[dev]"
python -m venv ./.venv
source ./.venv/bin/activate
make dev
make qa
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,16 +153,18 @@ Note:

**Parameters**

- **`value`** _(Any)_: The input value to validate.
- **`metadata`** _(dict)_: A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.
- **`value`** *(Any)*: The input value to validate.
- **`metadata`** *(dict)*: A dictionary containing metadata required for validation. Keys and values must match the expectations of this validator.


| Key | Type | Description | Default |
| --- | --- | --- | --- |
| `user_input` | String | The original user query passed into the RAG system. | N/A (Required) |
| `retrieved_context` | String | The context retrieved and used by the RAG system. | N/A (Required) |
| `min_range_value` | String | The minimum value for the rating range used by the LLM judge. | 0 (the default of the validator class) |
| `max_range_value` | String | The maximum value for the rating range used by the LLM judge. | 1 (the default of the validator class) |
</ul>
</ul>

**Returns**

**`ValidationResult`**: Returns a `PassResult` if the LLM judge's rating meets or exceeds the pass threshold, or a `FailResult` with detailed explanation if the rating is below the threshold.
**`ValidationResult`**: Returns a `PassResult` if the LLM judge's rating meets or exceeds the pass threshold, or a `FailResult` with detailed explanation if the rating is below the threshold.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "rag-context-evaluator"
version = "0.0.1"
version = "0.0.2"
description = "ML3 RAG Context Evaluator"
authors = [
{name = "MLcube", email = "info@mlcube.com"}
Expand Down
4 changes: 2 additions & 2 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions validator/main.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
from typing import Any, Callable, Dict, Optional
from guardrails import Guard
import openai
import logging
from guardrails.validator_base import (
FailResult,
Expand Down
Loading