Fix NoiseSensitivity ragas API compatibility

[nibalizer]

Pull Request Description

What and why?

Fixed a TypeError in the NoiseSensitivity test caused by incompatibility with the ragas library API. The ragas NoiseSensitivity metric no longer accepts a focus parameter in its constructor, which was causing the test to fail with TypeError: NoiseSensitivity.__init__() got an unexpected keyword argument focus.

Removed the focus=focus parameter from the noise_sensitivity() metric instantiation while keeping the parameter available in the wrapper function for backward compatibility.

How to test

Run the NoiseSensitivity test using the example code:

run_test(
    "validmind.model_validation.ragas.NoiseSensitivity",
    inputs={"dataset": vm_test_ds},
    params={
        "user_input_column": "question",
        "response_column": "rag_model_prediction",
        "reference_column": "ground_truth",
        "retrieved_contexts_column": "retrieval_model_prediction",
    },
).log()

The test should now execute without the TypeError.

What needs special review?

The focus parameter is still accepted by the wrapper function but is no longer passed to the underlying ragas metric. Verify that this doesn't break existing behavior or if the parameter should be removed entirely from the function signature.

Dependencies, breaking changes, and deployment notes

• Requires ragas version >=0.2.3,<=0.2.7 (already specified in pyproject.toml)
• No breaking changes to the public API

Release notes

Fixed compatibility issue with ragas library in NoiseSensitivity test that was causing TypeError when running the test.

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

[github-actions]

Pull requests must include at least one of the required labels: internal (no release notes required), highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

PR Summary

This PR removes the explicit 'focus' parameter from the call to the noise_sensitivity metric in the NoiseSensitivity test module. The change ensures that the metric is now invoked with its default configuration rather than using a specific focus. This update streamlines the metric invocation and aligns it with its revised parameter requirements as defined in the metric's implementation.

Test Suggestions

• Add tests to verify that noise_sensitivity produces the expected output using its default configuration.
• Test the functionality to ensure that removing the 'focus' parameter does not adversely affect overall behavior.
• Include regression tests that compare previous and current outputs to validate the change.

[juanmleng]

I am not able to reproduce this error. Using ragas 0.2.7

[AnilSorathiya]

I am not able to reproduce this either.

[juanmleng]

@nibalizer We can probably close this PR. We’ve capped the latest version of Ragas to 0.2.7, which implements the focus parameter. In newer versions—particularly version 0.2.14—the focus parameter was renamed to mode.

Ragas 0.2.7 (our version)

class NoiseSensitivity(MetricWithLLM, SingleTurnMetric):
    name: str = "noise_sensitivity"
    focus: t.Literal["relevant", "irrelevant"] = "relevant"
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {
            MetricType.SINGLE_TURN: {
                "user_input",
                "response",
                "reference",
                "retrieved_contexts",
            }
        }
    )

Ragas 0.2.14 (when focus was renamed to mode)

class NoiseSensitivity(MetricWithLLM, SingleTurnMetric):
    name: str = "noise_sensitivity"
    mode: t.Literal["relevant", "irrelevant"] = "relevant"
    _required_columns: t.Dict[MetricType, t.Set[str]] = field(
        default_factory=lambda: {
            MetricType.SINGLE_TURN: {
                "user_input",
                "response",
                "reference",
                "retrieved_contexts",
            }
        }
    )