Skip to content

[SC 11945] Demo notebook for agent testing documentation#422

Merged
AnilSorathiya merged 68 commits intomainfrom
anilsorathiya/sc-11945/demo-notebook-for-agent-testing-documentation
Sep 26, 2025
Merged

[SC 11945] Demo notebook for agent testing documentation#422
AnilSorathiya merged 68 commits intomainfrom
anilsorathiya/sc-11945/demo-notebook-for-agent-testing-documentation

Conversation

@AnilSorathiya
Copy link
Contributor

@AnilSorathiya AnilSorathiya commented Sep 4, 2025

Pull Request Description

What and why?

This PR introduces a comprehensive Banking AI Agent demo using LangGraph and ValidMind integration, along with supporting framework improvements. The changes include:

Before: The agents directory contained multiple incomplete and redundant notebook demos with inconsistent implementations.
After: A single, comprehensive banking-specific AI agent demo with:

  • Complete LangGraph-based agent implementation for banking use cases
  • Specialized banking tools for credit risk, fraud detection, and account management
  • Comprehensive test dataset with 160+ banking scenarios
  • Proper ValidMind integration for agent validation and testing
  • Cleaned up codebase with removed duplicate/incomplete demos

Key additions:

  • langgraph_agent_simple_banking_demo.ipynb - Complete banking agent demo notebook
  • banking_tools.py - specialized banking tools (credit risk analyzer, fraud detection, etc.)
  • banking_test_dataset.py - Comprehensive test dataset with 160+ banking scenarios
  • Enhanced @scorer decorator functionality for better test integration
  • Updated dependencies for better compatibility (numpy, scipy, statsmodels constraints)

How to test

Run notebooks/agents/langgraph_agent_simple_banking_demo.ipynb

What needs special review?

Dependencies, breaking changes, and deployment notes

Release notes

Checklist

  • What and why
  • Screenshots or videos (Frontend)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

Copy link
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of nitpicks

"\n",
"This notebook demonstrates how to build and validate a **banking-specific AI agent** using LangGraph integrated with ValidMind for comprehensive testing and monitoring.\n",
"\n",
"## What You'll Learn\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the intro should focus slightly more on how users can document and validate agents with ValidMind?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. The intro section makes users believe that the objective it to build a sample AI agent but really the objective is to learn how to validate an AI agent.

A prerequisite is to have an AI agent to test/document, therefore the notebook also does that for the user to simplify things, using a very simple and hypothetical use case, but building the demo use case should not be the objective of the notebook.

Copy link
Contributor

@juanmleng juanmleng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just left a couple of minor comments. Other than that, ltgm!

Copy link
Contributor

@johnwalz97 johnwalz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions
Copy link
Contributor

PR Summary

This pull request removes multiple legacy agent demo notebooks and associated code samples that were previously used to demonstrate the integration of LangChain, LangGraph, and banking-specific agent functionality with the ValidMind Library. In particular, the PR removes entire notebooks (e.g., the LangChain and LangGraph demo notebooks, both simple and complex versions, as well as the banking agent demo) and related utility modules (such as langchain_utils.py and agents/utils.py). In addition, some test decorators and a unit metric (RootMeanSquaredError) have been slightly refactored for conciseness. Overall, the changes are aimed at cleaning up outdated or redundant examples, thereby simplifying the codebase and reducing maintenance overhead.

Test Suggestions

  • Run the full suite of unit tests to ensure that removals have not inadvertently impacted any dependent modules.
  • Execute integration tests on ValidMind model and dataset functionality to verify that all tests still pass after the removal of old notebooks.
  • Perform manual smoke testing in environments using the updated library to ensure that new documentation and code paths function correctly.

@AnilSorathiya AnilSorathiya marked this pull request as ready for review September 26, 2025 22:12
@AnilSorathiya AnilSorathiya merged commit 745993e into main Sep 26, 2025
18 checks passed
@AnilSorathiya AnilSorathiya deleted the anilsorathiya/sc-11945/demo-notebook-for-agent-testing-documentation branch September 26, 2025 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants