Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
1b3f67a
support agent use case
AnilSorathiya Jun 24, 2025
723fcab
wrapper function for agent
AnilSorathiya Jun 24, 2025
28d9fbb
ragas metrics
AnilSorathiya Jun 30, 2025
ecf8e09
update ragas metrics
AnilSorathiya Jun 30, 2025
53e8879
fix lint error
AnilSorathiya Jun 30, 2025
1662368
create helper functions
AnilSorathiya Jul 1, 2025
cc84cbc
Merge branch 'main' into anilsorathiya/sc-10863/add-support-for-llm-a…
AnilSorathiya Jul 2, 2025
6f09780
delete old notebook
AnilSorathiya Jul 2, 2025
0bb731e
update description for each section
AnilSorathiya Jul 2, 2025
e758979
simplify agent
AnilSorathiya Jul 9, 2025
7c35cfe
simple demo notebook using langchain agent
AnilSorathiya Jul 10, 2025
9bb70e9
Update description of the simplified langgraph agent demo notebook
AnilSorathiya Jul 10, 2025
894d52a
add brief description to tests
AnilSorathiya Jul 14, 2025
d86a9af
add brief description to tests
AnilSorathiya Jul 14, 2025
884000f
Allow dict return type predict_fn
AnilSorathiya Jul 17, 2025
fbd5aa9
update notebook and refactor utils
AnilSorathiya Jul 18, 2025
daceabf
lint fix
AnilSorathiya Jul 18, 2025
5f8823a
Merge branch 'main' into anilsorathiya/sc-11324/extend-the-predict-fn…
AnilSorathiya Jul 18, 2025
70a5636
fix the test failure
AnilSorathiya Jul 18, 2025
33b06fb
new unit tests for multiple columns return in assign_predictions
AnilSorathiya Jul 18, 2025
8e12bd2
update notebooks to return multiple values in predict_fn
AnilSorathiya Jul 18, 2025
e38929d
general plotting and stats tests
AnilSorathiya Jul 23, 2025
e900a65
clear output
AnilSorathiya Jul 23, 2025
a08e881
Merge branch 'main' into anilsorathiya/sc-11380/add-generlize-plots-a…
AnilSorathiya Jul 24, 2025
16f4700
remove duplicate tests
AnilSorathiya Jul 24, 2025
bb9f9af
update notebook
AnilSorathiya Jul 24, 2025
5078a7a
Integration between deepeval and validmind
AnilSorathiya Jul 25, 2025
2eb6abb
Merge branch 'main' into anilsorathiya/sc-11452/support-for-the-deepe…
AnilSorathiya Aug 12, 2025
ad0b719
add MetricValues class for metric return type
AnilSorathiya Aug 15, 2025
94ca006
Return MetricValues in the unit tests
AnilSorathiya Aug 15, 2025
c4c885a
update all the unit metric tests
AnilSorathiya Aug 15, 2025
a1f3220
add unit tests for MetricValues class
AnilSorathiya Aug 15, 2025
1a7d0b6
update result to support MetricValues for unit metric tests
AnilSorathiya Aug 15, 2025
1d785ba
add copyright statement
AnilSorathiya Aug 15, 2025
271e85b
add deepeval lib as an extra dependency
AnilSorathiya Aug 15, 2025
f806fc6
fix the error
AnilSorathiya Aug 15, 2025
61c7ef6
demo draft change
AnilSorathiya Aug 18, 2025
b646d0b
demo draft change
AnilSorathiya Aug 18, 2025
dda4ced
fix api issue
AnilSorathiya Aug 18, 2025
dd8e0df
Merge branch 'main' into anilsorathiya/sc-11452/support-for-the-deepe…
AnilSorathiya Aug 21, 2025
81249c2
separate unit metrics and row metrics
AnilSorathiya Aug 22, 2025
794a322
draft notebook
AnilSorathiya Aug 22, 2025
a27bc48
Merge branch 'main' into anilsorathiya/sc-11452/support-for-the-deepe…
AnilSorathiya Aug 22, 2025
84dfa2f
update assign_score notebook
AnilSorathiya Aug 22, 2025
7aa2acc
update assign score notebook
AnilSorathiya Sep 1, 2025
247eacc
rename notebook
AnilSorathiya Sep 1, 2025
394c57c
update deepeval and VM integration notebook
AnilSorathiya Sep 1, 2025
a2ca13c
Merge branch 'main' into anilsorathiya/sc-11452/support-for-the-deepe…
AnilSorathiya Sep 4, 2025
5ebe51f
rename row metrics to scorer
AnilSorathiya Sep 4, 2025
15df53b
add scorer decorator
AnilSorathiya Sep 4, 2025
e28ba37
remove UnitMetricValue and RowMetricValues as they are not needed any…
AnilSorathiya Sep 4, 2025
d8a48c8
remove MetricValue class
AnilSorathiya Sep 5, 2025
d425576
support complex output for scorer
AnilSorathiya Sep 5, 2025
9c7e7e9
remove simple testcases
AnilSorathiya Sep 9, 2025
bbd6cd4
fix the list_scorers
AnilSorathiya Sep 9, 2025
c7b83f3
update notebook
AnilSorathiya Sep 9, 2025
a33f2a4
remove circular dependency of load_test
AnilSorathiya Sep 9, 2025
30c3abc
remove circular dependency of load_test
AnilSorathiya Sep 9, 2025
e91e6e4
move the AnswerRelevancy scorer in deepeval namespace
AnilSorathiya Sep 9, 2025
a284cd1
unit metric can return int and float only
AnilSorathiya Sep 9, 2025
1ec1c75
update notebook
AnilSorathiya Sep 9, 2025
427ddf5
fix lint error
AnilSorathiya Sep 9, 2025
917831c
remove scores listing from list_tests interface
AnilSorathiya Sep 10, 2025
58b3bde
add custom scorer support
AnilSorathiya Sep 10, 2025
cb52104
full path required to run scorer
AnilSorathiya Sep 11, 2025
36f2f96
remove circular dependency
AnilSorathiya Sep 11, 2025
439bd1d
make model parameter option in the assign_scores function
AnilSorathiya Sep 11, 2025
66dde16
fix lint error
AnilSorathiya Sep 11, 2025
b0fe22e
add tests
AnilSorathiya Sep 15, 2025
1fe452d
update notebook
AnilSorathiya Sep 15, 2025
5a101ff
Merge branch 'main' into anilsorathiya/sc-12254/add-new-deepeval-test…
AnilSorathiya Sep 15, 2025
730032c
Merge branch 'main' into anilsorathiya/sc-12254/add-new-deepeval-test…
AnilSorathiya Oct 1, 2025
dc2c743
add deeleval metrics as scorer
AnilSorathiya Oct 2, 2025
7b7a363
add copyright
AnilSorathiya Oct 2, 2025
472a16e
remove Geval test
AnilSorathiya Oct 7, 2025
b2d9a2a
add task completion test
AnilSorathiya Oct 7, 2025
b4c311f
update demo notebook
AnilSorathiya Oct 7, 2025
db63fe4
gitignore *.deepeval
AnilSorathiya Oct 7, 2025
8b43a77
update boxplot
AnilSorathiya Oct 7, 2025
d6c22df
update deepeval integration notebook
AnilSorathiya Oct 7, 2025
5376944
add GEval scorer
AnilSorathiya Oct 10, 2025
6c8b9c7
fix lint error
AnilSorathiya Oct 10, 2025
00afffa
remove space from geval metric name
AnilSorathiya Oct 10, 2025
e5108b5
update notebook
AnilSorathiya Oct 13, 2025
5130ef3
Merge branch 'main' into anilsorathiya/sc-12707/add-g-eval-test-in-lib
AnilSorathiya Oct 17, 2025
c69b3b3
merge notebook
AnilSorathiya Oct 17, 2025
e10c582
add Geval notebook
AnilSorathiya Oct 17, 2025
592917c
add Geval notebook
AnilSorathiya Oct 17, 2025
643ae38
update boxplot and geval notebook
AnilSorathiya Oct 20, 2025
a4fed89
fix the pyarrow version
AnilSorathiya Oct 20, 2025
0f44619
update lock file
AnilSorathiya Oct 20, 2025
fda284e
update lock file
AnilSorathiya Oct 20, 2025
a74ddf1
rollback changes
AnilSorathiya Oct 21, 2025
3acf92f
rollback poetry.lock
AnilSorathiya Oct 21, 2025
d01f98e
update lock file
AnilSorathiya Oct 21, 2025
b8dfcd3
update lock file
AnilSorathiya Oct 21, 2025
103f769
add criteria as column in the dataset
AnilSorathiya Oct 21, 2025
d5ee5b8
update Geval
AnilSorathiya Oct 21, 2025
ca51c49
update agent dataset object
AnilSorathiya Oct 21, 2025
2193329
update geval
AnilSorathiya Oct 21, 2025
27c1638
add reason in the geval
AnilSorathiya Oct 22, 2025
786da89
passing evaluation_params to select columns for geval
AnilSorathiya Oct 22, 2025
f6ba7e7
change criteria
AnilSorathiya Oct 23, 2025
5e198ed
Merge branch 'main' into anilsorathiya/sc-12707/add-g-eval-test-in-lib
AnilSorathiya Oct 23, 2025
1254c7b
update markup
AnilSorathiya Oct 23, 2025
58f3136
2.10.2
AnilSorathiya Oct 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@
" api_key=\"...\",\n",
" api_secret=\"...\",\n",
" model=\"...\",\n",
")"
")\n"
]
},
{
Expand Down Expand Up @@ -275,6 +275,11 @@
"from banking_tools import AVAILABLE_TOOLS\n",
"from validmind.tests import run_test\n",
"\n",
"pd.set_option('display.max_columns', None)\n",
"pd.set_option('display.max_colwidth', None)\n",
"pd.set_option('display.width', None)\n",
"pd.set_option('display.max_rows', None)\n",
"\n",
"# Load environment variables if using .env file\n",
"try:\n",
" from dotenv import load_dotenv\n",
Expand Down Expand Up @@ -722,15 +727,7 @@
"\n",
"print(\"Banking Test Dataset Initialized in ValidMind!\")\n",
"print(f\"Dataset ID: {vm_test_dataset.input_id}\")\n",
"print(f\"Dataset columns: {vm_test_dataset._df.columns}\")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f\"Dataset columns: {vm_test_dataset._df.columns}\")\n",
"vm_test_dataset._df.head(1)"
]
},
Expand All @@ -754,7 +751,8 @@
"vm_test_dataset.assign_predictions(vm_banking_model)\n",
"\n",
"print(\"Banking Agent Predictions Generated Successfully!\")\n",
"print(f\"Predictions assigned to {len(vm_test_dataset._df)} test cases\")"
"print(f\"Predictions assigned to {len(vm_test_dataset._df)} test cases\")\n",
"vm_test_dataset._df.head()"
]
},
{
Expand All @@ -772,11 +770,11 @@
"metadata": {},
"outputs": [],
"source": [
"pd.set_option('display.max_colwidth', 40)\n",
"pd.set_option('display.width', 120)\n",
"pd.set_option('display.max_colwidth', None)\n",
"print(\"Banking Test Dataset with Predictions:\")\n",
"vm_test_dataset._df.head()"
"# pd.set_option('display.max_colwidth', 40)\n",
"# pd.set_option('display.width', 120)\n",
"# pd.set_option('display.max_colwidth', None)\n",
"# print(\"Banking Test Dataset with Predictions:\")\n",
"# vm_test_dataset._df.head()"
]
},
{
Expand Down
Loading
Loading