Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions docs/_build/index.html

This file was deleted.

46 changes: 0 additions & 46 deletions docs/_build/search.js

This file was deleted.

893 changes: 0 additions & 893 deletions docs/_build/validmind.html

This file was deleted.

239 changes: 0 additions & 239 deletions docs/_build/validmind/__version__.html

This file was deleted.

244 changes: 0 additions & 244 deletions docs/_build/validmind/datasets.html

This file was deleted.

242 changes: 0 additions & 242 deletions docs/_build/validmind/datasets/classification.html

This file was deleted.

311 changes: 0 additions & 311 deletions docs/_build/validmind/datasets/classification/customer_churn.html

This file was deleted.

271 changes: 0 additions & 271 deletions docs/_build/validmind/datasets/classification/taiwan_credit.html

This file was deleted.

242 changes: 0 additions & 242 deletions docs/_build/validmind/datasets/credit_risk.html

This file was deleted.

448 changes: 0 additions & 448 deletions docs/_build/validmind/datasets/credit_risk/lending_club.html

This file was deleted.

311 changes: 0 additions & 311 deletions docs/_build/validmind/datasets/credit_risk/lending_club_bias.html

This file was deleted.

242 changes: 0 additions & 242 deletions docs/_build/validmind/datasets/nlp.html

This file was deleted.

288 changes: 0 additions & 288 deletions docs/_build/validmind/datasets/nlp/cnn_dailymail.html

This file was deleted.

255 changes: 0 additions & 255 deletions docs/_build/validmind/datasets/nlp/twitter_covid_19.html

This file was deleted.

242 changes: 0 additions & 242 deletions docs/_build/validmind/datasets/regression.html

This file was deleted.

386 changes: 0 additions & 386 deletions docs/_build/validmind/datasets/regression/fred.html

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/datasets/regression/lending_club.html

This file was deleted.

1,719 changes: 0 additions & 1,719 deletions docs/_build/validmind/errors.html

This file was deleted.

372 changes: 0 additions & 372 deletions docs/_build/validmind/test_suites.html

This file was deleted.

409 changes: 0 additions & 409 deletions docs/_build/validmind/test_suites/classifier.html

This file was deleted.

343 changes: 0 additions & 343 deletions docs/_build/validmind/test_suites/cluster.html

This file was deleted.

343 changes: 0 additions & 343 deletions docs/_build/validmind/test_suites/embeddings.html

This file was deleted.

307 changes: 0 additions & 307 deletions docs/_build/validmind/test_suites/llm.html

This file was deleted.

274 changes: 0 additions & 274 deletions docs/_build/validmind/test_suites/nlp.html

This file was deleted.

277 changes: 0 additions & 277 deletions docs/_build/validmind/test_suites/parameters_optimization.html

This file was deleted.

338 changes: 0 additions & 338 deletions docs/_build/validmind/test_suites/regression.html

This file was deleted.

307 changes: 0 additions & 307 deletions docs/_build/validmind/test_suites/statsmodels_timeseries.html

This file was deleted.

274 changes: 0 additions & 274 deletions docs/_build/validmind/test_suites/summarization.html

This file was deleted.

341 changes: 0 additions & 341 deletions docs/_build/validmind/test_suites/tabular_datasets.html

This file was deleted.

274 changes: 0 additions & 274 deletions docs/_build/validmind/test_suites/text_data.html

This file was deleted.

424 changes: 0 additions & 424 deletions docs/_build/validmind/test_suites/time_series.html

This file was deleted.

880 changes: 0 additions & 880 deletions docs/_build/validmind/tests.html

This file was deleted.

298 changes: 0 additions & 298 deletions docs/_build/validmind/tests/data_validation.html

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/ACFandPACFPlot.html

This file was deleted.

298 changes: 0 additions & 298 deletions docs/_build/validmind/tests/data_validation/ADF.html

This file was deleted.

305 changes: 0 additions & 305 deletions docs/_build/validmind/tests/data_validation/AutoAR.html

This file was deleted.

308 changes: 0 additions & 308 deletions docs/_build/validmind/tests/data_validation/AutoMA.html

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/tests/data_validation/AutoStationarity.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/data_validation/BivariateScatterPlots.html

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/data_validation/BoxPierce.html

This file was deleted.

This file was deleted.

310 changes: 0 additions & 310 deletions docs/_build/validmind/tests/data_validation/ClassImbalance.html

This file was deleted.

386 changes: 0 additions & 386 deletions docs/_build/validmind/tests/data_validation/DatasetDescription.html

This file was deleted.

303 changes: 0 additions & 303 deletions docs/_build/validmind/tests/data_validation/DatasetSplit.html

This file was deleted.

339 changes: 0 additions & 339 deletions docs/_build/validmind/tests/data_validation/DescriptiveStatistics.html

This file was deleted.

302 changes: 0 additions & 302 deletions docs/_build/validmind/tests/data_validation/DickeyFullerGLS.html

This file was deleted.

303 changes: 0 additions & 303 deletions docs/_build/validmind/tests/data_validation/Duplicates.html

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/EngleGrangerCoint.html

This file was deleted.

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/data_validation/HighCardinality.html

This file was deleted.

This file was deleted.

328 changes: 0 additions & 328 deletions docs/_build/validmind/tests/data_validation/IQROutliersBarPlot.html

This file was deleted.

320 changes: 0 additions & 320 deletions docs/_build/validmind/tests/data_validation/IQROutliersTable.html

This file was deleted.

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/JarqueBera.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/data_validation/KPSS.html

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/data_validation/LJungBox.html

This file was deleted.

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/data_validation/MissingValues.html

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/tests/data_validation/MissingValuesBarPlot.html

This file was deleted.

312 changes: 0 additions & 312 deletions docs/_build/validmind/tests/data_validation/MutualInformation.html

This file was deleted.

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/data_validation/PhillipsPerronArch.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

327 changes: 0 additions & 327 deletions docs/_build/validmind/tests/data_validation/RollingStatsPlot.html

This file was deleted.

305 changes: 0 additions & 305 deletions docs/_build/validmind/tests/data_validation/RunsTest.html

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/tests/data_validation/ScatterPlot.html

This file was deleted.

315 changes: 0 additions & 315 deletions docs/_build/validmind/tests/data_validation/ScoreBandDefaultRates.html

This file was deleted.

303 changes: 0 additions & 303 deletions docs/_build/validmind/tests/data_validation/SeasonalDecompose.html

This file was deleted.

302 changes: 0 additions & 302 deletions docs/_build/validmind/tests/data_validation/ShapiroWilk.html

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/data_validation/Skewness.html

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/data_validation/SpreadPlot.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/data_validation/TargetRateBarPlots.html

This file was deleted.

296 changes: 0 additions & 296 deletions docs/_build/validmind/tests/data_validation/TimeSeriesDescription.html

This file was deleted.

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/tests/data_validation/TimeSeriesFrequency.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/data_validation/TimeSeriesHistogram.html

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/data_validation/TimeSeriesLinePlot.html

This file was deleted.

This file was deleted.

305 changes: 0 additions & 305 deletions docs/_build/validmind/tests/data_validation/TimeSeriesOutliers.html

This file was deleted.

313 changes: 0 additions & 313 deletions docs/_build/validmind/tests/data_validation/TooManyZeroValues.html

This file was deleted.

303 changes: 0 additions & 303 deletions docs/_build/validmind/tests/data_validation/UniqueRows.html

This file was deleted.

309 changes: 0 additions & 309 deletions docs/_build/validmind/tests/data_validation/WOEBinPlots.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/data_validation/WOEBinTable.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/data_validation/ZivotAndrewsArch.html

This file was deleted.

248 changes: 0 additions & 248 deletions docs/_build/validmind/tests/data_validation/nlp.html

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/nlp/CommonWords.html

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/data_validation/nlp/Hashtags.html

This file was deleted.

303 changes: 0 additions & 303 deletions docs/_build/validmind/tests/data_validation/nlp/LanguageDetection.html

This file was deleted.

302 changes: 0 additions & 302 deletions docs/_build/validmind/tests/data_validation/nlp/Mentions.html

This file was deleted.

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/nlp/Punctuations.html

This file was deleted.

295 changes: 0 additions & 295 deletions docs/_build/validmind/tests/data_validation/nlp/Sentiment.html

This file was deleted.

311 changes: 0 additions & 311 deletions docs/_build/validmind/tests/data_validation/nlp/StopWords.html

This file was deleted.

323 changes: 0 additions & 323 deletions docs/_build/validmind/tests/data_validation/nlp/TextDescription.html

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/data_validation/nlp/Toxicity.html

This file was deleted.

256 changes: 0 additions & 256 deletions docs/_build/validmind/tests/model_validation.html

This file was deleted.

309 changes: 0 additions & 309 deletions docs/_build/validmind/tests/model_validation/BertScore.html

This file was deleted.

306 changes: 0 additions & 306 deletions docs/_build/validmind/tests/model_validation/BleuScore.html

This file was deleted.

This file was deleted.

307 changes: 0 additions & 307 deletions docs/_build/validmind/tests/model_validation/ContextualRecall.html

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/model_validation/FeaturesAUC.html

This file was deleted.

310 changes: 0 additions & 310 deletions docs/_build/validmind/tests/model_validation/MeteorScore.html

This file was deleted.

286 changes: 0 additions & 286 deletions docs/_build/validmind/tests/model_validation/ModelMetadata.html

This file was deleted.

This file was deleted.

301 changes: 0 additions & 301 deletions docs/_build/validmind/tests/model_validation/RegardScore.html

This file was deleted.

This file was deleted.

308 changes: 0 additions & 308 deletions docs/_build/validmind/tests/model_validation/RougeScore.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/model_validation/TokenDisparity.html

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/model_validation/ToxicityScore.html

This file was deleted.

274 changes: 0 additions & 274 deletions docs/_build/validmind/tests/model_validation/sklearn.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

308 changes: 0 additions & 308 deletions docs/_build/validmind/tests/model_validation/sklearn/ROCCurve.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/model_validation/sklearn/VMeasure.html

This file was deleted.

This file was deleted.

254 changes: 0 additions & 254 deletions docs/_build/validmind/tests/model_validation/statsmodels.html

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

246 changes: 0 additions & 246 deletions docs/_build/validmind/tests/prompt_validation.html

This file was deleted.

312 changes: 0 additions & 312 deletions docs/_build/validmind/tests/prompt_validation/Bias.html

This file was deleted.

298 changes: 0 additions & 298 deletions docs/_build/validmind/tests/prompt_validation/Clarity.html

This file was deleted.

298 changes: 0 additions & 298 deletions docs/_build/validmind/tests/prompt_validation/Conciseness.html

This file was deleted.

299 changes: 0 additions & 299 deletions docs/_build/validmind/tests/prompt_validation/Delimitation.html

This file was deleted.

304 changes: 0 additions & 304 deletions docs/_build/validmind/tests/prompt_validation/NegativeInstruction.html

This file was deleted.

302 changes: 0 additions & 302 deletions docs/_build/validmind/tests/prompt_validation/Robustness.html

This file was deleted.

300 changes: 0 additions & 300 deletions docs/_build/validmind/tests/prompt_validation/Specificity.html

This file was deleted.

305 changes: 0 additions & 305 deletions docs/_build/validmind/tests/prompt_validation/ai_powered_test.html

This file was deleted.

293 changes: 0 additions & 293 deletions docs/_build/validmind/unit_metrics.html

This file was deleted.

1,518 changes: 0 additions & 1,518 deletions docs/_build/validmind/vm_models.html

This file was deleted.

2 changes: 1 addition & 1 deletion docs/_sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ website:
- text: "---"
- text: "Python API"
# Root level items from validmind.qmd
- text: "<span class='version'>`2.8.12`</span>"
- text: "<span class='version'>`2.8.14`</span>"
file: validmind/validmind.qmd#version__
- text: "init<span class='suffix'></span>"
file: validmind/validmind.qmd#init
Expand Down
14 changes: 7 additions & 7 deletions docs/validmind.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ After you have pasted the code snippet into your development source code and exe

::: {.signature}

<span class="name">2.8.12</span>
<span class="name">2.8.14</span>

:::

Expand All @@ -66,7 +66,7 @@ If the API key and secret are not provided, the client will attempt to retrieve

**Arguments**

- `project (str, optional)`: The project CUID. Alias for model. Defaults to None. [DEPRECATED]
- `project (str, optional)`: The project CUID. Alias for model. Defaults to None. \[DEPRECATED\]
- `model (str, optional)`: The model CUID. Defaults to None.
- `api_key (str, optional)`: The API key. Defaults to None.
- `api_secret (str, optional)`: The API secret. Defaults to None.
Expand Down Expand Up @@ -225,11 +225,11 @@ Unit metrics are key-value pairs where the key is the metric name and the value
**Arguments**

- `key (str)`: The metric key
- `value (float)`: The metric value
- `inputs (list)`: A list of input IDs that were used to compute the metric.
- `params (dict)`: Dictionary of parameters used to compute the metric.
- `recorded_at (str)`: The timestamp of the metric. Server will use current time if not provided.
- `thresholds (dict)`: Dictionary of thresholds for the metric.
- `value (Union[int, float])`: The metric value
- `inputs (List[str])`: List of input IDs
- `params (Dict[str, Any])`: Parameters used to generate the metric
- `recorded_at (str)`: Timestamp when the metric was recorded
- `thresholds (Dict[str, Any])`: Thresholds for the metric

## preview_template<span class="suffix"></span>

Expand Down
21 changes: 21 additions & 0 deletions docs/validmind/errors.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -610,6 +610,27 @@ When an invalid metric results object is sent to the API.
- [APIRequestError<span class="suffix"></span>](#apirequesterror)
- builtins.BaseException with_traceback, add_note

### <span class="prefix"></span> InvalidParameterError

<!-- signatures.jinja2 -->

::: {.signature}

<span class="kw">class</span><span class="name">InvalidParameterError</span>(<span class="base">BaseError</span>):

:::

<!-- docstring.jinja2 -->

When an invalid parameter is provided.

<!-- errors.qmd.jinja2 - inherited members -->

**Inherited members**

- [BaseError<span class="suffix"></span>](#baseerror), [description<span class="suffix"></span>](#description)
- builtins.BaseException with_traceback, add_note

### <span class="prefix"></span> InvalidProjectError

<!-- signatures.jinja2 -->
Expand Down
45 changes: 34 additions & 11 deletions docs/validmind/tests.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,33 @@ ValidMind Tests Module

::: {.signature}

<span class="kw">def</span><span class="name">list_tests</span>(<span class="params"><span class="n">filter</span><span class="p">:</span><span class="n">Optional</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">task</span><span class="p">:</span><span class="n">Optional</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">tags</span><span class="p">:</span><span class="n">Optional</span><span class="p">\[</span><span class="n">List</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span><span class="p">\]</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">pretty</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">True</span><span class="muted">,</span></span><span class="params"><span class="n">truncate</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">True</span></span>)<span class="p"> → </span><span class="return-annotation"><span class="n">Union</span><span class="p">\[</span><span class="n">Dict</span><span class="p">\[</span><span class="nb">str</span><span class="p">, </span><span class="n">Callable</span><span class="p">\[</span><span class="n">...</span><span class="p">, </span><span class="n">Any</span><span class="p">\]</span><span class="p">\]</span><span class="p">, </span><span class="n">None</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">list_tests</span>(<span class="params"><span class="n">filter</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">task</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">tags</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">pretty</span><span class="o">=</span><span class="kc">True</span><span class="muted">,</span></span><span class="params"><span class="n">truncate</span><span class="o">=</span><span class="kc">True</span></span>):

:::

<!-- docstring.jinja2 -->

List all available tests with optional filtering
List all tests in the tests directory.

**Arguments**

- `filter (str, optional)`: Find tests where the ID, tasks or tags match the filter string. Defaults to None.
- `task (str, optional)`: Find tests that match the task. Can be used to narrow down matches from the filter string. Defaults to None.
- `tags (list, optional)`: Find tests that match list of tags. Can be used to narrow down matches from the filter string. Defaults to None.
- `pretty (bool, optional)`: If True, returns a pandas DataFrame with a formatted table. Defaults to True.
- `truncate (bool, optional)`: If True, truncates the test description to the first line. Defaults to True. (only used if pretty=True)

**Returns**

- list or pandas.DataFrame: A list of all tests or a formatted table.

## load_test<span class="suffix"></span>

<!-- signatures.jinja2 -->

::: {.signature}

<span class="kw">def</span><span class="name">load_test</span>(<span class="params"><span class="n">test_id</span><span class="p">:</span><span class="nb">str</span><span class="muted">,</span></span><span class="params"><span class="n">test_func</span><span class="p">:</span><span class="n">Optional</span><span class="p">\[</span><span class="n">Callable</span><span class="p">\[</span><span class="n">...</span><span class="p">, </span><span class="n">Any</span><span class="p">\]</span><span class="p">\]</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">reload</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">False</span></span>)<span class="p"> → </span><span class="return-annotation"><span class="n">Callable</span><span class="p">\[</span><span class="n">...</span><span class="p">, </span><span class="n">Any</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">load_test</span>(<span class="params"><span class="n">test_id</span><span class="p">:</span><span class="nb">str</span><span class="muted">,</span></span><span class="params"><span class="n">test_func</span><span class="p">:</span><span class="nb">callable</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">reload</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">False</span></span>):

:::

Expand All @@ -55,13 +67,20 @@ Test IDs are in the format `namespace.path_to_module.TestClassOrFuncName[:tag]`.

::: {.signature}

<span class="kw">def</span><span class="name">describe_test</span>(<span class="params"><span class="n">test_id</span><span class="p">:</span><span class="n">Optional</span><span class="p">\[</span><span class="n">TestID</span><span class="p"> (Union of </span><a href="/validmind/validmind/tests/data_validation.qmd">validmind.data_validation.\*</a><span class="p">, </span><a href="/validmind/validmind/tests/model_validation.qmd">validmind.model_validation.\*</a><span class="p">, </span><a href="/validmind/validmind/tests/prompt_validation.qmd">validmind.prompt_validation.\*</a><span class="p"> and </span><span class="nb">str</span><span class="p">)</span><span class="p">\]</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">raw</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">False</span><span class="muted">,</span></span><span class="params"><span class="n">show</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">True</span></span>)<span class="p"> → </span><span class="return-annotation"><span class="n">Union</span><span class="p">\[</span><span class="nb">str</span><span class="p">, </span><span class="n">HTML</span><span class="p">, </span><span class="n">Dict</span><span class="p">\[</span><span class="nb">str</span><span class="p">, </span><span class="n">Any</span><span class="p">\]</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">describe_test</span>(<span class="params"><span class="n">test_id</span><span class="p">:</span><span class="n">TestID</span><span class="p"> (Union of </span><a href="/validmind/validmind/tests/data_validation.qmd">validmind.data_validation.\*</a><span class="p">, </span><a href="/validmind/validmind/tests/model_validation.qmd">validmind.model_validation.\*</a><span class="p">, </span><a href="/validmind/validmind/tests/prompt_validation.qmd">validmind.prompt_validation.\*</a><span class="p"> and </span><span class="nb">str</span><span class="p">)</span><span class="o">=</span><span class="kc">None</span><span class="muted">,</span></span><span class="params"><span class="n">raw</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">False</span><span class="muted">,</span></span><span class="params"><span class="n">show</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">True</span></span>):

:::

<!-- docstring.jinja2 -->

Describe a test's functionality and parameters
Get or show details about the test

This function can be used to see test details including the test name, description, required inputs and default params. It can also be used to get a dictionary of the above information for programmatic use.

**Arguments**

- `test_id (str, optional)`: The test ID. Defaults to None.
- `raw (bool, optional)`: If True, returns a dictionary with the test details. Defaults to False.

## run_test<span class="suffix"></span>

Expand Down Expand Up @@ -112,41 +131,45 @@ This function is the main entry point for running tests. It can run simple unit

::: {.signature}

<span class="kw">def</span><span class="name">list_tags</span>()<span class="p"> → </span><span class="return-annotation"><span class="nb">Set</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">list_tags</span>():

:::

<!-- docstring.jinja2 -->

List all available tags
List unique tags from all test classes.

## list_tasks<span class="suffix"></span>

<!-- signatures.jinja2 -->

::: {.signature}

<span class="kw">def</span><span class="name">list_tasks</span>()<span class="p"> → </span><span class="return-annotation"><span class="nb">Set</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">list_tasks</span>():

:::

<!-- docstring.jinja2 -->

List all available tasks
List unique tasks from all test classes.

## list_tasks_and_tags<span class="suffix"></span>

<!-- signatures.jinja2 -->

::: {.signature}

<span class="kw">def</span><span class="name">list_tasks_and_tags</span>(<span class="param"><span class="n">as_json</span><span class="p">:</span><span class="nb">bool</span><span class="o">=</span><span class="kc">False</span></span>)<span class="p"> → </span><span class="return-annotation"><span class="n">Union</span><span class="p">\[</span><span class="nb">str</span><span class="p">, </span><span class="n">Dict</span><span class="p">\[</span><span class="nb">str</span><span class="p">, </span><span class="n">List</span><span class="p">\[</span><span class="nb">str</span><span class="p">\]</span><span class="p">\]</span><span class="p">\]</span></span>:
<span class="kw">def</span><span class="name">list_tasks_and_tags</span>(<span class="param"><span class="n">as_json</span><span class="o">=</span><span class="kc">False</span></span>):

:::

<!-- docstring.jinja2 -->

List all available tasks and tags
List all task types and their associated tags, with one row per task type and all tags for a task type in one row.

**Returns**

- A DataFrame with 'Task Type' and concatenated 'Tags'.

## test<span class="suffix"></span>

Expand Down
12 changes: 0 additions & 12 deletions docs/validmind/tests/data_validation/DatasetDescription.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -105,15 +105,3 @@ Will be used in favor of \_get_histogram in the future
<!-- docstring.jinja2 -->

Returns a collection of histograms for a numerical column, each one with a different bin size

<!-- function.qmd.jinja2 -->

## infer_datatypes<span class="suffix"></span>

<!-- signatures.jinja2 -->

::: {.signature}

<span class="kw">def</span><span class="name">infer_datatypes</span>(<span class="param"><span class="n">df</span></span>):

:::
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The examination invokes a series of steps:
1. For every numeric feature in the dataset, the 25th percentile (Q1) and 75th percentile (Q3) are calculated before deriving the Interquartile Range (IQR), the difference between Q1 and Q3.
1. Subsequently, the metric calculates the lower and upper thresholds by subtracting Q1 from the `threshold` times IQR and adding Q3 to `threshold` times IQR, respectively. The default `threshold` is set at 1.5.
1. Any value in the feature that falls below the lower threshold or exceeds the upper threshold is labeled as an outlier.
1. The number of outliers are tallied for different percentiles, such as [0-25], [25-50], [50-75], and [75-100].
1. The number of outliers are tallied for different percentiles, such as \[0-25\], \[25-50\], \[50-75\], and \[75-100\].
1. These counts are employed to construct a bar plot for the feature, showcasing the distribution of outliers across different percentiles.

### Signs of High Risk
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The test implements multiple threshold optimization methods:

- `dataset`: VMDataset containing features and target
- `model`: VMModel containing predictions
- `methods`: List of methods to compare (default: ['youden', 'f1', 'precision_recall'])
- `methods`: List of methods to compare (default: \['youden', 'f1', 'precision_recall'\])
- `target_recall`: Target recall value if using 'target_recall' method

**Returns**
Expand Down
2 changes: 1 addition & 1 deletion docs/validmind/version.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ sidebar: validmind-reference

::: {.signature}

<span class="name">2.8.12</span>
<span class="name">2.8.14</span>

:::
Loading