Conversation
* Remove ydata-profiling library dependency * poetry lock update * update infer_datatypes function to infer Text type * update test * remove lint error * remove print statements
* Fix index issue when using input_grid * bug fix : show the results of all the features --------- Co-authored-by: Anil Sorathiya <anil@validmind.ai>
* Remove ydata-profiling library dependency * poetry lock update * update infer_datatypes function to infer Text type * update test * remove lint error * remove print statements * resolve warning message
* Oops, I fudged up a link in the templates * New wording * Changing out old links in prep pt1 * Changing out old links in prep pt2 * Changing out old links in prep pt3 * Changing out old links in prep pt4 * Adding ToC extension to template README * Style guide tweak * 101 +intro * Update notebooks/tutorials/model_development/101-set_up_validmind.ipynb Co-authored-by: Nik Richers <nik@validmind.ai> * 2.8.13 --------- Co-authored-by: Nik Richers <nik@validmind.ai>
* support config option for logging test results * remove commented code * update the test * pass config to log_metric * remove usafe parameter from logging methods * remove config from the log_metric
* Add initial Quarto docs assets from experimental branch * Save point * Save point * Add API tree script * Save point * Save point * Save point before switching to Griffe API * Save point before splitting QMD template * Interim commit with mostly working reimplementation of template * Re-add more content, fix markup issues, and create sidebar template * Save point * Partially fix nested sidebar entries * Save point * Save ppoint docstring parsing * Save point before error message fixes * Save point * Save point * Save point * Save point * Save point for errors * Save point before switching to vm_models * Another save point * Got vm_models working * Save point with docstring issues * Save point with partial sidebar fixes * Save point with partial sidebar fixes * Sidebar fragment contains correct file embeds * Got sidebar working * Save point * Save point * Save point with first complete sidebar * Save point with first complete sidebar * Sidebar works * Add root-level classes * Undo save point * Sidebar fixes complete * Add codeblock template comments * Markup fixes * Fix sidebar regression * Fix sidebar regression * Undo stray commit * Save point * Save point * Save point * Likely safe save point * Halfway through switching to better signatures * Halfway through switching to better signatures * Continue to improve snignature blocks and remove duplication * Remove dead code * More signatures fine-tuning * More signatures fine-tuning * Save point for return annotations * Save point with most return annotations fixed but still getting NoneNoneNoneTuple * Save point with most return annotations fixed but still getting NoneNoneNoneTuple * Fix docstrings * Fix docstrings * Fixed docstring defaults when set to None * Save point for type annotations rework * Undo new processing functions in script * Fix docstring regression * Fix docstrings in Python API * Fix Python API docstring * Fix Python API docstring * Save point while fixing some docstrings * Fixed short and long description regression * Minor content update * Fix type class annotations * Fix type class annotations * Save point types fine-tuning * Save point types fine-tuning * Add missing default to docstring and fix grammar * Fix default handling in docstrings, fix class for type keywords * Fix default handling in docstrings, fix class for type keywords * Fix default handling in docstrings, fix class for type keywords * Fix wording * Output class methods on single line and append colon for classes and class methods * Output class methods on single line and append colon for classes and class methods * Remove colons for return annotations * Remove colons for return annotations * Fix variadic and positional parameters * Save point * Fix single line vs multiline signatures regression * Reapply fix for ExprList * Save point * Save point with debug comments in types.jinja2 * Fixed final raw JSON output for Expr* types * Save point before descending into experimental darkness * Minor type fix * Interim save point * Fixed ExprAttribute rendering for inherited members * Remove debug statements * Save point for class names in signatures for inherited members * Fix class for constructors that have multiple parameters * Fix raw JSON issue with ExprCall * Generalize Expr* type handling further * Better decorator formatting when >1 decorator * Add metadata and CSS files * Fix return annotation None and improve handling of @ in decorators * Fixed display of self in methods * Fix Python API warnings * Save point after fixing Python API, did it work? * Switch from reference/ to validmind/ path to avoid breaking links * CSS rename, latest version * Docstring edits * Generate QMD * Fix CSS embed in metadata file * Fix docstring handling bug with newlines * Improve alias handling to catch all root-level aliases safely * Clarify Torch TensorDataset is supported * Remove ToC and sidebar decoration to help with long names * Add special case for vm_models * Fix return type annotations * Improve type annotation coverage * Fix for missing vm_model classes and methods * Fix inherited members info * Improve formatting for type_info in docstrings * Fix ordering for class attributes * Add special case for cleaning up tests * Try to fix code test errors * Fix more code test errors * Fix yet more code test errors * Interim save point on better control over ordering * Remove unused _quarto.yml * Fix return annotation for run_test_suite * Fix some return annotations * Minor punctuation fix * Fix docstring for inspect() * Remove API tree script I used to explain the API docstrings to myself * Initial README.md draft * Move diagram in README.md * Start using template for errors * Update Quarto source * Save point for better error messages * Better API error handling * Error message formatting & fixes * Interim save point for fixing inherited BaseError content * Save point with signature and inherited member improvements * Save point for error messages * Interim save point for inherited member fixes * Remove inherited members for errors from test descriptions * Clean up inherited members * Finally fixed the elusive description method for BaseError * Fix minor formatting issue * Remove hardcoding * Save point before chewing out Sonnet * Remove more hardcoded content after a frank discussion with Sonnet * Consolidate signature handling * Improve handling for class param vs params * Improve sidebar * Hide test suite class info from main page * Improve filtering out non-public functions * Add version number to sidebar & hide ToC for root function topic * Add workflow for Quarto docs * Add test building Quarto docs to python.yaml * Ignore docs/** path in workflows * Make version in sidebar fancier * Fix missing RawData children in sidebar * Simplify inherited member formatting * Omit inherited members heading when there are none * Fix return annotation for pandas.DataFrame * Fix function signatures when async * Handle module-level aliases like describe_test_suite * Remove debug statement from script * ValidMind CSS tweaks * Update README.md * Update mermaid diagram * Update mermaid diagram, again * Update mermaid diagram, take three * Update mermaid diagram, take four * Update mermaid diagram, take five * Update mermaid diagram, take six * Edit README.md * Edit README.md * Add linter to README.md * Readme & CSS tweaks * CSS tweak for codeblocks * Fix flake8 whitespace error * Update Quarto Markdown source * Add dependencies to pyproject.toml and GitHub workflow * Switch .html links in titles to .qmd * Class prefix and function suffix CSS experiment * More class prefix and function suffix CSS experiments * Save point for sidebar fragment generation fixes after CSS experiments * Fix sidebar navigation fragment after CSS changes * Fix headings that link to themselves * Remove class prefixes from return annotation links * Improve handling for external types and return annotation linking * Enable parameter type linking for internal types * Fix case-sensitive external type link * Add HTML to external types * Improve TestID handling * Improve unit metric handling in signatures * Update docs/_metadata.yml Co-authored-by: Beck <164545837+validbeck@users.noreply.github.com> * Fix decorators macro * Update poetry.lock * Fix unit tests, round 1 * Fix unit tests, round 2 * Fix unit tests, round 3 * Apply # noqa: C901 to complex tests * Fix invalid table format errors * Try fixing pandas.io.formats error * Fix TableOutputHandler to address invalid table format errors * Fix C901 'TableOutputHandler.process' is too complex error * Refactor test to reduce complexity and undo # noqa: C901 * Fix annotation warnings in utils.py * Improve sidebar spacing * Found correct global fix for smart quotes --------- Co-authored-by: Beck <164545837+validbeck@users.noreply.github.com>
* Uploading Michael's notebook before testing * + champion model file & moved draft to tutorials * 111 draft * 111 notes * 111 adjusting * 111 +intro * 111 + intro * 102 accidentally deleted a header somehow * 111 new draft * 111 editing intro to be validator-focused * Edited the About ValidMind section to be more role agnostic * Missed one * 111 +Register a sample model * 111 +Getting to know ValidMind * 111 Set up for Import the champion model * 111 Shuffling things around * 111 Importing the champion model done? * 111 Setup for custom context * Moving LLM context stuff, changed my mind * 111 Added section on finding the validation report in the platform * 111 +ToC * 112 setup * 112 copied info over * 111 more context * 112 Setup for challenger models * 112 Challenger models v1 * 112 Setup for custom context * Changed dataset for custom context * 113 setup * 112 ToC * 113 title adjust * 114 setup * 113 Prereqs * 113 Setting up * 113 Setting up edit * 113 Setting up edit2 * 113 Setting up edit3 * 113 Setting up edit4 * 113 Set up for Identify qualitative tests * 113 Identify qualitative tests editing * 113 Run performance tests draft * 113 warning about missing text-driven blocks * 113 Evaluate performance of challengers WIP * 113 Wrapping up * 114 prereq * 114 transfered rest of code over * 114 transfered rest of code over edit * 114 cleanup wip * 114 cleanup wip2 * 114 cleanup wip3 * 114 Adjust a comparison test * 114 Next steps draft * 114 quick code cleanup * 114 Run diagnostic tests done * Implement a custom test done * 114 Verify test runs done * In summary done * 114 Add test results to reporting draft * 114 Finishing up * LLM custom context - session info * 114 quick edit to summary * Removing the old notebook * Tweak * Tweak2 * Typo * Comment on .gitignore * Updating the old intro notebook just in case someone finds it * Edits to validation key concepts + dataset transformations * Removing test suites as well * Tweak * Setup for advanced notebook * Moving the application scorecard advanced notebook * Moving 111 to new notebook * Moving 112 to new notebook * Moving 113 to new notebook * Moving 114 to new notebook * Moving the champion model * Shifting use case stuff around take 1 * Moving more stuff around * Adding individual tests * Editing * More editing * Adding ToC * Ugh * Cleaning up the What's next section * Removing the tests causing issues * Final edits? * xgb_model > xgb_model_developer_champion * Moving the introductory notebooks to own PR * 2.8.14
bug: reverting changes to load.py that broke listing and loading tests
…-logistic-regression [SC-7456] intro_for_model_developers.ipynb throws SHAPGlobalImportance.py error
…n and update Quarto documentation workflow to exclude specific branch.
Remove GH_TOKEN requirement
… sections (#339) * introduce log_text function to log qualitative text block * introduce log_text function to log qualitative text block * introduce log_text function to log qualitative text block * validate parameter types in log_text interface * convert str into text markup * format files * ux for log_text output * handle html text * checking if text is html * support html text input
… ClassImbalance and DescriptiveStatistics tests for improved categorization.
* Remove some pdoc remnants * Add GH_TOKEN again --------- Co-authored-by: Andres Rodriguez <andres@validmind.ai>
…g-in-rag-use-case [SC-9395] Add RAG benchmarking demo notebook
* Moved the introductory notebooks to their own PR * Renamed the old model development series * Renamed the new model validation series * Exporting the champion model * Editing validation 1 * Adjusting * Test * WIP new validator 2 * copying some code into validator 2 * Ugh * I give up for this week * Validation 2 - Verify data quality assessments setup * Validation 2 - Verify data quality assessments editing * Validation 2 - Document test results WIP * Validation 2 - Document test results - add * Validation 3 - setup * Validation 3 - Setting up WIP * Validation 3 - Testing... * Save point * Save point * Save point * Testing something * Testing more * Applying Andres' fixes * Validation 4 setup * Testing * Applying changes to subsequent notebooks * Deleting old draft for validation 2 * Save point * Save point * Save point * Save point * Testing? * Testing... * Editing... * Changing screencaps * Save point * Editing...... * Validation 2 cleanup * Validataion 3 editing * Save point * Save point * Save point * Save point * Validation 3 log a finding * Validation 3 Adjusted test * Validation 3 finding recommendation * Validation 3 more tests * Moving split dataset * Validation 4 Setting up * Validation 4 Summary & next steops * Copied custom test stuff over * Copied over verify test runs, adjusted next steps * Deleting old notebook * Validation 4 Editing custom tests * Validation 4 Custom inline test pt1 * Validation 4 Custom inline test pt2 * Validation 4 Custom inline test pt3 * Validation 4 Custom inline test pt4 * Validation 4 Custom inline test pt5 * Validation 4 Custom inline test pt6 * Validation 4 Custom test provider pt1 * Validation 4 Custom test provider pt2 * Validation 4 Next steps typo * Validation 4 Verify test runs setup * Validation 4 Verify test runs edit * Forgot the raw dataset * Fixing broken API links * Adding ToCs * Next steps - Work with your validation report * 2.8.18
* Adjustments for validation notebooks docs side * Updating the template * Test? * Update docs/templates/module.qmd.jinja2 Co-authored-by: Nik Richers <nik@validmind.ai> * Editing terminology * 2.8.19 --------- Co-authored-by: Nik Richers <nik@validmind.ai>
* Add text support in test result object * add custom test description example in the notebook
…-too-many-plots-for-test-description John6797/sc 9452/add check for too many plots for test description
|
|
PR SummaryThis pull request enhances the documentation generation process by integrating Quarto for generating API documentation. The changes include:
These changes aim to streamline the documentation process, making it more efficient and maintainable by leveraging Quarto's capabilities. Test Suggestions
|
johnwalz97
left a comment
There was a problem hiding this comment.
lgtm!
@cachafla @AnilSorathiya @juanmleng can you take a look and see if this looks good
|
BTW, next time remember you can use the |
Ah okay... I always forget about that 😬. Will do |
Internal Notes for Reviewers
External Release Notes