Skip to content

[SC-13270] Include both figures and tables in test result descriptions#450

Merged
juanmleng merged 3 commits intomainfrom
juan/sc-13270/include-both-figures-and-tables-in-test-result-descriptions
Nov 21, 2025
Merged

[SC-13270] Include both figures and tables in test result descriptions#450
juanmleng merged 3 commits intomainfrom
juan/sc-13270/include-both-figures-and-tables-in-test-result-descriptions

Conversation

@juanmleng
Copy link
Contributor

@juanmleng juanmleng commented Nov 21, 2025

Pull Request Description

What and why?

What

Updated test result description generation to include both figures and tables simultaneously. Previously, figures were excluded when tables were present ([] if tables else figures); now both are included when available.

Why

Tests often return both tabular data and visualizations. Excluding figures when tables exist limited the LLM’s ability to analyze both. Including both enables richer, more complete descriptions that combine insights from tables and figures.

How to test

Run a test that outputs both figures and tables, e.g., ClassImbalance and check that both figures and tables are included in the user message. You can check this by inspecting the trace in the Langfuse local project.

Screenshot 2025-11-21 at 12 16 51 Screenshot 2025-11-21 at 12 17 47

What needs special review?

Everything seems to work fine when including both, so I’m not sure whether excluding figures when tables are present was a deliberate choice.

Dependencies, breaking changes, and deployment notes

Release notes

Checklist

  • What and why
  • Screenshots or videos (Frontend)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

@juanmleng juanmleng self-assigned this Nov 21, 2025
@juanmleng juanmleng added bug Something isn't working internal Not to be externalized in the release notes labels Nov 21, 2025
steps:
- uses: actions/checkout@v4

- name: Free Disk Space
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will probably make these take much longer... i can take a look at it too see if theres another way to do this without using up a bunch of space

have you noticed it taking longer with this line?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right and it takes longer. So definitely worth taking a look. Thanks a lot @johnwalz97

Copy link
Contributor

@cachafla cachafla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@github-actions
Copy link
Contributor

PR Summary

This PR introduces several functional changes and enhancements:

  1. A new step in the GitHub Actions workflow (.github/workflows/dependency-testing.yaml) has been added to free disk space by running the disk_clean.sh script before setting up the Python environment. This helps ensure a cleaner and more reliable test environment, especially in environments with limited disk space.

  2. The test description generation logic in validmind/ai/test_descriptions.py has been updated. Previously, a conditional check involving tables was used to determine the list of figures, but now the expression has been streamlined to directly handle a potentially null or empty figures list by using the expression figures or []. This change improves robustness and clarity in handling the figures data.

  3. The version bump in pyproject.toml and validmind/__version__.py has been applied, although version updates are considered trivial regarding functional changes.

Test Suggestions

  • Verify that the workflow job successfully calls the disk_clean.sh script and that it correctly frees up disk space without affecting critical files.
  • Test the test description generation function with various scenarios: when figures is a valid list, is an empty list, or is None, to ensure that it handles all cases gracefully.
  • Run the full CI workflow to ensure that the added step integrates well with subsequent steps, particularly the Python setup.
  • Perform integration tests to check that downstream processes relying on figures rendering continue to operate correctly.

@juanmleng juanmleng merged commit 8afcfa6 into main Nov 21, 2025
17 checks passed
@juanmleng juanmleng deleted the juan/sc-13270/include-both-figures-and-tables-in-test-result-descriptions branch November 21, 2025 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants