Skip to content

Model integration testing#10

Open
brandomr wants to merge 1 commit intomainfrom
model-registration-tests
Open

Model integration testing#10
brandomr wants to merge 1 commit intomainfrom
model-registration-tests

Conversation

@brandomr
Copy link
Contributor

@mattprintz this is not ready for a PR since I have not tested this at all...I mostly just wanted to use this as a means to get your thoughts on this approach.

My simple idea here (how can we make this more automated?!) is this:

  1. Add a function to models.py to test a model by creating a default run.
  2. Add function to utils.py to do the default run for the model and store that info to a tests ES index
  3. Add function to runs.py which checks the status of test runs and updates tests index if the status is now success or failure (instead of running)

We'd basically run 2 crons:

  1. Something that periodically (weekly?) submits to /models/test for each model model_id that is published. Write the run_ids to a log file test_runs.log
  2. Something that periodically (daily?) reads test_runs.log and submits to /runs/test for each run_id

We'd then update the model overview page in the UI to add a column for the test run status.

NOTE to reiterate I've not tested any of this code, just wanted your take on the approach. Ideally we'd be doing this without crons and endpoint polling, but I'm fine with something quick and dirty if you don't mind it.

@brandomr brandomr requested a review from mattprintz January 26, 2022 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant