Skip to content

Latest commit

 

History

History
82 lines (47 loc) · 3.74 KB

testing.rst

File metadata and controls

82 lines (47 loc) · 3.74 KB

Testing Guidelines

Verify current Lithops status

Run lithops test to run an extensive inspection, which simulates your lithops installation on a variety of tests.

  • Get all available options by running lithops test --help.
  • Get all available test functions and their parent group by running lithops test -t help.
  • Run all test instances named <test name>, via lithops test -t <test name>. Run a test from a specific group by prefixing the test name with group name, e.g. : lithops test -t <test group>.<test name>. Run multiple tests by separating them with a comma, e.g. lithops test -t <test name1>,<test name2>.
  • To get all available groups, run lithops test -g help.
  • Run a single group of tests via lithops test -g <group name>. Run multiple tests by separating them with a comma, e.g. lithops test -g <test group1>,<test group2>.
  • To stop the test procedure upon first encountering a failed test, add the -f flag, e.g. lithops test -f.
  • To remove datasets, uploaded during the test procedure, use the -r flag, lithops test -r. WARNING - do not use this flag on a github workflow, due to race condition issues.
  • Get a complete list of the available flags by running lithops test --help.
  • A summarizing example: lithops test -t test_map,storage.test_cloudobject -g call_async -f.

Alternatively, you may run the tests via python3 -m lithops.tests.tests_main, followed by aforementioned flags.

Contribute

Add a test to an existing test group

Locate the matching test group in lithops/tests (indicated by file name) and add your test function appropriately: * Add your test function as a method inside the class inheriting from unittest.testcase.

  • Use existing util functions from lithops/tests/util_func or add new ones to that package.
  • A test that's simultaneously testing any two of the following functions: {async, map, map_reduce} will be placed in the proper file by complying with the following hierarchy: map_reduce > map > async.

Add a new test group

Before adding a test function that aims to test an untested feature:

  • Create a new file in lithops/tests using the template file as your guide:
    • Create a new copy of the template file and name it "test_feature_name", thus automatically creating a new test group named "feature_name".
    • Figure out which rows are necessary for your new test group, by following documentation beside the rows, then, proceed to un-comment said rows.
  • Continue to add the function by adhering to the instructions in the clause above.

Additional information

Whether you're adding a new function, or a new test group you'd be better off knowing the available resources at your disposal:

  • lithops/tests/util_func contains many functions, divided into categories, that you may find helpful.
  • lithops/tests contains a template_file called "test_template" which contains documentation regarding each common resource/import that may interest you.
  • Many examples utilizing said resources may be found across the "test_*" files of lithops/tests.
  • For a variety of evaluation functions ("assert*") belonging to unittest, browse this [documentation page](https://docs.python.org/3/library/unittest.html).

Examples

  • Access the functions in your chosen backend storage class via "STORAGE", e.g. STORAGE.put_object.
  • Access your bucket via STORAGE_CONFIG['bucket'].
  • Pass on "CONFIG" to your function executor instance, to allow your test function to work with users that provided a path to the config file via a flag, e.g. fexec = lithops.FunctionExecutor(config=CONFIG).