Documenting field testing and update automated testing in general

docs/dev_guide/testing.rst

@@ -4,6 +4,23 @@ Automated Testing
 
 Tripal 4 is being developed with automated testing as it is upgraded. This greatly improves the stability of our software and our ability to fix any bugs. We highly recommend developing automated testing alongside any extension modules you create! This guide is intended to explain how automated testing is working for Tripal 4 and help you develop similar tests for your extensions.
 
+You can find the tests for any Drupal module in the `tests` directory. Since Tripal core is composed of a number of submodules, you will find our test suite covers a number of directories:
+
+- Tripal Core (`tripal/tests`): these tests are focused on core functionality which is available to all Tripal sites.
+- Tripal Chado (`tripal_chado/tests`): here we test all interactions with the Chado database including database management, fields and Chado integration with core APIs such as Tripal DBX.
+- Tripal BioDB (`tripal_biodb/tests`): these tests focus on the Tripal BioTask API for managing database focused tasks which can benefit from advanced locking.
+- Tripal Layout (`tripal_layout/tests`): here we test the default layouts for Tripal Content forms and view displays.
+
+You will notice that in each of these test directories, there are a number of subdirectories. These are there by Drupal convention.
+
+- fixtures: this is where any SQL files or mock classes should go for setting up your tests.
+- src:
+
+    - Unit: PHPUnit-based tests with minimal dependencies. These should focus on testing specific methods using mock objects and should not require a fully bootstrapped Drupal site.
+    - Kernel: PHPUnit-based tests with a bootstrapped kernel, and a minimal number of extensions enabled.
+    - Functional: PHPUnit-based tests with a full bootstrapped Drupal instance. These tests include browser based tests and other tests looking at full subsystems.
+    - FunctionalJavascript: PHPUnit-based tests that use Webdriver to perform tests of Javascript and Ajax functionality in the browser.
+
 How run automated tests locally
 ---------------------------------
 
@@ -30,12 +47,29 @@ The following automated testing documentation and tutorials are focused on testi
 Additional Resources
 ----------------------
 
+*Note: The following docs are still relevant for Drupal 10.x to Drupal 11.2.x.*
+
+**Testing in General**
+
  - `Official Drupal: Testing Documentation <https://www.drupal.org/docs/testing>`_
  - `Official Drupal: PHPUnit file structure, namespace, and required metadata <https://www.drupal.org/docs/testing/phpunit-in-drupal/phpunit-file-structure-namespace-and-required-metadata>`_
  - `Official Drupal: Running PHPUnit Tests <https://www.drupal.org/docs/testing/phpunit-in-drupal/running-phpunit-tests>`_
+
+**Functional Testing**
+
  - `Official Drupal: PHPUnit Browser test tutorial <https://www.drupal.org/docs/testing/phpunit-in-drupal/phpunit-browser-test-tutorial>`_
  - `Official Drupal: PHPUnit JavaScript test writing tutorial <https://www.drupal.org/docs/automated-testing/phpunit-in-drupal/phpunit-javascript-test-writing-tutorial>`_
  - `Drupal 8, 9, 10 Functional and Unit Testing (Automation Testing) <https://gurinderpal.medium.com/drupal-8-9-10-functional-and-unit-testing-462993c3ce14>`_
+ - `Drupal API: BrowserTestBase <https://api.drupal.org/api/drupal/core%21tests%21Drupal%21Tests%21BrowserTestBase.php/class/BrowserTestBase/11.x>`_
+ - `Drupal API: UiHelperTrait <https://api.drupal.org/api/drupal/core%21tests%21Drupal%21Tests%21UiHelperTrait.php/trait/UiHelperTrait/11.x>`_
+ - `Drupal API: WebAssert <https://api.drupal.org/api/drupal/core%21tests%21Drupal%21Tests%21WebAssert.php/class/WebAssert/11.x>`_
+ - `Drupal API: FieldUiTestTrait <https://api.drupal.org/api/drupal/core%21modules%21field_ui%21tests%21src%21Traits%21FieldUiTestTrait.php/trait/FieldUiTestTrait/11.x>`_
+ - `Behat Mink: Traversing Pages <https://mink.behat.org/en/latest/guides/traversing-pages.html>`
+
+**Kernel and Unit Testing**
+
  - `Writing Automated Tests in Drupal 8, Part 4: Kernel tests <https://deninet.com/blog/2019/02/10/writing-automated-tests-drupal-8-part-4-kernel-tests>`_
  - `Writing Automated Tests in Drupal 8, Part 3: Unit tests <https://deninet.com/blog/2019/01/27/writing-automated-tests-drupal-8-part-3-unit-tests>`_
  - `Drupal 8: Writing Your First Unit Test With PHPUnit <https://www.axelerant.com/resources/team-blog/drupal-8-writing-your-first-unit-test-with-phpunit>`_
+ - `Envatotuts+: All About Mocking with PHPUnit <https://code.tutsplus.com/all-about-mocking-with-phpunit--net-27252t>`_
+ - `Official PHPUnit: Test Doubles <https://docs.phpunit.de/en/10.5/test-doubles.html>`_

docs/dev_guide/testing/chadoTestTrait.rst

@@ -5,26 +5,33 @@ Chado Testing Environment
 The chado testing environment builds upon the Tripal testing environment. I will describe the chado specific portions in detail below but for more detail on the other steps you should check out the documentation on the :ref:`Tripal Testing Environment`.
 
 1. Drupal sets up the testing environment including it's own database and fully functional site.
-2. Tripal does not make any new changes to the environment but the Chado Test bases do. Specifically, the add the chado_installations table to the drupal schema and initialize TripalDBX in the test environment.
+2. Tripal does not make any new changes to the environment but the Chado Test bases do. Specifically, they add the chado_installations table to the drupal schema and initialize TripalDBX in the test environment.
 3. The code inside your tests `setUp()` method is run. The first thing that should be done in the setup for any test interacting with chado is to intialize the chado database.
 
 
   .. code-block:: php
 
     // Initialize the chado instance with all the records that would be present after running prepare.
-    $chado = $this->getTestSchema(ChadoTestBrowserBase::PREPARE_TEST_CHADO);
+    $chado_connection = $this->getTestSchema(ChadoTestBrowserBase::PREPARE_TEST_CHADO);
 
   The capitalized portion indicates the type of chado to initialize. In the above example, `PREPARE_TEST_CHADO` indicates the resulting chado schema will have all the records that would be present after running prepare. The options available are:
 
    - `INIT_CHADO_EMPTY`: creates an empty chado schema for testing. All tables are there but no records.
    - `PREPARE_TEST_CHADO`: creates a chado schema with all tables and all records that would be present after running "prepare" through the UI.
    - `INIT_CHADO_DUMMY`: creates a prepared chado schema with everything that PREPARE_TEST_CHADO and with additional test data. You can see the test data here in `tripal_chado/tests/fixtures/fill_chado.sql <https://github.com/tripal/tripal/blob/4.x/tripal_chado/tests/fixtures/fill_chado.sql>`_
 
+   Additionally, in some cases you will want to specify a specific version of chado to install in the test environment. This can be done by passing the version number as the optional second parameter as is shown in the following example.
+
+   .. code-block:: php
+
+    // Initialize an empty chado version 1.3.3.013 instance with no records.
+    $chado_connection = $this->getTestSchema(ChadoTestBrowserBase::INIT_CHADO_EMPTY, '1.3.3.013');
+
 4. Finally your test method is called. Note: any services, plugins, etc. that you use here will only have the test environment available. You will not have access to any data in your main site, nor should this long term affect your main site. **That said, we do not recommend running tests on production sites!**
 5. Once your test is complete, the `tearDown()`` method is called to clean the entire development environment up. This includes dropping the development drupal tables including any changes made by your test.
 
-Retrieving the cvterm ID of a term in your test chado
--------------------------------------------------------
+Retrieving the cvterm ID of a term
+------------------------------------
 
 Often in your setup you will be using Tripal DBX to insert records into your test chado instance. There is a handly function to help you look up the cvterm_id based on the accession:
 
@@ -35,3 +42,19 @@ Often in your setup you will be using Tripal DBX to insert records into your tes
     $cvterm_id = $this->getCvtermID($idspace, $accession);
 
 The above example retrieves the cvterm_id for the gene term in the test chado database.
+
+Retrieving a cv, db, or cvterm record
+----------------------------------------
+
+When testing functionality you will often want to select a record from your test chado instance to confirm your functionality did what you expected. This can be done using Tripal DBX just as you would outside of the testing environment. Additionally, for a few often used tables we have helper methods to make it even easier. Each of the following will retrieve a single row in the specified table based on the parameters.
+
+  .. code-block:: php
+
+    $cvname = 'sequence';
+    $cv_record = $this->getChadoCvRecord($cvname);
+
+    $dbname = 'SO';
+    $db_record = $this->getChadoDbRecord($dbname);
+
+    $cvterm_name = 'gene';
+    $cvterm_record = $this->getChadoCvtermRecord($cvname, $cvterm_name);

docs/dev_guide/testing/fields.rst

@@ -16,4 +16,4 @@ For more information on how to test each of the above goals, see the following t
    :maxdepth: 2
 
    fields/chadoStorage
-
+   fields/fieldTypeCRUD

docs/dev_guide/testing/fields/chadoStorage.rst

@@ -3,7 +3,7 @@ Testing Chado Field storage
 =============================
 
 .. warning::
-  This documentation is still under development and is not complete.
+  This approach still works but can be slow. As such many of the core tests have moved to testing chado storage indirectedly through the field classes themselves.
 
 As described in the documentation for how to create fields, Chado Fields depend on the developer to define a number of properties in order to describe to ChadoStorage how to create, load and update the various biological data associated with that field. For example, when creating a field to describe the organism associated with a gene, you will define properties for the genus, species, infraspecific type, infraspecific name, etc. Then ChadoStorage will use the property definitions to pull these data out of Chado and make them available to your field. In this tutorial, we are focusing on testing that the properties you defined in your field, act as you expect and that ChadoStorage is understanding your intent properly.
 

docs/dev_guide/testing/fields/fieldTypeCRUD.rst

@@ -0,0 +1,13 @@
+Testing FieldType classes through CRUD operations
+====================================================
+
+.. warning::
+
+  All the following will assume you are developing these tests using the TripalDocker, with your module fully installed and mounted within the docker. For instructions on how to set this up see :doc:`Install with TripalDocker </install/docker>` documentation. In all the commands with CONTAINERNAME, replace it with the name of your container.
+
+Creating your testing FieldType Test Class
+-------------------------------------------
+
+All tests are encapsulated within their own class as dictated by PHPunit. As such lets create a class skeleton as follows:
+
+In `[yourmodule]/tests/src/Kernel/Plugin/ChadoField/FieldType` create a file with a descriptive name for your test. We recommend naming it using the following pattern `[FieldName]Test.php`. The "Test" suffix is required by PHPUnit so make sure to at least include that in your test name or the test runner will not be able to find your test.