Code Validator and Reference Validator enhancements

[LakshmiDintakurty]

Code Enhancements: Below are the list of code changes made for code-validator (and reference-ccda-validator). Please review the changes and let us know if you need any additional information and we can setup a meeting to discuss.

1. Async thread pool for code validation

   VocabularyValidationService uses a pool of ValidateWorker objects to perform code validation in parallel.

2. Fixed Thread safety issues in Vocab validator

3. VTD – Object pool pre-compiled XPaths

   a. VTD is an alternate high performance XML Parser.
   b. The auto pilot package (org.sitenv.vocabularies.validation.pool) encapsulates a pool of precompiled XPaths based on ccdaReferenceValidatorConfig.xml file
   c. The pool configuration is specified in CodeValidator.properties.

4. Switched from HSQL to H2 DB

   a. Configurable DatabaseConnection pool in CodeValidator.properties.
   b. Replaced sql.Connection with sql.DataSource.
   c. All the loader classes now use DataSource to load the valuesets/codesets from files to H2 DB and at the time of loading from H2 into HashSets.

5. Vocab lookup using Java HashSets/HashMaps instead of JPA queries

   a. Reimplemented the Repository classes to respective DAO classes (CodeSystemCodeDAO and ValueSetDAO)
   b. References to the CodeRepository and VsacValueSetRepository in CodeSystemCodeValidator & ValueSetCodeValidator are replaced with respective DAOs (CodeSystemCodeDAO and ValueSetDAO)

6. DB Cleanup after loading codes and valuesets to HashSets

   a. In order to reduce the memory footprint, after loading all the codes and valuesets to HashSets, all the data is deleted from H2DB.
   b. VocabularyLoadRunner performs the cleanup based on the flag ‘cleanUpDatabaseAfterLoadingHashSets’ set in CodeValidator.properties.
   c. Additional comments provided on the side effects/conditions w.r.t to setting the flag to true vs false in VocabularyLoadRunner class in the finally block of afterPropertiesSet().

7. ReferenceCCDAValidationService : Added new overloaded service methods to handle document validation

   Ex., support optional REST request parameter SeverityLevel (allowable values: error, warning, info). if SeverityLevel=”error”, don’t evaluate “warning” and “info” Vocab Conformance Rules

8. Added new functionality to support R1.1 CCDA doc Vocabulary Validation (VocabularyValidationService)

9. Ability to upload a compressed file for validation in addition to plain xml file to improve network latency

   Added functionality to support .zip format in addition to .xml file format in ReferenceCCDAValidationService. Useful when CCDA documents are large in size.

10. Perform MDHT & Vocab Validation of a document regardless whether the doc has schemaError or not, to better handle MU3 CERT Validator negative test cases

11. Support 100% accurate source CCDA file Line Numbers for each of the Vocabulary Validation error/warning/info results.

12. For supportability, added the Vocabulary Conformance Rule ID feature

    a. A unique Vocabulary Conformance Rule ID is defined for each configured in ccdaReferenceValidatorConfig.xml. Such as < validator id="1" >
    b. In each Vocabulary Validation error/warning/info result, output the conformance Rule ID it's violating. Such as "ruleID": "140"

[drbgfc]

Note: Please do not close this PR. Even though we are highly out of sync, and it is years later, it is an excellent reference that a lot of work went into.