Severity level should be configurable for each checker: - warning - severe - default If the level is warning a failing test should not result in an error code on exit.