test: refactor fake email heuristic to be more optimistic and allow offline tests

[art1f1c3R]

Summary

This PR refactors the FakeEmailAnalyzer focusing on making it more optimistic in passing emails, providing more information, and allowing tests to be run offline.

Description of changes

Previously, the FakeEmailAnalyzer.analyze method would return a FAIL result on the first detected invalid, or non-email, field for both the author and maintainer emails. This PR refactors this with the goal of reducing false positives to instead only return a FAIL if all emails are invalid, so if any of the emails are determined to be valid, then this heuristic passes. Previously, the FakeEmailAnalyzer.analyze method would also return immediately on a FAIL case and not analyze other emails if present. This PR refactors this to instead analyze all present emails regardless, to provide more information in its JSON return value.

The tests are refactored to run offline. Unit tests only test for valid and non-emails, as to run offline checking deliverability is turned off. The integration test email_checking_pypi_malware_analyzer has been added to check the deliverability of emails so invalid emails can be tested. It runs against the package smooth-operator, which has a known invalid email address with domain example.com, and email-validator, which has a known valid and deliverable email address.

Checklist

• I have reviewed the contribution guide.
• My PR title and commits follow the Conventional Commits convention.
• My commits include the "Signed-off-by" line.
• I have signed my commits following the instructions provided by GitHub. Note that we run GitHub's commit verification tool to check the commit signatures. A green verified label should appear next to all of your commits on GitHub.
• I have updated the relevant documentation, if applicable.
• I have tested my changes and verified they work as expected.

[art1f1c3R]

check_emails.sh has been run through ShellCheck and has no detected issues.

[behnazh-w]

check_emails.sh has been run through ShellCheck and has no detected issues.

Thanks for that. But going forward, we don't need to run ShellCheck because it is added already as a pre-commit hook 🙂

[tromai]

I think this fixture is defined but it's not used within the tests. Therefore, the logic of overriding check_deliverability = False didn't run, meaning that all FakeEmailAnalyzer instances in this test module had check_deliverability set to True. I have verified this by adding a breakpoint

@pytest.fixture(name="fake_email_defaults_override")
def set_defaults_(tmp_path: Path) -> None:
    """Disable check_deliverability in defaults.ini so we do not make network connections.

    Parameters
    ----------
    tmp_path: Path
        Pytest temporary path fixture.
    """
    import pdb; pdb.set_trace()
    defaults_file = Path(os.path.join(tmp_path, "config.ini"))
    content = """
    [heuristic.pypi]
    check_deliverability = False
    """
    defaults_file.write_text(content, encoding="utf-8")
    assert load_defaults(str(defaults_file)) is True

And ran this test module with

pytest --no-cov /home/tromai/dev/oracle/github/oracle/macaron/tests/malware_analyzer/pypi/test_fake_email.py

where the breakpoint wasn't triggered.

One fix could be using this fixture in the analyzer_ fixture:

@pytest.fixture(name="analyzer")
def analyzer_(fake_email_defaults_override: None) -> FakeEmailAnalyzer:
    """Pytest fixture to create a FakeEmailAnalyzer instance."""
    return FakeEmailAnalyzer()