CodeQuality: Doxygen regex doesn't work for a warning that's on two lines

[lit-af]

Hi,

I found a small bug with the Doxygen regex, the code-quality report generated description field is incomplete when there a warning message that's on two lines.

DoxygenWarningsLog.txt

C:/DEV/kog-ipc-common/src/kog_ipc.c:485: warning: The following parameter of Ipc_handleReceivedMessage(KogIpc_t *pIpc, IpcMessageEnvelope_t *pMessage, bool *pDisposeEnvelope) is not documented:
  parameter 'pDisposeEnvelope'

Doxygen_quality.json

    {
        "severity": "major",
        "location": {
            "path": "kog-ipc-common\\src\\kog_ipc.c",
            "lines": {
                "begin": 485
            }
        },
        "description": "The following parameter of Ipc_handleReceivedMessage(KogIpc_t *pIpc, IpcMessageEnvelope_t *pMessage, bool *pDisposeEnvelope) is not documented:",
        "fingerprint": "504e0ad956f6bf387431e0737ce0708a"
    },

I'm wondering if the issue comes from my doxygen configuration or from the regex.

EDIT:
I figured out what the regex should be!

r"(?:(?P<path1>(?:[/.]|[A-Za-z]).+?):(?P<line1>-?\d+):\s*(?P<severity1>[Ww]arning|[Ee]rror)|<.+>:(?P<line2>-?\d+)(?::\s*(?P<severity2>[Ww]arning|[Ee]rror))?): (?P<description1>.+(?:(?!\s*([Nn]otice|[Ww]arning|[Ee]rror): )[^/<\n][^:\n][^/\n].+)*[\s\n]*[^/<\n][^:\n][^/\n].+)|\s*\b(?P<severity3>[Nn]otice|[Ww]arning|[Ee]rror): (?!notes)(?P<description2>.+)\n?"

[Letme]

Great - can you open a Pull Request? You have a test case there so it should be quite straight forward and we welcome new contributions.

[lit-af]

I'd be willing to, but there's very little documentation on how to contribute and I'm mainly a C/C++ dev.

Is there any chance you could add instructions on how to add/run tests and how to run a development version of the package?

[lit-af]

I haven't been able to test this locally as I'm getting an error when when trying to install locally. I tested my regex online using the doxygen_warnings.txt log found in this repo.

FYI

The error is the following:

C:\dev\IoT\warnings-plugin> pip3 install -r .\docs\requirements.txt
Obtaining file:///C:/dev/IoT/warnings-plugin (from -r .\docs\requirements.txt (line 5))
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\build_meta.py", line 441, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 30, in <module>
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\discovery.py", line 127, in find
          convert_path(str(where)),
          ^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\AlexandreFrancoeur\AppData\Local\Temp\pip-build-env-dezlsa6y\overlay\Lib\site-packages\setuptools\_distutils\util.py", line 141, in convert_path
          raise ValueError("path '%s' cannot end with '/'" % pathname)
      ValueError: path 'src/' cannot end with '/'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

and I get the same error when running pip3 install -e warnings-plugin

[Letme]

That's great feedback - I will look into it along with the fact that we might be missing this installation/test guidelines in CONTRIBUTING.md file (you can help to improve as well).

Repo has tox for setting up your environment and installing stuff (so you simply run tox after you installed it), but it seems that is is not working at them moment since we relied on Continious Integration for too long (and also that is not running from your Pull Request). I will look into that as well in next few days.

[lit-af]

Thanks for actively responding to issues raised here. It's quite refreshing to see an OSS repo with active support for once.

I was able to install the package in a docker container running ubuntu 22.04. I figured out that git is a dependency, but it's not mentionned in the install instructions.

[Letme]

Yes, probably because if you manage to clone the repo, you should have git installed. Most of the time you should be using virtual environments (python -m venv venv and then source venv/bin/activate) for Python development, but I also use docker for keeping the C development environment.

We are glad to help. Since we use all our tools daily, then most of the time it is really cool to see other people using them as well - and we also learn about other usecases. Now I just need to find some time 😅

[lit-af]

Sadly, I'm on Windows at work and it looks like this package has been developed on linux only.

I'm testing my setup using the master branch in a freshly created and activated venv:

> python --version
Python 3.11.5
> git reflog
e12d16f (HEAD -> master, origin/master, origin/HEAD) HEAD@{0}: clone: from https://github.com/melexis/warnings-plugin.git

I'm getting the following error code when running tox -e py311

=========================================================================================================== test session starts ============================================================================================================
platform win32 -- Python 3.11.5, pytest-7.4.4, pluggy-1.3.0 -- C:\DEV\IoT\warnings-plugin\.tox\py311\Scripts\python.EXE
cachedir: .tox\py311\.pytest_cache
rootdir: C:\DEV\IoT\warnings-plugin
configfile: setup.cfg
plugins: cov-4.1.0
collecting ... collected 126 items

[...]

========================================================================================================= short test summary info ==========================================================================================================
FAILED tests/test_integration.py::TestIntegration::test_code_quality - AssertionError: False is not true : C:\DEV\IoT\warnings-plugin\tests\test_out\code_quality.json differs from C:\DEV\IoT\warnings-plugin\tests\test_in\code_quality....FAILED tests/test_integration.py::TestIntegration::test_code_quality_abspath_failure - AssertionError: ValueError not raised
FAILED tests/test_integration.py::TestIntegration::test_command_revtal_err - FileNotFoundError: [WinError 2] The system cannot find the file specified
FAILED tests/test_integration.py::TestIntegration::test_command_revtal_err_supress - FileNotFoundError: [WinError 2] The system cannot find the file specified
FAILED tests/test_integration.py::TestIntegration::test_command_to_stderr - FileNotFoundError: [WinError 2] The system cannot find the file specified
FAILED tests/test_integration.py::TestIntegration::test_command_with_its_own_arguments - FileNotFoundError: [WinError 2] The system cannot find the file specified
FAILED tests/test_integration.py::TestIntegration::test_cq_description_format - AssertionError: False is not true : C:\DEV\IoT\warnings-plugin\tests\test_out\code_quality_format.json differs from C:\DEV\IoT\warnings-plugin\tests\test_...FAILED tests/test_integration.py::TestIntegration::test_output_file_junit - AssertionError: False is not true : C:\DEV\IoT\warnings-plugin\tests\test_out\junit_double_fail_summary.txt differs from C:\DEV\IoT\warnings-plugin\tests\test...FAILED tests/test_integration.py::TestIntegration::test_output_file_robot_basic - AssertionError: False is not true : C:\DEV\IoT\warnings-plugin\tests\test_out\robot_double_fail_summary.txt differs from C:\DEV\IoT\warnings-plugin\test...FAILED tests/test_integration.py::TestIntegration::test_output_file_robot_config - AssertionError: False is not true : C:\DEV\IoT\warnings-plugin\tests\test_out\robot_double_fail_config_summary.txt differs from C:\DEV\IoT\warnings-plu...FAILED tests/test_integration.py::TestIntegration::test_output_file_sphinx - AssertionError: False is not true
FAILED tests/test_integration.py::TestIntegration::test_single_command_argument - FileNotFoundError: [WinError 2] The system cannot find the file specified
FAILED tests/test_integration.py::TestIntegration::test_two_command_arguments - FileNotFoundError: [WinError 2] The system cannot find the file specified
====================================================================================================== 13 failed, 113 passed in 0.93s ======================================================================================================
py311: exit 1 (1.36 seconds) C:\DEV\IoT\warnings-plugin> pytest --cov=mlx --cov-report=term-missing -vv tests/ pid=16788
.pkg: _exit> python C:\Users\AlexandreFrancoeur\AppData\Local\Programs\Python\Python311\Lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta
  py311: FAIL code 1 (25.19=setup[23.83]+cmd[1.36] seconds)
  evaluation failed :( (25.30 seconds)

I finally found the time to test in a ubuntu:22.04 docker container and I'm getting errors there too.

========================================================================================================== short test summary info ==========================================================================================================FAILED tests/test_integration.py::TestIntegration::test_code_quality - AssertionError: False is not true : /warnings-plugin/tests/test_out/code_quality.json differs from /warnings-plugin/tests/test_in/code_quality.json
FAILED tests/test_integration.py::TestIntegration::test_cq_description_format - AssertionError: False is not true : /warnings-plugin/tests/test_out/code_quality_format.json differs from /warnings-plugin/tests/test_in/code_quality_format.json
FAILED tests/test_integration.py::TestIntegration::test_output_file_junit - AssertionError: False is not true : /warnings-plugin/tests/test_out/junit_double_fail_summary.txt differs from /warnings-plugin/tests/test_in/junit_double_fail_summary.txt
FAILED tests/test_integration.py::TestIntegration::test_output_file_robot_basic - AssertionError: False is not true : /warnings-plugin/tests/test_out/robot_double_fail_summary.txt differs from /warnings-plugin/tests/test_in/robot_double_fail_summary.txt
FAILED tests/test_integration.py::TestIntegration::test_output_file_robot_config - AssertionError: False is not true : /warnings-plugin/tests/test_out/robot_double_fail_config_summary.txt differs from /warnings-plugin/tests/test_in/robot_double_fail_config_summary.txt
FAILED tests/test_integration.py::TestIntegration::test_output_file_sphinx - AssertionError: False is not true
======================================================================================================= 6 failed, 120 passed in 3.79s =======================================================================================================ERROR: InvocationError for command /warnings-plugin/.tox/py310/bin/pytest --cov=mlx --cov-report=term-missing -vv tests/ (exited with code 1)
__________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________ERROR:   py310: commands failed

[JasperCraeghs]

Your changes in PR #136 probably require changes to the reference/input files of the unit tests.

[lit-af]

I actually cloned your repo's master branch to make sure that I was able to run the tests first.