tools/ci: correcly report flake8 version.

[jcarrano]

This PR is rebased on top if #10251 for easy testing, but it is not required.

Contribution description

This PR contains two commits. I can split it if necessary.

Change the way version detection works

Detect command line tool versions without using "command". "command" may be a builting in some shells, leading to unportability.

The new version uses the status code to correctly detect a non-existent command. This allows it to differentiate between error in the tool and not-found errors.

It also works with compound commands, for example `python -m callable_module".

Fix flake8 detection in some systems (for example, riotdocker)

If the flake8 executable is not found, the static test script reports the tool as missing. It may happen that the flake8 module is installed, but the console entry point is not.

In the flake8 shell script, flake is invoked via python -m. The result is a confusing error message where static-test reports the tools as missing, yet the flake8 tests are run.

This patch makes the toolchain version script use the same command as the flake8 script.

Testing procedure

The easiest way to see the issue is to look at Murdock output from any PR (example) and see that flake8 is reported as missing, but the test that uses flake8 passes.

Travis uses a different image that does not show the problem, that's why I rebased on top of #10251. That PR's Travis output has the issue, while this one should not.

With this PR, the static-tests can detect flake8 without problems.

Why is this important?

Because if we have tests it is because we want to run them and if the output from the static test is contradictory, how do we know if the tests are running or not?

[jnohlgard]

See inline comment

[jnohlgard]

This way is not portable when the command is python -m flake8. It works in dash, but in bash this will look for the commands python, -m, and flake8 and print the paths found and return zero if any command is found. In zsh, it looks for the commands like in bash, but additionally, returns a non zero return code if any command is not found.

Shell test locally:

% command -v python3 -m flake8; echo return: $?
/usr/bin/python3
/usr/bin/flake8
return: 1

[jnohlgard]

How about adding optional extra arguments to this function instead, which can be used to pass arguments?

get_cmd_version() {
    if [ -z "$1" ]; then
        return
    fi

    local cmd="$1"
    shift 1
    local args=--version
    if [ $# -gt 0 ]; then
        args="$@"
    fi
    if command -v $cmd 2>&1 >/dev/null; then
        ver=$($cmd $args 2> /dev/null | head -n 1)
        # some tools (eg. openocd) print version info to stderr
        if [ -z "$ver" ]; then
            ver=$($cmd $args 2>&1 | head -n 1)
        fi
        if [ -z "$ver" ]; then
            ver="error"
        fi
    else
        ver="missing"
    fi

    printf "%s" "$ver"
}

and

printf "%23s: %s\n" "flake8" "$(get_cmd_version python3 -m flake8 --version)"

Though, on failure we get this error message instead (because of the captured stderr output):

flake8: /usr/lib/python-exec/python3.6/python3: No module named fake8

[jcarrano]

I think I fixed it now.

[smlng]

works for me now, i.e., on macOS:

 ./dist/tools/ci/print_toolchain_versions.sh 

Operating System Environment
-----------------------------
       Operating System: Mac OS X 10.14.2
                 Kernel: Darwin 18.2.0 x86_64 i386

Installed compiler toolchains
-----------------------------
             native gcc: gcc (Homebrew GCC 8.2.0) 8.2.0
      arm-none-eabi-gcc: arm-none-eabi-gcc (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204]
                avr-gcc: avr-gcc (GCC) 8.2.0
       mips-mti-elf-gcc: missing
             msp430-gcc: missing
   riscv-none-embed-gcc: missing
                  clang: Apple LLVM version 10.0.0 (clang-1000.10.44.4)

Installed compiler libs
-----------------------
   arm-none-eabi-newlib: "2.5.0"
    mips-mti-elf-newlib: missing
riscv-none-embed-newlib: missing
               avr-libc: "2.0.0" ("20150208")

Installed development tools
---------------------------
                  cmake: cmake version 3.13.2
               cppcheck: Cppcheck 1.86
                doxygen: 1.8.15
                    git: git version 2.20.1
                   make: GNU Make 3.81
                openocd: Open On-Chip Debugger 0.10.0
                 python: Python 2.7.15
                python2: Python 2.7.15
                python3: Python 3.7.2
                 flake8: 3.5.0 (mccabe: 0.6.1, pycodestyle: 2.3.1, pyflakes: 1.6.0) CPython 3.7.2 on Darwin
             coccinelle: spatch version 1.0.6 compiled with OCaml version 4.05.0

[jcarrano]

I rebased-squashed-fixed conflicts.

@gebart Are you OK with this PR?

[jnohlgard]

See comments, untested though

[jnohlgard]

Should this be "$@" instead of $@?
(combined with removing the quotes below at the get_cmd_version call for flake8

Suggested change

	VERSION_RAW=$( ($@ --version) 2>&1)
	VERSION_RAW=$( ("$@" --version) 2>&1)

[jcarrano]

"$@" will treat "a b" as the name of a command instead of command a with argument b.

[jnohlgard]

it is more reliable to use printf %s "$VERSION_RAW" instead of echo to avoid getting strange outputs in certain situations, especially since $VERSION_RAW may contain anything.

[jcarrano]

This has the head to make sure we get the fist line in commands that have more than one.

[jnohlgard]

Suggested change

	printf "%23s: %s\n" "flake8" "$(get_cmd_version "python3 -Wignore -m flake8")"
	printf "%23s: %s\n" "flake8" "$(get_cmd_version python3 -Wignore -m flake8)"

[jcarrano]

See comment above for $@.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[smlng]

@jcarrano can you have another look at @gebart comments, maybe we can get this in quick?!

[jcarrano]

@smlng I responded to @gebart's comment. He does not seem active at the moment, so if you think I answered satisfactorily dismiss his review and we merge.

no response

[smlng]

@cladmi can you have a quick look on this one, thx!

[cladmi]

Oops, sorry I missed your message, it got lost in the stale bot flood.