Skip to content

pkg/ccn-lite/relic: check for minimal cmake version#9720

Merged
miri64 merged 3 commits intoRIOT-OS:masterfrom
cladmi:pr/make/ccn-lite-relay/cmake_version
Aug 14, 2018
Merged

pkg/ccn-lite/relic: check for minimal cmake version#9720
miri64 merged 3 commits intoRIOT-OS:masterfrom
cladmi:pr/make/ccn-lite-relay/cmake_version

Conversation

@cladmi
Copy link
Contributor

@cladmi cladmi commented Aug 6, 2018

Contribution description

ccn-lite/relic do not build with cmake < 3.6.0. This checks for a minimal version
instead of failing to compile.

cmake version 3.5.2 is not >= to minimal required 3.6.0

Makefile:25: recipe for target '..cmake_version_supported' failed

Tested with versions 3.5.2, 3.6.0-rc1, 3.6.0, 3.6.3, 3.7.0-rc1, 3.7.0.

Note: the check used does not consider '-rcX' as 'sort -V' does not handle them
properly.

Testing

You can test this by compiling examples/ccn-lite-relay with different cmake versions:

Last 3.5 release

PATH=/tmp/cmake/cmake-3.5.2-Linux-x86_64/bin/:${PATH} make clean all BOARD=samr21-xpro

cmake version 3.5.2 is not >= to minimal required 3.6.0

Makefile:28: recipe for target '..cmake_version_supported' failed
make[1]: *** [..cmake_version_supported] Error 1

First 3.6 stable release

PATH=/tmp/cmake/cmake-3.6.0-Linux-x86_64/bin/:${PATH} make clean all BOARD=samr21-xpro
...
Build successfully

You can download cmake old versions here:

https://cmake.org/files/

Issues/PRs references

Fixes #9064 by making the problem clear

Found during release testing.

@cladmi cladmi requested review from cgundogan and smlng August 6, 2018 13:09
@cladmi cladmi added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 6, 2018
@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

@smlng I asked you to also have a test run on mac.

@miri64
Copy link
Member

miri64 commented Aug 6, 2018

@cladmi, some packet used in unittests causes a similar issue. Can you check?

@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

I thought about it just after submitting it ><

cp $(PKG_BUILDDIR)/src/lib/libccnl-riot.a $(BINDIR)/ccn-lite.a

$(PKG_BUILDDIR)/src/Makefile: $(TOOLCHAIN_FILE)
$(PKG_BUILDDIR)/src/Makefile: $(TOOLCHAIN_FILE) | ..cmake_version_supported
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is executed with every run without being cached, right? Can't we touch a file to skip subsequent version checks for an existing binary dir?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here TOOLCHAIN_FILE is also rebuilt everytime as it depends on git-download and so is Makefile created by cmake

A solution would still require to save command -v cmake to a file and verify it did not change since last call.
So I would more do it after I can finish working on #9634.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't you run the cmake version check before downloading or generating the toolchain file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was more concentrated on getting a working test than where I should put it.

Putting it as a check before downloading could be problematic if download gets done at a separate time or even a different environment. I still hope for a separate "prepare" step. But it could be moved again later.

So yes I can move it.

@smlng
Copy link
Member

smlng commented Aug 6, 2018

Doesn't work on macOS, I have cmake 3.12.0:

cmake --version
cmake version 3.12.0

CMake suite maintained and supported by Kitware (kitware.com/cmake).

but I get the error:

 make -C examples/ccn-lite-relay/
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
make[1]: Nothing to be done for `Makefile.include'.
make[1]: Nothing to be done for `Makefile.include'.
Building application "ccn-lite-relay" for "native" with MCU "native".

rm -Rf /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite
mkdir -p /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite
/Volumes/devel/github/smlng/RIOT/dist/tools/git/git-cache clone "https://github.com/cn-uofbasel/ccn-lite/" "48b17ebee7600b2e79b3acf0728217473d7a4ee8" "/Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite"
Cloning into '/Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite'...
remote: Counting objects: 22807, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 22807 (delta 6), reused 25 (delta 6), pack-reused 22772
Receiving objects: 100% (22807/22807), 15.77 MiB | 6.76 MiB/s, done.
Resolving deltas: 100% (15103/15103), done.
HEAD is now at 48b17ebe Merge pull request #269 from cgundogan/pr/fix_interest_evtimers
touch /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite/.git-downloaded
rm -Rf /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf
mkdir -p /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf
/Volumes/devel/github/smlng/RIOT/dist/tools/git/git-cache clone "https://github.com/mattconte/tlsf" "a1f743ffac0305408b39e791e0ffb45f6d9bc777" "/Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf"
Cloning into '/Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf'...
remote: Counting objects: 40, done.
remote: Total 40 (delta 0), reused 0 (delta 0), pack-reused 40
Unpacking objects: 100% (40/40), done.
HEAD is now at a1f743f Merge pull request #3 from velvitonator/large-alloc-corruption
touch /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf/.git-downloaded
git -C /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf checkout -f a1f743ffac0305408b39e791e0ffb45f6d9bc777
HEAD is now at a1f743f Merge pull request #3 from velvitonator/large-alloc-corruption
git -C /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf am --no-gpg-sign --ignore-whitespace "/Volumes/devel/github/smlng/RIOT/pkg/tlsf"/patches/*.patch
Applying: Fix warnining on implicit pointer conversion.
touch /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/tlsf/.git-patched
"/Library/Developer/CommandLineTools/usr/bin/make" -C /Volumes/devel/github/smlng/RIOT/pkg/ccn-lite
/Volumes/devel/github/smlng/RIOT/dist/tools/cmake/generate-xcompile-toolchain.sh > /Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/pkg/native/ccn-lite/xcompile-toolchain.cmake
sed: 1: "1 {s/cmake version //;s ...": extra characters at the end of p command

cmake version  is not >= to minimal required 3.6.0

make[1]: *** [..cmake_version_supported] Error 1
make: *** [/Volumes/devel/github/smlng/RIOT/examples/ccn-lite-relay/bin/native/ccn-lite.a] Error 2

Note: the sed command fails

@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

I have a quoting issue in the command, does this work ?

cmake --version | sed -n '1 {s/cmake version //;s/-rc.*//;p}'

@smlng
Copy link
Member

smlng commented Aug 6, 2018

@cladmi the p command must be closed with a ; too. Because this works for me:

cmake --version | sed -n '1 {s/cmake version //;s/-rc.*//;p;}'

@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

The issue for unittests is in relic that has the same issue.

I will move the test to a dedicated script to allow reusing it.

@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

Problem when doing the check before downloading, is that pkg-prepare errors are ignored:

pkg-prepare:
	-@for i in $(USEPKG) ; do "$(MAKE)" -C $(RIOTPKG)/$$i prepare ; done

So the error message is shown two times…

@cladmi
Copy link
Contributor Author

cladmi commented Aug 6, 2018

It still prevents to download the source though so could be kept this way.

@@ -0,0 +1,33 @@
#! /bin/bash
#
# usage: has_higher_version.sh <version> <minimal_version> [toolname]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't has_minimal_version be semantically (more) correct because the version doesn't have to be higher/greater but greater or equal, hence it should at least match the minimal version given

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will rename the scripts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also used "minimal version" everywhere but was not able to name the script accordingly ><

Copy link
Member

@smlng smlng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise: works now on macOS



HIGHEST=$(printf "%s\n%s\n" "${TOOLVERSION}" "${MINVERSION}" | sort -rV | head -n 1)
test "${HIGHEST}" = "${TOOLVERSION}" && exit 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to have a positiv message if version is good, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not output it in the build system. We say nothing about "gcc has a good version", "make is up to date", "wget is compatible".
Or I could print something to standard output and silent the error in Make.

HIGHEST=$(printf "%s\n%s\n" "${TOOLVERSION}" "${MINVERSION}" | sort -rV | head -n 1)
test "${HIGHEST}" = "${TOOLVERSION}" && exit 0

printf "\n%sversion %s is not >= to minimal required %s\n\n" "${TOOLSTR}" "${TOOLVERSION}" "${MINVERSION}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rephrase: printf "\nInvalid version, found %s%s, minimal required is %s\n\n" "${TOOLSTR}" "${TOOLVERSION}" "${MINVERSION}"

The >= should be implied by minimal required.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do, I should also print to stderr.

@cladmi cladmi changed the title pkg/ccn-lite: check for minimal cmake version pkg/ccn-lite/relic: check for minimal cmake version Aug 7, 2018
@cladmi cladmi force-pushed the pr/make/ccn-lite-relay/cmake_version branch from 5e1703f to 08f2ab0 Compare August 7, 2018 10:05
@cladmi
Copy link
Contributor Author

cladmi commented Aug 7, 2018

Updated. Tell me if you want an output that I silent in make.

@smlng smlng added Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation Area: build system Area: Build system Area: pkg Area: External package ports labels Aug 10, 2018
@cladmi
Copy link
Contributor Author

cladmi commented Aug 14, 2018

Can I squash ?

@smlng
Copy link
Member

smlng commented Aug 14, 2018

yes please!

cladmi added 3 commits August 14, 2018 15:26
    usage: has_minimal_version.sh <version> <minimal_version> [toolname]
      Checks that version >= minimal_version
      Version format MAJOR.MINOR.PATCH ex 3.1.4
ccn-lite does not build with cmake < 3.6.0. This checks for a minimal version
instead of failing to compile.

    cmake version 3.5.2 is not >= to minimal required 3.6.0

    Makefile:25: recipe for target '..cmake_version_supported' failed

Tested with versions 3.5.2, 3.6.0-rc1, 3.6.0, 3.6.3, 3.7.0-rc1, 3.7.0.

Note: the check used does not consider '-rcX' as 'sort -V' does not handle them
properly.
relic does not build with cmake < 3.6.0. This checks for a minimal version
instead of failing to compile.

    cmake version 3.5.2 is not >= to minimal required 3.6.0

    Makefile:25: recipe for target '..cmake_version_supported' failed
@cladmi cladmi force-pushed the pr/make/ccn-lite-relay/cmake_version branch from 08f2ab0 to 1aebe28 Compare August 14, 2018 13:27
@cladmi
Copy link
Contributor Author

cladmi commented Aug 14, 2018

The rebase has the same diff as before, except that ccn-lite version was bumped in between.

I retested compiling ccn-lite-relay on IoT-LAB with cmake version 3.0.2 and on my computer with cmake version 3.11.4.

Copy link
Member

@miri64 miri64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested Re-ACK

@miri64 miri64 merged commit 56d79b3 into RIOT-OS:master Aug 14, 2018
@cladmi cladmi deleted the pr/make/ccn-lite-relay/cmake_version branch August 15, 2018 15:42
@cladmi cladmi added this to the Release 2018.10 milestone Nov 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: build system Area: Build system Area: pkg Area: External package ports CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants