Different build behavior between `murdock` and `riot/riotbuild:latest` image

[cladmi]

Description

Compilation behavior is different between murdock the docker image with BUILD_IN_DOCKER=1 even with RIOT_CI_BUILD=1.

It makes things hard to debug and understand the root cause of problems.

Like for the following PRs:

#9362 Warnings only on murdock
#9605 Warnings only on murdock with llvm
#9507 Enough memory on murdock but not in the container

Sub pull requests

• pkg/libfixmath: replace FILE by RIOT_FILE_NOPATH pkg/libfixmath: replace __FILE__ by RIOT_FILE_NOPATH #9643
• makefiles/docker.inc.mk: export RIOT_CI_BUILD to docker makefiles/docker.inc.mk: export RIOT_CI_BUILD to docker #9895
• makefiles/docker.inc.mk: handle building in git worktree makefiles/docker.inc.mk: handle building in git worktree #10303
• makefiles/docker.inc.mk: Use directories in RIOT when possible makefiles/docker.inc.mk: Use directories in RIOT when possible #9646
  • Makefile.include: Fix BUILDRELPATH when RIOTPROJECT is CURDIR. Makefile.include: Fix BUILDRELPATH when RIOTPROJECT is CURDIR. #10302
• tests/*boot: handle building in docker tests/mcuboot: handle building in docker  #11083

Steps to reproduce the issue

To take one as an example:

Building tests/pkg_fatfs_vfs for BOARD=wsn430-v1_4 works in murdock as it is not complaining but this does not

make -C tests/pkg_fatfs_vfs/  BOARD=wsn430-v1_4 clean all RIOT_CI_BUILD=1 BUILD_IN_DOCKER=1 DOCKER="sudo docker"

Expected results

It has the same behavior in docker as in murdock, so either both fail or both succeed.

Actual results

Murdock is ok, but in docker it fails with:

/data/riotbuild/riotproject/tests/pkg_fatfs_vfs/bin/wsn430-v1_4/fatfs.a(ff.o):(.debug_loc+0x1656): relocation truncated to fit: R_MSP430_16_BYTE against `no symbol'
collect2: ld returned 1 exit status
/data/riotbuild/riotbase/Makefile.include:411: recipe for target '/data/riotbuild/riotproject/tests/pkg_fatfs_vfs/bin/wsn430-v1_4/pkg_fatfs_vfs.elf' failed
make: *** [/data/riotbuild/riotproject/tests/pkg_fatfs_vfs/bin/wsn430-v1_4/pkg_fatfs_vfs.elf] Error 1

Versions

Building with docker in master.

[cladmi]

@kaspar030 @smlng I think you are the persons who would know the most about it

[cladmi]

First thing I found, is that RIOT_CI_BUILD is not given to docker:

Without docker: -DRIOT_VERSION=\"buildtest\"
With Docker: -DRIOT_VERSION=\"2018.10-devel-104-g8832-9f7d272ad793\"

When building from a worktree, as the .git is not the real directory I even get:

With Docker: -DRIOT_VERSION=\"'UNKNOWN (builddir: /data/riotbuild/riotbase)'\"

So maybe the RIOT_VERSION should be parsed by the main repo and passed to docker.

[cladmi]

Other issue, RIOT_FILE_RELATIVE is not really relative in docker:

-DRIOT_FILE_RELATIVE=\"/data/riotbuild/riotproject/tests/libfixmath_unittests/bin/pkg/stm32f0discovery/libfixmath/libfixmath/fix16_str.c\"

Outside I get:

-DRIOT_FILE_RELATIVE=\"tests/libfixmath_unittests/bin/pkg/stm32f0discovery/libfixmath/libfixmath/fix16_str.c\"

Which can easily explain build size difference when RIOT_FILE_RELATIVE is used.

[cladmi]

Also if it's possible to have an archive of the currently used image. I could diff with my local docker image and try to find the reason.

[miri64]

At least for the LLVM thing @kaspar030 found a reason: #9398 (comment)

[ZetaR60]

Any updates on this? The new Docker image in RIOT-OS/riotdocker#42 is waiting on this to be resolved.