test suite failure: nl-test-util.c:Core:route_1 The list of 2 routes did not find a one-to-one match with the list of 3 expected routes

[kanavin]

The Yocto Project CI is seeing an intermittent failure on arm64 hosts:

local fe80::a87d:e0ff:fe4b:9308 dev v1 table main proto kernel scope global metric 0 pref medium
unicast fe80::/64 dev v1 table main proto kernel scope global metric 256 pref medium
multicast ff00::/8 dev v1 table main proto kernel scope global metric 256 pref medium
FAIL: nl-test-util.c:Core:route_1 The list of 2 routes did not find a one-to-one match with the list of 3 expected routes

This does not seem to be happening on x86 hosts.

[KanjiMonster]

I have a dejavu, and the previous issue for the same test failing was even by you as well.

For some reason printf() capturing doen't seem to work within libcheck, so the output of passed on routes wasn't collected.

But luckily the issue is "obvious" from the FAIL line:

FAIL: nl-test-util.c:Core:route_1 The list of 2 routes did not find a one-to-one match with the list of 3 expected routes

Since the ip output contains 3 routes, but it claims it only got 2, my first guess here would be that this is a race condition between the kernel assigning adding routes when bringing the interface up, and retrieving the route cache.

If the route cache is queried before all routes were added, we will only get e.g. 2 instead of the expected 3 routes, and then the check fails.

Edit: all three routes are only added on bring up, so it races against all three.

[KanjiMonster]

@thom311 should printf() in tests be fprintf(stderr, ...) so it gets captured in the test logs? AFAICT most of them do only print on failure, which would seem relevant.