util: move utf8 utils to a separate header

[fabiobaltieri]

Hi, bumped into a build regression since #94779, one of our downstream application uses an interesting combination of configs that triggered a circular dependency build failure (which was a lot of fun to troubleshoot).

Managed to reproduce the issue upstream with

diff --git a/samples/basic/blinky/src/main.c b/samples/basic/blinky/src/main.c
index 4cab4969d94..5ebcbcf8a97 100644
--- a/samples/basic/blinky/src/main.c
+++ b/samples/basic/blinky/src/main.c
@@ -4,6 +4,9 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
+#define _GNU_SOURCE
+#include <sys/types.h>
+
 #include <stdio.h>
 #include <zephyr/kernel.h>
 #include <zephyr/drivers/gpio.h>

and west build -p -b nrf52dk/nrf52832 samples/basic/blinky -DCONFIG_POSIX_API=y -DCONFIG_NEWLIB_LIBC=y

May look into adding a test case for this but for now I think the right thing to do is to split these on their own header.

--

This moves the declaration of the utf8 utils defined in lib/utils/utf8.c in their own header. Main reason to do this is that the current setup requried adding an include for sys/types.h in util.h, which can result in a build falure due to a circular header depdenecy when using:

CONFIG_POSIX_API=y
CONFIG_NEWLIB_LIBC=y
_GNU_SOURCE

the loop and error are:

• include/sys/types.h:50: <- this is a NEWLIB one
• include/zephyr/posix/sys/select.h:9:
• include/zephyr/posix/posix_types.h:30:
• include/zephyr/kernel.h:17:
• include/zephyr/kernel_includes.h:25:
• include/zephyr/sys/atomic.h:18:

include/zephyr/sys/util.h:705:1: error: unknown type name 'ssize_t'

[fabiobaltieri]

cc @iliar

[fabiobaltieri]

I do not think this should be treated as a hotfix, this creates new headers, moves things around etc and has the chance of breaking existing users. Is this blocking anythjing significan anywhere in zephyr upstream?

No I caught this downstream, but that's just because nothing hit the particular combination of options that triggers the build fail, fine by me giving it one more day, just concerned about other users bumping into it, was a bit of a pain to troubleshoot would gladly spare other users from doing the same.

[fabiobaltieri]

CI test for this: #95395

[cfriedt]

@fabiobaltieri - my 2¢: leave the declarations in sys/util.h but remove <sys/types.h> and change the ssize_t to int.

The reasoning: Zephyr does occasionally get into trouble using POSIX types, especially the kind that have a length dependent on the arch.

In ISO C, time_t and size_t fall into that category.

In POSIX, ssize_t, off_t, and possibly others.

[fabiobaltieri]

@fabiobaltieri - my 2¢: leave the declarations in sys/util.h but remove <sys/types.h> and change the ssize_t to int.

I can do that, but looking at this file I think that these utf8 utilities should have never been in this file in the first place, util.h should really just be basic utilities that can find use anywhere, these utf8 string analysis/manipulation things belongs to their own header, how about keeping this to move them out and then you can do whatever you want with the type?

Feels like util.h became a bit of a dumping ground, these functions have their own C file, they should have their own header.

[fabiobaltieri]

Swapped to int, kept the separate header.

[fabiobaltieri]

Sorry one more tweak: converted the local variable in utf8.c as well and drop the header from there too. Nothing is ever easy.

[fabiobaltieri]

You know what, I may as well cherry pick the test while I'm at it.

[fabiobaltieri]

Ok so this is now a:

• split the header on their own file
• stop using types.h
• add a test for the nasty use case that led to this

@cfriedt happy with me adding that file in the posix tests?

@nashif this is going to fail compliance because _GNU_SOURCE so it'll need org superpower to merge

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cfriedt]

LGTM - in general, we should avoid exposing "gnuisms" that become visible with _GNU_SOURCE.

For test purposes, I think it's OK to do as evidence to show that we don't break in the presence of that macro.

[fabiobaltieri]

LGTM - in general, we should avoid exposing "gnuisms" that become visible with _GNU_SOURCE.

For test purposes, I think it's OK to do as evidence to show that we don't break in the presence of that macro.

Yeah of course, that's just for testing purposes, downstream users may do it, and we already have other tests that do it, and surprisingly some drivers code too.

[fabiobaltieri]

@nashif? (note that there's a CI false positive)