[RFC] POSIX subsys vs libc: playing along or against each other

[pfalcon]

This is related to #13054 and #12965, and made a separate ticket to capture the essence of the problem and have a focused discussion.

The basic fact:

C library specification is a part of POSIX. See e.g. http://pubs.opengroup.org/onlinepubs/9699919799/functions/memset.html for POSIX specification for memset(). Note that POSIX's libc specification is a superset of ANSI C standard library specification.

In other words, we'll never have a no-nonsense POSIX subsystem, until along with it we provide a decent libc implementation. Currently, Zephyr support 2 libc implementations:

• Minimal libc, fully part of Zephyr source tree. But implementing a tiny subset of ANSI C standard library, and far less of POSIX libc.
• newlib libc, which is pretty full-fledged implementation of ANSI C and POSIX libc (though some parts are missing, e.g. BSD Sockets API implementation). newlib is an external project, shipping along with toolchain as part of the Zephyr SDK.

The essence of the proposal is: Given that minimal libc is clearly inadequate for POSIX compatibility, make newlib a requirement for POSIX subsystem.

The risk of this solution: This effectively also make Zephyr SDK a requirement for POSIX subsys, i.e. POSIX subsys won't be available for 3rd-party toolchains (unless those toolchains include compatible build of newlib (compatible, because newlib itself comes in different versions, there can be changes/incompatibilities and POSIX subsys may depend on a particular version)).

Alternative solution 1:

Support both newlib and minimal libc. This would require pretty heavy development on minimal libc. And not just development, but development in a way to be compatible with newlib (because otherwise POSIX subsys will be ridden with #ifdef for one vs another libc; instead, we should take newlib as a pattern, and make minimal follow it). Put it in plain words, we either will need to create a header set for minimal which would match that of newlib "from scratch", or just copy it from newlib. And that doesn't even talk about actual code to implement the function. In the end of long (or not so long, if we indeed just copy newlib header set) path, we'll end up with 2 copies of newlib: one in the main tree, another in SDK tree, to maintain. That clearly sounds like monkey work to do once, and then over along.

Alternative solution 2:

Try to preserve minimal support for those parts of POSIX subsys which have it now. And that's literally just pthread and fs. This would require the same nature of work as for Alternative 1 (making minimal libc headers compatible with newlib), but at least it will be bounded.

[pfalcon]

@galak, @nashif: FYI

[pfalcon]

Regarding alternatives 1 and 2: there should be clear usecases, and clear stakeholders who will pursue that work.

The usecase of the main proposed solution should be clear: to make POSIX subsys to actually deliver (i.e. be able to build existing POSIX apps against Zephyr), with realistic effort.

And note that selecting the main solution doesn't precluding working on Alternatives 1/2 at later time. Again, the main topic of the proposal is to get really-working POSIX subsys "soon". Once that is achieved, we can "optimize" it for usecases where less than the full newlib is needed (well, they can optimize, the stakeholders who need that).

[stephanosio]

NOTE: Since we are starting to add the support for more toolchains (e.g. Arm Compiler aka. armclang, ARC MWDT) with their own C standard libraries, we will need consider this not just in the context of newlib but for all supported third-party C standard libraries.