RTOS issues with newlib (and newlib nano)
GNU ARM Embedded Toolchain distributions include a non-polluting reduced-size runtime library called newlib (or newlib-nano for the smallest variant). Unfortunately, newlib has some challenges for embedded development:
(1) newlib uses free storage (malloc/free) in startling places within the C runtime library. Thus, newlib (v2.5.0) free storage routines get dragged in and used unexpectedly as follows:
- Even before main() is called, some allocation is required by runtime:
sbrk (2552+3592) malloc(2528+2528)...
Ooops: Not MISRA-compliant, if you believe in that stuff.
- printf family uses _malloc_r(), and %f format code (16
bytes for trapped first call) uses malloc(). Initial printf
(using float %f) allocates:
- 428 bytes using the global context _reent impure_data. The global context is used before task switching sets a task-specific context, and also for certain newlib buffers used globally (TBD: why? how is this safe?).
- (1542-428)=1114 bytes using the thread-specific _reent context.
- so a minimum of 1542 bytes, plus at least 1114 for every additional thread using printf family.
- strtok (commonly used in embedded applications as is snprintf).
- Probably additional functions (TBD: exhaustive check; I only checked the functions I needed).
(2) newlib's free storage relies on external implementation of sbrk to provide storage doled out by the malloc-family (see newlib sbrk requirements). sbrk in turn requires the linker control file to provide the underlying memory boundaries. Freescale did this incorrectly in at least some examples; consequently free storage calls return wild pointers. Because of (1) and (2), a 1-line Freescale application that printf’s a floating point number corrupts memory. This can be seen with a new project or modifying one of Freescale's LED-blinky samples (no RTOS). See: BUG: malloc overruns heap, returns invalid pointer, and corrupts memory (This was KDS; not retested with MCUXpresso).
(3) newlib’s malloc-family implementation provides hooks for external locking procedures (__malloc_lock/unlock) to prevent reentrant execution of malloc routines. If an RTOS-based application does not replace newlib's internal malloc family, _malloc_lock/unlock must be provided for thread safety. Freescale/NXP FreeRTOS examples neither replace the malloc-family nor provide __malloc_lock/unlock. Hence they are not thread-safe, and can corrupt memory because of improper malloc support explained above. Guess how I found out...
(4) To support thread-safe operations, newlib provides for thread-specific storage of newlib-internal buffers (for example the storage used by printf family above, strtok, errno, etc.). In FreeRTOS setting configUSE_NEWLIB_REENTRANT to 1 in FreeRTOSconfig.h supports this. When set, FreeRTOS allocates a newlib reentrancy storage area in each task control block, and sets newlib’s _impure_ptr to this thread-specific context each task switch. See FreeRTOS newlib support feature discussion. configUSE_NEWLIB_REENTRANT is not set in any of Freescale/NXP FreeRTOS examples. Hence their examples are not thread-safe if you use any newlib functions relying on thread-specific reentrancy storage in more than one task...
Using newlib safely with FreeRTOS - Possible Approaches
If you are certain your application does not use any newlib functions that internally use the malloc-family and/or depend on thread-specific reentrant context, you could do nothing. But, are you really sure??? The only way to safe preclude accidental use is to search newlib sources for all such forbidden functions, and provide HCF stubs for each (or just provide a linker -wrap command for each forbidden functions but don't implement the wrappers, so link fails if forbidden routines are ever referenced). Anyway, you really need to be sure.
To avoid using printf and dragging in some of the newlib components, there are a number of cut-down printf implementations available (that do not use malloc), for example:
- printf-stdarg.c distributed in the FreeRTOS Lab TCPIP example; this one only uses stack storage but does not implement floating point.
- https://github.com/ksstech/support_common .
Another option is wrap newlib’s malloc-family to use FreeRTOS free storage (ie heap_4.c), and specify newlib support for FreeRTOS. Tell the linker to wrap all newlib's malloc-family functions (using -Xlinker --wrap=malloc etc.), and provide a wrapper function that calls the FreeRTOS functions. I tried that, but newlib's printf family uses realloc, which is not supported in FreeRTOS heap implementations.
In the end (thanks to Richard Damon for encouraging this approach), I implemented the FreeRTOS memory API on top of newlib's malloc family, and provided all the hooks newlib's malloc family requires.
Using newlib safely with FreeRTOS - Recommended Solution Details
If your application needs a complete
malloc family implementation,
or you are using any newlib functions that require malloc
printf family or
do the following
(I've provided an implementation below):
- Implement the hooks required by newlib (
Make sure your linker file matches the
- Provide a heap implementation that implements the FreeRTOS memory API using the malloc family of newlib.
If your application needs a complete
or other newlib functions requiring reentrancy support:
- Configure FreeRTOS for newlib support. In FreeRTOSconfig.h, add the line:
#define configUSE_NEWLIB_REENTRANT 1
newlib has a larger faster version, and a slower but more compact nano version.
To use newlib nano, add the linker command-line option
Unfortunately some vendors distribute toolchains with incorrect examples of FreeRTOS/newlib.
However, with a bit of care newlib works well and safely with FreeRTOS.
Best Regards, Dave
PS: newlib provides facilities for wrapping stdio functions, not covered in this article. You will want to use these, for example, if your application uses posix IO functions to read and write to a USB stick using a local FAT implementation.
PS: Hope Richard Barry and the FreeRTOS team will consider adding this to FreeRTOS ;-)