September-2019 version: focus on STM's STMcubeIDE toolchain (newlib 3.0.0 on ST32F4xx).
See 2017 version, with focus on NXP MCUxpresso toolchain with newlib 2.5.0 on Kinetis K64F (Cortex-M4)

Background

For reliability, smaller embedded systems typically don't use dynamic memory, thus avoiding associated problems of leakage, fragmentation, and resulting out-of-memory crashes. In Nadler & Associates smaller embedded projects (and avionics products), we've historically used no runtime dynamic memory.

Some tool chains (and especially using C++, the topic of a separate article) require dynamic memory internally, even if your application uses no free storage calls. The newlib C-runtime library (used in many embedded toolchains) internally uses it's own malloc-family routines. newlib maintains some internal buffers and requires some support for thread-safety.

This article explains how to use newlib safely in a FreeRTOS project with GNU toolchain.
Warning: This article discusses newlib version 3.0.0 - later versions may be different...

newlib requirements

GNU ARM Embedded Toolchain distributions include a non-polluting reduced-size runtime library called newlib (or newlib-nano for the smallest variant). Unfortunately, newlib internally uses free storage (malloc/free) in startling places within the C runtime library. Thus, newlib free storage routines get dragged in and used unexpectedly.

The most common functions that bite unsuspecting embedded developers are sprintf using %f and strtok. Here's a list of newlib functions with reentrancy support (and hence using malloc).

newlib requires platform-specific support to:

  • provide thread-safety,
  • switch thread contexts, and
  • obtain memory to be doled out and managed by malloc/free/etc.

Here's the detailed list of support functions required by newlib.
If this support is properly implemented, newlib works well in a threaded environment like FreeRTOS. CubeMX does not properly set up these support functions for a FreeRTOS project.
See: Bug Report to STM regarding CubeMX FreeRTOS projects and the STM bug discussion below.

Example newlib use of malloc/malloc_r:
Placing a breakpoint within malloc_r (innermost malloc routine) shows the following:

  • newlib 3.0.0 does not use malloc before calling main() (unlike earlier NXP/newlib 2.5 startup).
    Verified for all four newlib variants.
  • With simple decimal output, sprintf does not call malloc
  • For %f output, sprintf does 4 malloc totalling ~200 bytes (only the first time it is called per task/thread).
    %f also requires proper linker arguments for float support.
  • printf or similar function expected to do IO (as opposed to string operation) allocates an IO control structure of 428 bytes.

Tip: How can I see who is calling malloc ?
The basic steps are:

  • create a wrapper function malloc_r,
  • add the wrapper options to the linker, and
  • place a breakpoint in malloc_r.

A complete example with instructions is included in the code below.

newlib under the hood
newlib maintains information it needs to support a single context (thread/task/ISR) in a reentrancy structure. This includes things like a thread-specific errno, thread-specific pointers to allocated buffers, etc. The active reentrancy structure is pointed at by global pointer _impure_ptr, which initially points to a statically allocated structure. If you use no threading or tasks, and you don't use malloc/free/etc or any of the reentrancy-dependent functions in multiple execution contexts, nothing more is required to use newlib in your application (newlib will maintain its required info in the single static structure). If your application or any library you use requires malloc/free/etc or any of the reentrancy-dependent functions in multiple contexts, newlib requires:

  • concurrency protection for malloc/free/etc. The free storage pool will be corrupted if multiple threads call into these functions concurrently! To prevent reentrant execution of malloc/free/etc routines newlib requires hook procedures __malloc_lock/unlock If an RTOS-based application does not replace newlib's complete internal malloc family, _malloc_lock/unlock must be provided for thread safety.
  • multiple reentrancy structures (one per context), and a mechanism to create, initialize, and cleanup these structures, plus switching _impure_ptr to point at the correct reentrancy structure each time the context changes.

FreeRTOS support for newlib
FreeRTOS provides support for newlib's context management. In FreeRTOSconfig.h, add:
#define configUSE_NEWLIB_REENTRANT 1 // Required for thread-safety of newlib sprintf, strtok, etc...
With this option FreeRTOS does the following (in task.c):

  • For each task, allocate and initialize a newlib reentrancy structure in the task control block (TCB).
    This adds 96 bytes overhead per task (plus anything else newlib allocates).
  • Each task switch, set _impure_ptr to point to the newly active task's reentrancy structure.
  • On task destruction, clean up the reentrancy structure (help newlib free any associated memory).

FreeRTOS memory management
FreeRTOS internally uses its own memory management scheme (implemented by your choice of heapxx.c routines). Some FreeRTOS applications only use FreeRTOS-provided memory management. Unfortunately many libraries use the standard "C" routines internally, including newlib itself and many others (for example STM's USB stack LL and HAL routines). For these cases I've implemented the FreeRTOS memory management API on top of the "C" standard (using newlib) in the module heap_useNewlib.c below.

Bugs in projects generated by STM's CubeMX
There are a number of problems in projects generated with STM's CubeMX tool, which lead to memory corruption and other unfortunate happenings:

  1. STM does not provide __malloc_lock/unlock. Hence memory management is not thread-safe.
    In a stunning bit of coding malpractice, STM's USB stack calls malloc from within an ISR (see for example function USBD_CDC_Init). Normally thread-safety is provided by suspending task-switching during memory management. Because of this incompetent use of malloc inside an ISR, interrupts need to be suspended during memory management. Unfortunately this can greatly increase interrupt latency.
    ToDo: Verify priority set by taskENTER_CRITICAL_FROM_ISR actually blocks USB interrupt.
  2. newlib's free storage requires an external implementation of sbrk to provide the heap storage then doled out by the malloc-family (see newlib sbrk requirements).
    The sbrk implementation provided by STM cannot work with FreeRTOS.
    Any malloc request after the scheduler starts will fail.
  3. STM did not enable (in FreeRTOSconfig.h):
    #define configUSE_NEWLIB_REENTRANT 1 // Required for thread-safety of newlib sprintf, strtok, etc...

Using newlib safely with FreeRTOS - Possible Approaches

If you are certain your application does not use any newlib functions or any library that internally use the malloc-family and/or depend on thread-specific reentrant context, you could do nothing. But, are you really sure??? What about the libraries (for example STM's USB stack LL and HAL routines)? For some hints, see this list of newlib functions with reentrancy support. The only really safe way to preclude accidental use is to provide HCF stubs for every one of these functions. Alternatively you can provide a linker -wrap command for each forbidden function but don't implement the wrappers, so link fails if forbidden routines are ever referenced.
You really need to be sure!

To avoid using newlib's printf and dragging in newlib reentrancy components, there are many cut-down printf implementations available (that do not use malloc), for example:

You still need to be sure you don't accidentally use any newlib facilities requiring reentrancy support and/or malloc-family.

Another option is wrap newlib’s malloc-family to use FreeRTOS free storage (ie heap_4.c), and specify newlib support for FreeRTOS. Tell the linker to wrap all newlib's malloc-family functions (using -Xlinker --wrap=malloc etc.), and provide a wrapper function that calls the FreeRTOS functions. I tried that, but newlib's printf family uses realloc, which is not supported in FreeRTOS heap implementations.

In the end (thanks to Richard Damon for encouraging this approach), I implemented the FreeRTOS memory API on top of newlib's malloc family, and provided all the hooks newlib's malloc family requires.

Using newlib safely with FreeRTOS - Recommended Solution Details

If your application needs a complete malloc family implementation, or you are using any newlib functions that require malloc (for example printf family or strtok), do the following (I've provided an implementation below):

  • Implement the hooks required by newlib (sbrk, __malloc_lock/unlock).
    Make sure your linker file matches the sbrk implementation!
  • Provide a heap implementation that implements the FreeRTOS memory API using the malloc family of newlib.

To use the implementation I've provided in your project:

  • Exclude from all builds any current FreeRTOS heap implementation, typically something like:
    Middlewares\Third_Party\FreeRTOS\Source\portable\MemMang\heap_4.c
  • Add the module heap_useNewlib.c I've provided.
  • If your application needs a complete snprintf implementation, strtok, or other newlib functions requiring reentrancy support, or you're not really sure...
    Configure FreeRTOS for newlib support. In FreeRTOSconfig.h, add the line:
    #define configUSE_NEWLIB_REENTRANT 1


Download Code

Summary

Unfortunately some vendors distribute toolchains with incorrect examples of FreeRTOS/newlib.
However, with a bit of care newlib works well and safely with FreeRTOS.
Enjoy!
Best Regards, Dave

PS: newlib provides facilities for wrapping stdio functions, not covered in this article. You will want to use these, for example, if your application uses posix IO functions to read and write to a USB stick using a local FAT implementation.

PS: Hope Richard Barry and the FreeRTOS team will add heap_useNewlib.c to FreeRTOS ;-)

Additional References

http://www.billgatliff.com/newlib.html
http://wiki.osdev.org/Porting_Newlib
http://www.embecosm.com/appnotes/ean9/ean9-howto-newlib-1.0.html