Serial printing from flash memory (F() macro, PROGMEM, sprintf_P, SPTR)

Question

I have been testing various techniques on an Arduino Uno for serial printing from flash memory instead of RAM. At the same time, I have been using the freeMemory() function from Adafruit.

I created a program that prints the word "test", with various methods:

Simple print:

Serial.print("test");
Serial.print(freeMemory());

This call reports free RAM = 2267

Using the F() Macro:

Serial.print(F("test"));
Serial.print(freeMemory());

This call reports free RAM = 2299

Using sprintf_P()

char buf[100];
sprintf_P(buf,PSTR("test"));
Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

Using strcpy_P()

 char buf[100];
 strcpy_P(buf, (PGM_P)F("test"));
 Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

Using strcpy_P and const char[] PROGMEM

char buf[100];
const char buff[] PROGMEM = "test";
strcpy_P(buf, buff);
Serial.print(buf);
Serial.print(freeMemory());

This call reports free RAM = 2267

The basic point of this question is to gather all the possible flash memory printing techniques. Also it would be really useful if you could explain in detail what exactly each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM does.

IMPORTANT EDIT, 1:

Depending on where and how I use the call Serial.print(freeMemory()); the results change. For example, at the last example (PROGMEM, strcpy_P) when I included another Serial.print(freeMemory()); before the call of the printing functions, then both freeMemory() calls report 2299!

Another interesting problem is that the freeMemory() reports 2299 as freeRam, while the SRAM of my Arduino Uno is only 2 kbytes.

EDIT 2:

Following the suggestion in comments, I changed "test" to "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttest". Now all different methods report a free ram of 2299, including the simple Serial.print().

freeMemory():

#ifdef __arm__
// should use uinstd.h to define sbrk but Due causes a conflict
extern "C" char* sbrk(int incr);
#else // __ARM__
extern char *__brkval;
#endif // __arm__
int freeMemory() {
 char top;
#ifdef __arm__
 return &top - reinterpret_cast<char*>(sbrk(0));
#elif defined(CORE_TEENSY) || (ARDUINO > 103 && ARDUINO != 151)
 return &top - __brkval;
#else // __arm__
 return __brkval ? &top - __brkval : &top - __malloc_heap_start;
#endif // __arm__
}

• Michel Keijzers: Nice report, but I miss a question; I guess you don't use a Uno as that model has 2 KB only.
• Majenko: All those routines use 100 bytes on the stack and nothing else, except maybe transient space within the functions you call which is also on the stack. None of that is relevant to storing data in and reading data from flash. All the things you ask about are macros. You can find what they expand to in the source code of either the core or the compiler.
• Edgar Bonet: I suggest you try with a message much longer than "test". There are many details in the compiler optimization process that can make small variations in the memory consumption. Those can make your measurements hard to interpret with such a small string.
• NickG: Will try immediately, and report back with the results. @MichelKeijzers Strangely enough, I am using an UNO. Any ideas?
• NickG: @EdgarBonet your suggestion really seems to have an impact. I changed "test" to "testtesttesttesttesttesttesttesttesttesttesttesttesttesttesttest". However the results seem to be even weirder, as all different methods (including the simple Serial.print) report a free RAM of 2299. Let me also point out that this is the freeRam I get when I only call the freeMemory() in my code.

Answer 1

First, the freeMemory() function that you used from Adafruit was originated from a GitHub repository.

With the code related to the __arm__ implementation aside, for the code related to the AVR, it is incomplete for handling the corner case when the program does not use the malloc() function. If your program never uses malloc(), the formula used in freeMemory() will produce the wrong result. A simple corrected version for AVR that I used is here:

extern unsigned int __heap_start;
extern char *__brkval;
int freeMemory() {
 char top_of_stack;
 if (__brkval == 0) {
 Serial.println(((int)&top_of_stack - (int)&__heap_start));
 } else {
 Serial.println((int)&top_of_stack - (int)__brkval);
 }
}

explain in detail what exactly does each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM.

PSTR("string literal")

PSTR came into Arduino from part of avr-libs (a C library) defined in avr/pgmspace.h. It is actually a macro, not a function as many think it was, and is defined as:

#define PSTR(s) ((const PROGMEM char *)(s))

It looks a little bit intimidating, but it is actually quite simple. It not only tells the avr-gcc that the string literal s should be kept in program memory (i.e., flash memory in the case of Arduino), but it allows the program to convert it into a const char *s PROGMEM pointer, so that it can be pass around into some PROGMEM-aware functions as a parameter.

// This is a PROGMEM-aware function in C
printMsg_P(const char* str) {
 char buf[strlen_P(str)+1];
 strcpy_P(buf, str);
 puts(buf);
}
// This handles the normal string array function
printMsg(const char* str) {
 puts(str);
}

F("string literal")

F() is also a macro, but it is not part of the avr/progmem.h. It is actually part of the String class defined in WString.h together with a definition of a class called __FlashStringHelper:

class __FlashStringHelper;
#define F(string_literal) (reinterpret_cast<const __FlashStringHelper *>(PSTR(string_literal)))

To understand the F() macro and the difference between F() and PSTR() take a little bit explanation.

As you can see from the definition of F(), it casts a string literal into a const char* variable using PSTR(), and then recast it into a class __FlashStrngHelper. If you further look at the __FlashStringHelper, it is actually an empty class without any construct or method. It is literally empty, so why is this useful? And why can't it just pass the string literal around by just using PSTR()?

This has to do with C++'s function overload. Function overload is a feature of C++ where multiple methods could have the same function name, but each with different parameters. Within all the Serial.print() methods, there is a method Serial.print(const char* str) that will take in a string literal and print it to serial monitor, but it is not PROGMEM-aware. If you pass in a PROGMEM string literal Serial.print(PSTR("string literal"));, the same overloaded method we just mentioned will accept what you pass in because it meet the type checking (remember PSTR is actually const char*), but it will not print out the correct string for PSTR("string literal"), because it simply expecting a normal string.

In C, as we see before, we solve the problem by using two functions with different names for handling normal string and PROGMEM string, but Serial is a C++ class. It needs another function overloaded method to be PROGMEM-aware, and PSTR() is clearly not the solution as we just mentioned.

WSting.h and Print.cpp create an overload method that is PROGMEM-aware by accepting a class as parameter, so that you can pass a F() wrapped string literal which has a data type of class __FlashStringHelper into the Serial.print() and get the print out correctly from program memory. See the source code yourself.

Serial.print(const __FlashStringHelper *ifsh)

So in summary, the F() and PSTR() are alike. Both are telling the compiler to keep the string literal in the program memory, and allow the string literal to be pass around into some function as parameter. But it was designed for different purpose from different programming paradigm (C versus C++). F() is Serial.print()-friendly, while PSTR() is not. If you are getting PSTR() in a function and yet you want to send it to Serial.print(), you will have to explicitly cast it into __FlashStringHelper like this:

Serial.print((__FlashStringHelper*)PSTR("test"));

PGM_P

PGM_P is a macro defined as:

#define PGM_P const char *

Whether you want to use PGM_P or const char * is a personal preference. For example, this is copy from PROGMEM (Arduino reference):

const char *const string_table[] PROGMEM = {string_0, string_1, string_2, string_3, string_4, string_5};

can be written as:

PGM_P const string_table[] PROGMEM = {string_0, string_1, string_2, string_3, string_4, string_5};

This is a lengthy explanation, but I hope this will help with better understanding the subject and therefore be able to use the PROGMEM and all the associated macros effectively.

• Edgar Bonet: Re "PSTR [...] is defined as [...]": This definition is introduced by the comment "The #define below is just a dummy that serves documentation purposes only." The actual definition is somewhat more convoluted.

Answer 2

explain in detail what exactly does each of the keywords F(), (PGM_P)F, PSTR, const PROGMEM.

• F() wraps the string literal in PSTR(...) and then casts it to __FlashStringHelper. This forces it to remain in flash and also gives it a type that the Arduino core can identify as a string in flash for overloading.
• PGM_P is a "Pointer to ProGraM mememory". This is the type that all the _P variants of functions require. It's basically const char * provided by avr/pgmspace.h
• const PROGMEM is actually two things. const means you're not changing the value, and PROGMEM is another name for __ATTR_PROGMEM__ which itself is an alias for __attribute__((__progmem__)) which is a flag for the compiler to tell it to keep the variable in flash and not copy it to RAM.

So F() and PGM_P are used when you are passing a string literal to a function that expects one of those formats. PROGMEM is used when you're setting up a constant "variable" (oxymoron there, I know...) that points to data in flash.

• NickG: Could you please explain: sprintf_P(buf,PSTR("test")); ? Is this wrong? Should it be like this: sprintf_P(buf,PGM_P("test")); ? Also what about this command: strcpy_P(buf, (PGM_P)F("test")); which combines both PGM_P and F()?
• Majenko: (PGM_P)F("test") is what I usually use.
• hcheung: (PGM_P)F("test") is equivalent to PSTR("test"). What happen is that F("test") cast the string literal into a class __FlashStringHelper, and then recast it to PGM_P which basically is a macro as const char*. This is the same as PSTR("test") that converts a string literal s[] to `const char* s variable.