Conditional assignment of array

Question

I have some really long global variable arrays filled with data.

There is a single function that processes the data. The function works on only one array everytime. This arrays value changes every iteration and gets the value from the global variable arrays.

Since i declare lots of global variable arrays, in order to pass them to the array_to_be_processed_by_the function, the memory is filled up really quickly.

What is the best way to conditionally give values to the array?

For example, there could be a counter in the program, and if the counter is a particular value, the array would be initialized with certain data. Of course, i would also have to call the function, and i would have to keep track of the counter and increase it and zero it at the beginning of the loop, to start from the beginning.

In this approach, only one variable would always be initialized.

PSEUDOCODE:
int looper = 0;
void loop()
{
 switch(looper)
 {
 case 0:
 my_array[] = {DATA HERE};
 exec_func(myarray);
 looper++;
 continue;
 case 1:
 ......
 case 'last_case':
 ...
 looper = 0;
 break;
 }
}

However, i am not sure if this approach is the correct one. Especially for a microcontroller.

One obvious issue, is that the array values are not necessarily the same size. Therefore, the base array should be destroyed at each case? If this is good approach, should new/delete be used on an arduino?

What is the best approach to conditionally initialize an array, so that i don't get out of memory

• chrisl: Are your base arrays constant? If yes, you could put them in flash/program memory instead of RAM. As I've understood you would copy the data into an extra arasy anyway to work on the data there
• 6v6gt: In the example pseudocode, my_array[] defined in case 1 would be automatically destroyed at the end of case 1. Incidentally, if you define variables within a case, the case block should be enclosed in brackets { } or you get compiler warnings.
• hcheung: In order to answer your question, we need to know how you define the exec_func(myarray)? What is the data type of my_array[]? Does the function take in a my_array[] or does it take in a pointer to the array? Whether the my_array[] only be used within the case 0 or does it uses somewhere else? Meanwhile I'd suggest you read Learn C++. BTW, the new/delete is used for dynamic allocated memory using command like malloc(), it does not applicable to static array creation.
• the busybee: Would you mind to edit your question and provide a minimal, reproducible, and complete example, please? It should show your use case, even if it is not optimized as you like.
• user1584421: @chrisl Yes they are constant. I am learning about PROGMEM right now! Thank you for the tip!

Answer 1

Let's consider this:

case 0:
 my_array[] = {DATA HERE};
 exec_func(myarray);

This is not quite valid C++, but let's pretend you get the syntax right to make it work. The problem is: doing this will not help you save memory. Where do you think the compiler is going to store {DATA HERE}? In memory, right. It will be stored as an anonymous array. Then the initialization of my_array will make a second copy of the data in memory, this time in a named array.

You can avoid this second copy by naming your arrays of constants, and passing the correct one to your function:

// At global scope:
const int my_array_0[] = {...};
const int my_array_1[] = {...};
// Within the switch/case:
case 0:
 exec_func(my_array_0);

You can even avoid the switch/case altogether by using an array of pointers, but that is irrelevant to your current problem.

Your idea of initializing only the array you actually need can work if you manage to do it procedurally, i.e. implementing some recipe with instructions, rather than copying a set of constant:

case 0: {
 int my_array[array_size];
 for (int i = 0; i < array_size; i++) {
 my_array[i] = some_expression_to_compute_this_array_item;
 }
 exec_func(myarray);
}

This is not always feasible though.

If you really have to store the constants in the program, and you are using an AVR-based Arduino (like the Uno, Mega, Micro...), then you can save RAM by storing the arrays of constants in flash memory, and copying only the one you need to RAM:

// At global scope:
const int my_array_0[] PROGMEM = {...};
const int array_size_0 = sizeof my_array_0 / sizeof my_array_0[0];
const int my_array_1[] PROGMEM = {...};
const int array_size_1 = sizeof my_array_1 / sizeof my_array_1[0];
// Within the switch/case:
case 0: {
 int my_array[array_size_0]; // array in RAM
 memcpy_P(my_array, my_array_0, sizeof my_array);
 exec_func(my_array);
}
case 1: {
 int my_array[array_size_1]; // array in RAM
 memcpy_P(my_array, my_array_1, sizeof my_array);
 exec_func(my_array);
}

Check the documentations of PROGMEM and memcpy_P() for the details.

If you go this route, you may consider modifying the function exec_func() so that it expects its parameter to be a pointer to flash instead of a pointer to RAM. Then you will completely avoid the copy in RAM.

---

Edit: expanding on the idea of passing a pointer to flash.

The C++ compiler doesn't really know the difference between a pointer to RAM and a pointer to flash. If you want to pass a pointer to flash to a function, you have to write the function in such a way that it expects a pointer to flash. For example, this function prints out the contents of a flash-based array:

// The argument should be a pointer to flash.
void exec_func(const int *data)
{
 for (int i = 0; i < array_size; i++) {
 int number = pgm_read_word(&data[i]);
 Serial.println(number);
 }
}

Note that the array is not accessed directly (evaluating data[i] would give garbage). Instead, the address of the element you want (&data[i], a flash address) is passed to the macro pgm_read_word(), which uses inline assembly to get the relevant data from the flash.

Now you can call this function passing it the address of a PROGMEM array, as in exec_func(my_array_0);.

Just for completeness, I will show you how to use an array of pointers to avoid the switch/case construct:

const int my_array_0[] PROGMEM = {...};
const int my_array_1[] PROGMEM = {...};
...
const int *arrays[] = {my_array_0, my_array_1, ...};
int looper = 0;
void loop()
{
 exec_func(arrays[looper]);
 if (++looper == number_of_arrays) looper = 0;
}

Note that here arrays is a RAM-based array. That's why you can access the elements directly as arrays[looper]. These elements, however, are pointers to flash-based arrays. If you have so many arrays that arrays gets too big, you might consider putting it also in flash.

• user1584421: Thank you very much for this detailed answer! The problem is: doing this will not help you save memory. Where do you think the compiler is going to store {DATA HERE}? Well, my logic behind this thought, was that I have seen much larger codebases in arduino. I thought that the CODE part was different than the variable assignment. I am reading about PROGMEM right now. One more question. If i pass a pointer to flash location, then i don't need the my_array[] at all? I just give a pointer to the my_array_0[] (for example)?
• chrisl: @user1584421 That depends on if you want to change the data. exec_func() can be written to read the data from flash, but you cannot change it there. If you need to do transformations to the raw array data, then you still need to copy it to RAM like Edgar did in his last code snippet.
• user1584421: @chrisl No they arrays would be constants. So i don't even need my_array[]. Just pointers for the const arrays in Flash memory. Thanks for the tip!
• user1584421: LOL! You just edited it in the same time i managed to get it to work! I will give it a read and mark it as accepted it later! Thank you so much for your answer!
• user1584421: I tried the pointer to array to flash before but it didnt work. Of course, maybe i did something wrong. However, i managed to get it to work with the code snippet you provided just before the edit. With the extra work of having to deduce the number of elements in the array in the flash. int ArrayElements = sizeof(my_array_0) / sizeof(int); And then i would create the empty array, just like your example, passing the arrayelements as its size. Yes, maybe its not as fast and memory efficient, but its three lines of code. And only one array in memory. I can get away with it.