Skip to main content
Arduino

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

Why is it considered bad practice to use the 'new' keyword in Arduino?

I previously asked this question:

Is it required to delete variables before going to sleep?

On that question, @Delta_G posted this comment:

... Really on a microcontroller I would create the object in a smaller scope and try to do everything in my power to avoid having to use new or any other form of dynamic allocation. .... etc.

That comment got three likes and when I google about dynamic allocation using Arduino, everyone also tries to stay away from that. In summary from all the research I did, my conclusion is now Do not allocate memory unless you really really have to.

I am using the Visual Studio IDE to create my C++ libraries that I intend to use with Arduino. On the Arduino IDE I just reference those libraries and the code compiles great. Visual Studio is very powerful and it enables me to create really nice code, because I can test it on my computer before running it on Arduino. For example, I created this library:

// MyQueue.h
typedef struct QueueItem
{
 void* item;
 QueueItem* next;
 QueueItem()
 {
 item = nullptr;
 next = nullptr;
 }
} QueueItem;
class Queue
{
public:
 unsigned char count; /* Number of items on queue */
 QueueItem* first; /* Points to first item on the queue */
 Queue() /* Constructor */
 {
 count = 0;
 first = nullptr;
 }
 void enqueue(void* item) /* Enqueue an object into the queue */
 {
 count++;
 if (first == nullptr)
 {
 first = new QueueItem();
 first->item = item;
 // Log message because we are using the "new" keword. We need to make sure we dispose QueueItem later
 #ifdef windows
 std::cout << "Creating " << first << endl;
 #endif // windows
 }
 else {
 // Find last item
 QueueItem* current = first;
 while (current->next != NULL)
 {
 current = current->next;
 }
 QueueItem* newItem = new QueueItem();
 newItem->item = item;
 // Log message because we are using the "new" keyword. We need to make sure we dispose QueueItem later
 #ifdef windows
 std::cout << "Creating " << newItem << endl;
 #endif // windows
 current->next = newItem;
 }
 }
 void* dequeue()
 {
 if (count == 0)
 return nullptr;
 QueueItem* newFirst = first->next;
 void* pointerToItem = first->item;
 // Log message we are deleting an object because we created it with the 'new' keyword
 #ifdef windows
 std::cout << "Deleting " << first << endl;
 #endif // windows
 delete first;
 first = newFirst;
 count--;
 return pointerToItem;
 }
 void clear() /* Empty queue */
 {
 while (count > 0)
 {
 dequeue();
 }
 }
 ~Queue() /* Destructor. Dispose everything */
 {
 clear();
 }
};

Now on my Arduino sketch, I can have the following code if I reference that header file.

typedef struct Foo
{
 int id;
} Foo;
void someMethod()
{
 Queue q;
 // Create items
 Foo a;
 a.id = 1;
 Foo b;
 b.id = 2;
 // Enqueue a,b and c
 q.enqueue(&a);
 q.enqueue(&b);
 // Deque
 Foo * pointerTo_a = (Foo*)q.dequeue();
 int x = pointerTo_a->id; // =1
 Foo * pointerTo_b = (Foo*)q.dequeue();
 int y = pointerTo_b->id; // =2
 // Error
 Foo * test = (Foo*)q.dequeue();
 // test == null pointer
}

Most people say do not use void pointers. Why!? Because I am using void pointers I can now use this queue class with whatever object I want!

So I guess my question is: Why is everyone trying to stay away and avoid code like this?

I am using the NRF24L01 radio module to send messages to several Arduinos. It is convenient to have a queue of messages to be sent. I would be able to code the same program without allocating memory and avoiding the new keyword. But that code will look ugly in my opinion.

In this quarantine I decided to learn C++ and that has changed the way I code Arduino. The moment I learned C++ I stopped using the Arduino IDE. I been a backed developer for 12 years, and that is the reason why I learned C++ in a couple of months. Arduino is just a hobby for me. I am still very new to microcontrollers and I will like to understand why people stay away from the full power of C++ when it comes to microcontrollers. I know I have only 2 kilobytes of RAM. I will not be allocating that much memory. I still want to take advantage of the C++ programming language by using the new , delete , poineters and destructors`. I want to keep using Visual Studio to write powerful C++ libraries.

In C++ I write interfaces like this

// Note I use uint32_t instead of 'unsigned long' because an unsigned long is different size on Windows than on Arduino. Also I use an unsigned short instead of an int because an unsigned short is the same size on Windows and Arduino.
class IArduinoMethods
{
public:
 // Unsigned long in Arduino
 virtual void delay(uint32_t delayInMilliseconds) = 0;
 virtual void print(const char* text) = 0;
 virtual uint32_t millis() = 0; // Get elapsed time in milliseconds
};

And I then implement the classes like this. For example, this is the class that I will use when testing my code on a windows computer:

// Class to be run on Windows.
class ArduinoMockWindows : public IArduinoMethods
{
public:
 // Inherited via IArduinoMethods
 virtual void delay(uint32_t delayInMilliseconds) override
 {
 // This code will be different on Arduino, and that is why I need this dependency
 Sleep(delayInMilliseconds); // Windows
 }
 virtual uint32_t millis()
 {
 //clock_begin = std::chrono::steady_clock::now();
 std::chrono::steady_clock::time_point now = std::chrono::steady_clock::now();
 auto duration = now.time_since_epoch();
 // etc..
 return someDuration;
 }
};

Because a Windows computer cannot send NRF24 radio messages, I can implement an interface (dependency) that will write to a file, for example instead of sending a real radio packet just for testing.

The caveat is that my libraries will require these dependencies. For my library to work, I will have to pass it an object of type IArduinoMethods and INrfRadio. If I am running my code on Windows I will pass it a class that will implement those methods that can run on windows. Anyways the point is not to show how C++ works. I am just showing how I use pointers and allocate memory for a lot of things.

Because I allocated memory I was able to test my library on Windows and on Arduino for example. I may also create unit tests. I see so many benefits by allocating memory. If I am organized and remember to free the objects I no longer use, I can gain all this benefits. Why people do not code like this when it comes to Arduino?


Edit 1


Now that I understand how heap fragmentation works, I know I have to be careful when using the new keyword.

I hate when people do what they are told to do without understanding how things work. For example, the answer https://arduino.stackexchange.com/a/77078/51226 from Why is the queue library in this question for starters?. There are going to be times when a ring buffer works better and other times when the new keyword works better. Probably the ring buffer will work best for most cases.

Take the following scenario where you only have 1 KB of memory left.

  1. There is a hierarchy of nodes where a node have a child and a sibling. For example, node A can have child B and sibling C. Then child B can have another child, etc.

(I will be storing this in memory)

  1. I have a queue of work that needs to be done.

(I will have to store this work somewhere)

  1. I will have a queue of events

(I will have to store this somewhere)

If I use what most people say I should do then I will be have to:

  1. Reserve 500 kB to be able to store nodes (I will be limited to n number of nodes)

  2. Reserve 250 kB for the queue of work that needs to be done.

  3. Reserve 250 kB for the queue of events.

This is what most people will do and it will work great with no problems of heap fragmentation.

Now this is what I will do

  1. Ensure that everything that I allocate is of size 12 bytes. A node only has its id (unsigned int), child (pointer), type (unsigned char), etc.. with a total of 12 bytes.

  2. Ensure that all the work that will be enqueued is of size 12 bytes as well.

  3. Ensure that all the events that will be enqueued is of size 12 bytes as well.

Now if I have more work than events, this will work. I just have to program in my code that I never allocate more than 70 items. I will have a global variable that has that count of allocations. My code will be more flexible. I will not have to be stuck with strictly 20 events, 20 work and 30 nodes. If I have fewer nodes then I will be able to have more events. **Anyways my point is that one solution is not better than the other. There are going to be scenarios when one solution is better.

In conclusion, just understand how heap fragmentation works and you will gain a lot of power by using the new keyword. Do not be a sheep and do what people tell you to do without understanding how things work.**.


Edit 2


Thanks to @EdgarBonet, I ended up storing Nodes on the stack. Here is why:

I have a hierarchy of nodes that can be represented as:

typedef struct Node
{
 unsigned short id;
 Node * sibling;
 Node * child;
} Node;

As you can see every node is only 6 bytes. That is another reason why I did not care to much about allocating Nodes at the beginning. If I allocate this node on the heap I will be losing 2 more bytes (33%) for every allocation because on every allocation the size of the node has to be stored. As a result I created these two methods and a buffer:

// For this to work a node can never have an id of 0 !!!
Node nodeBuffer[50]; /* Buffer to store nodes on stack */
Node* allocateNode(Node nodeToAllocate) /* Method to store a node */
{
 // Find first available spot where a node can be saved
 for (char i = 0; i < 50; i++)
 {
 if (nodeBuffer[i].id == 0)
 {
 nodeBuffer[i] = nodeToAllocate;
 return & nodeBuffer[i];
 }
 }
 return nullptr;
}
void freeNode(Node* nodeToFree) /* Method to delete a node */
{
 nodeToFree->id = 0; // If the id of a node is 0 this is my convention of knowing it is deleted.
}

And on my code I used to have things like:

Node * a = new Node();
a->id = 234423;
// ....
// .. etc
// ..
delete a;

Now I just have to replace that code with:

Node * a = allocateNode({});
a->id = 234423;
// ....
// .. etc
// ..
freeNode(a);

And my code works exactly the same without having to use the new keyword. I thought it was going to be complicated to refactor the code and create a buffer.

I made this modification because I wanted to be able to store more nodes on my code. By loosing that 33% I was not going to be able to create that many. If I only allocate objects of the same size and I do not allocate that many it is perfectly fine to use the new keyword. > Also in the case of the queue I will allocate and delete objects very fast. Because the objects will not persist on memory for too long, and the chances of having heap fragmentation are very low.

Answer*

Draft saved
Draft discarded
Cancel
6
  • 3
    A fixed-size FIFO queue is typically implemented as a ring buffer. Commented Jul 24, 2020 at 10:24
  • @EdgarBonet Yes it is. The OP should be able to find plenty of existing code for FIFOs online though, so I guess I'm more interested in helping them getting the concept than exactly how they do the implementation. Commented Jul 24, 2020 at 11:54
  • The FIFO fix sized as ring buffer is a great sollution. There are going to be times when the ring buffer is better (probably most of the time) and there are others when it is not. Take a look at the edit of the question. If you keep track of allocations and you never exceed the count of allocations all of the same size you should have no problem allocating a new queue item. Commented Jul 24, 2020 at 15:14
  • Exactly graham your answer is probably the best solution for most of the cases and also the safest. But for cases when I am building a hierarchy of nodes it will be harder to create with a buffer. If I am already allocating Nodes I guess I can keep allocating them using the queue. My point is that if you know how things work it is ok to not go with the convention. Also the probability of an alloc not working is very small since I am not using interupts or running my code in different places. For the rest of my projects I will probably use your solution. Commented Jul 24, 2020 at 17:55
  • @TonoNam You are assuming there that your allocation granularity goes down to at least 4 bytes. If the granularity is 8 bytes at best, then each 12-byte allocation will actually occupy 16 bytes of RAM. So your heap-based code would only be able to store 3/4 of what could be stored with a statically-allocated queue. Plus whatever overheads are involved in running the heap, which can be significant when you're allocating small amounts of data (and 12 bytes is usually "small"). You can't just divide RAM length by structure length and assume that's how many you get. Commented Jul 24, 2020 at 21:19

lang-cpp

AltStyle によって変換されたページ (->オリジナル) /