-
Notifications
You must be signed in to change notification settings - Fork 7.7k
-
Board
ESP32 Dev Module
Device Description
No other board. Just ESP32 Dev Module,
Hardware Configuration
None
Version
latest stable Release (if not listed below)
IDE Name
Arduino IDE 2.3.5
Operating System
Windows 11
Flash frequency
80 Mhz
PSRAM enabled
no
Upload speed
115200
Description
I was trying to upgrade an existing ESP32 Program but even with all program options disabled the program size was 90%.
I decided to write a minimal program and report the program size for different ESP32 Core, from 2.0.13 to 3.2.0
Here are the results
Arduino IDE 2.3.5
ESP32 Dev Module
nullprog.ino as shown in next section
ESP32 Core 2.0.13
Sketch uses 257465 bytes (19%) of program storage space.
Global variables use 21432 bytes (6%) of dynamic memory
ESP32 Core 2.0.17
Sketch uses 258113 bytes (19%) of program storage space.
Global variables use 21432 bytes (6%) of dynamic memory
ESP32 Core 3.0.0
Sketch uses 889873 bytes (67%) of program storage space.
Global variables use 40724 bytes (12%) of dynamic memory,
ESP32 Core 3.1.3
Sketch uses 903108 bytes (68%) of program storage space.
Global variables use 44724 bytes (13%) of dynamic memory
ESP32 Core 3.2.0
Sketch uses 905882 bytes (69%) of program storage space.
Global variables use 46304 bytes (14%) of dynamic memory
My Question then is
Why the huge jump in program size from ESP32 Core 2.0.13 to Core 3.2.0
19% Program space to 69% program space,
I can no longer program the esp32 boards with the application (Weather station) because the program size is now (using Core 3.2.0) in excess of 100%.
Is there something I am missing here or perhaps not understanding the huge jump in program size from 2.x to 3.x of the ESP32 Core versions?
Any suggestions most welcome.
regards
Robert
Sketch
//nullprog.ino #include <Arduino.h> #include <WiFiServer.h> #include <WiFiClient.h> #include <ArduinoJson.h> #include "SPIFFS.h" #include <WiFi.h> #include <Math.h> #include <Wire.h> void setup() {} void loop() {}
Debug Message
None, comparing output program size for different Core versions
Other Steps to Reproduce
No, just run the nullprog.ino
test results speak for themselves
I have checked existing issues, online documentation and the Troubleshooting Guide
- I confirm I have checked existing issues, online documentation and Troubleshooting guide.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
I dug deep into this and here is my conclusion:
- Size increase comes from the WiFi library after the network layer refactoring in v3
- It is present on empty sketch, because we always instantiate the
WiFiobject - This was true even before, but now we include proper destructors for the AP and STA interfaces
- The destructors call
WiFi.mode()internally, which in turn links to all low level WiFi and Netif init/deinit methods - Commenting out the calls to
WiFi.mode()in the destructors, brings back the flash size to 300+KB
Now with that said:
- There is no real scenario, where you will include the WiFi library, but never use it. If you will use only ETH/PPP, then include
Network.hinstead. - Any use ...
Replies: 9 comments 4 replies
-
You can resize the partitions by allocating more space for app0 and app1 by reducing SPIFFS. For example app 1408 Spiffs 828:
`# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x160000,
app1, app, ota_1, 0x170000,0x160000,
spiffs, data, spiffs, 0x330000,0xD0000,`
I myself do not really understand why the idf kernel has grown so much in size. The second surprise for you will be the use of RAM. Personally, I will have to switch to esp32 with a larger memory size of 8-16 MB. But the fact that there is little RAM left threatens the use of esp32 at all. Something simple is still possible, but multifunctional devices with the current psram policy have already reached 60 KB and less free for me. Which no longer allows using https.
Beta Was this translation helpful? Give feedback.
All reactions
-
Thank you for your comment. I appreciate the response and your thoughts.
Yes, I know about partitions and heap size and all about that and SPIFFS etc. I have been using esp32 for a number of years now (as well as esp8266 before that). The partition scheme really does not help here, as the project also uses OTA updates. As the project is now stable and been in place for some time, partioning was explored and changed over time to where it is today. Been there, done that, took home the medal so to speak.
RAM is not an issue for me. But code space is. The problem is if all the project options are enabled, the resultant code space significantly exceeds 100%. With just 1 option enabled, a web server, program space is 90%.
To be honest, none of it addresses the concern I expressed, let alone explain why the increase.
I also feel that moving to a discussion could be interpreted as a "its not a real convern - live it - all your fault - write better code".
I come back to the fact the the explosion of program space used by core now renders my solutions totally worthless if moving to the latest core version. The project was often referred to as an esp32 killer app. Now its progably gonna be referred to the app that esp32 core killed. Just my thoughts. Not blaming anyone.
Regards
Robert
Beta Was this translation helpful? Give feedback.
All reactions
-
I partly understood that yes, this is our problem and we have to live with it.
I moved to version 3.x.x only because of Matter, I still keep my main version on the distant 1.0.2 just because of memory issues. Now I decided to switch to ESP32 with 8-16 MB of PSRAM, because in addition to the web server, web client, websocket server and client, I need to implement Matter support. I also wrote for the 8266, there was still plenty of space there, and strangely enough, RAM was enough. I haven’t written off the 8266 yet.
Good luck to you!!!
Beta Was this translation helpful? Give feedback.
All reactions
-
I dug deep into this and here is my conclusion:
- Size increase comes from the WiFi library after the network layer refactoring in v3
- It is present on empty sketch, because we always instantiate the
WiFiobject - This was true even before, but now we include proper destructors for the AP and STA interfaces
- The destructors call
WiFi.mode()internally, which in turn links to all low level WiFi and Netif init/deinit methods - Commenting out the calls to
WiFi.mode()in the destructors, brings back the flash size to 300+KB
Now with that said:
- There is no real scenario, where you will include the WiFi library, but never use it. If you will use only ETH/PPP, then include
Network.hinstead. - Any use of the WiFi library will inevitably call
WiFi.mode()which will link to all those things anyway and bring the size back to the same amount.
With all of the above, I conclude that this is not a real world issue. Yes, the WiFi stack has increased in size in ESP-IDF, but we have no control over that.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 2
-
Thank you so much for your work!!! I suspected that the memory problem is in the IDF itself and does not depend on porting for Arduino.
If you have the opportunity, please consider more aggressive use of PSRAM in Arduino’s menuconfig settings. With the current default settings, PSRAM is used by the system in rare cases.
Beta Was this translation helpful? Give feedback.
All reactions
-
me-no-dev
Appreciate the reply.
That being the case, the flow on effect is that the selected partition scheme (ESP32 Dev Module) is now rendered useless unless NOT using Wifi. Because there is no program space left to add anything of substance before the program space exceeds 100%.
I have observed the same issues with other available partiton schemes. The majority of those suffer from the same spave issues.
It is increasingly looking like the only real solution (apart from moving away from ESP32) is to use a custom partition definition.
Because this increase affects all the available partition spaces, perhaps it might be prudent to look at existing partition definitions and modify them to hold a larger size of program space, considering the IDF changes increased program space from 19% (old IDF) to 63% (new IDF)?
My concern (apart from the projects) is that new users moving to the ESP32 could encounter these same issues and be deterred from using ESP32 in their projects because program space can now be easily become a major issue for them (and apart from the frustration that also brings)
Regards
Robert
Beta Was this translation helpful? Give feedback.
All reactions
-
There are plenty of partition options. For example "no-fs" gives you almost 2MB APP partition. If we are to modify current partitions, this will cause other issues, especially with partitions that have file systems. The result will be that FS will no longer mount, it will need to be re-formatted and data will be lost.
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.
All reactions
-
Facing a similar issue. The trouble is, there are a large number of devices that were orignally shipped in one configuration (1.2MB APP, 1.5MB spiffs, with OTA). When the base firmware size went overboard, it makes it harder to push updates. For newer ones, I have moved to bigger flash. Older ones out in the field poses some challenge because of this. Wonder if anyone else is on the same boat.
Beta Was this translation helpful? Give feedback.
All reactions
-
I have the same problem. I had to install a portable version of Arduino with an old core for old devices. In the code, I have to handle the differences and exclude what is not supported. This way, I can write one project and compile it in different versions of Arduino for both old and new devices.
#define oldVersionCode 1 #if oldVersionCode //code for old version #else ///code for new version #endif
Beta Was this translation helpful? Give feedback.
All reactions
-
@brownrb - Is this project open source? I'd like to build it using Arduino Core 3.2.0 and check what can be done to reduce the final binary size. Thanks!
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.
All reactions
-
Beta Was this translation helpful? Give feedback.