-
Couldn't load subscription status.
- Fork 7.7k
-
Board
ESp32S3, IDF 5.5
Device Description
Image
I was hoping to speed up the builds via docker. It worked actually really well to get around pathological platformio redownloading.
10GB, I know it says ~14GB but the uno build is 4GB Docker Image and that's tiny so 14 - 4 = 10 GB payload for IDF.
What makes it so big? Are you including precompiled libs for all the dev boards as a monobuild?
Feel free to close this issue or move it to discussion. But wow, 10GB.
Hardware Configuration
ESP32S3 on platformio
Version
latest stable Release (if not listed below)
Type
Task
IDE Name
platformio
Operating System
windows
Flash frequency
40
PSRAM enabled
yes
Upload speed
115200
Description
Big payload for a tiny board
Sketch
N/A
Debug Message
N/A
Other Steps to Reproduce
No response
I have checked existing issues, online documentation and the Troubleshooting Guide
- I confirm I have checked existing issues, online documentation and Troubleshooting guide.
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 10 comments
-
Thats git. In previous versions the framework was completely stored in git with all binaries. So the repo is big with this history.
Solution do not a full clone of arduino repo. A git clone with depth 1 is enough. Since no submodules used without --recursive. Just a few MB will be downloaded.
Beta Was this translation helpful? Give feedback.
All reactions
-
👍 1
-
This is the platformio.ini
Thought it was platform-espressif32.zip, but when I optimized that I got negligible speed increase. Is the git repository coming from the framework link?
[env:esp32s3]
platform = https://github.com/pioarduino/platform-espressif32/releases/download/54.03.20/platform-espressif32.zip
framework = arduino
upload_protocol = esptool
monitor_filters =
default
esp32_exception_decoder ; Decode exceptions so that they are human readable.
lib_deps = FastLED
Beta Was this translation helpful? Give feedback.
All reactions
-
None of my pioarduino releases downloads anything via git (clone). All needed packages are stripped down and uses either zip or xz tar balls. The biggest zip is the arduino lib with the precompiled binaries ~350Mbyte.
Beta Was this translation helpful? Give feedback.
All reactions
-
I use pioardiuno for everything.
I'm telling you man, it's 10GB. I have no idea why yet. I may dive into the container today to find out where all the bloat is coming from.
This explains why it's taking 15-25 minutes to install esp32XX
Beta Was this translation helpful? Give feedback.
All reactions
-
I did some auditing of the container and wanted to share the results. I don't see a github repo being installed. I do see a multi platform deploy
Container deep dive (host in this case is mapped in and doesn't count)
# sudo du -h --max-depth=1 / \
--exclude=/proc \
--exclude=/sys \
--exclude=/run \
--exclude=/mnt \
--exclude=/media \
--exclude=/dev \
2>/dev/null | sort -hr | head -n 20> > > > > > > ^C
# bash
root@d03f3dbf2070:/fastled# du -h --max-depth=1 / \
--exclude=/proc \
--exclude=/sys \
--exclude=/run \
--exclude=/mnt \
--exclude=/media \
--exclude=/dev \
2>/dev/null | sort -hr | head -n 20
21G /
8.2G /root
7.8G /host
3.7G /fastled
457M /usr
9.0M /var
1.4M /etc
24K /tmp
4.0K /srv
4.0K /opt
4.0K /home
4.0K /boot
root@d03f3dbf2070:/fastled# cd /root
root@d03f3dbf2070:~# ls
root@d03f3dbf2070:~# ls
root@d03f3dbf2070:~# ls -al
total 60
drwx------ 1 root root 4096 Oct 7 21:55 .
drwxr-xr-x 1 root root 4096 Oct 7 22:09 ..
-rw-r--r-- 1 root root 597 Oct 6 13:09 .bashrc
drwxr-xr-x 1 root root 4096 Oct 7 22:25 .cache
drwxr-xr-x 4 root root 4096 Oct 6 13:09 .config
drwxr-xr-x 1 root root 4096 Oct 7 21:55 .fastled
drwxr-xr-x 3 root root 4096 Oct 6 13:09 .local
drwxr-xr-x 1 root root 4096 Oct 7 22:14 .platformio
-rw-r--r-- 1 root root 187 Oct 6 13:09 .profile
-rw------- 1 root root 0 Sep 30 00:38 .python_history
-rw-r--r-- 1 root root 252 Oct 7 21:38 .wget-hsts
-rw-r--r-- 1 root root 26 Oct 6 13:09 .zshrc
root@d03f3dbf2070:~# diskaudit
Scanning for files...
Found 79,413 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 8,850,342,870, number of files: 79,413:
D .platformio: 4,704,047,774 (53.2%)
D .fastled: 2,920,632,609 (33.0%)
D .cache: 1,172,586,794 (13.2%)
D .local: 53,073,186 (0.6%)
F .bashrc: 1,194 (0.0%)
F .wget-hsts: 504 (0.0%)
D .config: 383 (0.0%)
F .profile: 374 (0.0%)
F .zshrc: 52 (0.0%)
F .python_history: 0 (0.0%)
root@d03f3dbf2070:~# cd .platformio
root@d03f3dbf2070:~/.platformio# ls
appstate.json packages platforms
root@d03f3dbf2070:~/.platformio# diskaudit
Scanning for files...
Found 35,818 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 4,705,756,628, number of files: 35,818:
D packages: 4,187,040,929 (89.0%)
D .cache: 477,699,097 (10.2%)
D platforms: 41,016,302 (0.9%)
F appstate.json: 300 (0.0%)
root@d03f3dbf2070:~/.platformio# cd packages
root@d03f3dbf2070:~/.platformio/packages# ls
contrib-piohome framework-arduinoespressif32-libs tool-esptoolpy tool-scons
framework-arduinoespressif32 tool-esp_install tool-mklittlefs toolchain-riscv32-esp
root@d03f3dbf2070:~/.platformio/packages# diskaudit
Scanning for files...
Found 34,650 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 4,187,100,349, number of files: 34,650:
D toolchain-riscv32-esp: 2,485,735,083 (59.4%)
D framework-arduinoespressif32-libs: 1,642,126,358 (39.2%)
D framework-arduinoespressif32: 49,681,281 (1.2%)
D tool-scons: 3,516,144 (0.1%)
D contrib-piohome: 3,402,541 (0.1%)
D tool-esptoolpy: 2,309,106 (0.1%)
D tool-esp_install: 211,725 (0.0%)
D tool-mklittlefs: 118,111 (0.0%)
root@d03f3dbf2070:~/.platformio/packages# cd tool
tool-esp_install/ tool-esptoolpy/ tool-mklittlefs/ tool-scons/ toolchain-riscv32-esp/
root@d03f3dbf2070:~/.platformio/packages# cd tool
tool-esp_install/ tool-esptoolpy/ tool-mklittlefs/ tool-scons/ toolchain-riscv32-esp/
root@d03f3dbf2070:~/.platformio/packages# cd tool
tool-esp_install/ tool-esptoolpy/ tool-mklittlefs/ tool-scons/ toolchain-riscv32-esp/
root@d03f3dbf2070:~/.platformio/packages# cd toolchain-riscv32-esp
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp# ls
bin include lib libexec package.json picolibc riscv32-esp-elf share
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp# diskaudit
Scanning for files...
Found 2,782 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 2,547,281,077, number of files: 2,782:
D riscv32-esp-elf: 1,352,462,181 (53.1%)
D picolibc: 774,960,733 (30.4%)
D libexec: 244,917,374 (9.6%)
D bin: 96,643,978 (3.8%)
D lib: 74,408,193 (2.9%)
D share: 3,887,128 (0.2%)
F package.json: 988 (0.0%)
F .piopm: 502 (0.0%)
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp# cd riscv32-esp-elf
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp/riscv32-esp-elf# diskaudit
Scanning for files...
Found 1,633 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 1,354,425,765, number of files: 1,633:
D lib: 1,322,741,988 (97.7%)
D bin: 16,213,320 (1.2%)
D include: 15,470,457 (1.1%)
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp/riscv32-esp-elf# cd lib
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp/riscv32-esp-elf/lib# ls
crt0.o libc_nano.a libm_nano.a libstdc++.a nano.specs rv32i_zicsr_zifencei semihost.specs
crt1-sim.o libg.a libnosys.a libstdc++.a-gdb.py no-rtti rv32imac_zicsr_zifencei sim.specs
default.specs libg_nano.a libpthread_stubs.a libstdc++exp.a nosys.specs rv32imafc_zicsr_zifencei sys.qemu.specs
ldscripts libgloss.a libsemihost.a libsupc++.a picolibc.specs rv32imafc_zicsr_zifencei_zba_zbb_zbc_zbs
libc.a libm.a libsim.a libsys_qemu.a picolibcpp.specs rv32imc_zicsr_zifencei
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp/riscv32-esp-elf/lib# diskaudit
Scanning for files...
Found 415 files.
Collecting file sizes...
Waiting for 8 tasks to complete.
Partitioning results...
Total size: 1,439,982,474, number of files: 415:
D rv32i_zicsr_zifencei: 224,265,798 (15.6%)
D rv32imc_zicsr_zifencei: 222,453,770 (15.4%)
D rv32imac_zicsr_zifencei: 222,381,416 (15.4%)
D rv32imafc_zicsr_zifencei: 221,605,544 (15.4%)
D rv32imafc_zicsr_zifencei_zba_zbb_zbc_zbs: 220,132,636 (15.3%)
D no-rtti: 110,592,964 (7.7%)
F libstdc++.a: 85,261,312 (5.9%)
F libg.a: 28,545,520 (2.0%)
F libc.a: 28,545,520 (2.0%)
F libg_nano.a: 22,991,264 (1.6%)
F libc_nano.a: 22,991,264 (1.6%)
F libm_nano.a: 7,631,600 (0.5%)
F libm.a: 7,624,040 (0.5%)
F libstdc++exp.a: 5,251,596 (0.4%)
F libsupc++.a: 4,142,064 (0.3%)
F libnosys.a: 1,656,064 (0.1%)
F libpthread_stubs.a: 1,200,064 (0.1%)
D ldscripts: 1,177,938 (0.1%)
F libgloss.a: 535,832 (0.0%)
F libsim.a: 524,648 (0.0%)
F libsemihost.a: 415,804 (0.0%)
F libsys_qemu.a: 23,812 (0.0%)
F crt1-sim.o: 7,264 (0.0%)
F crt0.o: 7,232 (0.0%)
F libstdc++.a-gdb.py: 4,958 (0.0%)
F picolibcpp.specs: 4,798 (0.0%)
F picolibc.specs: 4,750 (0.0%)
F nano.specs: 1,048 (0.0%)
F nosys.specs: 554 (0.0%)
F sys.qemu.specs: 410 (0.0%)
F semihost.specs: 404 (0.0%)
F sim.specs: 372 (0.0%)
F default.specs: 214 (0.0%)
root@d03f3dbf2070:~/.platformio/packages/toolchain-riscv32-esp/riscv32-esp-elf/lib
And here's some AI analsyis:
You’re looking at a multilib RISC-V toolchain from Espressif. The huge space is coming from the same runtime libraries being compiled for multiple ISA variants/ABIs, plus a couple of "flavors" (full vs nano, RTTI vs no-RTTI), and most of them are LTO-enabled archives (which are big).
Here’s what those "mystery" chunks are:
rv32i_zicsr_zifencei / rv32imc_zicsr_zifencei / rv32imac_zicsr_zifencei / rv32imafc_zicsr_zifencei / rv32imafc_zicsr_zifencei_zba_zbb_zbc_zbs (≈ 220–224 MB each)
These are the multilib directories: copies of the core static libraries built for different RISC-V ISA sets:
rv32i = base integer
m = mul/div, a = atomics, f = single-precision float, c = compressed
zicsr/zifencei = mandatory control/status & fence extensions
zba/zbb/zbc/zbs = bit-manip extensions (used on newer cores like C6/P4)
Each folder contains per-variant builds of things like libstdc++.a, libsupc++.a, libgcc.a, libc.a (picolibc), libm.a, start files, and sometimes specs/ldscripts. This is how one toolchain supports ESP32-C3 (rv32imc), ESP32-C6 (often rv32imafc+Zb), ESP32-P4*, etc.
libstdc++.a (85 MB), libg.a (28.5 MB), libc.a (28.5 MB), libg_nano.a/libc_nano.a (≈23 MB each), libm*.a (≈7.6 MB), libsupc++.a (4.1 MB)
These are the top-level default-variant archives (one chosen multilib gets duplicated at the root for convenience).
libstdc++.a + libsupc++.a = C++ standard library and low-level C++ runtime.
libc*.a, libm*.a = picolibc (or nano builds) and math.
The "nano" variants are size-tuned builds (fewer features, smaller stdio, etc.).
Size is large because these are static libraries with LTO sections (bitcode/IR) and lots of templates; the archives pack a ton of object files so the linker can pick what it needs.
no-rtti/ (111 MB)
Alternative C++ libs compiled without RTTI/exceptions, for minimal footprint. Good for embedded if you avoid exceptions/typeid.
ldscripts/, *.specs, crt0.o, crt1-sim.o
Linker scripts, GCC specs files (select nano/nosys/etc.), and C runtime start objects.
libnosys.a (stubs), libgloss.a/libsemihost.a/libsim.a/libsys_qemu.a, libpthread_stubs.a
Syscall stubs and support for semihosting/simulators; no full pthreads here—just stubs for linking.
picolibc/ (775 MB at the package root)
Headers, archives, and again multilib copies for the C runtime—this duplicates a lot of what you see under riscv32-esp-elf/lib, which is why it’s another massive wedge.
Why so big overall?
Multilib explosion: ~5 ISA/ABI variants ×ばつ (libstdc++ + supc++ + libc + libm + libgcc + ...).
LTO archives: objects store intermediate representation, which bloats .a size even though the final ELF you build may still be small.
Both full and "nano" flavors, and no-RTTI flavor too.
Duplicate placement (per-variant dirs plus a default copy at the lib root; plus the separate picolibc/ tree).
Quick ways to verify what’s inside (pick any .a or a file in a multilib dir):
Show the multilib matrix this GCC supports
riscv32-esp-elf-gcc -print-multi-lib
See which ISA/ABI a given object was built for
riscv32-esp-elf-readelf -A rv32imafc_zicsr_zifencei/lib/libstdc++.a | head
List members in a fat archive
riscv32-esp-elf-ar t libstdc++.a | head -n 50
Check if LTO/IR is present (look for .gnu.lto_ or LLVM bitcode)
riscv32-esp-elf-nm -a libstdc++.a | head
strings libstdc++.a | grep -i -E 'lto|bitcode|llvm' | head
If you want to reclaim space (without breaking your builds):
Keep only the multilib you actually use (e.g., for ESP32-C3: rv32imc_zicsr_zifencei; for ESP32-C6/P4: the rv32imafc...zba_zbb_zbc_zbs dir). You can move the others out of the way.
Use "nano" and/or no-RTTI: point GCC to nano.specs or pick libs from no-rtti/ to reduce final binary size (and you can prune the unused full variants).
Strip debug info from archives (careful, test this):
riscv32-esp-elf-strip -g path/to/libsomething.a
Deduplicate with hardlinks if multiple copies are byte-identical across dirs (sometimes they are). Tools like rdfind/fdupes can help.
If you control the toolchain build: disable LTO in shipped archives or provide a "slim toolchain" with only one multilib.
Bottom line: those 200-220 MB folders are per-ISA copies of the same standard libraries; the 85 MB libstdc++.a and ~23-28 MB libc variants are LTO-enabled static archives; no-rtti is an alternative C++ lib flavor; and the rest are syscall/semihost/sim support libraries and specs.
Beta Was this translation helpful? Give feedback.
All reactions
-
Recommended sample esp32p4 "thin" build
-Os
-flto
-ffunction-sections -fdata-sections -Wl,--gc-sections
-fno-rtti -fno-exceptions -fno-use-cxa-atexit
-specs=nano.specs
-Wl,-u,_printf_float
-g1
-fno-omit-frame-pointer
That would take out the largest redudancy. I wondered if there was xtensa and riscv, but it seems that surprisingly there's only riscv installed. I'm not sure how platformio is able to just generate one platform for a url that supports multiple platforms. Is it by any chance compiling riscv on the fly? (update no it's not compiling) That would explain why it takes so long, if it's being built.
I may dive more into this later.
Beta Was this translation helpful? Give feedback.
All reactions
-
No, only the toolchain which is needed is installed. Build for esp32 and the xtensa toolchain is downloaded and installed.
As said nothing from git is installed. Only what is needed.
btw. the toolchains tarballs are not so big to download. Source is here https://github.com/espressif/crosstool-NG/releases
Beta Was this translation helpful? Give feedback.
All reactions
-
To minimize downloads, you could preload the Docker Image with all tools and toolchains matching for the OS. All this stuff is downloaded into folder .platformio/tools Provide this folder preloaded with all needed and it will be fast. The tools and toolchains will be installed from this folder when needed.
Beta Was this translation helpful? Give feedback.
All reactions
-
D rv32i_zicsr_zifencei: 224,265,798 (15.6%)
D rv32imc_zicsr_zifencei: 222,453,770 (15.4%)
D rv32imac_zicsr_zifencei: 222,381,416 (15.4%)
D rv32imafc_zicsr_zifencei: 221,605,544 (15.4%)
D rv32imafc_zicsr_zifencei_zba_zbb_zbc_zbs: 220,132,636 (15.3%)
AI says this is the same toolset, but with slightly different flavors. Instead of 1.2GB it could be 250mb
Beta Was this translation helpful? Give feedback.
All reactions
-
Yes, and needed since the espressif riscv mcu have different hardware features and extensions.
The xz riscv toolchain package is around 280MB to download. Imho its size is fully okay.
Beta Was this translation helpful? Give feedback.