Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Native USB debugging on ESP32 in PlatformIO without ESP-IDF toolchain? #7879

Closed Answered by Jason2866
papakpmartin asked this question in Q&A
Discussion options

I have a custom ESP32-S2-based circuit board with USB-C which does not have a USB-serial converter IC like a common dev board might. On this board, USB(-) is on GPIO19, and USB(+) is on GPIO20. USB works great for powering the board and for uploading firmware.

The board works well and I have access to good old-fashioned serial console via a USB-to-logic level serial interface, but it would be nice to be able to get serial out to USB along with the firmware upload (like a dev board).

I’m using PlatformIO in Visual Studio Code, and writing with arduino-esp32 rather than ESP-IDF.

I understand by these instructions from Espressif that when using ESP-IDF I can configure log output to go to USB CDC rather than UART, and this has been done successfully on my custom board. But I would like to be able to do this without having to switch over to ESP-IDF.

I’m presuming that one of the partitions that PlatformIO is building for me is this configuration with some nice common sense defaults, but I can’t see how I might alter those defaults to do what I’m looking for.

Any thoughts or pointers?

You must be logged in to vote

Use a custom boards.json for your DIY board. Like this assuming no PSRAM. Name it for example esp32s2-cdc.json Make a new folder boards in your root folder of the project. Place your created json file there

{
 "build": {
 "arduino":{
 "ldscript": "esp32s2_out.ld"
 },
 "core": "esp32",
 "extra_flags": [
 "-DARDUINO_USB_CDC_ON_BOOT=1"
 ],
 "f_cpu": "240000000L",
 "f_flash": "80000000L",
 "flash_mode": "qio",
 "mcu": "esp32s2",
 "variant": "esp32s2"
 },
 "connectivity": [
 "wifi"
 ],
 "debug": {
 "openocd_target": "esp32s2.cfg"
 },
 "frameworks": [
 "arduino",
 "espidf"
 ],
 "name": "ESP32-S2 CDC",
 "upload": {
 "flash_size": "...

Replies: 8 comments 6 replies

Comment options

This functionality is already available. @Jason2866 could you please help with PIO instructions?

You must be logged in to vote
0 replies
Comment options

Use a custom boards.json for your DIY board. Like this assuming no PSRAM. Name it for example esp32s2-cdc.json Make a new folder boards in your root folder of the project. Place your created json file there

{
 "build": {
 "arduino":{
 "ldscript": "esp32s2_out.ld"
 },
 "core": "esp32",
 "extra_flags": [
 "-DARDUINO_USB_CDC_ON_BOOT=1"
 ],
 "f_cpu": "240000000L",
 "f_flash": "80000000L",
 "flash_mode": "qio",
 "mcu": "esp32s2",
 "variant": "esp32s2"
 },
 "connectivity": [
 "wifi"
 ],
 "debug": {
 "openocd_target": "esp32s2.cfg"
 },
 "frameworks": [
 "arduino",
 "espidf"
 ],
 "name": "ESP32-S2 CDC",
 "upload": {
 "flash_size": "4MB",
 "maximum_ram_size": 327680,
 "maximum_size": 4194304,
 "require_upload_port": true,
 "speed": 460800
 },
 "url": "https://www.myboard.com",
 "vendor": "DIY"
}

and create as usual an env in your platformio.ini and add as board your board

...
board = esp32s2-cdc
....

Looking in the predefined boards.json will help to understand the different settings https://github.com/platformio/platform-espressif32/tree/develop/boards

If your board has PSRAM add in section extra_flags the entry "-DBOARD_HAS_PSRAM"

The needed options can be set in platformio.ini too. But i like it more to define a custom boards.json which exactly meets the boards specs. So one time done no more thinking about which settings in platformio.ini are needed when a new env is done with this board...

You must be logged in to vote
4 replies
Comment options

This is fantastic. Thank you @Jason2866 and @me-no-dev! I will try this shortly and mark as answer when i see it work. This makes perfect sense.

Comment options

I had been using esp32-s2-saola-1.json. What does -DARDUINO_ESP32S2_DEV do? Is there a reference for these flags? Should I ditch that one when adding -DARDUINO_USB_CDC_ON_BOOT=1?

Comment options

Do I also need to set Serial up differently in setup()? Currently I'm doing...

 Serial.begin(115200);
 while (!Serial);
 Serial.println("SERIAL: Setup complete");

Asking because it appears I'm getting "stuck" here.

Comment options

Ah... interesting (to me)...

I was able to upload via USB as always and eventually got...

WARNING: ESP32-S2FH4 (revision v0.0) chip was placed into download mode using GPIO0.
esptool.py can not exit the download mode over USB. To run the app, reset the chip manually.

If I do this with "Upload and Monitor", the monitor doesn't "catch" when I hit the reset button.

However, if I use "Monitor" and if I manually choose the "Set upload/monitor/test port", then Monitor does pick it up and the board can start normally and get me the logging I was looking for!

Thanks so much! :)

Answer selected by papakpmartin
Comment options

Might using "-DARDUINO_USB_CDC_ON_BOOT=1" have unexpected consequences or implications?

My new config is essentially identical to the esp32-s2-saola-1.json that I was using before, with only that flag added. However, with the old device config everything works as expected, and with the new device config I'm unable to connect to an MQTT broker as expected... just stops there... my logging is...

Running task__handleReporting
WIFI: About to connect
FIRMWARE: Version 2
Available Ram: 103256 bytes
WIFI: Status: 3
WIFI: Connecting...
WIFI: CONNECTED
MQTT: About to connect

...and I get nothing more than that even with this in place:

build_flags = 
 -DCORE_DEBUG_LEVEL=5

Immediately after that last log I'm running...

 if (!m_mqttClient.connected()) {
 Serial.println("MQTT: Setting server");
 m_mqttClient.setServer(MQTT_BROKER, MQTT_PORT);
 Serial.println("MQTT: Setting callback");
 ...

I'm using knolleary/PubSubClient@^2.8 for MQTT, in case that's helpful.

Again, rolling back to a plain esp32-s2-saola-1 build works perfectly.

Any thoughts would be appreciated! :)

You must be logged in to vote
0 replies
Comment options

@papakpmartin Mhh, the mqtt issue is strange and cant explain.
You can try our modified mqtt pubsubclient version. Working for all esp

https://github.com/arendst/Tasmota/tree/development/lib/default/pubsubclient-2.8.13

Have seen this too:

WARNING: ESP32-S2FH4 (revision v0.0) chip was placed into download mode using GPIO0.
esptool.py can not exit the download mode over USB. To run the app, reset the chip manually.

Maybe adding this helps

 "use_1200bps_touch": true,
 "wait_for_upload_port": true,
 "require_upload_port": true,

Finally you can try:

Modified (forked) Platformio platform-espressif32 and changed to make this settings possible in boards.json

 "require_upload_port": true,
 "before_reset": "usb_reset",
 "after_reset": "no_reset",
 "speed": 460800

You can try if it works for you. If yes i will provide a PR currently we use this change only for project Tasmota

platform = https://github.com/tasmota/platform-espressif32/releases/download/2023.02.00/platform-espressif32.zip
You must be logged in to vote
0 replies
Comment options

Well, I was able to give this a go... it was weird. It mostly did not make a difference. Same symptom of starting nicely (as long as USB was plugged in and Monitor window was open) and running until approximately the same point, then pause for about 8-10 seconds, then crash/restart the ESP. No stacktrace.

My device file is basically...

{
 "build": {
 "arduino":{
 "ldscript": "esp32s2_out.ld"
 },
 "core": "esp32",
 "extra_flags": [
 "-DARDUINO_ESP32S2_DEV",
 "-DARDUINO_USB_CDC_ON_BOOT=1"
 ],
 "f_cpu": "240000000L",
 "f_flash": "80000000L",
 "flash_mode": "qio",
 "mcu": "esp32s2",
 "variant": "esp32s2"
 },
 "connectivity": [
 "wifi"
 ],
 "debug": {
 "openocd_target": "esp32s2.cfg"
 },
 "frameworks": [
 "arduino",
 "espidf"
 ],
 "name": "My Custom ESP32-S2",
 "upload": {
 "flash_size": "4MB",
 "maximum_ram_size": 327680,
 "maximum_size": 4194304,
 "speed": 460800,
 "use_1200bps_touch": true,
 "wait_for_upload_port": true,
 "require_upload_port": true
 },
 "url": "...",
 "vendor": "..."
 }

...and my platformio.ini contains basically...

[env:my-custom-esp32s2]
; platform = espressif32
platform = https://github.com/tasmota/platform-espressif32/releases/download/2023.02.00/platform-espressif32.zip
board = my-custom-esp32s2
framework = arduino
monitor_speed = 460800
monitor_filters =
 esp32_exception_decoder
 log2file
lib_deps = 
 knolleary/PubSubClient@^2.8
 ; others
; build_flags = 
; -DCORE_DEBUG_LEVEL=5

Oddly, with that set up once in a while it worked perfectly for a few minutes, but would eventually revert back to the same behavior.

At one point I thought it was maybe xTaskCreatePinnedToCore as when I replaced the single call to that with...

 xTaskCreate(
 task__handleReporting,
 "task__handleReporting",
 25000,
 NULL, // parameters
 10, // priority, higher number is higher priority
 &taskHandle__handleReporting
 // CONFIG_ARDUINO_RUNNING_CORE // Wi-Fi tasks must run on same core and Arduino
 );

...it worked. But it did not reliably work.

Not sure what to say, but I wanted to at least share what happened when I tried.

You must be logged in to vote
2 replies
Comment options

Oh, but I can say that...

"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true

...did help me reliably get upload and monitor connections.

Even when using "normal" platform = espressif32 and not the tasmota/platform-espressif32. (I mean, it seemed to be better in both cases.)

Comment options

OK, so I at least have it down to:

  • Using the esp32-s2-saola-1.json that I have been using all along works perfectly
  • Using a custom board identical to that one also works perfectly
  • Adding "-DARDUINO_USB_CDC_ON_BOOT=1" to the custom board (and doing nothing else differently)...
    • Requires monitor to be connected in order to get through setup()
    • Only works maybe 5% of the time, but instead usually reboots (watchdog timeout?)
  • Keeping that flag and using the custom platform and extra attributes has no obvious effect on performance
  • Keeping that flag and moving the Wi-Fi/MQTT code's task to xTaskCreateCore rather than the original xTaskCreatePinnedToCore might have a slightly positive effect, perhaps working 10% of the time instead of 5% of the time; but still it usually simply reboots.

So in the end, it appears that adding "-DARDUINO_USB_CDC_ON_BOOT=1" is doing something unexpected (by me) that interferes with my code.

Maybe in a couple of weeks I could try to come up with a simplified test case.

I didn't have a chance to try swapping out the MQTT library either. If it fails, it does mostly fail there (rarely before that in the Wi-Fi connection).

Comment options

@papakpmartin Thinking that the issue you see is caused from your sketch. We had a "funny" bug in mqtt lib which caused weird issues.

You must be logged in to vote
0 replies
Comment options

Well, I'm not sure what to make of this, but commenting out all of the Serial.p... lines in the library doing the MQTT connection allowed everything to work as expected! But these debug lines from the library just before the MQTT stuff did work prior to me commenting everything out:

WIFI: Status: 3
WIFI: Connecting...
WIFI: CONNECTED

So essentially, all Serial lines in Comms.c here were commented out. However, unlike that example project, I have other libraries that do use Serial just fine... in setup() anyway.

Weird. But thought I'd share as it seemed interesting. :)

You must be logged in to vote
0 replies
Comment options

The issue was indeed my code. Having fixed that, this USB debug logging works wonderfully.

The problem was this while having no delay:

 while (!m_mqttClient.connect(mac_address.c_str(), MQTT_USER, MQTT_PASS)) {
 Serial.println("MQTT: Connecting...");
 Serial.print("MQTT: State: ");
 Serial.println(m_mqttClient.state());
 }

So in brief, @Jason2866's advice above worked perfectly.

You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet

AltStyle によって変換されたページ (->オリジナル) /