esp-idf-monitor does not support escape characters (IDFGH-13646)

[Smity]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate
• Read the documentation to confirm the issue is not addressed there and your configuration is set correctly
• Tested with the latest version to ensure the issue hasn't been fixed

How often does this bug occurs?

always

Expected behavior

Documentation states that esp-idf-monitor should support escape characters.

Actual behavior (suspected bug)

Using IDF 5.2.2 in VSCode. Flashing "Basic Console Example" (esp_console_repl) to an ESP32S3. I haven't made any changes to the example code, except for trying each of the USB_JTAG, USB_CDC, and UART interfaces. Each one reads and writes console just fine, but they all report "Your terminal application does not support escape sequences.".

Error logs or terminal output

--- esp-idf-monitor 1.4.0 on \\.\COM11 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x12b4
load:0x403c9700,len:0x4
load:0x403c9704,len:0xaf0
load:0x403cc700,len:0x2be4
entry 0x403c9898
I (202) cpu_start: Multicore app
I (212) cpu_start: Pro cpu start user code
I (212) cpu_start: cpu freq: 160000000 Hz 
I (212) cpu_start: Application information: 
I (215) cpu_start: Project name:     console
I (220) cpu_start: App version:      1      
I (224) cpu_start: Compile time:     Sep  5 2024 17:25:04
I (230) cpu_start: ELF file SHA256:  1f279153d...        
I (235) cpu_start: ESP-IDF:          v5.2.2
I (240) cpu_start: Min chip rev:     v0.0  
I (245) cpu_start: Max chip rev:     v0.99 
I (250) cpu_start: Chip rev:         v0.2  
I (255) heap_init: Initializing. RAM available for dynamic allocation:
I (262) heap_init: At 3FCA3588 len 00046188 (280 KiB): RAM
I (268) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (274) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM 
I (280) heap_init: At 600FE070 len 00001F78 (7 KiB): RTCRAM
I (287) spi_flash: detected chip: winbond
I (291) spi_flash: flash io: dio
W (295) spi_flash: Detected size(16384k) larger than the size in the binary image header(4096k). Using the size in the binary image header.
I (309) sleep: Configure to isolate all GPIO pins in sleep state
I (315) sleep: Enable automatic switching of GPIO sleep configuration
I (323) main_task: Started on CPU0   
I (333) main_task: Calling app_main()
I (373) example: Command history enabledn

Type 'help' to get the list of commands.
Use UP/DOWN arrows to navigate through command history.
Press TAB when typing command name to auto-complete.   

Your terminal application does not support escape sequences.

Line editing and history features are disabled.

On Windows, try using Putty instead.
esp32s3> I (873) main_task: Returned from app_main()

### Steps to reproduce the behavior

1. Build & flash `esp_console_repl` to ESP32S3
2. Click `ESP-IDF: Monitor Device`

### Project release version

1.4.0

### System architecture

Intel/AMD 64-bit (modern PC, older Mac)

### Operating system

Windows

### Operating system version

19045.4780

### Shell

PowerShell

### Additional context

Perhaps I'm missing something here and my configuration's off, but I don't think so because all the documentation and related forum threads I've read on this suggest I have everything set up correctly.

[peterdragun]

Hi @Smity,
thank you for the report. As you mentioned this should work even on Windows. I have a couple of questions to better identify what might be the issue:

You mentioned that you are running VS Code, so I assume you are using the integrated terminal. Please verify that you get the same issue in the regular Windows Powershell or Command line. You can run the monitor with the following command: python -m esp_idf_monitor. You either need to be in active ESP-IDF env to run this command or you can install the monitor separately using pip.

except for trying each of the USB_JTAG, USB_CDC, and UART interfaces.

Do you mean the physical interfaces -> USB ports? If you change the port you also need to make sure you updated the following setting in the menuconfig to match your interface: Component config -> ESP System Settings -> Channel for console output. This is often the cause for failing to start the interactive console.

ESP32-S3 devkits usually have two USB ports, one marked UART and the other one marked USB, can you please check which one are you using?

Can you please check that this works for you using other serial monitoring applications such as Putty as mentioned in the console output?

[Smity]

Thanks for your response.

Do you mean the physical interfaces -> USB ports?

I mean the menuconfig setting. My devkit indeed has two physical USB ports — one that connects directly to the D+ and D- pins, and one that connects to UART0 through an on-board UART-USB chip — and I've been making sure to use the one that correctly matches my menuconfig settings.

The console also doesn't work if I invoke the same command that VSCode uses in a plain PowerShell window:
& '%USERPROFILE%\.espressif\python_env\idf5.2_py3.11_env\Scripts\python.exe' '%USERPROFILE%\esp\v5.2.2\esp-idf\tools\idf_monitor.py' -p COM11 -b 112500 --toolchain-prefix xtensa-esp32s3-elf- --target esp32s3 '%USERPROFILE%\...\ESP_CONSOLE_2\build\ESP_CONSOLE_2.elf'

PuTTY works if I'm using UART0 and my board's built-in UART-USB chip, but it only recognizes escape characters after I reset the board (I have to connect the board to my computer, then open a PuTTY terminal (the console will connect, but escape characters will be disabled), then press the RST button on my dev board to reconnect and enable escape characters). If I try PuTTY with the USB_JTAG or USB_CDC configuration, resetting the dev board disconnects the USB and ends the PuTTY session.

I expect that if I disabled the call to linenoiseProbe() in esp_console_repl.c:341, I wouldn't have to restart the board to get the console to work on PuTTY. I have tried doing that with esp-idf-monitor, but the monitor still wasn't able to send escape characters, resulting in a lot of garbage data getting printed. I haven't tried that with PuTTY yet.

Since I made this bug report I have seen the console work correctly in esp-idf-monitor on Windows 11 (I'm on Windows 10), but I haven't been able to figure out what my colleague did differently to get it working.

[peterdragun]

PuTTY works if I'm using UART0 and my board's built-in UART-USB chip, but it only recognizes escape characters after I reset the board (I have to connect the board to my computer, then open a PuTTY terminal (the console will connect, but escape characters will be disabled), then press the RST button on my dev board to reconnect and enable escape characters). If I try PuTTY with the USB_JTAG or USB_CDC configuration, resetting the dev board disconnects the USB and ends the PuTTY session.

That is expected behavior, ESP will test the console on boot (as you later mentioned with the linenoiseProbe function), and then it will continue in a mode that was detected (either it got a response if it supports escape sequences or no response). In USB modes, the port disappearing is also expected, because of chip reset.

I am not sure what might be the issue that you are running into. We have tested this on multiple Windows computers (all Windows 11) and it works without any issues even in the VS Code terminal. It seems like an issue with your environment/console. The interactive console uses VT100 escape characters and the only thing that the monitor does regarding this is to send those characters as soon as it gets them from the chip.

Here are a couple of suggestions to try:

1. Seems like you were using Powershell, is the behavior the same for the cmd line?
2. Check that you are not using a legacy mode of the command line
3. Check for an option to enable VT100 escape sequence (or virtual terminal) on your system, those should work by default AFAIK, but you may have them disabled for some reason.

[Smity]

Same behaviour on CMD; legacy mode is off.

Apparently, VT100 escape sequences are not enabled by default on Windows 10 (they are on Windows 11, though), and it seems they have to be enabled separately for each terminal instance. I couldn't find any global setting that would permanently enable them. I even tried adding a registry key, but to no effect.

https://stackoverflow.com/questions/16081639/how-to-use-vt100-code-in-windows
https://stackoverflow.com/questions/51680709/colored-text-output-in-powershell-console-using-ansi-vt100-codes

[dobairoland]

@Smity, so did it start to work for you when you enabled VT100 escape sequences?

[peterdragun]

Hi @Smity, I have an update regarding this issue.
Thanks to @fhrbata, we have been able to find out what the issue is. Even if the terminal supports Virtual Terminal processing and it is turned on, it does not support the Device Status Report (0x1b[5n), which is used in IDF to identify if the console supports VT100 or not. (See https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences#query-state)
In this case, I would recommend using the new Windows Terminal app, which is the default on Windows 11, and that is also the reason why it works there without any issues. It can also be installed on Windows 10. We will try to update the message in the example, so it will be more clear.