[native_sim] Zephyr 4.1 breaks GPIO_EMUL_SDL and INPUT_SDL_TOUCH

[PetervdPerk-NXP]

Describe the bug

I've got this application that runs natively on the NXP FRDM-MCXN947 but also has a native/posix/64 target.
https://github.com/NXPHoverGames/Doom-MCX
When upgrading from Zephyr 4.0.0 to Zephyr 4.1.0 the input (GPIO SDL emulation and SDL touch) for native/posix/64 is not working anymore.

To Reproduce

# initialize  for the example-application (main branch)
west init -m https://github.com/nxphovergames/doom-mcx --mr main doom-mcx-workspace
# update Zephyr modules
cd doom-mcx-workspace
west update
cd doom-mcx
west build -p always -b native_sim/native/64
./build/zephyr/zephyr.exe

When modifying west.yml to point to Zephyr v4.0.0 8469084dfae85f854555f0607f2c838dad097235
Then input is working again

Impact
No SDL input on native_sim

Environment (please complete the following information):

• OS: (Ubuntu 22.04 under wsl2)
• Toolchain (Zephyr sdk 0.17.0)

[faxe1008]

Does it work with CONFIG_INPUT=y?

[PetervdPerk-NXP]

CONFIG_INPUT=y already, see

Again the same code is working fine on native hardware.

[aescolar]

CC @fabiobaltieri @pdgendt

[fabiobaltieri]

Can you bisect this?

[PetervdPerk-NXP]

Can you bisect this?

I think I've narrowed it down this PR #81184
The commit 86a126d before is working fine.

EDIT: Reverting 02d562e on top Zephyr 4.1 gives me input again.

[JarmouniA]

FYI @Finomnis

[Finomnis]

Huh? I didn't change anything regarding the input, at least not that I know of...

Very weird.

I'm sadly very limited on time right now, would someone else mind dissecting that change in more detail to see which line exactly broke it?

[faxe1008]

@PetervdPerk-NXP your code does not build for me:

/tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_stuff.c: In function ‘HU_Start’:
/tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_stuff.c:372:19: error: passing argument 5 of ‘HUlib_initSText’ from incompatible pointer type [-Wincompatible-pointer-types]
  372 |                 _g->hu_font,
      |                 ~~^~~~~~~~~
      |                   |
      |                   const patch_t **
In file included from /tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_stuff.c:39:
/tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_lib.h:175:18: note: expected ‘const patch_t *’ but argument is of type ‘const patch_t **’
  175 |   const patch_t *font,
      |   ~~~~~~~~~~~~~~~^~~~
/tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_stuff.c:384:19: error: passing argument 4 of ‘HUlib_initTextLine’ from incompatible pointer type [-Wincompatible-pointer-types]
  384 |                 _g->hu_font,
      |                 ~~^~~~~~~~~
      |                   |
      |                   const patch_t **
/tmp/doom-mcx-workspace/doom-mcx/prboom2/src/hu_lib.h:152:18: note: expected ‘const patch_t *’ but argument is of type ‘const patch_t **’
  152 |   const patch_t *f,
      |   ~~~~~~~~~~~~~~~^

Is this somehow reproducible with an intree-sample?

EDIT:// Hmm looking at the resulting autoconf.h tho it seems like CONFIG_LV_Z_POINTER_INPUT is not enabled for me, which might be the culprit. Can you check this on your end with a working build @PetervdPerk-NXP ?

EDIT-2:// @PetervdPerk-NXP try adding CONFIG_LVGL=y this is missing from your prj.conf: https://github.com/NXPHoverGames/Doom-MCX/blob/main/prj.conf
If that fixes your issue, I fail to understand what the commit you linked has to do with it.

[PetervdPerk-NXP]

Is this somehow reproducible with an intree-sample?

EDIT:// Hmm looking at the resulting autoconf.h tho it seems like CONFIG_LV_Z_POINTER_INPUT is not enabled for me, which might be the culprit. Can you check this on your end with a working build @PetervdPerk-NXP ?

EDIT-2:// @PetervdPerk-NXP try adding CONFIG_LVGL=y this is missing from your prj.conf: https://github.com/NXPHoverGames/Doom-MCX/blob/main/prj.conf If that fixes your issue, I fail to understand what the commit you linked has to do with it.

1. There's no in-tree sample that only uses input it seems
2. I'm not using LVGL just zephyr framebuffer and zephyr input, this works without LVGL
3. I tried CONFIG_LGVL=y, doesn't fix it.
4. The error/warning you're showing when compiling is a warning treated as error, which compiler are you using? kinda weird though it enforces this now. You could try something like set(CMAKE_COMPILE_WARNING_AS_ERROR OFF)

[Finomnis]

@PetervdPerk-NXP would it be possible for you to strip down that project to a minimal reproducible example? Which demonstrates the problem and works if you revert said commit? So we can argue about it more focused?

[PetervdPerk-NXP]

@PetervdPerk-NXP would it be possible for you to strip down that project to a minimal reproducible example? Which demonstrates the problem and works of you revert said commit? So we can argue about it more focused?

Kinda hard to do tbh, overall a simple fb with input sample is missing unfortunately.

Btw I've narrowed it down to the line causing it.

zephyr/drivers/display/display_sdl_bottom.c

Line 56 in 02d562e

SDL_SetTextureBlendMode(*texture, SDL_BLENDMODE_BLEND);

Uncommenting this gives me input again.

EDIT2: @faxe1008 if you could share the cmake output showcasing the GCC version that would help me fixing the compile error you're having.

[Finomnis]

OK wtf :D if that is true then something very weird is going on.

Do we cause undefined behavior somewhere?

[Finomnis]

@PetervdPerk-NXP are you aware that the SDL driver is not multi threading capable? If you interact with the SDL driver from multiple threads, that might cause undefined behavior. Just checking, this is an open issue at the moment if I remember right.

[PetervdPerk-NXP]

@PetervdPerk-NXP are you aware that the SDL driver is not multi threading capable? If you interact with the SDL driver from multiple threads, that might cause undefined behavior. Just checking, this is an open issue at the moment if I remember right.

I do have 2 threads but that's working since like zephyr 3.6.0, however I'm running with SDL_DISPLAY_USE_HARDWARE_ACCELERATOR=n which was always fine.

I'm thinking it could be perf related thing, without SDL_SetTextureBlendMode disabled CPU load is ~60% and with SDL_SetTextureBlendMode enabled it's around ~95%.

[Finomnis]

So what happens if you SDL_DISPLAY_USE_HARDWARE_ACCELERATOR=y?

[PetervdPerk-NXP]

Then I don't have a screen due to multiple threads, just a gray background frames don't get updated.

Hacking my code to be single threaded I get a screen again and input works. But Cpu load skyrockets to 600% (multi-core) though.

So my hypothesis, SDL_DISPLAY_USE_HARDWARE_ACCELERATOR=n, is just single threaded and the alpha blending introduced #81184 eats so much cpu thus blocking input events.

[Finomnis]

Hmm. That sucks.

I think the proper fix would be to finally tackle #71410.

[Finomnis]

Also, why do you get such a high CPU load? I'm pretty sure SDL uses OpenGL if available, there should not be much CPU involved in the SDL draw calls.

[Finomnis]

I mean we could add a kconfig option that disables transparency, that would be another "fix". But for your case it feels more like a workaround tbh.

[PetervdPerk-NXP]

I'm okay keeping this open for now and linking to #71410 when that gets fixed this one would be fixed as well.

Regarding cpu load, no clue maybe due to wsl2 that does opengl software rendering in the end.

[fabiobaltieri]

So my hypothesis, SDL_DISPLAY_USE_HARDWARE_ACCELERATOR=n, is just single threaded and the alpha blending introduced #81184 eats so much cpu thus blocking input events.

Is it on LVGL or on the zephyr threads? Not sure this changes anything in native_sim but the input thread has a very low priority by default, maybe try

CONFIG_INPUT_THREAD_PRIORITY_OVERRIDE=y
CONFIG_INPUT_THREAD_PRIORITY=-2

[Finomnis]

Is it on LVGL or on the zephyr threads?

@fabiobaltieri No lvgl involved here, to my understanding.

[fabiobaltieri]

@fabiobaltieri No lvgl involved here, to my understanding.

Right, does SDL have a thread that could starve everything else? Though IIUC on native_sim all Zephyr threads map to pthread threads, doubt it would just starve the input thread to the point it would never run.

Only SDK thread I see is the one in boards/native/common/sdl/sdl_events.c and has a sleep in the loop, so yeah doubt there's any problem there.

[Finomnis]

That's why I feel like it's a user thread priority problem. OP states that they have two threads, so I assume one for rendering and one for input. I suspect the rendering thread has a higher priority than the input thread.

[PetervdPerk-NXP]

That's why I feel like it's a user thread priority problem. OP states that they have two threads, so I assume one for rendering and one for input. I suspect the rendering thread has a higher priority than the input thread.

Well this the fun thing I don't have special input thread. Main thread samples input periodically and there's an input callback for touch. No clue how zephyr handles this, but again on native hardware this works fine.

I think it's more related to the sdl implementation itself not forwarding input events.

[PetervdPerk-NXP]

So my hypothesis, SDL_DISPLAY_USE_HARDWARE_ACCELERATOR=n, is just single threaded and the alpha blending introduced #81184 eats so much cpu thus blocking input events.

Is it on LVGL or on the zephyr threads? Not sure this changes anything in native_sim but the input thread has a very low priority by default, maybe try

CONFIG_INPUT_THREAD_PRIORITY_OVERRIDE=y
CONFIG_INPUT_THREAD_PRIORITY=-2

Thank you for the suggestion, just tried this but doesn't fix the input.

[Finomnis]

Well this the fun thing I don't have special input thread. Main thread samples input periodically and there's an input callback for touch.

So your main thread also renders the image and draws it on screen? You mention you have two threads, what does the other thread do?