Skip to content

Commit 2a103b9

Browse files
congzhangzhcongzhangzh
committed
add readme & design note
1 parent d690b9e commit 2a103b9

File tree

2 files changed

+231
-0
lines changed

2 files changed

+231
-0
lines changed

‎design_en.md

+204
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
# Win32 and Asyncio Dual-Thread Collaboration Design
2+
3+
## Architecture Diagram
4+
5+
```mermaid
6+
graph TB
7+
subgraph "UI Thread (Win32)"
8+
A[Win32 Message Loop] --> B[Handle GUI Events]
9+
B --> C[Execute asyncio Tasks]
10+
C --> D[Release Semaphore]
11+
12+
M[TRIO_MSG/ASYNCIO_MSG Handler] --> N[Process Task Queue]
13+
N --> O[Run asyncio.run_once]
14+
O --> P[Release Semaphore]
15+
end
16+
17+
subgraph "Backend Thread (Asyncio)"
18+
E[Wait for Semaphore] --> F[Call epoll.poll]
19+
F --> G[Detect Events]
20+
G --> H[Request UI Processing]
21+
H --> E
22+
end
23+
24+
subgraph "Coordination Mechanisms"
25+
I[Semaphore/Lock]
26+
J[Asyncio Ready Task Queue]
27+
K[Wake Pipe]
28+
end
29+
30+
D --> I
31+
I --> E
32+
P --> I
33+
C --- J --- H
34+
35+
subgraph "Timer Events"
36+
L[asyncio.call_later] --> Q[wake_backend_4_timer]
37+
Q --> R[Write to Wake Pipe]
38+
R --> S[Wake Up Backend Thread]
39+
S --> T[Process Timer Event]
40+
end
41+
42+
R --> K --> F
43+
```
44+
45+
## Core Design Principles
46+
47+
### 1. Dual-Thread Model
48+
49+
- **UI Thread**: Responsible for Win32 message loop and GUI event handling
50+
- **Backend Thread**: Responsible for monitoring events in the asyncio event loop
51+
52+
This separation allows the GUI to remain responsive while asynchronous operations run in the background.
53+
54+
### 2. Thread Coordination Mechanisms
55+
56+
```mermaid
57+
flowchart LR
58+
A[UI Thread] -->|releases| B[Semaphore]
59+
B -->|acquired by| C[Backend Thread]
60+
C -->|monitors events| D[epoll]
61+
D -->|events occur| E[Task Queue]
62+
E -->|processed by| A
63+
F[Timer Events] -->|writes to| G[Wake Pipe]
64+
G -->|wakes up| D
65+
```
66+
67+
1. **Semaphore**: Ensures the two threads execute alternately, preventing race conditions
68+
2. **Task Queue**: Thread-safe queue for passing functions that need to be executed on the UI thread
69+
3. **Wake Pipe**: Allows the backend thread to be awakened when new events (like timers) occur
70+
71+
### 3. Message Passing Mechanisms
72+
73+
```mermaid
74+
sequenceDiagram
75+
participant B as Backend Thread
76+
participant Q as Task Queue
77+
participant M as Message Window
78+
participant U as UI Thread
79+
80+
B->>Q: Add function to queue
81+
B->>M: PostMessage(TRIO_MSG/ASYNCIO_MSG)
82+
M->>U: Process message
83+
U->>Q: Get and execute tasks
84+
U->>U: Run asyncio.run_once()
85+
```
86+
87+
1. **TRIO_MSG/ASYNCIO_MSG**: Custom Win32 message used to trigger asyncio task processing in the UI thread
88+
2. **PostMessage**: Safely sends messages from the backend thread to the UI thread
89+
3. **run_sync_soon_threadsafe**: Adds functions to the queue and notifies the UI thread to execute them
90+
91+
## Key Event Flows
92+
93+
### 1. Initialization Process
94+
95+
```mermaid
96+
sequenceDiagram
97+
participant U as UI Thread
98+
participant S as Semaphore
99+
participant B as Backend Thread
100+
101+
U->>U: Create coordination mechanisms
102+
U->>B: Start thread
103+
B->>S: Wait for semaphore
104+
U->>U: run_events_on_ui_thread()
105+
U->>S: Release semaphore
106+
S->>B: Semaphore acquired
107+
B->>B: Begin monitoring
108+
```
109+
110+
### 2. Normal Event Loop
111+
112+
```mermaid
113+
sequenceDiagram
114+
participant U as UI Thread
115+
participant Q as Task Queue
116+
participant S as Semaphore
117+
participant B as Backend Thread
118+
119+
U->>S: Release semaphore
120+
S->>B: Acquire semaphore
121+
B->>B: Call epoll.poll(timeout)
122+
Note over B: Event occurs
123+
B->>Q: Add task to queue
124+
B->>U: Send TRIO_MSG/ASYNCIO_MSG
125+
U->>Q: Process all tasks
126+
U->>U: Run asyncio.run_once()
127+
U->>S: Release semaphore
128+
S->>B: Acquire semaphore (loop continues)
129+
```
130+
131+
### 3. New Timer Event Flow
132+
133+
```mermaid
134+
sequenceDiagram
135+
participant U as UI Thread
136+
participant T as Asyncio Timer
137+
participant W as Wake Pipe
138+
participant B as Backend Thread
139+
140+
Note over U: loop.call_later() called
141+
U->>T: Create new timer
142+
T->>W: Write data to wake pipe
143+
W->>B: Wake up from epoll.poll()
144+
B->>U: Request process timer events
145+
U->>U: Process timer callbacks
146+
```
147+
148+
### 4. Timeout Handling Flow
149+
150+
```mermaid
151+
sequenceDiagram
152+
participant U as UI Thread
153+
participant B as Backend Thread
154+
155+
B->>B: Get timeout from loop.get_backend_timeout()
156+
B->>B: epoll.poll(timeout)
157+
Note over B: Timeout occurs
158+
B->>U: Request task processing
159+
U->>U: Execute expired timer callbacks
160+
```
161+
162+
### 5. Task Completion Flow
163+
164+
```mermaid
165+
sequenceDiagram
166+
participant AT as Asyncio Task
167+
participant CB as Completion Callback
168+
participant U as UI Thread
169+
170+
AT->>AT: Task completes
171+
AT->>CB: Call on_task_done
172+
CB->>U: Send result via run_sync_soon_threadsafe
173+
U->>U: Call done_callback
174+
```
175+
176+
## Implementation Considerations
177+
178+
1. **Error Handling**: Both backend and UI threads need robust exception catching and handling
179+
2. **Resource Cleanup**: Properly close event loop, pipes, and epoll when tasks complete
180+
3. **Compatibility**: Handle API differences between asyncio versions
181+
4. **Deadlock Prevention**: Ensure semaphores are correctly released in all paths
182+
5. **Timer Precision**: Use wake mechanism to ensure timers are processed promptly
183+
184+
## Key Function Descriptions
185+
186+
### asyncio_guest_run
187+
188+
Main entry function responsible for setting up the dual-thread environment and starting the asyncio task.
189+
190+
### run_events_on_ui_thread
191+
192+
Executes one cycle of the asyncio event loop on the UI thread, handling ready callbacks and I/O events.
193+
194+
### backend_thread_loop
195+
196+
Main loop of the backend thread, responsible for monitoring file descriptor events and timer timeouts.
197+
198+
### wake_backend_4_timer
199+
200+
Wakes up the backend thread when new timers are created, ensuring timer events are processed promptly.
201+
202+
## Summary
203+
204+
This design coordinates the Win32 message loop and asyncio event loop through careful orchestration, enabling the use of Python's asynchronous programming in GUI applications. Semaphores ensure synchronization between threads, while wake mechanisms and message queues provide efficient communication channels, solving the limitations of the traditional single-threaded asyncio model in GUI environments.

‎readme.md

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Implement asyncio guest mode and with sample code work with it
2+
3+
## Purpose
4+
5+
1. Make asyncio work with all GUI frameworks, sample code be implemented in tornado, pygame, tkinter, gtk, qt5, win32, pyside6
6+
2. Make webview_python library support call async python from javascript
7+
a. https://github.com/congzhangzh/webview_python
8+
b. https://github.com/congzhangzh/webview_python/issues/1
9+
10+
### Design
11+
12+
## Solution inspire by:
13+
1. https://www.electronjs.org/blog/electron-internals-node-integration
14+
2. https://trio.readthedocs.io/en/stable/reference-lowlevel.html#using-guest-mode-to-run-trio-on-top-of-other-event-loops
15+
3. https://github.com/sunoru/Webviews.jl/blob/main/src/platforms/windows/Impl.jl#L59
16+
17+
## Samples inspire by
18+
1. https://github.com/richardsheridan/trio-guest
19+
20+
## Discuss with other guys
21+
1. https://discuss.python.org/t/connecting-asyncio-and-tkinter-event-loops/14722/33
22+
2. https://github.com/congzhangzh/webview_python/issues/1
23+
3. https://github.com/webview/webview_deno/issues/185
24+
25+
## Some early attempts:
26+
1. https://github.com/congzhangzh/python_gui_with_asyncio
27+
2. Where all important iteration happend: https://github.com/congzhangzh/asyncio_guest_run/tree/main/v2

0 commit comments

Comments
 (0)