Skip to content

Commit f57f30b

Browse files
benoitcbenoitc
committed
Add OWN_GIL internals documentation
1 parent 68edf93 commit f57f30b

1 file changed

Lines changed: 282 additions & 0 deletions

File tree

‎docs/owngil_internals.md‎

Lines changed: 282 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,282 @@
1+
# OWN_GIL Mode Internals
2+
3+
## Overview
4+
5+
OWN_GIL mode provides true parallel Python execution using Python 3.12+ per-interpreter GIL (`PyInterpreterConfig_OWN_GIL`). Each OWN_GIL context runs in a dedicated pthread with its own subinterpreter and GIL.
6+
7+
## Architecture
8+
9+
```
10+
┌─────────────────────────────────────────────────────────────────────┐
11+
│ Erlang VM │
12+
├─────────────────────────────────────────────────────────────────────┤
13+
│ │
14+
│ Process A Process B │
15+
│ py_context:call(Ctx1, ...) py_context:call(Ctx2, ...) │
16+
│ │ │ │
17+
│ ▼ ▼ │
18+
│ ┌─────────────┐ ┌─────────────┐ │
19+
│ │ Dirty Sched │ │ Dirty Sched │ │
20+
│ └──────┬──────┘ └──────┬──────┘ │
21+
│ │ │ │
22+
└──────────┼───────────────────────────┼──────────────────────────────┘
23+
│ │
24+
│ dispatch_to_owngil_thread │
25+
▼ ▼
26+
┌──────────────────────┐ ┌──────────────────────┐
27+
│ OWN_GIL Thread 1 │ │ OWN_GIL Thread 2 │
28+
│ ┌────────────────┐ │ │ ┌────────────────┐ │
29+
│ │ Subinterpreter │ │ │ │ Subinterpreter │ │
30+
│ │ (own GIL) │ │ │ │ (own GIL) │ │
31+
│ └────────────────┘ │ └──┴────────────────┘ │
32+
│ Parallel Execution! │ │ Parallel Execution! │
33+
└──────────────────────┘ └──────────────────────┘
34+
```
35+
36+
## Comparison with Other Modes
37+
38+
| Mode | Thread Model | GIL | Parallelism |
39+
|------|-------------|-----|-------------|
40+
| `worker` | Dirty scheduler | Main interpreter GIL | None |
41+
| `subinterp` | Dirty scheduler | Shared GIL | None (isolated namespaces) |
42+
| `owngil` | Dedicated pthread | Per-interpreter GIL | True parallel |
43+
44+
## Key Data Structures
45+
46+
### py_context_t (OWN_GIL fields)
47+
48+
```c
49+
typedef struct {
50+
// ... common fields ...
51+
52+
bool uses_own_gil; // OWN_GIL mode flag
53+
pthread_t own_gil_thread; // Dedicated pthread
54+
PyThreadState *own_gil_tstate; // Thread state
55+
PyInterpreterState *own_gil_interp; // Interpreter state
56+
57+
// IPC synchronization
58+
pthread_mutex_t request_mutex;
59+
pthread_cond_t request_ready; // Signal: request available
60+
pthread_cond_t response_ready; // Signal: response ready
61+
62+
// Request/response state
63+
int request_type; // CTX_REQ_* enum
64+
ErlNifEnv *shared_env; // Zero-copy term passing
65+
ERL_NIF_TERM request_term;
66+
ERL_NIF_TERM response_term;
67+
bool response_ok;
68+
69+
// Process-local env support
70+
void *local_env_ptr; // py_env_resource_t*
71+
72+
// Lifecycle
73+
_Atomic bool thread_running;
74+
_Atomic bool shutdown_requested;
75+
} py_context_t;
76+
```
77+
78+
### Request Types
79+
80+
```c
81+
typedef enum {
82+
CTX_REQ_CALL, // Call Python function
83+
CTX_REQ_EVAL, // Evaluate expression
84+
CTX_REQ_EXEC, // Execute statements
85+
CTX_REQ_REACTOR_READ, // Reactor on_read_ready
86+
CTX_REQ_REACTOR_WRITE, // Reactor on_write_ready
87+
CTX_REQ_REACTOR_INIT, // Reactor init_connection
88+
CTX_REQ_CALL_WITH_ENV, // Call with process-local env
89+
CTX_REQ_EVAL_WITH_ENV, // Eval with process-local env
90+
CTX_REQ_EXEC_WITH_ENV, // Exec with process-local env
91+
CTX_REQ_CREATE_LOCAL_ENV,// Create process-local env dicts
92+
CTX_REQ_SHUTDOWN // Shutdown thread
93+
} ctx_request_type_t;
94+
```
95+
96+
## Request Flow
97+
98+
### 1. Context Creation
99+
100+
```
101+
nif_context_create(env, "owngil")
102+
└── owngil_context_init(ctx)
103+
├── Initialize mutex/condvars
104+
├── Create shared_env
105+
└── pthread_create(owngil_context_thread_main)
106+
└── owngil_context_thread_main(ctx)
107+
├── Py_NewInterpreterFromConfig(OWN_GIL)
108+
├── Initialize globals/locals
109+
├── Register py_event_loop module
110+
└── Enter request loop
111+
```
112+
113+
### 2. Request Dispatch
114+
115+
```
116+
nif_context_call(env, ctx, module, func, args, kwargs)
117+
│
118+
├── [ctx->uses_own_gil == true]
119+
│ └── dispatch_to_owngil_thread(env, ctx, CTX_REQ_CALL, request)
120+
│ ├── pthread_mutex_lock(&ctx->request_mutex)
121+
│ ├── Copy request term to shared_env
122+
│ ├── Set ctx->request_type = CTX_REQ_CALL
123+
│ ├── pthread_cond_signal(&ctx->request_ready)
124+
│ ├── pthread_cond_wait(&ctx->response_ready) // Block
125+
│ ├── Copy response from shared_env
126+
│ └── pthread_mutex_unlock(&ctx->request_mutex)
127+
│
128+
└── [ctx->uses_own_gil == false]
129+
└── Direct execution with GIL (worker/subinterp mode)
130+
```
131+
132+
### 3. Request Processing (OWN_GIL Thread)
133+
134+
```
135+
owngil_context_thread_main(ctx)
136+
while (!shutdown_requested) {
137+
pthread_cond_wait(&ctx->request_ready)
138+
139+
owngil_execute_request(ctx)
140+
switch (ctx->request_type) {
141+
case CTX_REQ_CALL: owngil_execute_call(ctx); break;
142+
case CTX_REQ_EVAL: owngil_execute_eval(ctx); break;
143+
case CTX_REQ_EXEC: owngil_execute_exec(ctx); break;
144+
// ... other cases
145+
}
146+
147+
pthread_cond_signal(&ctx->response_ready)
148+
}
149+
```
150+
151+
## Process-Local Environments
152+
153+
OWN_GIL contexts support process-local environments for namespace isolation:
154+
155+
```
156+
Erlang Process A Erlang Process B
157+
│ │
158+
▼ ▼
159+
┌───────────────┐ ┌───────────────┐
160+
│ py_env_res_t │ │ py_env_res_t │
161+
│ globals_A │ │ globals_B │
162+
│ locals_A │ │ locals_B │
163+
└───────┬───────┘ └───────┬───────┘
164+
│ │
165+
└─────────┬───────────────┘
166+
▼
167+
┌─────────────────────┐
168+
│ OWN_GIL Context │
169+
│ (shared context, │
170+
│ isolated envs) │
171+
└─────────────────────┘
172+
```
173+
174+
### Creating Process-Local Env
175+
176+
```
177+
py_context:create_local_env(Ctx)
178+
└── nif_create_local_env(CtxRef)
179+
└── dispatch_create_local_env_to_owngil(env, ctx, res)
180+
└── owngil_execute_create_local_env(ctx)
181+
├── res->globals = PyDict_New()
182+
├── res->locals = PyDict_New()
183+
└── res->interp_id = ctx->interp_id
184+
```
185+
186+
### Using Process-Local Env
187+
188+
```erlang
189+
{ok, Env} = py_context:create_local_env(Ctx),
190+
CtxRef = py_context:get_nif_ref(Ctx),
191+
ok = py_nif:context_exec(CtxRef, <<"x = 1">>, Env),
192+
{ok, 1} = py_nif:context_eval(CtxRef, <<"x">>, #{}, Env).
193+
```
194+
195+
## Thread Lifecycle
196+
197+
### Startup
198+
199+
1. `Py_NewInterpreterFromConfig` with `PyInterpreterConfig_OWN_GIL`
200+
2. Save thread state and interpreter state
201+
3. Initialize `__builtins__` in globals
202+
4. Register `py_event_loop` module for reactor callbacks
203+
5. Release GIL and enter request loop
204+
205+
### Request Loop
206+
207+
```c
208+
while (!shutdown_requested) {
209+
pthread_mutex_lock(&request_mutex);
210+
while (!request_pending && !shutdown_requested) {
211+
pthread_cond_wait(&request_ready, &request_mutex);
212+
}
213+
214+
if (shutdown_requested) break;
215+
216+
// Process request (GIL already held within subinterpreter)
217+
owngil_execute_request(ctx);
218+
219+
pthread_cond_signal(&response_ready);
220+
pthread_mutex_unlock(&request_mutex);
221+
}
222+
```
223+
224+
### Shutdown
225+
226+
1. Set `shutdown_requested = true`
227+
2. Signal `request_ready` to wake thread
228+
3. Thread exits loop, acquires GIL
229+
4. Call `Py_EndInterpreter` to destroy subinterpreter
230+
5. pthread terminates
231+
232+
## Memory Management
233+
234+
### Shared Environment
235+
236+
- `ctx->shared_env` is used for zero-copy term passing
237+
- Request terms copied into shared_env by caller
238+
- Response terms created in shared_env by OWN_GIL thread
239+
- Caller copies response back to their env
240+
241+
### Process-Local Env Cleanup
242+
243+
```c
244+
py_env_resource_dtor(env, res) {
245+
if (res->pool_slot >= 0) {
246+
// Shared-GIL subinterpreter: DECREF with pool GIL
247+
} else if (res->interp_id != 0) {
248+
// OWN_GIL subinterpreter: skip DECREF
249+
// Py_EndInterpreter cleans up all objects
250+
} else {
251+
// Worker mode: DECREF with main GIL
252+
}
253+
}
254+
```
255+
256+
## Performance Characteristics
257+
258+
| Operation | Shared-GIL | OWN_GIL |
259+
|-----------|-----------|---------|
260+
| Call overhead | ~2.5μs | ~10μs |
261+
| Throughput (single) | 400K/s | 100K/s |
262+
| Parallelism | None | True |
263+
| Resource usage | Lower | Higher (1 pthread per context) |
264+
265+
Use OWN_GIL when:
266+
- CPU-bound Python work that benefits from parallelism
267+
- Long-running computations
268+
- Need true concurrent Python execution
269+
270+
Use shared-GIL (subinterp) when:
271+
- I/O-bound or short operations
272+
- High call frequency
273+
- Resource constraints
274+
275+
## Files
276+
277+
| File | Description |
278+
|------|-------------|
279+
| `c_src/py_nif.h` | Structure definitions, request types |
280+
| `c_src/py_nif.c` | Thread main, dispatch, execute functions |
281+
| `src/py_context.erl` | Erlang API for context management |
282+
| `test/py_owngil_features_SUITE.erl` | Test suite |

0 commit comments

Comments
 (0)