Releases: benoitc/erlang-python
v2.2.0
Added
-
OWN_GIL Mode - True parallel Python execution with Python 3.14+ subinterpreters. Each subinterpreter runs with its own GIL in a dedicated thread, enabling true parallelism for CPU-bound workloads.
-
Process-Bound Python Environments - Per-Erlang-process Python namespaces with isolated globals/locals that persist across calls.
-
Event Loop Pool -
py_event_loop_pooldistributes async tasks with scheduler-affinity routing. -
ByteChannel API - Raw byte streaming without term serialization. Ideal for HTTP bodies, file streaming, binary protocols.
-
PyBuffer API - Zero-copy buffer for WSGI input streams with file-like interface.
-
True streaming API -
py:stream_start/3,4andpy:stream_cancel/1for event-driven streaming from Python generators. -
erlang.whereis(name)- Lookup registered Erlang PIDs from Python. -
erlang.schedule_inline(callback)- Inline continuation scheduling. -
py:spawn_call/3,4,5- Fire-and-forget with result delivery. -
Explicit bytes conversion -
{bytes, Binary}tuple for round-trip safety. -
Import caching API -
py:import/1,2,py:add_import/1,2,py:add_path/1. -
Per-interpreter preload code - Execute code in new interpreters with inherited globals.
Fixed
- Channel notification for create_task
- Channel waiter race condition
- Event loop isolation and resource safety
- Python 3.14 venv activation
- OWN_GIL safety fixes (mutex leak, deadlock prevention, env validation)
Changed
py:castis now fire-and-forget (usepy:spawn_callfor results)- OWN_GIL requires Python 3.14+
- Removed auto-started io pool
- Removed py_event_router
- Config-based initialization for imports/paths
Performance
- Direct NIF channel operations (up to 1760x speedup)
- nif_process_ready_tasks optimization (~15% improvement)
See CHANGELOG.md for full details.
v2.1.0 - Async Task API
Added
-
Async Task API - uvloop-inspired task submission from Erlang
py_event_loop:run/3,4- Blocking run of async Python functionspy_event_loop:create_task/3,4- Non-blocking task submission with referencepy_event_loop:await/1,2- Wait for task result with timeoutpy_event_loop:spawn_task/3,4- Fire-and-forget task execution- Thread-safe submission via
enif_send(works from dirty schedulers) - See Async Task API docs
-
erlang.spawn_task(coro)- Spawn async tasks from sync and async contexts- Works where
asyncio.get_running_loop()fails - Returns
asyncio.Taskfor optional await/cancel
- Works where
-
Explicit Scheduling API - Control dirty scheduler release from Python
erlang.schedule(callback, *args)- Release scheduler, continue via Erlang callbackerlang.schedule_py(module, func, args, kwargs)- Release scheduler, continue in Pythonerlang.consume_time_slice(percent)- Check if NIF time slice exhaustedScheduleMarkertype for cooperative long-running tasks
-
Distributed Python Execution - Run Python across Erlang nodes
- Documentation and Docker-based demo
- See Distributed Execution docs
Changed
- Event Loop Performance
- Growable pending queue (256 to 16384)
- Snapshot-detach pattern to reduce mutex contention
- Callable cache (64 slots) avoids PyImport/GetAttr per task
- Task wakeup coalescing
Fixed
ensure_venvalways installs deps, even if venv existserlang.sleep()timing in sync contexttime()returns fresh value when loop not running- Handle pooling bugs in ErlangEventLoop
- Task wakeup race causing batch task stalls
v2.0.0
Highlights
- Dual Pool Support - Separate pools for CPU-bound and I/O-bound operations with registration-based routing
- Channel API - Bidirectional message passing between Erlang and Python (8x faster than Reactor for small messages)
- OWN_GIL Subinterpreter Thread Pool - True parallelism with Python 3.12+ subinterpreters
- Reactor API - FD-based protocol handling for building custom servers
- Virtual Environment Management - Automatic venv creation with
py:ensure_venv/2,3
Added
py:ensure_venv/2,3- Automatic venv creation and activationpy:dup_fd/1- Safe socket handoff from Erlang to Python- Dual pool support (
defaultandiopools) with registration-based routing - Channel API (
py_channel) for bidirectional message passing - OWN_GIL subinterpreter thread pool for true parallelism
erlang.reactormodule for FD-based protocol handling- ETF encoding for PIDs and References
erlang.send(pid, term)for fire-and-forget message passing- Audit hook sandbox blocking fork/exec operations
- Process-per-context architecture
Changed
py:call_asyncrenamed topy:cast- Unified
erlangPython module (removed separateerlang_asyncio) - Async worker backend replaced with event loop model
SuspensionRequirednow inherits fromBaseException
Deprecated
py_asgimodule - use Channel API or Reactor API insteadpy_wsgimodule - use Channel API or Reactor API instead
Removed
- Context affinity functions (
py:bind,py:unbind,py:is_bound,py:with_context,py:ctx_*) - Signal handling support in ErlangEventLoop
- Subprocess support in ErlangEventLoop
Fixed
- Reactor context extending erlang module in subinterpreters
- FD stealing and UDP connected socket issues
- Timer scheduling for standalone ErlangEventLoop
- Subinterpreter cleanup and thread worker re-registration
- ProcessError exception class identity in subinterpreters
See CHANGELOG.md for full details.
v1.8.1
Fixed
- ASGI scope caching bug - HTTP method was not treated as a dynamic field in the scope template cache. This caused incorrect method values when the same path was accessed with different HTTP methods (e.g., GET /path followed by POST /path would return method="GET" for both requests).
v1.8.0
Added
-
ASGI NIF Optimizations - Six optimizations for high-performance ASGI request handling
- Direct Response Tuple Extraction - Extract
(status, headers, body)directly without generic conversion - Pre-Interned Header Names - 16 common HTTP headers cached as PyBytes objects
- Cached Status Code Integers - 14 common HTTP status codes cached as PyLong objects
- Zero-Copy Request Body - Large bodies (≥1KB) use buffer protocol for zero-copy access
- Scope Template Caching - Thread-local cache of 64 scope templates keyed by path hash
- Lazy Header Conversion - Headers converted on-demand for requests with ≥4 headers
- Direct Response Tuple Extraction - Extract
-
erlang_asyncio Module - Asyncio-compatible primitives using Erlang's native scheduler
erlang_asyncio.sleep(delay, result=None)- Sleep using Erlang'serlang:send_after/3erlang_asyncio.run(coro)- Run coroutine with ErlangEventLooperlang_asyncio.gather(*coros)- Run coroutines concurrentlyerlang_asyncio.wait_for(coro, timeout)- Wait with timeouterlang_asyncio.wait(fs, timeout, return_when)- Wait for multiple futureserlang_asyncio.create_task(coro)- Create background task- Event loop functions:
get_event_loop(),new_event_loop(),set_event_loop(),get_running_loop()
-
Erlang Sleep NIF - Synchronous sleep primitive for Python
py_event_loop._erlang_sleep(delay_ms)- Sleep using Erlang timer- Releases GIL during sleep, no Python event loop overhead
-
Scalable I/O Model - Worker-per-context architecture
py_event_worker- Dedicated worker process per Python context- Combined FD event dispatch and reselect via
handle_fd_event_and_reselectNIF
-
New Test Suite -
test/py_erlang_sleep_SUITE.erlwith 8 tests
Performance
- ASGI marshalling optimizations - 40-60% improvement for typical ASGI workloads
- Eliminates event loop overhead for sleep operations (~0.5-1ms saved per call)
- Sub-millisecond timer precision via BEAM scheduler (vs 10ms asyncio polling)
- Zero CPU when idle - event-driven, no polling
See CHANGELOG.md for full details.
v1.7.1
v1.7.0
Added
-
Shared Router Architecture for Event Loops
- Single
py_event_routerprocess handles all event loops (both shared and isolated) - Timer and FD messages include loop identity for correct dispatch
- Eliminates need for per-loop router processes
- Handle-based Python C API using PyCapsule for loop references
- Single
-
Isolated Event Loops - Create isolated event loops with
ErlangEventLoop(isolated=True)- Default (
isolated=False): uses the shared global loop managed by Erlang - Isolated (
isolated=True): creates a dedicated loop with its own pending queue - Full asyncio support (timers, FD operations) for both modes
- Useful for multi-threaded Python applications where each thread needs its own loop
- See
docs/asyncio.mdfor usage and architecture details
- Default (
v1.6.1
Fixed
- ASGI headers now correctly use bytes instead of str - Fixed ASGI spec compliance issue where headers were being converted to Python
strobjects instead ofbytes. The ASGI specification requires headers to belist[tuple[bytes, bytes]]. This was causing authentication failures and form parsing issues with frameworks like Starlette and FastAPI, which search for headers using bytes keys (e.g.,b"content-type").- Added explicit header handling in
asgi_scope_from_map()to bypass generic conversion - Headers are now correctly converted using
PyBytes_FromStringAndSize() - Supports both list
[name, value]and tuple{name, value}header formats from Erlang - Fixes GitHub issue #1
- Added explicit header handling in
v1.6.0
Added
-
Python Logging Integration - Forward Python's
loggingmodule to Erlang'sloggerpy:configure_logging/0,1- Setup Python logging to forward to Erlangerlang.ErlangHandler- Python logging handler that sends to Erlangerlang.setup_logging(level, format)- Configure logging from Python- Fire-and-forget architecture using
enif_send()for non-blocking messaging - Level filtering at NIF level for performance
- Thread-safe - works from any Python thread
-
Distributed Tracing - Collect trace spans from Python code
py:enable_tracing/0,py:disable_tracing/0- Enable/disable span collectionpy:get_traces/0- Retrieve collected spanserlang.Span(name, **attrs)- Context manager for creating spanserlang.trace(name)- Decorator for tracing functions- Automatic parent/child span linking via thread-local storage
-
New Erlang modules:
py_logger,py_tracer
Performance
-
Type conversion optimizations - Faster Python ↔ Erlang marshalling
- Use
enif_is_identicalfor atom comparison instead ofstrcmp - Stack allocate small tuples/maps (≤16 elements) to avoid heap allocation
- Use
enif_make_map_from_arraysfor O(n) map building vs O(n²) puts
- Use
-
Fire-and-forget NIF architecture - Log and trace calls never block Python execution
Fixed
-
Python 3.12+ event loop thread isolation - Fixed asyncio timeouts on Python 3.12+
ErlangEventLoopnow only used for main thread; worker threads getSelectorEventLoop- Per-call
ErlNifEnvfor thread-safe timer scheduling in free-threaded mode - Fail-fast error handling in
erlang_loop.pyinstead of silent hangs - Added
gil_acquire()/gil_release()helpers to avoid GIL double-acquisition
-
Intermittent test failures on free-threaded Python - Added startup synchronization
v1.5.0
Added
-
py_asgimodule - Optimized ASGI request handling with:- Pre-interned Python string keys (15+ ASGI scope keys)
- Cached constant values (http type, HTTP versions, methods, schemes)
- Thread-local response pooling (16 slots per thread, 4KB initial buffer)
- Direct NIF path bypassing generic py:call()
- ~60-80% throughput improvement over py:call()
- Configurable runner module via
runneroption - Sub-interpreter and free-threading (Python 3.13+) support
-
py_wsgimodule - Optimized WSGI request handling with:- Pre-interned WSGI environ keys
- Direct NIF path for marshalling
- ~60-80% throughput improvement over py:call()
- Sub-interpreter and free-threading support
-
Web frameworks documentation - New documentation at
docs/web-frameworks.md