Skip to content

Releases: benoitc/erlang-python

v2.2.0

24 Mar 01:36

Choose a tag to compare

Added

  • OWN_GIL Mode - True parallel Python execution with Python 3.14+ subinterpreters. Each subinterpreter runs with its own GIL in a dedicated thread, enabling true parallelism for CPU-bound workloads.

  • Process-Bound Python Environments - Per-Erlang-process Python namespaces with isolated globals/locals that persist across calls.

  • Event Loop Pool - py_event_loop_pool distributes async tasks with scheduler-affinity routing.

  • ByteChannel API - Raw byte streaming without term serialization. Ideal for HTTP bodies, file streaming, binary protocols.

  • PyBuffer API - Zero-copy buffer for WSGI input streams with file-like interface.

  • True streaming API - py:stream_start/3,4 and py:stream_cancel/1 for event-driven streaming from Python generators.

  • erlang.whereis(name) - Lookup registered Erlang PIDs from Python.

  • erlang.schedule_inline(callback) - Inline continuation scheduling.

  • py:spawn_call/3,4,5 - Fire-and-forget with result delivery.

  • Explicit bytes conversion - {bytes, Binary} tuple for round-trip safety.

  • Import caching API - py:import/1,2, py:add_import/1,2, py:add_path/1.

  • Per-interpreter preload code - Execute code in new interpreters with inherited globals.

Fixed

  • Channel notification for create_task
  • Channel waiter race condition
  • Event loop isolation and resource safety
  • Python 3.14 venv activation
  • OWN_GIL safety fixes (mutex leak, deadlock prevention, env validation)

Changed

  • py:cast is now fire-and-forget (use py:spawn_call for results)
  • OWN_GIL requires Python 3.14+
  • Removed auto-started io pool
  • Removed py_event_router
  • Config-based initialization for imports/paths

Performance

  • Direct NIF channel operations (up to 1760x speedup)
  • nif_process_ready_tasks optimization (~15% improvement)

See CHANGELOG.md for full details.

v2.1.0 - Async Task API

12 Mar 12:28

Choose a tag to compare

Added

  • Async Task API - uvloop-inspired task submission from Erlang

    • py_event_loop:run/3,4 - Blocking run of async Python functions
    • py_event_loop:create_task/3,4 - Non-blocking task submission with reference
    • py_event_loop:await/1,2 - Wait for task result with timeout
    • py_event_loop:spawn_task/3,4 - Fire-and-forget task execution
    • Thread-safe submission via enif_send (works from dirty schedulers)
    • See Async Task API docs
  • erlang.spawn_task(coro) - Spawn async tasks from sync and async contexts

    • Works where asyncio.get_running_loop() fails
    • Returns asyncio.Task for optional await/cancel
  • Explicit Scheduling API - Control dirty scheduler release from Python

    • erlang.schedule(callback, *args) - Release scheduler, continue via Erlang callback
    • erlang.schedule_py(module, func, args, kwargs) - Release scheduler, continue in Python
    • erlang.consume_time_slice(percent) - Check if NIF time slice exhausted
    • ScheduleMarker type for cooperative long-running tasks
  • Distributed Python Execution - Run Python across Erlang nodes

Changed

  • Event Loop Performance
    • Growable pending queue (256 to 16384)
    • Snapshot-detach pattern to reduce mutex contention
    • Callable cache (64 slots) avoids PyImport/GetAttr per task
    • Task wakeup coalescing

Fixed

  • ensure_venv always installs deps, even if venv exists
  • erlang.sleep() timing in sync context
  • time() returns fresh value when loop not running
  • Handle pooling bugs in ErlangEventLoop
  • Task wakeup race causing batch task stalls

v2.0.0

09 Mar 14:28

Choose a tag to compare

Highlights

  • Dual Pool Support - Separate pools for CPU-bound and I/O-bound operations with registration-based routing
  • Channel API - Bidirectional message passing between Erlang and Python (8x faster than Reactor for small messages)
  • OWN_GIL Subinterpreter Thread Pool - True parallelism with Python 3.12+ subinterpreters
  • Reactor API - FD-based protocol handling for building custom servers
  • Virtual Environment Management - Automatic venv creation with py:ensure_venv/2,3

Added

  • py:ensure_venv/2,3 - Automatic venv creation and activation
  • py:dup_fd/1 - Safe socket handoff from Erlang to Python
  • Dual pool support (default and io pools) with registration-based routing
  • Channel API (py_channel) for bidirectional message passing
  • OWN_GIL subinterpreter thread pool for true parallelism
  • erlang.reactor module for FD-based protocol handling
  • ETF encoding for PIDs and References
  • erlang.send(pid, term) for fire-and-forget message passing
  • Audit hook sandbox blocking fork/exec operations
  • Process-per-context architecture

Changed

  • py:call_async renamed to py:cast
  • Unified erlang Python module (removed separate erlang_asyncio)
  • Async worker backend replaced with event loop model
  • SuspensionRequired now inherits from BaseException

Deprecated

  • py_asgi module - use Channel API or Reactor API instead
  • py_wsgi module - use Channel API or Reactor API instead

Removed

  • Context affinity functions (py:bind, py:unbind, py:is_bound, py:with_context, py:ctx_*)
  • Signal handling support in ErlangEventLoop
  • Subprocess support in ErlangEventLoop

Fixed

  • Reactor context extending erlang module in subinterpreters
  • FD stealing and UDP connected socket issues
  • Timer scheduling for standalone ErlangEventLoop
  • Subinterpreter cleanup and thread worker re-registration
  • ProcessError exception class identity in subinterpreters

See CHANGELOG.md for full details.

v1.8.1

25 Feb 02:56

Choose a tag to compare

Fixed

  • ASGI scope caching bug - HTTP method was not treated as a dynamic field in the scope template cache. This caused incorrect method values when the same path was accessed with different HTTP methods (e.g., GET /path followed by POST /path would return method="GET" for both requests).

v1.8.0

25 Feb 01:23

Choose a tag to compare

Added

  • ASGI NIF Optimizations - Six optimizations for high-performance ASGI request handling

    • Direct Response Tuple Extraction - Extract (status, headers, body) directly without generic conversion
    • Pre-Interned Header Names - 16 common HTTP headers cached as PyBytes objects
    • Cached Status Code Integers - 14 common HTTP status codes cached as PyLong objects
    • Zero-Copy Request Body - Large bodies (≥1KB) use buffer protocol for zero-copy access
    • Scope Template Caching - Thread-local cache of 64 scope templates keyed by path hash
    • Lazy Header Conversion - Headers converted on-demand for requests with ≥4 headers
  • erlang_asyncio Module - Asyncio-compatible primitives using Erlang's native scheduler

    • erlang_asyncio.sleep(delay, result=None) - Sleep using Erlang's erlang:send_after/3
    • erlang_asyncio.run(coro) - Run coroutine with ErlangEventLoop
    • erlang_asyncio.gather(*coros) - Run coroutines concurrently
    • erlang_asyncio.wait_for(coro, timeout) - Wait with timeout
    • erlang_asyncio.wait(fs, timeout, return_when) - Wait for multiple futures
    • erlang_asyncio.create_task(coro) - Create background task
    • Event loop functions: get_event_loop(), new_event_loop(), set_event_loop(), get_running_loop()
  • Erlang Sleep NIF - Synchronous sleep primitive for Python

    • py_event_loop._erlang_sleep(delay_ms) - Sleep using Erlang timer
    • Releases GIL during sleep, no Python event loop overhead
  • Scalable I/O Model - Worker-per-context architecture

    • py_event_worker - Dedicated worker process per Python context
    • Combined FD event dispatch and reselect via handle_fd_event_and_reselect NIF
  • New Test Suite - test/py_erlang_sleep_SUITE.erl with 8 tests

Performance

  • ASGI marshalling optimizations - 40-60% improvement for typical ASGI workloads
  • Eliminates event loop overhead for sleep operations (~0.5-1ms saved per call)
  • Sub-millisecond timer precision via BEAM scheduler (vs 10ms asyncio polling)
  • Zero CPU when idle - event-driven, no polling

See CHANGELOG.md for full details.

v1.7.1

22 Feb 23:51

Choose a tag to compare

Fixed

  • Hex package missing priv directory - Added explicit files configuration to include priv/erlang_loop.py and other necessary files in the hex.pm package

v1.7.0

22 Feb 23:38

Choose a tag to compare

Added

  • Shared Router Architecture for Event Loops

    • Single py_event_router process handles all event loops (both shared and isolated)
    • Timer and FD messages include loop identity for correct dispatch
    • Eliminates need for per-loop router processes
    • Handle-based Python C API using PyCapsule for loop references
  • Isolated Event Loops - Create isolated event loops with ErlangEventLoop(isolated=True)

    • Default (isolated=False): uses the shared global loop managed by Erlang
    • Isolated (isolated=True): creates a dedicated loop with its own pending queue
    • Full asyncio support (timers, FD operations) for both modes
    • Useful for multi-threaded Python applications where each thread needs its own loop
    • See docs/asyncio.md for usage and architecture details

v1.6.1

22 Feb 22:01
7326786

Choose a tag to compare

Fixed

  • ASGI headers now correctly use bytes instead of str - Fixed ASGI spec compliance issue where headers were being converted to Python str objects instead of bytes. The ASGI specification requires headers to be list[tuple[bytes, bytes]]. This was causing authentication failures and form parsing issues with frameworks like Starlette and FastAPI, which search for headers using bytes keys (e.g., b"content-type").
    • Added explicit header handling in asgi_scope_from_map() to bypass generic conversion
    • Headers are now correctly converted using PyBytes_FromStringAndSize()
    • Supports both list [name, value] and tuple {name, value} header formats from Erlang
    • Fixes GitHub issue #1

v1.6.0

22 Feb 20:57

Choose a tag to compare

Added

  • Python Logging Integration - Forward Python's logging module to Erlang's logger

    • py:configure_logging/0,1 - Setup Python logging to forward to Erlang
    • erlang.ErlangHandler - Python logging handler that sends to Erlang
    • erlang.setup_logging(level, format) - Configure logging from Python
    • Fire-and-forget architecture using enif_send() for non-blocking messaging
    • Level filtering at NIF level for performance
    • Thread-safe - works from any Python thread
  • Distributed Tracing - Collect trace spans from Python code

    • py:enable_tracing/0, py:disable_tracing/0 - Enable/disable span collection
    • py:get_traces/0 - Retrieve collected spans
    • erlang.Span(name, **attrs) - Context manager for creating spans
    • erlang.trace(name) - Decorator for tracing functions
    • Automatic parent/child span linking via thread-local storage
  • New Erlang modules: py_logger, py_tracer

Performance

  • Type conversion optimizations - Faster Python ↔ Erlang marshalling

    • Use enif_is_identical for atom comparison instead of strcmp
    • Stack allocate small tuples/maps (≤16 elements) to avoid heap allocation
    • Use enif_make_map_from_arrays for O(n) map building vs O(n²) puts
  • Fire-and-forget NIF architecture - Log and trace calls never block Python execution

Fixed

  • Python 3.12+ event loop thread isolation - Fixed asyncio timeouts on Python 3.12+

    • ErlangEventLoop now only used for main thread; worker threads get SelectorEventLoop
    • Per-call ErlNifEnv for thread-safe timer scheduling in free-threaded mode
    • Fail-fast error handling in erlang_loop.py instead of silent hangs
    • Added gil_acquire()/gil_release() helpers to avoid GIL double-acquisition
  • Intermittent test failures on free-threaded Python - Added startup synchronization

v1.5.0

18 Feb 09:45

Choose a tag to compare

Added

  • py_asgi module - Optimized ASGI request handling with:

    • Pre-interned Python string keys (15+ ASGI scope keys)
    • Cached constant values (http type, HTTP versions, methods, schemes)
    • Thread-local response pooling (16 slots per thread, 4KB initial buffer)
    • Direct NIF path bypassing generic py:call()
    • ~60-80% throughput improvement over py:call()
    • Configurable runner module via runner option
    • Sub-interpreter and free-threading (Python 3.13+) support
  • py_wsgi module - Optimized WSGI request handling with:

    • Pre-interned WSGI environ keys
    • Direct NIF path for marshalling
    • ~60-80% throughput improvement over py:call()
    • Sub-interpreter and free-threading support
  • Web frameworks documentation - New documentation at docs/web-frameworks.md