WASM Rendering Backend: Transition to Canvas 2D Direct / WebGL

[jserv]

Background

The current WebAssembly rendering backend relies on a framebuffer-based model, where pixel buffers are produced in WASM memory and then copied into a browser rendering surface. While functionally correct, this approach introduces avoidable overhead:

• Repeated memory copies between WASM and JavaScript
• Increased pressure on the JS–WASM boundary
• Limited ability to leverage browser-native graphics acceleration paths

As the WebAssembly port matures, these costs increasingly dominate rendering time and constrain scalability.

Problem Statement

The framebuffer-based rendering path is suboptimal for browser environments. It was originally designed for portability, not for tight integration with browser graphics APIs. As a result:

• Per-frame latency is higher than necessary
• Performance does not scale well with resolution or frame rate
• Advanced rendering paths (e.g., GPU acceleration) are harder to integrate cleanly

A more browser-native rendering strategy is required.

Design Goals

This proposal aims to:

• Reduce per-frame overhead in the WASM rendering pipeline
• Minimize unnecessary memory copies
• Improve rendering performance and responsiveness
• Preserve broad browser compatibility
• Provide a clear evolution path toward GPU-accelerated rendering

Proposed Design

We propose transitioning the rendering backend to use Canvas 2D Direct as the primary rendering path, with WebGL as an optional acceleration backend.

Canvas 2D Direct as Baseline

Canvas 2D Direct offers a simple, well-supported API that integrates cleanly with WebAssembly via JavaScript glue code (e.g., EM_ASM). Key properties:

• Direct rendering to the browser’s 2D canvas context
• Avoids intermediate framebuffer copies where possible
• Supported by all modern browsers, including IE11 and evergreen browsers

This makes Canvas 2D Direct the default, lowest-common-denominator solution.

Optional WebGL Backend

For environments where GPU acceleration is desired, WebGL (specifically WebGL2 where available) can be layered on top of the same abstraction:

• Enables hardware-accelerated rendering
• Suitable for higher resolutions or complex visual effects
• Available in modern browsers since approximately 2015

WebGL is explicitly optional and not required for correctness.

Compatibility Considerations

• Canvas 2D Direct: Works across all modern browsers, including legacy enterprise environments.
• SharedArrayBuffer: Requires HTTPS and appropriate COOP/COEP headers. Usage remains optional and orthogonal to this proposal.
• WebGL2: Available on modern browsers; graceful fallback to Canvas 2D Direct is required.

The proposed design prioritizes correctness and portability first, with acceleration as an additive feature.

Expected Benefits

• Lower rendering overhead compared to framebuffer-based rendering
• Reduced memory bandwidth usage
• Improved frame latency and responsiveness
• Cleaner separation between rendering policy and backend implementation
• Easier future extensions (e.g., WebGPU, offscreen canvas)

Alternatives Considered

• Retain framebuffer-only rendering: Simple but increasingly inefficient; does not address core performance issues.
• WebGL-only rendering: Higher performance potential but significantly reduces compatibility and increases implementation complexity.
• SharedArrayBuffer-based pixel sharing: Useful in specific cases but constrained by security requirements and deployment complexity.

The proposed Canvas 2D Direct baseline offers the best balance between performance, simplicity, and compatibility.

References

• MDN Canvas API
• MDN WebGL API
• Emscripten `EM_ASM