Skip to content

[mypyc] Add librt.random module#21433

Merged
JukkaL merged 34 commits intomasterfrom
librt-random
May 8, 2026
Merged

[mypyc] Add librt.random module#21433
JukkaL merged 34 commits intomasterfrom
librt-random

Conversation

@JukkaL
Copy link
Copy Markdown
Collaborator

@JukkaL JukkaL commented May 7, 2026

The stdlib random module is fairly often used in performance critical code, and it's not super efficient. Add librt.random with a subset of the stdlib module interface that is optimized for performance when compiled.

Use ChaCha8 as the algorithm. Based on some research, this is a modern, high-quality PRNG algorithm. It's used by Go math/rand/v2, among others.

This is a non-cryptographic PRNG, similar to the stdlib random module (but this uses a different algorithm).

I used Claude Code and Codex to write all the code, but I iterated on it quite a lot and did a bunch of manual validation and code review. I also asked Codex to explicitly check that the ChaCha8 implementation is correct by comparing it to a reference implementation.

Use thread-local RNG state for module-level functions to enable good scaling in free-threaded builds. There's some extra complexity from having to free the state at thread exit.

I asked a LLM to generate and run a benchmark, and here are the results on 3.14:

  ┌─────────────┬─────────────────────┬────────────────────────┐
  │  Function   │ vs stdlib(compiled) │ vs stdlib(interpreted) │
  ├─────────────┼─────────────────────┼────────────────────────┤
  │ random()    │        3.2x         │          4.8x          │
  ├─────────────┼─────────────────────┼────────────────────────┤
  │ randint()   │        16.6x        │         18.0x          │
  ├─────────────┼─────────────────────┼────────────────────────┤
  │ randrange() │        14.5x        │         16.2x          │
  ├─────────────┼─────────────────────┼────────────────────────┤
  │ choice()    │        12.9x        │         10.3x          │
  └─────────────┴─────────────────────┴────────────────────────┘

choice() was replaced with randrange when using librt, since we don't provide it as part of this fairly minimal API.

@JukkaL JukkaL marked this pull request as draft May 7, 2026 13:55
@github-actions

This comment has been minimized.

@JukkaL JukkaL marked this pull request as ready for review May 8, 2026 10:28
@JukkaL JukkaL changed the title [WIP] Add librt.random [mypyc] Add librt.random May 8, 2026
@JukkaL JukkaL changed the title [mypyc] Add librt.random [mypyc] Add librt.random module May 8, 2026
@github-actions

This comment has been minimized.

Comment thread mypyc/build.py Outdated
ModDesc(
"librt.random",
["random/librt_random.c"],
["random/librt_random.h", "random/librt_random_api.h", "random/librt_random_api.c"],
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the _api files shouldn't be necessary when building the module as a shared library.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@JukkaL JukkaL merged commit 02d917d into master May 8, 2026
24 checks passed
@JukkaL JukkaL deleted the librt-random branch May 8, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants