Skip to content

Improve import startup with lazy top-level exports#2950

Open
fede-kamel wants to merge 4 commits intoopenai:mainfrom
fede-kamel:feat/import-startup-lazy-2819
Open

Improve import startup with lazy top-level exports#2950
fede-kamel wants to merge 4 commits intoopenai:mainfrom
fede-kamel:feat/import-startup-lazy-2819

Conversation

@fede-kamel
Copy link
Copy Markdown

Summary

This PR addresses import-time startup regression tracked in #2819 by making heavy top-level exports lazy.

Key changes:

  • Lazily resolve top-level exports in openai.__init__ for:
    • AzureOpenAI
    • AsyncAzureOpenAI
    • pydantic_function_tool
    • AssistantEventHandler
    • AsyncAssistantEventHandler
  • Add an eager-import debug/CI mode: OPENAI_EAGER_IMPORT=1
    • In eager mode, all lazy exports are resolved at import time to catch deferred-import breakages.
  • Lazily create the internal Azure module-client subclass only when Azure mode is actually used.
  • Add an import benchmark helper script: scripts/bench_import.py.
  • Add tests for lazy/eager import behavior:
    • tests/test_import_surface.py
    • tests/lib/test_import_surface_live.py (live-gated)

Why this relates to #2819

Issue #2819 reports that import openai is too slow and that eager loading of type-heavy/internal surfaces is a major contributor. This PR reduces startup cost in the default path by moving expensive imports behind first-use access while preserving API compatibility.

Testing done

Unit / behavior tests

  • PYTHONPATH=src python3 -m pytest tests/test_import_surface.py -q -o addopts='' -> passed
  • PYTHONPATH=src python3 -m pytest tests/test_module_client.py -q -o addopts='' -> passed

Live API-key integration test

  • OPENAI_LIVE=1 PYTHONPATH=src OPENAI_API_KEY=... python3 -m pytest tests/lib/test_import_surface_live.py -q -o addopts='' -> passed
  • This validates eager-mode import + real API request flow (client.models.list()).

Performance benchmarks

Using scripts/bench_import.py:

  • WSL/Linux run (patched branch):
    • lazy avg: 0.1533s
    • eager avg (OPENAI_EAGER_IMPORT=1): 0.2601s
    • delta: +0.1068s (+69.7%)
  • Native Windows run from patched branch path:
    • lazy avg: 0.7019s
    • eager avg: 2.9429s
    • delta: +2.2410s (+319.3%)

Interpretation: eager mode is intentionally slower, and the default lazy path materially reduces startup work compared with forced eager loading.

Notes

  • This PR keeps behavior-focused tests deterministic and keeps live testing opt-in via OPENAI_LIVE=1 and OPENAI_API_KEY.
  • Existing unrelated local worktree changes were excluded from this PR.

@fede-kamel fede-kamel requested a review from a team as a code owner March 10, 2026 02:34
@fede-kamel
Copy link
Copy Markdown
Author

Validation summary for #2819:

  1. Unit/behavior tests (rebased branch)
  • PYTHONPATH=src python3 -m pytest tests/test_import_surface.py -q -o addopts='' -> 2 passed
  • PYTHONPATH=src python3 -m pytest tests/test_module_client.py -q -o addopts='' -> 12 passed
  1. Live API-key integration test
  • OPENAI_LIVE=1 PYTHONPATH=src OPENAI_API_KEY=... python3 -m pytest tests/lib/test_import_surface_live.py -q -o addopts='' -> 1 passed
  • This test validates eager-mode import + a real API request (client.models.list()).
  1. Performance measurements
  • WSL/Linux (patched branch):
    • lazy avg: 0.1533s
    • eager avg (OPENAI_EAGER_IMPORT=1): 0.2601s
    • delta: +0.1068s (+69.7%)
  • Native Windows (patched branch path):
    • lazy avg: 0.7019s
    • eager avg: 2.9429s
    • delta: +2.2410s (+319.3%)

Interpretation: default lazy mode reduces import startup work significantly; eager mode remains available for CI/dev verification.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2bd0b97246

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/openai/__init__.py Outdated
@g0t4
Copy link
Copy Markdown

g0t4 commented Mar 11, 2026

Two things, about the env var for eager load

  • The point of CI is to reproduce the user experience and validate it works. Having an env var change something fundamental with imports will lead to a nightmare of bugs just in CI and then missed bugs in reality.
  • Second, if CI needs eager imports for a special set of tests, why not have a script or pytest startup function that performs the imports? That way it is explicit what CI is testing.

I think it's great you're working on a fix for this btw... thank you

@fede-kamel
Copy link
Copy Markdown
Author

Took this feedback and removed the runtime eager-import env var path from openai.__init__.

The branch now keeps lazy imports as the only product behavior and makes the coverage explicit in tests instead:

  • unit tests explicitly resolve the lazy exports
  • the live test explicitly resolves the lazy exports and performs a real request
  • openai.lib re-exports were also made lazy so import openai stays on the lazy path

Validated locally with:

  • PYTHONPATH=src .venv/bin/python -m pytest tests/test_import_surface.py tests/test_lazy_types_import.py -q -o addopts=''
  • OPENAI_LIVE=1 PYTHONPATH=src .venv/bin/python -m pytest tests/lib/test_import_surface_live.py -q -o addopts=''

@fede-kamel
Copy link
Copy Markdown
Author

@RobertCraigie @rachellim @g0t4 — Friendly ping! Just re-validated the branch: all tests pass (test_import_surface 2/2, test_module_client 12/12), all lazy exports resolve correctly, and the benchmark shows ~0.18s avg cold import on macOS. This addresses #2819 with a significant startup improvement — on Windows the delta was over 300%. Would love to get a review when you get a chance. Thanks!

The 2.32.0 release added 'from .types.websocket_reconnection import ...'
at the top of openai/__init__.py, which transitively loads openai.types
through openai._client as well. That invalidates the 'openai.types is
lazy' assertion the earlier test relied on, so the lazy-types live test
is removed and the remaining tests assert the boundaries that still
hold (lib.azure, lib.streaming, lib._tools).

Also stabilize pyright under strict:
 - preserve static visibility of AzureOpenAI / AsyncAzureOpenAI /
   pydantic_function_tool / AssistantEventHandler under TYPE_CHECKING
   so callers keep their signatures
 - rename _AZURE_MODULE_CLIENT_CLASS to a non-constant symbol and type
   the module-client factory as type[AzureOpenAI] to silence strict
   reportCallIssue / reportConstantRedefinition warnings
 - tidy ruff imports / I001 noise in scripts/bench_import.py and the
   test files added in this branch
@fede-kamel fede-kamel force-pushed the feat/import-startup-lazy-2819 branch from 2cb445c to b0fb0f4 Compare April 20, 2026 14:56
@fede-kamel
Copy link
Copy Markdown
Author

Friendly bump for maintainer review — rebased onto main (e507a4eb, 2.32.0) and CI-ready.

What changed since last review

  • Clean rebase onto 2.32.0. The release added from .types.websocket_reconnection import ReconnectingEvent, ReconnectingOverrides at the top of openai/__init__.py, which transitively pulls openai.types through openai._client. openai.types therefore can't stay lazy without a deeper refactor, so I dropped the lazy-types live test and kept the tests that still hold (lazy openai.lib.azure, openai.lib.streaming, openai.lib._tools).
  • Strict pyright cleanups. Kept AzureOpenAI / AsyncAzureOpenAI / pydantic_function_tool / AssistantEventHandler / AsyncAssistantEventHandler statically visible under TYPE_CHECKING, so callers keep their signatures. Renamed _AZURE_MODULE_CLIENT_CLASS (strict-mode reportConstantRedefinition) and typed the factory as type[AzureOpenAI] so Azure-specific kwargs type-check at the call site.

Integration benchmark (Python 3.13, cold import openai, 30 samples each)

metric origin/main (2.32.0) this branch speedup
min 346 ms 288 ms 1.20×
median 401 ms 325 ms 1.23×
mean 397 ms 338 ms 1.18×

Python's own -X importtime cumulative time for the openai module:

  • origin/main: 390 ms
  • this branch: 232 ms1.68× faster / ~158 ms shaved

Repro: python scripts/bench_import.py (also in the branch).

What still stays lazy after the rebase

Verified on import openai:

  • openai.lib.azure — not loaded
  • openai.lib.streaming — not loaded
  • openai.lib._tools — not loaded

These are the biggest non-transitively-required submodules, which is where the ~160 ms comes from.

Happy to split further or squash to the original 3 commits if preferred. Would love to get this in — the import-time win is meaningful for CLI / Lambda / short-lived workers.

@fede-kamel
Copy link
Copy Markdown
Author

@mcgrew-oai @hintz-openai — friendly ping in case this slipped off the review queue. Just rebased onto 2.32.0 (see benchmark comment above: ~1.18–1.23× faster cold import openai, ~160 ms shaved off -X importtime). Happy to split commits or narrow scope if that helps review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants