Embedded GraalPy fails with LookupError after upgrading requests to 2.33.1 / charset_normalizer to 3.4.6

[timfel]

Summary

Upgrading the embedded Python dependencies from requests==2.32.5 / charset_normalizer==3.4.5 to requests==2.33.1 / charset_normalizer==3.4.6 causes an embedded GraalPy application to fail at runtime with:

LookupError: no such codec is supported.

We hit this in the graalvm/graal-languages-demos repo in the GraalPy Micronaut and Spring Boot guides. The checked-in GraalPy/GraalVM version did not change; only the resolved Python dependency set drifted.

Environment

• GraalPy plugin / runtime: 25.0.2
• JVM test runtime: Oracle GraalVM JDK 21.0.8+12.1
• Embedded mode via GraalPy build tools / VFS
• Reproduced locally on 2026-04-01

Dependency change

Last passing embedded env resolved:

• requests==2.32.5
• charset_normalizer==3.4.5
• urllib3==2.6.3
• idna==3.11
• certifi==2026.2.25
• vader-sentiment==3.2.1.1

Failing embedded env resolved:

• requests==2.33.1
• charset_normalizer==3.4.6
• urllib3==2.6.3
• idna==3.11
• certifi==2026.2.25
• vader-sentiment==3.2.1.1

Reproduction shape

In Java, we create an embedded GraalPy context and evaluate:

from vader_sentiment.vader_sentiment import SentimentIntensityAnalyzer
SentimentIntensityAnalyzer()

With the newer dependency set, bean initialization fails and the embedded eval throws:

Caused by: LookupError: LookupError: no such codec is supported.
    at <python>.<module>(Unknown)
    at org.graalvm.polyglot.Context.eval(Context.java:449)

This is the local failure we reproduced in the Micronaut example:

Message: LookupError: no such codec is supported.
Path Taken:
new o.e.SentimentAnalysisController(SentimentAnalysis sentimentAnalysis)
---> new o.e.SentimentAnalysisController([SentimentAnalysis sentimentAnalysis])
Caused by: LookupError: LookupError: no such codec is supported.
    at <python>.<module>(Unknown)
    at org.graalvm.polyglot.Context.eval(Context.java:449)
    at org.example.SentimentAnalysis.<init>(SentimentAnalysis.java:20)

Workaround

Pinning the embedded Python dependencies back to:

• requests==2.32.5
• charset_normalizer==3.4.5

makes the same local reproduction pass again.

Notes

The exact codec name is not included in the thrown exception text, but vader_sentiment initializes using codecs.open(..., encoding="utf-8"), so this may be related to codec availability / registration during embedded startup or transitive imports in this dependency combination.

If useful, I can provide a reduced reproducer based on the GraalPy Micronaut guide setup.

[BigMichi1]

i'm currently using this workaround in my script(s). it must be at the very top before anything else

import importlib
import codecs
import unicodedata

# ============================================================================
# GraalPy compatibility patches
#
# These patches are ONLY needed when running under GraalPy (GraalVM's Python
# implementation). They are not required on CPython and would be harmless no-ops
# there since the patched functions already behave correctly.
#
# Background: GraalPy does not implement the full CPython codec registry.
# CPython ships dozens of "codec search functions" for legacy encodings
# (e.g. shift_jis, big5, euc_kr, latin-1 variants). GraalPy only registers a
# subset of codecs and raises LookupError for the rest.  Several third-party
# libraries (notably charset-normalizer >= 3.x and pydantic >= 2.x) perform
# speculative codec imports or lookups during module initialisation or
# during encoding detection — causing hard ImportError / LookupError crashes
# inside GraalPy's VFS even though the codec is never actually used.
# ============================================================================

# --- Patch 1: safe importlib.import_module ---
#
# When is this needed?
#   charset-normalizer >= 3.0 dynamically imports one module per supported
#   encoding (e.g. `encodings.shift_jis`) to test codec availability.
#   On CPython all of these modules exist; on GraalPy many are absent and
#   `import_module` raises LookupError instead of ModuleNotFoundError,
#   which is not caught by the library's own try/except ImportError guard.
#
# What does this patch do?
#   Wraps import_module so that any LookupError is swallowed and a minimal
#   stub module is returned.  The stub exposes IncrementalDecoder /
#   IncrementalEncoder from the stdlib `codecs` module so that isinstance
#   checks inside charset-normalizer don't crash with AttributeError.
_real_import_module = importlib.import_module

def _safe_import_module(name, package=None):
    try:
        return _real_import_module(name, package)
    except LookupError:
        import types
        stub = types.ModuleType(name)
        stub.IncrementalDecoder = codecs.IncrementalDecoder
        stub.IncrementalEncoder = codecs.IncrementalEncoder
        return stub

importlib.import_module = _safe_import_module

# --- Patch 2: unicodedata.decomposition ---
#
# When is this needed?
#   GraalPy (as of 24.x) does not expose `unicodedata.decomposition()`.
#   charset-normalizer's encoding-detection logic calls this function to
#   identify multi-codepoint characters (e.g. accented letters in NFD form)
#   when scoring candidate encodings.  Without the patch the call raises
#   AttributeError and the detection silently fails or crashes.
#
# What does this patch do?
#   Provides a pure-Python fallback: normalise the character to NFD and
#   return the decomposed codepoints as space-separated uppercase hex strings,
#   which matches the format CPython's C-extension returns (e.g. "0041 0301"
#   for "Á").  An empty string is returned for characters that have no
#   decomposition, consistent with CPython behaviour.
if not hasattr(unicodedata, "decomposition"):
    def _decomposition(char: str) -> str:
        # Normalize to NFD and return the decomposed form as space-separated hex codepoints
        nfd = unicodedata.normalize("NFD", char)
        if nfd == char:
            return ""
        return " ".join(f"{ord(c):04X}" for c in nfd)

    unicodedata.decomposition = _decomposition

maybe this helps