Search results

[Roadmap] Q3 2026 Roadmap

roadmap

Status: Open.

#15044 In NVIDIA/TensorRT-LLM;

· laikhtewari opened on Jun 6, 2026

[Bug]: Guided decoding (xgrammar) + EAGLE-3 + draft_len_schedule reaching 0 crashes during CUDA graph capture, "bitmask must have the same batch size as logits"

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

#15022 In NVIDIA/TensorRT-LLM;

· chungen04 opened on Jun 5, 2026

[Bug]: quantize.py --device_map has no working value for GB10 (unified memory / NVML)

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

#15021 In NVIDIA/TensorRT-LLM;

· ElessarWillomoont opened on Jun 5, 2026

Piecewise Cudagraph support for Gemma4 VSWA cache pools

CUDA Graph

Status: Open.

#14975 In NVIDIA/TensorRT-LLM;

· bmarimuthu-nv opened on Jun 4, 2026

[Feature]: Kernel factory and step3.7 optimizations

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuning

Status: Open.

#14967 In NVIDIA/TensorRT-LLM;

· suyoggupta opened on Jun 4, 2026

[AutoDeploy]: Add Ultra model to btk

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14966 In NVIDIA/TensorRT-LLM;

· marinayanov opened on Jun 4, 2026

[AutoDeploy]: Add gpt-oss-120b model to btk

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14965 In NVIDIA/TensorRT-LLM;

· marinayanov opened on Jun 4, 2026

[Feature]: AutoDeploy vLLM prototype

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#14963 In NVIDIA/TensorRT-LLM;

· suyoggupta opened on Jun 4, 2026

[auto-dev]: auto-dev wheel discovery improvements

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Infra<NV>automated tests, build checks, github actions, system stability & efficiency.<NV>automated tests, build checks, github actions, system stability & efficiency.

Status: Open.

#14951 In NVIDIA/TensorRT-LLM;

· galagam opened on Jun 4, 2026

[Bug]: Gemma 3 / Gemma 4 fail in TensorRT-LLM 1.3.0rc17 container on NVIDIA L4: FlashInfer unsupported architecture, kv_layout metadata error, AutoDeploy use_cache mismatch

bugSomething isn't workingSomething isn't working

Status: Open.

#14942 In NVIDIA/TensorRT-LLM;

· ylnhari opened on Jun 4, 2026

[RFC]: Semantic KV Cache Reuse

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

RFC

Status: Open.

#14918 In NVIDIA/TensorRT-LLM;

· zbennett10 opened on Jun 3, 2026

[Bug][Disagg] KvCacheAwareRouter silently degrades to load-only routing (per-server block table not populated; matched_tokens≡0) — ~90% vs 96% prefix-cache hit

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inference

Status: Open.

#14882 In NVIDIA/TensorRT-LLM;

· lishicheng1996-nv opened on Jun 3, 2026