Skip to content

Add additionalContext support to MLXLanguageModel#145

Merged
mattt merged 9 commits intohuggingface:mainfrom
noorbhatia:mlx-custom-generation-options
Mar 24, 2026
Merged

Add additionalContext support to MLXLanguageModel#145
mattt merged 9 commits intohuggingface:mainfrom
noorbhatia:mlx-custom-generation-options

Conversation

@noorbhatia
Copy link
Contributor

@noorbhatia noorbhatia commented Mar 13, 2026

Add CustomGenerationOptions struct to MLXLanguageModel with an additionalContext: [String: JSONValue]? property for injecting variables into Jinja2 chat templates

Usage

var options = GenerationOptions()
options[custom: MLXLanguageModel.self] = .init(
    additionalContext: ["enable_thinking": .bool(true)]
)
let response = try await session.respond(to: "Hello", options: options)

@noorbhatia
Copy link
Contributor Author

Hi @mattt , can you take a look at this. Thanks!

Copy link
Collaborator

@mattt mattt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for not reviewing sooner. #147 also introduces MLXLanguageModel.CustomGenerationOptions, so we'll have to do some traffic control here. Should have both of these merged today.

@mattt mattt force-pushed the mlx-custom-generation-options branch from df19229 to d27310d Compare March 23, 2026 13:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for passing an additionalContext dictionary through GenerationOptions into MLXLMCommon’s chat-template rendering context, enabling callers to inject custom variables for Jinja2-based templates.

Changes:

  • Extends MLXLanguageModel.CustomGenerationOptions with additionalContext and threads it into MLXLMCommon.UserInput construction paths.
  • Adds a basic MLX test that exercises setting additionalContext via GenerationOptions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
Sources/AnyLanguageModel/Models/MLXLanguageModel.swift Introduces additionalContext custom option and forwards it into MLX user input creation for standard, streaming, and structured generation flows.
Tests/AnyLanguageModelTests/MLXLanguageModelTests.swift Adds a regression test ensuring additionalContext can be set and a response is produced.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mattt
Copy link
Collaborator

mattt commented Mar 24, 2026

@noorbhatia Alright, I think this is in really good shape now! Thanks for your help on this and your patience 🙇

@mattt mattt merged commit 46187c2 into huggingface:main Mar 24, 2026
7 checks passed
@noorbhatia
Copy link
Contributor Author

Amazing, thanks @mattt !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants