fix(chat): support tool_choice in Qwen3.5-VL chat handler#108
fix(chat): support tool_choice in Qwen3.5-VL chat handler#108abdullah-cod9 wants to merge 1 commit intoJamePeng:mainfrom
Conversation
- 'none': tools are not injected into the prompt - 'required': model is instructed to always call a tool - 'auto'/None: model follows each tool's description to decide Fixes: tools were always injected regardless of tool_choice value
|
The original Qwen 3.5 chat template didn't have any logic regarding https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/chat_template.jinja |
I didn't notice that, thank you for pointing it out! Is If the original chat template doesn't support it, what would be the downside of adding the logic directly in I'm asking to better understand the design philosophy here — is there a reason to stay strictly in sync with the original template? |
|
Current adaptations primarily involve injecting the image_url into the multimodal component and passing in Jinja variables. However, it is strongly recommended to retain the original chat template. This may have a smaller impact on models with large parameters, but could potentially affect models with small parameters. Of course, you could also share your code examples and why this variable is so important, so I can see if there are any other potential issues. |
|
Thank you for the explanation! My use case: I have a set of tools with a button in the UI to enable or disable them via I tested the change on Qwen3.5 2B and it worked fine without any noticeable impact on the model's behavior. That said, I understand your concern about staying close to the original template, especially for smaller models. If you'd prefer to keep the template unchanged, feel free to close the PR — I can implement a higher-level workaround that prevents tools from being injected when |
Problem
tool_choiceparameter is ignored inQwen35ChatHandler.Tools are always injected into the prompt regardless of the value passed,
because the chat template never references
tool_choice.Root Cause
The
CHAT_FORMATstring inQwen35ChatHandleronly checks:with no condition on
tool_choice.Solution
Added
tool_choicehandling in the template:"none""required""auto"/NoneTesting
Closes #107