serve: enable OpenAI auto tool-calling (qwen3_xml) + reasoning split by default
A forum user hit HTTP 400 "'auto' tool choice requires --enable-auto-tool-choice
and --tool-call-parser to be set" — the server shipped without tool-calling
enabled, which is exactly what agent/tool workloads (the whole point) need.
- serve.sh + mtp_serve.sh now pass --enable-auto-tool-choice --tool-call-parser
qwen3_xml --reasoning-parser qwen3 by default (overridable/disable via
TOOL_PARSER= / REASONING_PARSER=).
- Parser matters: Qwen3.5 emits the XML tool format
(<function=name><parameter=k>v</parameter></function>), so hermes returns 200
but with EMPTY tool_calls (the call lands in content). qwen3_xml parses it.
Verified on the box: tool_choice=auto -> tool_calls=[get_weather {city:Paris}];
plain chat unaffected; <think> -> reasoning_content.
- README: document tool-calling defaults + that the server binds 0.0.0.0
(LAN-reachable at http://<spark-ip>:8000; no auth).
This commit is contained in:
@@ -174,6 +174,17 @@ Selected with `--profile`:
|
||||
The server is OpenAI-compatible (`/v1/chat/completions` with tool calls + SSE,
|
||||
`/v1/completions`, `/v1/models`) and serves under the model name `qwen`.
|
||||
|
||||
**Tool calling is enabled by default** (`--enable-auto-tool-choice
|
||||
--tool-call-parser qwen3_xml --reasoning-parser qwen3`), so clients can send `tools`
|
||||
with `tool_choice="auto"`. Qwen3.5 emits the *XML* tool format, so `qwen3_xml` is the
|
||||
correct parser — `hermes` returns 200 but with empty `tool_calls` (the call lands in
|
||||
`content` instead). `<think>` reasoning is split into `reasoning_content`. Override
|
||||
with `TOOL_PARSER=` / `REASONING_PARSER=` (empty disables; `qwen3_coder` also works).
|
||||
|
||||
**Network:** the server binds `0.0.0.0` in `--net=host`, so it is reachable from the
|
||||
LAN at `http://<spark-ip>:8000` — the `127.0.0.1` in the examples is just the local
|
||||
default. There is no auth; put it behind a reverse proxy / firewall for shared use.
|
||||
|
||||
## Benchmarks
|
||||
|
||||
All single-stream (c=1), temperature 0, GB10. "Hermes" regenerates the next
|
||||
|
||||
Reference in New Issue
Block a user