# Fixing interleaved thinking for local Claude Code
2026-07-23 · https://tomvaucourt.com/posts/llamacpp-interleaved-thinking/
There's [a thread on r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/s/2yq8a7PraH) where someone builds a pet project with
Qwen3.5 35B A3B running inside Claude Code, tallies up what the session would
have cost on Sonnet 4.6 ($10.85, per ccusage), and lands on their actual bill:
two minutes of 400W electricity. That's the promise of pointing Claude Code at
your own hardware, and it mostly delivers.
Mostly. Because the same thread carries a recurring complaint: the model just
stops. It works for a while, makes a few tool calls, then goes silent right
where the next tool call should be. No error. The turn ends at the tail of a
thinking block, and Claude Code sits there waiting.
The workaround people converged on was to disable thinking. It works. It's
also the wrong lesson, and the actual bug sat in llama.cpp's Anthropic
compatibility layer, where I eventually found and fixed it.
## The symptom
I was running the model in Claude Code, testing how far its agentic coding
could go in a frontier harness. My first suspicion was the chat template: newly
released models ship with template bugs often enough that it's the reasonable
first guess. But the same model drove OpenClaw fine. Same weights, same server,
different harness. So I opened a Claude Code session with Opus, handed it my
llama.cpp container, and it had the cause in about fifteen minutes.
The model stopped abruptly before emitting a tool call, at the end of a
thinking block. An `<|im_end|>` where a `tool_use` should be, with empty
content. I hit it on both Qwen 3.5 models I run; nothing about it is
Qwen-specific, and without the fix it would most likely have hit Qwen 3.6 and
every other interleaved-thinking model the same way.
## What the server was actually doing
With thinking enabled, an agent's single logical turn is a chain: thinking →
tool_use → thinking → tool_use, all the way to the final answer. That's
interleaved thinking, and it's how the Anthropic API expects agentic loops to
run — Claude Code sends the whole chain back to the server on every request so
the model can see its own reasoning trail.
llama.cpp's `/v1/messages` endpoint speaks the Anthropic Messages format on
the wire — that's what Claude Code sends — but the rest of the server thinks
in OpenAI-style messages, so the endpoint's real job is translation. The
mapping is mostly mechanical: a `thinking` block should land in
`reasoning_content`, a `tool_use` block becomes an entry in `tool_calls`, and
a `tool_result` comes back as a `role: "tool"` message. Except the first one
didn't happen. The conversion dropped the thinking blocks. Every one of them:
So here's what the model saw: a transcript where its previous tool calls in
the current turn appear to have happened with no thinking lead-in, because it
was deleted. The conversion rewrites the chat history so the model believes it
never emitted thinking. Then, when it's time for the next tool call, the model
begins a fresh thinking block, trying to re-establish its plan. But the prompt
now teaches it that tool calls happen without the reasoning scaffolding. That mismatch shifts the distribution: instead of transitioning
from thinking into a tool_use, it often terminates the message right there.
EOS.
From the user's perspective that reads as "Claude Code stalls after tool
calls." The mechanical cause is that dropping interleaved thinking blocks
breaks the learned cadence that leads into the next tool call, so the model
ends its turn prematurely.
This is also why disabling thinking "fixes" it: with no thinking blocks to
drop, the tool-call sequence stays internally consistent, and the model keeps
emitting tool calls instead of bailing out. You've traded the model's
reasoning for its stability, and you shouldn't have to.
There's a template subtlety underneath that confused the diagnosis further.
Qwen's chat template doesn't keep all historical thinking: it strips ``
blocks from older turns while keeping them for the current tool-use chain.
That's correct behavior: the model was trained that way. The bug was upstream
of the template: the endpoint stripped thinking *everywhere*, so the template
never even got the blocks it needed for the active chain.
## The fix
[PR #20120](https://github.com/ggml-org/llama.cpp/pull/20120): in
`convert_anthropic_to_oai()`, collect `thinking` blocks into
`reasoning_content` on the converted message instead of discarding them, and
stop dropping assistant messages whose only content was thinking. The diff is
8 lines added, 2 removed. The tests are 133 lines.
It merged fast, and another user independently confirmed it also resolved the
autoparser 500 errors they'd been seeing. Those 133 lines only exist because a
maintainer reminded me to write tests. I shouldn't have needed reminding: the
whole point of a fix this small is pinning the behavior down so it can't
quietly regress.
## If your local agent "randomly" stalls
Check what your serving layer does to the conversation before the model sees
it. The failure mode generalizes: any proxy or compatibility layer that
silently edits chat history is gaslighting the model — showing it a version
of its own behavior that never happened, then acting surprised when its
next-token distribution goes somewhere strange. Reasoning models make this
worse, because the part most likely to be "cleaned up" in conversion is
exactly the part that carries the model's plan.
Model releases always come with quirks like this one, and every one of them
makes local models look weaker than they are. The only way they get fixed is
the people actually running this stuff sending the patch upstream. Fix a few,
and you find out the local models were far more capable than they looked at
first sight.