I put two Tesla V100-SXM2-32GB cards in ripper, my main homelab linux box. They’re old datacenter GPUs; the V100 came out in 2017. Together they have 64 GB of VRAM, which is enough to serve Qwen3.8-27B with vLLM at a 64K context window.
So the obvious question was whether that model could run chatJPt, the Zerto/HPE assistant I’ve been building, in place of Claude. I stood up a second chatJPt instance on my dev box, pointed it at the vLLM server instead of the Anthropic API, and asked both of them the same eight questions.
The setup
Both instances run the same chatJPt build and connect to the same docs MCP servers, so they have the same tools: search_docs, get_page, list_versions, interop_check, the pairing-rules lookup and so on. The only difference is the model behind them.
- Claude: Sonnet 5.5 through the Anthropic API, prompt caching on, list price $2 per million input tokens and $10 per million output.
- Qwen: Qwen3.8-27B in NVFP4, tensor-parallel across the two V100s, served by vLLM with an OpenAI-compatible API. Qwen’s thinking mode is off; it answers directly. Prefix caching is on in vLLM, but there’s no bill, so nobody’s counting.
To make Qwen work I added a small translation layer to chatJPt. The app was written around Anthropic’s message format: system prompt, messages and tool definitions. That layer converts all of it to OpenAI chat completions and converts the streamed reply back. Claude wrote most of it in an evening. It also wrote a bug where the server said “missing API key” because the startup check still looked for the Anthropic key. That took five minutes to fix and about twenty to find.
The questions
I took the questions I’d already been throwing at the Qwen instance and added one about firewall ports:
- What is the latest Zerto version? Cite the page.
- What is the latest version of Zerto for AWS?
- Can you create a setup guide for Zerto on AWS?
- What are the limitations of Zerto with VAIO vs zDriver in 10.9?
- Can I use zDriver with VCF 9 and ESXi 9?
- How do I suppress the guest agent not responding warning on the VRA helpers?
- Tell me about what is new in Zerto 10.9.
- What ports need to be open between a ZVM and a VRA?
The numbers
| Question | Sonnet time | Sonnet tools | Qwen time | Qwen tools |
|---|---|---|---|---|
| Latest version, cite page | 6.4 s | 1 | 8.9 s | 2 |
| Latest for AWS | 5.1 s | 1 | 1.8 s | 1 |
| AWS setup guide | 65.0 s | 14 | 39.6 s | 9 |
| VAIO vs zDriver | 17.9 s | 3 | 17.7 s | 5 |
| zDriver on VCF 9 / ESXi 9 | 12.0 s | 3 | 9.5 s | 3 |
| Suppress VRA-H warning | 10.5 s | 2 | 22.8 s | 7 |
| What’s new in 10.9 | 18.9 s | 3 | 10.0 s | 2 |
| ZVM to VRA ports | 13.5 s | 2 | 8.1 s | 2 |
Qwen finished all eight in 118 seconds. Sonnet took 149. I’ll be honest, I didn’t expect a pair of 2017 GPUs to win on total time. Qwen was faster on five questions, tied on one and slower on two.
Sonnet starts typing sooner, though. Its first words showed up in about 2 seconds every time, usually a line like “I’ll check the interoperability matrix.” Qwen’s first text arrived anywhere from 0.3 to 23.7 seconds in, depending on how many searches it ran before saying anything.
The Sonnet runs cost $0.53 for all eight questions. 86% of Sonnet’s input tokens were cache reads at $0.10 per million, which is what kept it that low. Without caching the same eight turns would have cost about $1.96.
Qwen cost nothing per question. Ripper is on anyway; the extra power draw is the real cost, which is about 600 watts under full load and 100w at idle while the models are loaded and vLLM is ready for requests.
How the answers compared
This is the part I cared about. A fast, free, wrong answer is worse than no answer.
On five of the eight, the answers were close enough that I’d be fine handing either one to a customer. The 10.9 what’s-new list had the same items in nearly the same order: the AI Assistant, the Zerto MCP server, Recovery Plans, the Defender XDR integration, HashiCorp Vault, the Cyber Recovery VPG. Both models listed the same VAIO restrictions: no RDMs, vVols or FCDs, no Clone, FLR, Extended Journal Copy or Instant Restore. Both got the ZVM-to-VRA ports right (4005, 4006 and 4009) and cited the 10.9 ZVM Appliance firewall page.
Where they split:
Qwen followed “cite the page” better. On question 1 Qwen linked the 10.9 release notes. Sonnet told me the answer came from a tool, not a page, so it had no URL to give, and offered to go find one. Technically honest. Not what I asked for.
Qwen filled in a table Sonnet left blank. On VAIO vs zDriver, Sonnet put “not checked” in the zDriver VPGs-per-ZVM cell and said it hadn’t opened the non-VAIO sizing page. Qwen opened that page and filled in 10,000.
Sonnet answered the whole zDriver question. I asked about VCF 9 and ESXi 9. Both models said no to zDriver on VCF 9 and quoted the VCF install guide. Sonnet then went through the interop matrix for standalone ESXi 9.0 through 9.1, found every row marked “VAIO variant only”, and said so. Qwen answered the VCF half and offered to check plain vSphere 9 if I asked.
Sonnet did better on the VRA helper question, and Qwen wandered. Both got the core fact right: a VRA helper has no OS and no VMware Tools, so the guest agent warning is expected. Qwen ran seven tool calls in 23 seconds, and one of the pages it cited was from the Hyper-V admin guide for a question about vCenter. Sonnet ran two, cited the actual VRA-H KB article, and gave the practical answer: scope the vCenter alarm so it skips the Z-VRAH* VMs, and don’t power the helper off to clear it, because that triggers alert ZVM0021.
Both narrate. Each one writes “Let me search for…” lines into the answer before tool calls. On the VRA helper question Qwen’s answer opens with six of them. Sonnet’s AWS guide opens with five. That’s a chatJPt prompt problem I need to fix for both.
Things that broke along the way
The first real failure was Qwen going quiet. I asked it to look over my Zerto and Morpheus servers and the HVM hosts. It made 13 tool calls and then just stopped, with no answer and no error. chatJPt allowed 8 model rounds per question. Claude usually finishes in that budget; Qwen, with its smaller context and more searching, didn’t. It ran out mid-research, and the app never told the model to wrap up.
The fix, which only applies to the local model:
- 16 rounds instead of 8.
- The last round tells the model to stop calling tools and answer with what it has.
- Once a turn passes 50K tokens of context, chatJPt summarizes the research so far into notes and keeps going on the notes. Everyone calls that compacting. A “compacting research notes” chip shows in the UI so you know why it paused.
That 64K window is the real constraint. Sonnet 5.5 has a 1M window, so a long AWS setup guide with 14 tool calls never comes close. Qwen has to be careful about every page it pulls in.
So which one?
For my own lab and for testing, Qwen on the V100s is good enough that I’ll keep the instance running. On factual docs questions it gets the facts right, cites real pages and is often faster. It costs me nothing per question.
For chatJPt in production, I’m staying on Sonnet. The gap isn’t speed or cost. It’s judgment on the messy questions. On the VRA helper question Sonnet found the right KB and gave practical advice. Qwen stayed on the documentation and pulled in a page for the wrong hypervisor. And the 1M context window means Sonnet doesn’t need the compacting machinery at all.
What’s next
- Let users pick the provider per chat. That’s filed as an issue, and it’ll ship as a beta flag first. The advantage is that queries never go to a model provider… which could matter in some cases.
- Turn Qwen’s thinking mode on and rerun these same eight questions. Thinking costs tokens and time, but those are free here.
If you’re running a local model behind your own MCP tools, I’d like to hear how it’s going, especially if you’ve found a context-window trick that beats compacting.
![]()