Description
Unity Gateway on Databricks puts one gateway in front of every coding agent, so Claude Code, Codex and Hermes Agent can run on the same model and be measured the same way. All three passed every hidden test on the same task, but on the same Opus model one agent's bill was 2.4x another's.
⏰ Chapters
00:00 - Same Model, Three Harnesses: What a Harness Is
01:34 - What the Research Says About Harnesses
02:24 - The Setup: One Model, One Gateway, Stock Installs
03:36 - Demo: The Repo, the Tests and the Gateway Configs
04:18 - The Real Run: Overhead, Three Agents, Grading
05:40 - Demo: The Raw Recordings of All Three Runs
06:48 - Results on Kimi K3: A Tie on Correctness, a Spread on Cost
07:57 - Each Harness on Its Own Model: The Caching Surprise
09:22 - Demo: The Gateway Usage Table in Databricks
09:58 - How to Choose a Coding Agent If You're Starting Today
🔀 How Unity Gateway measures every harness
A harness is everything around the model: the loop, the tools, the system prompt and context, memory, and permissions. Each harness keeps its own API, so Unity Gateway gives Claude Code an Anthropic route, Codex a Responses route and Hermes a chat completions route, and all three reach the same model served on Databricks, the open-weight Kimi K3. Every request carries a tag with the harness and the run, so the gateway's usage table records calls, prompt tokens, cached tokens and output for every run with the same counting. Recent research found that swapping harnesses moves pass rates by only about 5 points on 447 SWE-bench Verified tasks but changes cost per task by up to 3x, and the usage table shows where that difference comes from.
🛠️ What you'll see in the demo
One task: add refunds to a small coffee shop codebase, with 6 visible tests and 7 hidden tests the agents never see.
• The setup on screen: the repo, 3 of 6 tests failing before any agent runs, and the config that points each harness at its Unity Gateway route
• Before any work: Codex sends about 7,800 tokens to reply "ok", Claude Code 10,500, Hermes almost 12,000
• The real runs: Claude Code reads every file and stops at 6 passed, and Codex prints "tokens used 14,897" while the gateway counted 158,499 input tokens
• 9 runs on Kimi K3, all 7/7 hidden: Claude Code averaged 43 seconds and 14 cents, Codex 65 seconds and 17 cents, Hermes 72 seconds and 19 cents, with overlapping ranges
• Why the spread: every model call resends the conversation, and 80 to 90% of the prompt tokens came from the cache
• Each harness on its own model: Claude Code on Claude Opus 5.5 cost about 20 cents a task, Hermes on the same model 49 cents with zero cached tokens, and Codex with GPT-6.1 Sol was the cheapest run at 7 cents
• The usage table queried live in the Databricks SQL editor, one row per graded run
Whether you're a developer choosing a coding agent, a platform team rolling agents out, or a lead watching the AI bill, Unity Gateway gives you one bill, one usage table and a tag on every run, so you can swap models without touching your tools. This was one task with 2 or 3 runs per pair at list prices, and Anthropic doesn't officially support non-Claude models in Claude Code, so treat the numbers as a method, not a leaderboard.
🔗 Resources
📘 Unity Gateway documentation: https://docs.databricks.com/aws/en/unity-gateway
🤖 Get started with coding agents: https://docs.databricks.com/aws/en/ai-gateway/coding-agent-integration
📊 Track model usage: https://docs.databricks.com/aws/en/ai-gateway/usage-tracking
📄 What Does a Harness Buy? Tokens, Mostly: https://arxiv.org/abs/2610.04433
🤝 Stay connected
Website: https://databricks.com
Twitter: https://x.com/databricks/
LinkedIn: https://www.linkedin.com/company/databricks/
Instagram: https://www.instagram.com/databricksinc/
Facebook: https://www.facebook.com/databricksinc/