Harness Tuner.
Improve your agent automatically.
Tunes tools, prompts, skills, model and hyperparams.
A real production agent, after tuning
Measured on held-out data, on a smaller model.
Most optimizers tune the prompt.
Harness Tuner tunes everything around the model.
Tools
The biggest win we see. Your tools probably wrap APIs written for code, not for agents. It rewrites them for the agent: smaller responses, better descriptions, no repeated calls. When a script should do the job, it writes a new code tool.
Prompts and skills
It cuts contradictions, adds the requirements nobody wrote down, and moves long instructions into skills that load only when needed.
Model and settings
It searches models and providers, then tunes thinking level, temperature and the rest for the one it picks.
RAG, fine-tunes and sub-agents
Harness Tuner can integrate fine-tunes, RAG or sub-agents you build in Kiln, and measure whether they make an impact.
How it works
Optimize for quality, cost or speed. When it cuts cost or latency, it holds quality. Tell it what to focus on and what not to touch, in plain English.
Bring all your evals. It tracks every one.
Each experiment runs only the evals it targets, so trying an idea is cheap. Before anything is promoted, it's checked against every eval. Nothing regresses quietly.
Experiments branch from any earlier version, often from several parents, and the winners merge back.
Agents only ever see train. Val decides promotions. Test is never read, so the number you report is one it never optimized against.
It finds the best quality, the cheapest at good quality, a balanced option and the fastest. Compare them side by side in Kiln.
Claude Code, Codex or OpenCode, on your existing subscription.
Our free app tracks every experiment and shows the trade-offs side by side.
Optimize the demo project.
Downloads a small moderation task into a new folder, adds it to Kiln, and tunes it: 624 labeled forum posts, and an eval. The baseline prompt doesn't know the forum's house rules, so there's plenty to find and fix.
uv tool install harness-tuner
harness-tuner --demoFrequently asked
How is this different from prompt optimization (DSPy, GEPA, etc.)?
Prompt optimizers tune the prompts inside a program. Harness Tuner tunes the whole agent you already have: its tools, skills, prompts, model and settings, against every eval you care about.
How does it compare to asking Claude Code or Codex to optimize your agent?
In a way, that's what it does, with the rigor and scale built in:
- a code loop runs the process, so it can't get off track
- real experiments, each one tracked
- batches of experiments planned and run in parallel
- many evals, each tracked on its own
- cost control: experiments run only the evals they target, winners merge, and ideas that aren't working stop early
- val and test sets withheld from the agents
- skills that guide your coding agent through each step
How does it compare to fine-tuning?
A tuned harness is easier to deploy (no GPUs, off-the-shelf models), easier to update when new product requirements come up, quicker to move to the latest model releases, and cheaper. Kiln supports fine-tuning too, and the tuner can use your fine-tuned models in advanced cases.
How does it rewrite tools without changing your code?
It doesn't touch your original tools. It writes wrappers: small code tools that call your existing tools and reshape what goes in and out. A wrapper can trim a response to the fields that matter, loop over a set of results, fold many calls into one, or do the math so the model doesn't have to. Your tools keep their current signature, so nothing outside Kiln changes.
Do the experiments need to run against production or staging?
Experiments call your agent's tools, so they need an environment where those tools can run. The best option is a synthetic world for your evals, built with Kiln Seahaven. It gives you a safe, reproducible and highly parallel place to run them.
What does a run cost, and how long does it take?
It depends on the scope of the project. A small agent with a small eval can cost a few dollars. A large agent with hundreds of tools can cost several hundred. You set a budget and a time limit, and it stops starting new work when either runs out.
Harness Tuner does several things to keep costs down:
- Targeted experiments: each experiment runs only the evals it's trying to improve, and stops early when an idea isn't working.
- No wasted runs: if your evals can't detect a real win, it says so before spending anything.
- Subscription agents: the tuner's own agents run on your existing Claude Code, Codex or OpenCode subscription. It uses some of your quota, but that's usually much cheaper than per-token billing.
Evals need direct API access and can't run on a coding agent subscription, so their tokens bill to your provider keys in Kiln. Details: Costs.
Tune your harness. Get above the frontier.
Free and open source. Runs on the coding agent you already use, with Kiln tracking every experiment.