Same agent, same prompt, same model. Swapping eight typed browser tools for one CLI tool cut a run from 351 seconds to 102, and 1.5M input tokens to 97.5k.
Eight typed tools351 s · 26 screenshots
One CLI tool102 s · 0 screenshots
0 s60 s120 s180 s240 s300 s360 s
screenshot
scroll, press, click
everything else
model call
Every tool call in both runs, drawn to the same time scale. Hover a block for the tool and its duration.
TL;DR: I only changed the tools. One CLI tool was about 3x faster than eight typed ones and used 15x fewer input tokens, in one run each.
What happened
I asked Athena, my Rust agent on Telegram, to read a Kaggle competition page. It took 26 screenshots, 17 PageDown presses and almost six minutes.
Athena drove the browser through eight typed tools I had wrapped around agent-browser. Each one checked its inputs and did one small thing. It looked tidy. Then I ran the same commands myself and saw what the model had actually been given:
The snapshot ran before Kaggle finished rendering, so it came back empty.
browser_read always passed a URL, so it fetched raw HTML instead of the rendered tab.
The interactive snapshot lists buttons, not page text.
So the model did what I would have done. It took screenshots and scrolled. Two of those are plain bugs, and I did not fix them and keep the typed tools, so I cannot say how much of the gap is design and how much is bugs.
The cache surprise
Every model call resends the whole conversation, and providers cache the start of it. 23 of the 58 calls threw that cache away and wrote everything again. Each one came right after an image.
read from cache
written again
Input tokens for each model call, in order. Every mostly-yellow bar is a call that threw the cache away. Hover a bar for its numbers.
My guess: Athena keeps only the newest four images. From the fifth one on, every new screenshot swaps an old image for a placeholder, which edits an early message and breaks the cached start. I have not tested this by changing the cap, but the pattern is gone in the run with no screenshots.
The change
I deleted the eight tools and added one, agent_browser, which passes its arguments straight to the CLI. Same prompt, same model (openai/gpt-6-luna), same page.
Measure
Eight typed tools
One CLI tool
Wall time
351 s
102 s
Model calls
58
14
Screenshots
26
0
Input tokens
1.50M
97.5k
Cache resets
23
0
One run of each.
This time the model read pages as text, with read on the open tab and eval document.body.innerText, and moved between pages by URL.
Caveats
One run of each, and I did not grade the answers, only time and tokens. The traces record cost as 0, and cache reads and writes bill at different rates, so 15x fewer tokens is not 15x cheaper.
The new tool is also unrestricted. No URL checks, and eval and downloads are allowed. The sandbox is now the only guardrail.
What I would check next time
Count tool calls per task. 26 screenshots for one page was sitting right there in the trace.
Look at cache reads per call, not just totals. A reset shows up as a sawtooth.
Run the tool yourself with the same flags before blaming the model.