My AI Workflow Learned to Check Its Toolbelt

I shipped the first version of awesome-ai-workflow with a very specific obsession: accountability.

Can the agent explain what it was asked to do? Can it show the files it read? Can it map requirements to tests? Can it prove the work is done without waving vaguely at a green command it half remembers?

That was the flight recorder idea. Tiny bootloader files. Deeper workflow loaded only when the task deserves it. Tiers for how much ceremony the work needs. Specs, plans, verification ledgers, review records, and archives when the blast radius is large enough.

That solved a real problem.

It did not solve speed.

More specifically, it did not solve the moment where an agent lands in a large repo, starts with rg, reads too much, loses the plot, and then starts treating a 900 line file like a sacred text because it happened to open it first.

I do not want agents that read everything.

I want agents that know when they need a better tool.

The gap was large codebase retrieval

The workflow already had the discipline parts.

It knew when to plan. It knew when to use TDD. It knew not to claim completion without fresh evidence. It knew to keep AGENTS.md and CLAUDE.md small instead of turning every session into a prompt loading exercise.

What it did not have was a sane answer to this question:

When should the agent stop grepping and use something like Serena?

That matters because large codebases are not just “more files.” They change the failure mode.

In a small repo, rg is usually enough. Search the string. Read the nearby file. Make the change. Run the test. Done.

In a large repo, exact search can turn into a swamp. The agent finds 200 matches, opens five files, guesses which one matters, and accidentally builds its mental model around whatever it saw first. If the task is “who calls this,” “where is this implemented,” “what breaks if I rename this,” or “how does this symbol move across packages,” plain text search is the wrong first tool.

That is where Serena is interesting.

Serena gives the agent symbol-level navigation through MCP. Definitions, references, overviews, callers, edits around symbols. IDE-shaped context instead of line-number archaeology.

But “interesting” does not mean “install it everywhere.”

That distinction is the whole update.

The workflow now has a search routing table

I added an on-demand large codebase reference that tells the agent which retrieval mode to reach for:

Known string       -> rg
Symbol graph       -> Serena or native LSP tooling
Structural pattern -> ast-grep or tree-sitter
Fuzzy concept      -> semantic search, last resort

This sounds obvious when you write it down. It was not obvious enough to be encoded.

Without a routing rule, agents do what agents do: they overuse whatever tool is most familiar. If rg worked once, they keep using rg. If an MCP server exists, they start poking it for everything. If semantic search sounds fancy, they use it before doing the boring exact search that would have found the answer in 300 milliseconds.

The new rule is deliberately unglamorous.

Start with text when you know the text.

Use symbol tools when the question is about code structure.

Use structural search when syntax shape matters.

Use semantic search only after the boring tools fail.

That last part matters. I do not want “AI search” to become another magic box agents hide behind. grepai is useful. WarpGrep might be useful in the right setting. But if the agent knows the function name and still reaches for semantic search, something has gone wrong.

I changed my mind about “preflight”

When we talked about this, my first instinct was a preflight command.

Check whether rg, Serena, grepai, Ollama, and the rest are installed. Print a report. Maybe later, generate an install plan.

That is fine, but it is not quite the right mental model.

This is supposed to be an AI-first workflow. The agent should not run a giant checklist because a checklist exists. It should look at the repo and ask, “Do I need more capability for this task?”

So the new script is not a generic preflight. It is a readiness check the agent runs when the repo looks large, unfamiliar, polyglot, or cross-file heavy:

python3 ultimate-agentic-workflow/scripts/large_codebase_tools.py --project-root . --json

It counts project signals while ignoring generated and vendor directories. It detects languages. It looks for monorepo-ish layouts. It checks whether uv, rg, serena, ast-grep, grepai, and ollama exist on PATH. It knows WarpGrep/Morph is hosted and treats that as explicit approval only.

Then it decides.

If the repo is small, it does not ask to install Serena just because Serena is missing.

That is important. On this repo, the script currently reports:

large_codebase: false
serena installed: false
serena action: not-needed-yet
approval requests: 0

Good.

The workflow should not pester you to install a symbol server for a tiny docs repo.

On a synthetic large Python repo with Serena missing, it reports ask-to-install and drafts the exact approval request:

This repository shows large-codebase signals (file-count-threshold).
Serena may help with symbol-level navigation, but it is not installed.
May I install/configure Serena for Codex and/or Claude?

And then it lists the commands it would need:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install -p 3.13 serena-agent
serena init
serena setup codex
claude mcp add --scope user serena -- serena start-mcp-server --context claude-code --project-from-cwd

That is the shape I wanted.

The agent can detect the gap. The agent can draft the request. The human still approves the mutation.

Tools are part of the threat model

This is where the security part of my brain gets loud.

An AI coding agent that can install its own tools is both useful and terrifying.

Useful because missing tools waste time. The Ralph-NG maiden voyage had exactly this problem: a task burned multiple attempts because rg was not installed and the executor kept generating rg commands anyway. A smarter workflow should notice that and either use a portable fallback or ask to install the missing binary.

Terrifying because “install this helpful tool” is a supply chain event.

It may touch your home directory. It may edit .codex or .claude. It may add an MCP server. It may download packages. It may clone a GitHub repo. It may introduce a hosted service that sees part of your code. If you let an agent do all of that silently because it sounded productive, congratulations, you built a very polite persistence mechanism.

So the workflow draws a hard line.

The readiness script is read only. It inspects files and PATH. It does not install packages. It does not edit config. It does not clone GitHub repos. It does not start MCP servers.

If it thinks a tool is useful but missing, it produces an approval request with:

• the exact commands;
• the paths that may be written;
• the network or hosted-service risk;
• any credentials or API keys required;
• rollback notes.

The agent can be proactive. It cannot be sneaky.

That is the difference between autonomy and nonsense.

The install catalog is boring on purpose

The new large codebase reference now documents the install path for the tools I actually want agents to think about:

• uv, because Serena’s recommended install path uses it;
• rg, because exact search is still the default first move;
• Serena, because symbol navigation matters in large repos;
• ast-grep, because syntax-shaped search is different from text search;
• grepai, because local semantic search is sometimes useful when names do not match concepts;
• Ollama, because grepai’s local default needs embeddings;
• WarpGrep/Morph, because hosted search exists and the data path needs to be explicit.

I resisted the urge to make this a plugin marketplace.

That is not the job.

The workflow should know enough to say, “This task would benefit from Serena, Serena is missing, here is the install and config request.” It should not become a package manager with opinions.

For Codex, the Serena setup is either serena setup codex or a [mcp_servers.serena] entry in ~/.codex/config.toml or a trusted project .codex/config.toml.

For Claude Code, the documented path is:

claude mcp add --scope user serena -- serena start-mcp-server --context claude-code --project-from-cwd

or a project-pinned variant.

For hosted Morph/WarpGrep, the workflow calls out the thing people like to skip over: code or query context may leave the machine. That does not mean “never use it.” It means ask first, and be honest about the data path.

Why this belongs in the workflow

Some of this sounds like tool documentation.

It is, but the documentation is not the interesting part.

The interesting part is making the agent responsible for noticing when its current retrieval strategy is becoming stupid.

An agent should be able to say:

I am in a large TypeScript monorepo. This task asks for callers and implementations across packages. rg can help, but symbol navigation would reduce the chance I read the wrong files. Serena is missing. Here is the approval request.

That is useful.

An agent should also be able to say:

This repo is small. Serena is not installed, but I do not need it for this task.

That is just as useful.

Most workflow systems only encode what the agent should do after it starts coding. I want this one to encode when the agent should improve its own operating conditions, and when it should leave them alone.

That is a different kind of judgment.

The anti-bloat pass mattered too

While I was in there, I added a simplicity gate to the review phase.

Before the agent adds new code, dependencies, generated files, MCP servers, or external repos, it has to ask a few boring questions:

• does the behavior already exist in the repo?
• can the standard library or an installed dependency solve it?
• is this the smallest safe diff?
• did the change preserve correctness, security, tests, and the user’s scope?

This is the Ponytail lesson without importing a whole persona.

Agents love adding machinery. They especially love adding machinery when the prompt sounds important. “Large codebase acceleration” is exactly the sort of phrase that can turn into five new abstraction layers and a YAML file nobody asked for.

The simplicity gate is there to make the agent prove the extra machinery is buying something.

In this case, I think it is. The readiness script is small, read only, covered by tests, and tied directly to a workflow decision.

That is the bar.

The tests are part of the feature

I added tests for the boring promises, because those are the promises agents tend to break first.

The tests check that:

• a large project with Serena missing produces ask-to-install;
• the tool catalog has install instructions and risk notes for the identified tools;
• the CLI JSON says it is read-only;
• the approval request includes commands and write targets;
• the approval request does not include prose pretending to be a shell command.

That last one came from catching my own sloppy first pass. I had a command list that included “or edit config…” as text. That is fine in documentation. It is not fine in an approval payload that an agent may later try to execute.

This is why I like tests even for workflow code.

They catch the dumb stuff before the agent turns it into automation.

What changed in the repo

The update added:

• references/large-codebase.md, with search routing, context-budget policy, install catalog, Serena setup, semantic search notes, and permission-gated setup rules;
• references/pilot-measurement.md, because efficiency claims stay hypotheses until we measure them;
• scripts/large_codebase_tools.py, the adaptive readiness check;
• template changes so generated Codex and Claude instructions tell agents to ask before touching .codex, .claude, dependencies, or GitHub checkouts;
• workflow pointers so large or unfamiliar repos route through the new reference and readiness script;
• tests for the behavior.

I also corrected the research report that kicked this off. A few claims were too strong. Serena efficiency numbers are still hypotheses. Semantic search should not be treated as equal to lexical and symbol search. Ponytail’s exact ladder was not confirmed as written, but the anti-bloat mechanism is worth keeping.

That matters.

If the whole point of the workflow is evidence before claims, the workflow’s own documentation has to live by the same rule.

Where this goes next

The obvious next step is an apply mode.

Not today.

The shape I want is something like:

large_codebase_tools.py --project-root . --json
large_codebase_tools.py --write-plan serena
large_codebase_tools.py --apply approved-plan.json

But I am not rushing that.

The read-only readiness check gives the agent enough agency to notice missing tools and ask for them. That already closes the worst gap. Actually applying changes to ~/.codex/config.toml, .codex/config.toml, .claude, package caches, or cloned GitHub repos needs a tighter plan format and better rollback handling.

I want that, but I want it boring.

The good version is not a dramatic autonomous installer. The good version is a cautious agent saying, “This would help. Here is the exact plan. Do you approve?”

That is the pattern I keep coming back to with AI coding agents.

Give them initiative.

Give them evidence.

Give them tools.

Do not give them silent authority over your machine.

That is not fear. That is engineering.