Capability, not competence: agent skills as maps and GPS

Published October 17, 2026.

Ask an agent how many of the 185 skill folders in a large public catalog contain a scripts/ directory. If it reads the listing and answers, it's estimating. A script answers by counting: 3. The model didn't lack intelligence. It lacked an instrument.

That's the whole argument in one task. I don't care whether a model can do the job after I take its tools away. I care how much it can do with them. Some skills are maps: they show the route and the branch points, a script measures what the model can't see by reading, and the model still makes the call. Some are GPS: they drive deterministic work end to end. Most real work needs both, in sequence.

The rule

Build instruments for facts a program can measure. Keep judgment with the agent when objectives or evidence are ambiguous. Automate end to end when postconditions are checkable and failure is contained. Measure task quality, cost, and recoverability against a no-skill baseline.

The common shape is a three-step hybrid:

+---------------------+     +----------------------+     +--------------------------+
| 1. Script extracts  | --> | 2. Agent judges the  | --> | 3. Script validates and  |
|    the facts        |     |    ambiguous cases   |     |    commits the result    |
|    (deterministic)  |     |    (judgment)        |     |    (checkable)           |
+---------------------+     +----------------------+     +--------------------------+

Four questions decide where each step sits:

• Ambiguity. Is the objective or the evidence open to interpretation? Then the agent holds that step.
• Reversibility. Can you undo a wrong result? Then automating it is safer.
• Error cost. What does a wrong answer break, and who notices?
• Checkable postconditions. Can a separate program confirm the result? Then a script can own it end to end.

Map or GPS: who holds the decision?

Map and GPS are the two ends of a spectrum, not two boxes. A skill that drives a task end to end can beat a map on the exact task it was built for. So the choice comes down to who should hold the decision, and who checks it.

Those are two different authorities. Execution authority is who takes the step. Verification authority is who confirms it was right. A map keeps both with the model: the model reads the evidence a script prints, compares it with what it expected, and chooses the next branch. When the route is wrong, it can notice. A GPS takes execution authority. A GPS that executes without independent checks may hide errors; a GPS with explicit checks can remain auditable. Counting, parsing, and checking a reference are good GPS work, because a script can prove the answer.

Which arrangement wins on a given task is an empirical question, and the test is below.

Capability vs competence

Capability is what a tool lets a model do while it has the tool. Competence is what the model can still do after the tool is taken away. For agents, I optimize capability.

David Krakauer's paper, Competitive and Complementary Tools (arXiv 2607.18460), gives both terms precise meanings. Competence is "the performance that remains when the tool is withdrawn." Capability "is what tools add."

Here I need to separate the model, the session, and the system. Without weight updates, using a skill does not increase the base model's learned competence; a persistent agent system can still accumulate knowledge outside its weights. When I say competence after withdrawal, I mean the base model with the skills, scripts, and external memory all removed. Measured that way, no skill I give an agent is complementary, because nothing transfers into the weights. That's fine. I'm trying to raise capability, and capability is measured with the tool present.

Where the frame comes from

Krakauer, president of the Santa Fe Institute, discussed whether AI tools are complementary or competitive on StarTalk in What Is Intelligence? With David Krakauer (Oct 2, 2026). His paper defines the split. "I call a tool complementary when competence transfers to the user and persists after withdrawal, and competitive when the user grows more dependent the more it is used and tool-free competence is compromised." An abacus or a map is complementary. A GPS is competitive. He doesn't put language models in a single column: "When its output is reconstructed and verified, it transfers a method and augments intelligence. When its answers are accepted uncritically ... it diminishes intelligence." The paper also quotes Henri Bergson's definition of intelligence as "the faculty of manufacturing artificial objects, especially tools to make tools." Tools that build tools is the goal of ai-agents.

Research on people and AI argues for human independence. The GPS for Thinking proposes a compass model so AI will "strengthen, not supplant, the student's inner compass." Not all cognitive offloading is equal separates dependent offloading from autonomous offloading. That goal is right for a student. For an agent, dependence on a good tool is the goal.

Claude skills with scripts: maps with instruments

A map with no instruments tells the model where to look but gives it nothing to look with. The script is the instrument. It does the deterministic work and reports what it found, and the model makes the call.

Anthropic's skill guidance already favors scripts. Its engineering post says "many applications require the deterministic reliability that only code can provide." Its building guide says "Code is deterministic; language interpretation isn't." That's the reliability argument. Mine adds one thing: a script that prints evidence keeps the model checking the route.

In ai-agents, skill-bootstrap produces workflow skills with a first useful action, discriminating observations, decision branches, and outcome checks (.claude/skills/skill-bootstrap/references/authoring-and-gates.md). The model still chooses the branch. ai-agents PR #6642, open as of Oct 17, 2026, adds the instrument side. Each verified item is sorted as a scaffold (prose the model reasons with), an instrument (a script that computes the answer), or both. Then bootstrap_forms.py check rejects a computable item that has no script.

The skill started as a gist I adapted in July from a prompt Fable 5 generated for u/Rodbourn, shared in an r/ClaudeAI thread started by u/oj93-rd. It asks a stronger model to write skills so smaller models can "carry this project forward without you." It also asks for instruments: "MEASURE instead of eyeball ... ship actual scripts inside the skill's scripts/ dir where they exist or where you can write them."

Skill catalogs: count is not capability

wshobson/agents has 185 skill folders, and at commit 044b4b2 (Oct 15, 2026), 3 of them contain a scripts/ directory. In ai-agents, 56 of 121 skill folders do, at commit e2b9c1e (Oct 17, 2026). Treat those counts as inventory, not evidence of capability. A count tells you how broad a catalog is. It doesn't tell you whether any one skill makes the model more capable.

The primary measure is outcome quality: the task scored with the skill and without it. Cost, latency, variance, and how often a person has to step in are guardrails. A skill that raises quality but makes runs erratic hasn't earned its place yet.

For every skill, I want five answers. Which baseline failure does it fix? What part is deterministic? Who makes the ambiguous decisions? How are failures detected? What measured improvement justifies maintaining it?

How to test whether a skill helps

The ai-agents scenario runner already compares with and without. It scores a baseline cell with no skill, a description cell, and a full cell. A skill passes only when its best cell averages at least 3.5 and beats baseline by at least 0.5 (scripts/eval/eval-rule-activation.py).

That runner measures whether the model follows the skill's guidance. This argument also needs to know whether the task came out right, so PR #6642 adds a capability eval that scores the task itself, with the instruments and without them. Its first run showed no demonstrated accuracy gain on these four tasks with Haiku. F1 went from 0.867 to 0.872 on one task and stayed at 1.000 on the other three. Cost per run went up on all four.

That doesn't mean instruments have no benefit. For a crisp question, Haiku wrote its own throwaway script, so the task got an instrument either way and a shipped script had no gap to fill. Accuracy is one outcome. The next runs also need execution time, consistency across runs, and recovery after a failure.

What I measure next

1. Outcome quality on an unfiltered, representative task set, plus a harder subset the model fails without help.
2. Repeated paired runs with confidence intervals, success criteria declared before the run, and held-out scenarios.
3. Four conditions with equal tool-access budgets: no skill, map, GPS, and the hybrid.
4. Guardrails for each condition: cost, latency, variance, intervention rate, and wrong turns the agent catches on its own.

I'll publish the numbers when I have them, including any that go against me. The skills, scripts, and evals are in ai-agents. To try it on your own repo, start with skill-bootstrap.

What task does your agent keep failing until you hand it a tool? If you've measured a skill with and without it, or have one you think should be a GPS, open an issue on ai-agents.