Agent approvals in tiers: why too many prompts make things less safe

On 23 October 2026 Anthropic described how its agent Claude Duet handles approvals. One sentence in it applies to anyone running agents in production: approval fatigue is a serious problem, because too many requests teach people to tap through them. Duet itself is a product for US subscribers. The question behind it concerns every environment where an agent may change something, ours included.

This post keeps two things apart. What Anthropic and an arXiv preprint write is sourced. From our own operation we describe only what our configuration and operating notes contain. We have no measurement of our own approval rate, so it appears below as an open point.

At a glance

Anthropic calls approval fatigue a serious problem, and Duet blocks certain actions even with approval. A preprint shows in a model that more human review can lower safety. We split agent actions into four tiers: no prompt, a narrow standing permission, approval per operation, and never. How often people actually click through here, we have not measured.

An approval tier defines which actions of an agent run without a prompt, which need a human confirmation, and which never run even with confirmation. The goal is to keep confirmations rare enough that people still read them. The tier is a decision somebody made beforehand and wrote down.

What Anthropic writes about approvals

The Duet safety post of 23 October describes four things that matter here.

• Before a sensitive message, a payment or a deletion, Duet shows the action, the destination and the content including every recipient, and waits for confirmation.
• A separate automated check runs alongside, built on the auto mode classifier from Claude Code. It runs outside the model doing the work, so instructions hidden in a page cannot influence it.
• Some actions stay off-limits even with approval: trading securities, banking actions such as moving money between accounts, and accessing someone else's account.
• Paying or sending money, deleting an order and creating an account are not covered by a "don't ask me again" rule, however often they were approved before.

On availability, the announcement of 21 October names a research preview on a waitlist, starting with US Pro, Max and Ultra subscribers. Anyone quoting only Pro and Max leaves Ultra out.

The engineering post on auto mode from 25 March 2026 supplies the tier logic. Tier 1 is a list of read-only tools plus your own allow rules. In tier 2, file writes inside the project folder run without a classifier. In tier 3 a classifier reviews everything else, including shell commands and web fetches. By default it blocks, among other things, production deploys, installing persistence such as cronjobs, and changes to the agent's own permission configuration. That post also contains the figure many reports rely on: users accept 93 percent of manual prompts anyway. The post does not say how Anthropic measured it.

What a preprint calculates

Emre Turan posted a preprint on arXiv on 8 June 2026, with no journal or conference listed. It evaluates 125 hand-labeled agent actions. The reviewers only moderately agreed on what counts as risky (Fleiss' kappa 0.52). So there is no single ground truth about which action deserves a prompt.

The second result is a model. If reviewer fatigue grows with the number of prompts, realized safety follows an inverted U over the escalation rate. More human oversight can make the system less safe, and the safest setting does not escalate everything. The author describes this as a modeling result meant to motivate a study with real reviewers. The paper also describes a flooding attack: whoever floods the reviewer with requests can slip a harmful action through. That too is a modeling result meant to motivate a study with humans.

How we tier actions

The table below states rules and contains no measurements. They come from our operating notes and the configuration of our workstations, as of 5 November 2026, and each note carries its own date.

Three observations from those notes matter for the rest of this post.

An approval does not transfer. On 20 August 2026 a host was restarted twice into a running setup. A machine that had been explicitly excluded counted as released afterwards, because a later list named it. Nobody had lifted the exclusion. Since then an approval does not carry over to other hosts or to later operations on the same host, and lifting an exclusion has to be stated explicitly. That is the counterpart to "don't ask again": permissions stay small enough that they cannot grow unnoticed.

A human changes the agent's rights. The classifier rejected a production deploy, a new systemd timer and the agent writing its own settings. These are three of the categories Anthropic lists as default blocks. The fix was a narrow entry in the auto mode configuration, which the user sets personally. When something is rejected, our note says not to work around it, not to split it up, to prepare the result and say what is missing.

The click barrier is not everywhere. The configuration also contains a blanket entry that SSH commands from the workstation do not ask. There the limit sits in the zone rule and not in the dialog. The decision was made deliberately on 4 September 2026, and it is where our tiers are thinnest.

One thing we did measure is the size of the configuration. The mirrored settings file of our workstations holds seven auto mode permissions (one of them a reference to the defaults) and three soft denies. The local file lists two additional allow rules.

We measured with a read-only Python call that counts entries in the JSON files. These are our own figures and no vendor claims. They say something about the size of the rule set and nothing about how often prompts appear.

Where fatigue and the cost of asking show up for us

Our notes document two failure directions, both without a count.

On 2 November 2026 an overnight test was requested, and the agent asked back instead of starting the harmless part at once. The night was lost. The rule derived from it: start the part that changes nothing immediately, ask only about the steps behind a real approval, and name those steps in the same message.

The other direction is the note of 12 August 2026. Three escalations in one day, two of them unfounded, cost time and, in one case, a statement that had been written into several documents and had to be withdrawn. Since then a check order applies: does an earlier statement depend on it, does the suspicion have a signature, can the business owner explain it in one sentence.

Both are single cases. They show that prompts have costs, and that nobody here counts the rate.

Still to measure: approval rate and response time

Anthropic gives 93 percent for Claude Code overall. For our operation there is no figure, and we are not supplying one.

Method for the measurement, which has not run yet: over 14 working days, write every permission request of the agent on the workstation to a log with timestamp, tool, decision and seconds until the answer. This can be done through Claude Code's hook events. Whether those events capture the decision completely is part of the groundwork and has not been checked. Evaluate per tool class: read, file write, shell, SSH. Response time is the more interesting value than the rate. Very short response times would hint that requests are not read, which is our assumption and something the measurement would have to test.

A caveat for the evaluation: auto mode decisions do not appear as prompts and are missing from the count. The rate would therefore not be comparable to Anthropic's figure, and we would say so.

What is established and what is open

Established, with a primary source:

• Duet is a research preview on a waitlist, first for US Pro, Max and Ultra subscribers (Anthropic, 21 October 2026).
• Approvals show action, destination and content. The check runs outside the model. Some actions stay off-limits even with approval, others are excluded from "don't ask again" (Anthropic, 23 October 2026).
• Anthropic calls approval fatigue a serious problem and says it will keep adjusting how often it asks (same post).
• In Claude Code 93 percent of manual prompts are accepted, with no statement on the measurement (Anthropic, 25 March 2026).
• 125 actions, moderate reviewer agreement, an inverted U in the model (Turan, arXiv 2606.08919, preprint).

Open:

• The safety post gives no figure for how many prompts Duet raises or how often they are confirmed.
• The inverted U is a model. A trial with real reviewers is missing, according to the author.
• How often people click through here is not measured.
• Whether our tier 4 is complete, we have not checked.

Check it in ten minutes

1. Write down which identities in your setup use agents (developer account, service account, chat agent).
2. Create three lists: runs without a prompt, needs confirmation, never runs.
3. Remove from the confirmation list everything that only reads or undoes itself. It belongs in the first list.
4. For every standing permission, check that it names host, operation and date. If one of the three is missing, the permission is too broad.
5. Put everything that moves money, creates an account or deletes irreversibly on the third list, even if the agent has never tried it.

How such lists connect with rights and logging in a larger environment is covered in our post on Claude mods and their permissions. We collect the topic in the hub AI in production. If you want to set up a tier model for your own agents, our offer is described under Corporate LLM.

Sources last checked: 5 November 2026.

Further Reading

• Anthropic: Claude Duet Safety, Privacy, and Security (23 October 2026)
• Anthropic: Introducing Claude Duet (21 October 2026)
• Anthropic Engineering: How we built Claude Code auto mode
• Turan: Oversight Has a Capacity, arXiv 2606.08919