AI agent guardrails: read-only by construction

One of our products lets customers define their own AI agents: they write the instructions, we run them against their data. The obvious risk, and the reason AI agent guardrails matter, is that a prompt gets widened over time until an agent that was supposed to summarize things can also change things.

The obvious fix, a hand-maintained allow-list of safe tools, fails quietly. Someone adds a new tool that writes data, forgets to update the list, and nothing breaks until something breaks badly.

Agent permissions: make the tool declare itself

Instead, every tool in the registry declares whether it mutates state. Custom agents get a tool set that is derived from that flag, not curated:

type Tool = {
  name: string
  mutates: boolean
  run: (args: unknown, ctx: Context) => Promise<unknown>
}
 
export function readOnlyTools(registry: Tool[]): Tool[] {
  return registry.filter((t) => !t.mutates)
}

Now the safe default is automatic. A new write tool is excluded from user-authored agents the moment it's registered, whether or not anyone remembers this code exists. The only way to get a write tool into a custom agent is to change the tool's own definition, which is exactly where that decision should be reviewed.

Budgets and deadlines are part of the contract

Every custom agent runs with a fixed tool-call budget and a wall-clock deadline, and the result says whether either was hit:

export const CUSTOM_AGENT_BUDGET = 8
export const CUSTOM_DEADLINE_MS = 45_000
 
export interface CustomAgentResult {
  output: string
  guardrails: {
    checked: number
    ok: boolean
    violations: { rule: string; severity: string; note: string }[]
  }
  budgetExhausted: boolean
  deadlineHit: boolean
}

Report, don't hide

The subtle design choice here: guardrail checks are reported alongside the output, not enforced by silently withholding it. If an answer trips a rule, the user sees the answer and the violation.

Silent enforcement feels safer but makes the system unexplainable. Users see an empty or vague response and can't tell whether the agent failed, the data was missing, or a rule fired. Visible guardrails build the kind of trust you need before anyone lets an agent near real work.

Related: LLM guardrails: assert before you render · AI agent testing: catch the failures that look like success