MCP gateway: 116 tools at constant context cost

The first version of every agent integration I've built looks the same. You write a handful of tools, register them all, and the model gets better. Then you add twenty more, and it gets worse.

The reason is boring: every tool schema you advertise over MCP is serialized into the model's context on every single turn. Names, descriptions, argument shapes. At a few dozen tools you're burning thousands of tokens before the user has said anything, and the model now has to pick the right tool out of a long, noisy list.

For Harbormaster, an authenticated MCP gateway we built this year to keep the MCP context window small, the registry holds 116 tools. The model sees seven.

Hot path vs. long tail

Tools don't get used evenly. In a RAG-heavy product, four tools (search, fetch, and a couple of retrieval variants) account for most calls. Everything else is long tail: admin operations, integrations, one-off utilities.

So we split them:

Hot-path tools are promoted to native MCP tools. The model sees them directly and calls them in one hop.
Everything else hides behind a three-step discovery loop: search_tools, describe_tool, invoke_tool.

That's the whole trick. tools/list returns the same seven entries whether the registry has 116 tools or 10,000.

The MCP tool discovery loop

Each step returns only what the model needs to take the next one.

src/lib/tools/discovery.ts

export const DEFAULT_DISCOVERY_LIMIT = 10;
export const MAX_DISCOVERY_LIMIT = 25;
export const MAX_DISCOVERY_QUERY_CHARS = 2000;
 
export type SearchToolsResult = {
  total_registry_size: number;
  namespaces: { namespace: string; count: number }[];
  results: {
    name: string;
    description: string;
    simulated: boolean;
    relevance: number;
  }[];
  next_step: string; 
};

Two details matter more than they look:

next_step is part of the payload. The search result tells the model what to call next, in plain language. You're not relying on the system prompt to explain the protocol; the protocol explains itself at the moment it's needed.
Limits are hard. Search returns at most 25 results and caps the query length. An agent in a loop will happily ask for everything. Don't let it.

describe_tool then returns the argument schema for exactly one tool, and invoke_tool runs it through the same auth and validation path as a native call.

Keep a trace

Every discovery run records its steps:

export type DiscoveryStep = {
  tool: "search_tools" | "describe_tool" | "invoke_tool";
  args: Record<string, unknown>;
  result: unknown;
  error?: string;
};
 
export type DiscoveryTrace = {
  query: string;
  picked: string | null;
  steps: DiscoveryStep[];
};

This turned out to be the most useful part of the system. When the model picks the wrong tool, the trace shows you whether search ranked it badly, the description was ambiguous, or the model ignored a good result. Those are three different fixes, and without the trace they all look the same: "the agent did something dumb."

What it costs

Discovery adds two round trips for long-tail tools. In practice that's fine, because long-tail tools are, by definition, rare. The trade is a small latency hit on uncommon calls in exchange for a smaller, cleaner context on every call.

The broader lesson applies well beyond MCP: context is a budget, and anything that costs you on every turn deserves more scrutiny than anything that costs you once.