Tool Ingestion Engineering for LLMs: Building Secure Agent Execution Infrastructure

Large language models are powerful reasoning and generation engines, but enterprise autonomy begins only when a model can interact safely with systems outside its context window. An agent that can search documentation, query a database, open a ticket, modify a customer record, trigger a workflow, or call an internal API has crossed an architectural boundary: it is no longer only generating text. It is participating in a software execution environment.

That boundary requires more than function-calling syntax. Production systems need a disciplined process for discovering tools, acquiring their schemas, validating provenance, normalizing heterogeneous interfaces, classifying risk, binding permissions, registering versions, exposing only relevant capabilities to the model, validating arguments, brokering credentials, executing calls, validating results, and recording the entire transaction. This broader discipline is tool ingestion engineering for LLMs.

The distinction matters because tool calling is only one runtime mechanism inside a much larger capability supply chain. A model may generate a syntactically valid function call and still invoke the wrong capability, target an unauthorized resource, retry a non-idempotent mutation, ingest a poisoned tool response, or act on a silently changed schema. The surrounding infrastructure must therefore enforce strict validation and approval controls and bounded execution policies independently of model behavior.

This infrastructure also complements the memory and state layers described in the Stateful Agentic AI Systems Architecture. Stateful agents retain task context and execution history; tool-ingestion systems determine which external capabilities are available, under what policy, with which schema, and through which trusted execution path. Together, they form the operational substrate for reliable multi-step and multi-agent orchestration.

Key Takeaways

• Tool calling is not tool ingestion. Tool ingestion covers the entire lifecycle from discovery and trust validation to runtime execution and retirement.
• The model must never be the security boundary. Authentication, authorization, schema validation, approval, credential brokering, and sandboxing belong in deterministic infrastructure.
• Tool metadata and tool results are untrusted inputs. External schemas, descriptions, and returned content can contain malicious or misleading instructions.
• Large tool catalogs should be retrieved dynamically. Enterprises should not inject every available tool definition into every prompt.
• Retries require idempotency semantics. Blindly retrying mutations can duplicate financial, operational, or customer-impacting actions.
• Tool execution must be observable. Every invocation should be correlated with identity, policy decisions, schema version, latency, cost, result status, and trace identifiers.

What Is Tool Ingestion Engineering for LLMs?

Tool ingestion engineering is the process of converting heterogeneous executable capabilities into a governed internal representation that an AI agent can discover and use safely. The source capability may be a REST endpoint, an MCP tool, a GraphQL mutation, a database procedure, a Python function, a SaaS action, or a long-running workflow. The agent should not reason directly against all of those interfaces in their native forms. Instead, the platform translates them into a canonical tool contract.

A useful mental model is:

Raw Capability
      ↓
Discovery / Acquisition
      ↓
Canonical Tool Contract
      ↓
Security + Policy Binding
      ↓
Versioned Registry Entry
      ↓
Model-Visible Capability
      ↓
Validated Runtime Execution

The canonical contract is the key abstraction. It allows the rest of the platform to treat tools uniformly even when the underlying transports differ. An MCP server might expose JSON Schema natively. An OpenAPI service might require extracting request and response definitions. An internal Python function might derive its contract from type annotations and Pydantic models. A legacy database procedure may require a hand-authored wrapper. Once normalized, however, every capability can pass through the same registry, policy, observability, and execution layers.

Tool Ingestion vs. Tool Calling vs. Tool Execution

These terms are often used interchangeably, but production architecture benefits from separating them.

Concept	Primary Responsibility	Typical Components
Tool Ingestion	Transforms an external capability into a governed platform asset	Discovery, schema parsing, normalization, provenance, risk classification, registry, versioning
Tool Selection	Determines which approved tools are relevant to the current task	Capability retrieval, semantic ranking, policy filters, context budgeting
Tool Calling	Allows the model to express an intention to invoke a tool	Structured output, tool-use tokens, constrained decoding, JSON/function call payload
Tool Execution	Performs the actual external operation	Execution gateway, API client, credential broker, queue/worker, sandbox
Tool Result Ingestion	Safely returns external data to the model	Output validation, field projection, size limits, sanitization, context hydration

This separation prevents a common architectural mistake: assuming that because a model can emit a valid JSON object, the system has solved tool integration. Structured generation improves syntax, but it does not prove that a tool is trustworthy, that a user is authorized to invoke it, that the selected operation is appropriate, or that returned content is safe to place back into the model’s context.

The Enterprise Tool Ingestion Lifecycle

A mature tool platform should treat tool definitions as governed software artifacts. The lifecycle begins before the model ever sees a function name.

1. Discover the Capability Source

Capabilities can enter the platform through an approved MCP server, OpenAPI document, code repository, service catalog, database adapter, event system, or manually registered integration. Discovery should record where the tool came from and which organization, repository, server, or service owner is accountable for it.

2. Verify Provenance and Trust

Tool definitions are executable supply-chain inputs. A tool with a plausible name and description can still be malicious or compromised. Microsoft recommends treating MCP tool descriptions and responses as untrusted context and reviewing schema changes before they reach production. OWASP similarly identifies tool poisoning, schema poisoning, scope creep, secret exposure, and command injection as distinct MCP risks. See the Microsoft Azure MCP security guidance and the OWASP MCP Top 10.

Production registries should therefore retain provenance metadata such as source URI, server identity, owner, signature or hash, discovery timestamp, approval status, and the definition fingerprint used during the invocation.

3. Acquire and Parse the Schema

The platform extracts the tool name, human-readable description, input schema, output schema when available, transport metadata, and any source-specific annotations. This is where OpenAPI, MCP, internal code annotations, and other source formats diverge.

4. Normalize Into a Canonical Contract

Normalization translates source-specific details into a common internal model. This reduces downstream complexity because authorization, ranking, approval, execution, and logging can reason against stable fields rather than protocol-specific structures.

5. Classify Risk and Side Effects

The tool is categorized by whether it reads data, writes data, performs destructive actions, communicates externally, handles money, touches privileged infrastructure, or crosses trust boundaries. Risk classification drives default policies and approval requirements.

6. Bind Identity, Authorization, and Policy

A tool should have explicit scopes, tenant restrictions, environment constraints, resource limits, and approval rules before it becomes model-visible. The registry should describe what the tool can do; the policy engine should decide who can do it now.

7. Version, Test, Approve, and Publish

Schema changes should flow through CI and contract tests rather than becoming immediately available. Silent definition changes are particularly dangerous in agent systems because the model may receive new instructions or parameter semantics without a corresponding application deployment.

Canonical Tool Contracts and Schema Normalization

A canonical tool contract should contain more than a name and JSON Schema. It needs enough metadata for the platform to reason about operational and security consequences.

{
  "tool_id": "crm.customer.update",
  "version": "3.2.1",
  "source": "mcp://crm-prod",
  "description": "Update permitted customer profile attributes.",
  "input_schema": {},
  "output_schema": {},
  "risk_class": "reversible_write",
  "side_effects": true,
  "destructive": false,
  "idempotent": false,
  "approval_required": true,
  "required_scopes": ["customer.write"],
  "credential_mode": "delegated_oauth",
  "timeout_ms": 5000,
  "max_retries": 1,
  "environment": "production",
  "owner": "crm-platform",
  "definition_hash": "sha256:..."
}

The exact fields vary by organization, but the principle is stable: the tool contract should carry the information required by infrastructure, not just the information needed by the model.

Avoid Universal Schema-Depth Rules

Shallow schemas can improve model reliability and reduce token overhead, but a universal rule such as “never exceed two levels of nesting” is too rigid. Modern MCP tooling supports full JSON Schema 2020-12 constructs, including references, composition, and conditional structures. The 2026-07-28 MCP release candidate explicitly expanded tool input and output schemas to full JSON Schema 2020-12. See the MCP 2026-07-28 release-candidate notes.

A better production policy is to bound schema complexity using measurable controls: maximum serialized size, maximum reference depth, maximum validation time, maximum property count, and approved composition patterns. Complex domain objects can legitimately require nesting; the platform should manage complexity rather than pretend it does not exist.

MCP, OpenAPI, APIs, and Internal Functions as Tool Sources

MCP is increasingly important because it standardizes how clients discover and invoke tools, but an enterprise tool-ingestion layer should remain broader than any one protocol. OpenAPI remains common for service contracts. Internal applications often expose typed functions directly. Data platforms may use SQL or RPC wrappers. Legacy systems may require adapters.

Source	Strength	Ingestion Concern
MCP	Native tool discovery, schemas, standardized invocation	Server trust, tool-definition drift, authorization, result poisoning
OpenAPI / REST	Widely deployed enterprise interfaces	Overly broad specs, endpoint selection, auth mapping, response reduction
GraphQL	Flexible typed queries and mutations	Query complexity, mutation risk, field-level authorization
gRPC	Strong contracts and efficient service calls	Protocol translation and message-schema exposure
Database Tools	Direct access to authoritative data	Least privilege, SQL safety, tenant isolation, transaction boundaries
Internal Functions	Low integration overhead	Implicit permissions, local side effects, poor lifecycle governance
Workflow Engines	Durable long-running processes	Job identity, callback state, retries, cancellation, compensation

MCP 2026-07-28 and Dynamic Tool Infrastructure

The MCP 2026-07-28 specification materially changes how architects should think about production MCP infrastructure. The release moves MCP toward a stateless request/response core, introduces routable method and tool headers, makes selected list responses cacheable, hardens authorization, formalizes extensions, and moves Tasks into an extension for durable long-running operations.

For tool-ingestion platforms, several changes are especially relevant:

• Stateless core: requests can be routed across ordinary scalable HTTP infrastructure without affinity to a long-lived protocol session.
• Header-based routing: method and tool names can be available to gateways for routing, metering, and authorization decisions.
• Cacheable tool catalogs: list responses can include cache hints, reducing repeated catalog fetches and helping maintain stable tool-definition prefixes.
• Authorization hardening: the specification strengthens issuer validation and credential isolation semantics.
• Tasks extension: long-running work can be represented independently of a synchronous tool-response loop.

These changes make MCP increasingly suitable as a transport and discovery layer, but they do not replace enterprise governance. MCP tool annotations are also explicitly hints, not trustworthy security assertions. The specification warns clients not to make sensitive decisions based solely on annotations from untrusted servers. A production platform should treat metadata such as read-only, destructive, or idempotent as claims that may require verification or policy override.

Tool Registry Architecture and Capability Catalogs

Hardcoding every tool directly into agent application code works for prototypes. It becomes brittle when an organization has hundreds of capabilities, multiple environments, changing versions, multiple teams, and different permission scopes. A registry decouples tool lifecycle from agent deployment.

The registry should answer questions such as:

• Which version of this tool is approved for production?
• Which agents or user roles may see it?
• Which scopes are required to invoke it?
• Is it read-only, reversible, destructive, financial, external-facing, or privileged?
• Which schema hash was approved?
• Which transport or execution adapter should handle it?
• What timeout, retry, and idempotency rules apply?
• Who owns the tool and who approved the current definition?

Registry-based routing also fits naturally with the LLM Gateway Orchestration pattern: the LLM gateway governs model access and routing, while the tool gateway governs capability access and execution.

Dynamic Tool Selection and Context Hydration

One of the most important scaling principles is that the full enterprise tool catalog should not be injected into every request. The original function-calling pattern often assumes that the model receives the complete set of tool schemas during prompt construction. That approach can work for a handful of functions, but it becomes expensive and noisy at enterprise scale.

Instead, tool selection should resemble retrieval:

Enterprise Tool Registry
        │
        │ thousands of registered capabilities
        ▼
Task / User Intent
        ↓
Identity & Tenant Filter
        ↓
Policy Filter
        ↓
Capability Retrieval
        ↓
Semantic / Metadata Ranking
        ↓
Top Relevant Tools
        ↓
Token Budget Allocation
        ↓
LLM Context

This approach reduces prompt overhead, lowers tool-selection ambiguity, and prevents irrelevant high-risk capabilities from entering the model’s decision space. The same context-hydration discipline used for state and memory can therefore be extended to tools: retrieve only what is fresh, authorized, relevant, and valuable for the current node in the workflow.

Pydantic AI’s current tool capabilities illustrate this direction by supporting deferred tool loading and search for a long tail of tools rather than eagerly loading everything. See the Pydantic AI advanced tool documentation.

The Core Runtime Mechanics: The Intercept-and-Execute Loop

Once relevant tools have been selected, the runtime loop begins. The model still does not execute code directly. It expresses an intention; deterministic infrastructure decides whether and how that intention becomes an external action.

Stage 1: Context Hydration

The model receives the current conversation or state plus the subset of tool contracts selected for this turn. Schema text consumes input tokens and contributes to prefill work. The exact token overhead depends on the tokenizer, description length, property count, examples, and schema complexity—not the parameter count of the model itself.

For self-hosted inference, these costs intersect with the LLM serving stack, KV-cache capacity, batching strategy, and prompt-prefix reuse. Schema optimization is therefore both a reasoning-quality concern and an inference-efficiency concern.

Stage 2: Structured Tool Intent

The model emits a structured tool request rather than ordinary prose. Different providers and open-source stacks implement this through combinations of tool-use training, provider-specific control tokens, JSON schema guidance, grammar-constrained decoding, or structured-output mechanisms. Architects should avoid assuming that every platform relies on the same “function-calling fine-tune.”

Stage 3: Interception and Parsing

The host runtime recognizes the tool-call payload and stops treating the model output as final user-facing text. The payload is parsed into a typed request, assigned an execution identifier, and passed to deterministic validation and policy layers.

Stage 4: Validation and Authorization

Input schema validation verifies structure. Business validation checks semantic rules. Authorization confirms the caller’s identity and permission. A risk engine determines whether human approval is required. These checks occur before credentials are acquired and before any downstream system is contacted.

Stage 5: Programmatic Execution

The execution gateway calls the external API, database, MCP server, workflow, or sandbox. This step may be synchronous or asynchronous and must enforce its own timeout and resource budget independently of the LLM generation timeout.

Stage 6: Result Validation and Re-Injection

The raw result is validated, reduced to approved fields, checked for excessive size or unsafe content, and serialized back into the agent context. Tool results are data, not trusted instructions.

KV-Cache Behavior Is Implementation-Dependent

It is inaccurate to assume that every tool call forces the inference server to rebuild the entire KV cache or always incurs two full prefill passes. In append-only conversations, a serving system may reuse the cached prefix and process only newly appended tool-result tokens. Whether that optimization is available depends on provider behavior, routing, cache retention, batching, prefix stability, and the serving engine.

A more accurate architecture is:

Initial Request:
Prompt + Selected Tool Schemas
        ↓
Prefill → Decode Tool Call

Tool Returns Data
        ↓
Reusable Cached Prefix (when available)
        +
New Tool Result Tokens
        ↓
Incremental Prefill
        ↓
Decode Continues

When prefix reuse is unavailable, context movement between workers, schema changes, or cache eviction can increase recomputation. That makes cache-aware routing and stable tool-definition ordering valuable infrastructure concerns, especially in high-throughput agent platforms. The same cost-management principles connect to enterprise semantic caching and broader strategies to reduce AI API and inference cost.

Strict Input Schema Validation Without False Confidence

Loose schemas force the model to infer structure. Strict schemas reduce ambiguity, but syntactic validity is not semantic correctness. A payload can satisfy JSON Schema and still request the wrong customer, an unauthorized account, an excessive refund, or a destructive action.

Validation Layer	Question Answered	Example
Syntax	Can the payload be parsed?	Valid JSON object
Schema	Are fields and types allowed?	customer_id is integer
Field Constraint	Are values within legal bounds?	refund_amount > 0 and ≤ configured maximum
Business Rule	Does the action make sense?	Customer account is eligible for refund
Authorization	May this identity perform it?	Caller has refund.write scope
Risk Policy	Does it require approval?	Refund above threshold pauses for human review

Regex constraints, numeric bounds, enumerations, discriminated unions, and explicit required fields can all improve reliability. Pydantic models are particularly useful in Python because validation happens before business code executes. However, validation should be paired with secure database and service access patterns, not treated as a substitute for them.

Output Validation and Result Sanitization

Input validation receives most of the attention in tool-calling examples, but output validation is equally important. A tool response may be malformed, excessively large, contain secrets, include cross-tenant data, or carry hidden natural-language instructions designed to manipulate the model.

OWASP describes MCP tool poisoning as an indirect prompt-injection pattern in which malicious tool output enters the model context and influences subsequent actions. See OWASP MCP Tool Poisoning.

A production result pipeline should look like:

External Tool
     ↓
Raw Result
     ↓
Content-Type + Size Validation
     ↓
Output Schema Validation
     ↓
Tenant / Permission Projection
     ↓
Secret and Sensitive-Field Filtering
     ↓
Injection / Content Risk Inspection
     ↓
Whitelisted Field Projection
     ↓
Bounded Serialization
     ↓
Agent Context

Microsoft’s current Zero Trust guidance for AI systems similarly recommends treating user prompts, retrieved documents, tool responses, and memory as untrusted input. See Input, Context, and Retrieval Hygiene.

Authentication, Authorization, RBAC, and Tool Visibility

Authentication establishes who is acting. Authorization determines what that identity may do. Tool visibility determines which capabilities are worth showing the model. These are related but different controls.

Identity
   ↓
Authentication
   ↓
Authorization
   ↓
Tenant / Environment Policy
   ↓
Tool Visibility Filter
   ↓
Invocation Authorization
   ↓
Execution Policy

Tool invisibility is not authorization. Hiding an unauthorized tool from the model reduces confusion and attack surface, but the execution gateway must still reject a forged invocation. Model context is not a security boundary.

Authorization should be evaluated at invocation time against the user, agent identity, tenant, tool, resource, environment, and requested operation. Long-lived shared service accounts undermine attribution and widen blast radius. OWASP’s MCP guidance specifically warns against scope creep and recommends least privilege, scoped identities, expiration, and auditable entitlement changes.

Tool Risk Classification and Human Approval

Not every tool deserves the same execution path. A read-only documentation lookup is fundamentally different from deleting a production database, sending an external email, issuing a refund, or rotating infrastructure credentials.

Risk Class	Example	Default Control
Read-only	Search documentation	Automatic if authorized
Low-risk additive write	Add CRM note	Scoped authorization + audit
Reversible mutation	Update ticket status	Validation + audit + optional approval threshold
Destructive	Delete customer record	Human approval + strong policy gate
Financial	Issue refund	Transaction limit + approval + idempotency
Privileged infrastructure	Restart production service	Strong authentication + approval + isolated runner
External communication	Send customer email	Recipient policy + content review + audit

MCP tool annotations include hints such as read-only, destructive, idempotent, and open-world behavior, but the specification explicitly warns that annotations from untrusted servers should not be trusted as security facts. Enterprise risk classification should therefore be derived from approved policy and verification, not blindly copied from upstream metadata.

High-impact operations should integrate with human-in-the-loop AI workflows so the approval happens outside the model’s own reasoning channel.

Tool Poisoning, Indirect Prompt Injection, and Result-Side Attacks

Tool ingestion creates a software supply-chain problem as well as a runtime prompt-injection problem. A malicious or compromised tool can attack the system through its definition, description, schema, behavior, or returned content.

Important threat classes include:

• Tool-definition poisoning: a tool description contains instructions intended to manipulate selection or behavior.
• Schema rug pull: a previously approved tool changes its schema or semantics after approval.
• Tool shadowing: a malicious capability imitates a trusted tool name or purpose.
• Indirect prompt injection: external content returned by a tool contains instructions that the model mistakes for trusted directives.
• Command injection: agent-generated arguments are embedded unsafely into shell commands or queries.
• SSRF: network-capable tools are manipulated into contacting internal or sensitive endpoints.
• Credential exfiltration: secrets leak into prompts, outputs, logs, or downstream calls.
• Confused deputy: an agent with elevated privileges performs an action on behalf of untrusted input.
• Cross-tenant leakage: a tool returns records outside the current identity’s authorization boundary.

OWASP recommends server-side enforcement, least privilege, tool isolation, schema validation, approved-server allowlists, and explicit confirmation for sensitive actions. Its command-injection guidance also warns against shell construction from model-controlled strings and recommends structured parameters, allowlists, sandboxing, and non-root execution. See OWASP MCP Command Injection & Execution.

Credential Isolation and Secret Brokering

The model should not receive raw API keys, database passwords, cloud credentials, refresh tokens, or service-account secrets. Credentials belong to the execution infrastructure, not the prompt, conversation history, state checkpoint, tool definition, or model-visible result.

LLM / Agent
     │
     │ logical tool request
     ▼
Tool Gateway
     │
     ├─ Identity Context
     ├─ Authorization Decision
     └─ Secret Broker
            │
            ▼
      Short-Lived Credential
            │
            ▼
       External Service

Where possible, the gateway should use delegated or workload identity with short-lived, scope-limited credentials. The secret broker should return credentials directly to the execution adapter, not to the model-facing process. Logs and traces must redact secret-bearing headers and payload fields.

Synchronous vs. Asynchronous Tool Execution

Dynamic tool ingestion does not imply that every call should flow through Kafka or RabbitMQ. Interactive, low-latency tools often work best through a direct synchronous path:

Agent → Tool Gateway → External API → Validated Result → Agent

Queues and durable workers are more appropriate when operations are long-running, bursty, retryable, or must survive process restarts:

Agent
  ↓
Submit Durable Job
  ↓
Task Queue / Workflow Engine
  ↓
Worker
  ↓
External System
  ↓
State Store / Completion Event
  ↓
Agent Resumes or Polls

The MCP 2026-07-28 Tasks extension provides a protocol-level mechanism for long-running operations, but the same architectural principle applies outside MCP: separate interactive inference from durable execution when the task lifecycle exceeds the conversational request window.

Idempotency, Retries, and the Exactly-Once Illusion

Retries are one of the most dangerous areas in autonomous execution. Consider a tool that issues a $500 refund. The external service processes the refund but the network connection times out before the agent receives a response. A naive retry can issue a second refund.

Every mutating tool should declare or derive retry semantics. Useful fields include:

• execution_id
• correlation_id
• idempotency_key
• retry_policy
• timeout_ms
• max_attempts
• reconciliation_strategy

Tool Request
    ↓
Assign Execution ID
    ↓
Check Idempotency Store
    ↓
Execute Mutation
    ↓
Timeout / Unknown Outcome?
      │
      ├─ No → Record Result
      │
      └─ Yes → Reconcile Status
                    ↓
             Retry Only If Safe

“Exactly once” is usually an application-level effect created through idempotency, deduplication, durable state, and reconciliation—not a guarantee provided by the network. A registry should therefore capture whether a tool is idempotent and whether a retry is permitted after an ambiguous outcome.

Tool Versioning, Contract Testing, and Lifecycle Governance

Production tools evolve. Fields are added, renamed, deprecated, or given new semantics. Security scopes change. Downstream services migrate. An ingestion platform should make these changes explicit.

Source Change
    ↓
Schema Diff
    ↓
Static Validation
    ↓
Security / Policy Tests
    ↓
Contract Tests
    ↓
Human Approval for High-Risk Changes
    ↓
Registry Version
    ↓
Canary Exposure
    ↓
Production
    ↓
Observe → Deprecate → Revoke

Recommended tests include schema conformance, invalid-boundary tests, authorization tests, tenant-isolation tests, timeout behavior, retry behavior, idempotency tests, output-schema tests, prompt-injection tests, dependency-failure tests, backwards-compatibility tests, and approval-path tests.

Tool-definition fingerprints are especially valuable. If a tool description, schema, or policy-relevant annotation changes, the platform can withhold the new version until it passes review. OWASP’s tool-poisoning guidance recommends provenance metadata and definition fingerprinting to detect silent redefinitions.

Code-Level Implementation: A Safer Pydantic Tool Boundary

The original prototype pattern—validate a Pydantic object and execute an UPDATE statement—is useful but incomplete. Partial updates create a subtle problem: a field that is omitted by the caller must be distinguished from a field explicitly set to null or false. Otherwise an agent trying to update one property can accidentally overwrite others.

Define an Explicit Patch Model

from decimal import Decimal
from pydantic import BaseModel, EmailStr, Field, ConfigDict

class CustomerPatch(BaseModel):
    model_config = ConfigDict(extra="forbid")

    customer_id: int = Field(gt=0)
    email: EmailStr | None = None
    account_balance: Decimal | None = Field(default=None, ge=0)
    is_active: bool | None = None

extra="forbid" prevents the model from inventing unknown properties. However, optional fields alone are not sufficient; execution code must inspect which fields were actually supplied.

Separate Identity and Policy From the Model Payload

from dataclasses import dataclass

@dataclass(frozen=True)
class ExecutionContext:
    user_id: str
    tenant_id: str
    trace_id: str
    execution_id: str
    scopes: set[str]

The model should not be trusted to provide tenant_id, authorization scopes, or service credentials. Those values come from the authenticated platform context.

Validate Authorization Before Mutation

async def execute_customer_patch(
    patch: CustomerPatch,
    ctx: ExecutionContext,
) -> dict:
    if "customer.write" not in ctx.scopes:
        raise PermissionError("Missing customer.write scope")

    supplied = patch.model_fields_set - {"customer_id"}
    if not supplied:
        return {"status": "no_change"}

    allowed_fields = {"email", "account_balance", "is_active"}
    if not supplied.issubset(allowed_fields):
        raise ValueError("Attempted update of disallowed field")

    values = patch.model_dump(
        include=supplied,
        exclude_unset=True,
    )

    return await customer_gateway.update_fields(
        tenant_id=ctx.tenant_id,
        customer_id=patch.customer_id,
        fields=values,
        execution_id=ctx.execution_id,
    )

This approach avoids the earlier failure mode in which missing values could be written as NULL or a default Boolean could silently change account state. The gateway remains responsible for parameterized database access, tenant enforcement, transactions, audit logging, and output normalization.

Add an Execution Gateway Instead of Direct Model-to-Database Access

The agent-facing function should never hold raw SQL privileges broader than necessary. The Secure Database Connections for LLMs pattern should mediate database access through a service or policy-controlled data layer with parameterized statements and least privilege.

Pydantic AI supports structured tools and toolsets while keeping execution in application code rather than inside the model. Its documentation also distinguishes tool return values from metadata that does not need to enter model context, which is useful for separating operational telemetry from model-visible content. See the Pydantic AI advanced tool features.

Hardcoded Script Integrations vs. Dynamic Tool Ingestion

Architectural Dimension	Hardcoded Integration	Dynamic Tool Ingestion Platform
Registration	Tool definitions embedded in application code	Versioned registry and discovery pipeline
Schema Changes	Require application edits or may drift silently	Schema diff, fingerprint, approval, versioning
Tool Visibility	Usually static	Identity-, task-, tenant-, and policy-aware
Authorization	Often implemented per script	Central policy enforcement at gateway
Risk Controls	Ad hoc	Classified and policy-driven
Retries	Local try/catch behavior	Explicit idempotency and reconciliation semantics
Observability	Application-specific logging	Correlated traces, audit events, cost and latency metrics
Scale	Suitable for small fixed toolsets	Designed for hundreds or thousands of capabilities

The point is not that hardcoded functions are always wrong. They are often the simplest correct solution for small, stable systems. Dynamic ingestion becomes valuable when capability count, ownership, policy complexity, environment segmentation, or independent versioning makes static binding operationally expensive.

Tool Execution Observability and Distributed Tracing

Autonomous execution without observability is operationally unsafe. Every tool call should produce enough telemetry to reconstruct who requested the action, which model or agent selected it, which tool definition was used, which policy decision allowed it, how long it took, what it cost, and whether the result was accepted or rejected.

Recommended telemetry includes:

• tool.selection.count
• tool.call.count
• tool.call.success_rate
• tool.call.failure_rate
• tool.call.latency
• tool.validation.failure
• tool.authorization.denied
• tool.approval.requested
• tool.approval.denied
• tool.retry.count
• tool.timeout.count
• tool.input.tokens and tool.output.tokens
• tool.version and tool.definition_hash
• agent.id, tenant.id, trace.id, and execution.id

OpenTelemetry’s GenAI semantic conventions include agent and tool attributes and an execute_tool operation name, providing a useful foundation for interoperable traces. See the OpenTelemetry GenAI semantic conventions.

User Request [trace_id=123]
        ↓
Agent Inference
        ↓
Tool Retrieval
        ↓
Policy Evaluation
        ↓
Tool Execution
        ↓
External API / Database
        ↓
Result Validation
        ↓
Agent Continuation

These traces should feed the same operational layer used by an AI Token Observability Dashboard, allowing platform teams to correlate model cost with tool latency, failures, retries, approval waits, and downstream service behavior.

Complete Enterprise Tool Ingestion Reference Architecture

A mature platform separates the control plane that governs capabilities from the runtime plane that executes them.

Control Plane

• Tool-source connectors for MCP, OpenAPI, RPC, databases, SaaS, and code
• Discovery and provenance validation
• Schema acquisition and normalization
• Risk and side-effect classification
• Policy and RBAC binding
• Definition fingerprinting and signing
• Contract tests and security tests
• Versioned tool registry
• Approval, publication, deprecation, and revocation workflows

Runtime Plane

• Authenticated user and agent identity
• Agent orchestrator and typed shared state
• Capability retrieval and ranking
• Tool visibility filtering
• Invocation authorization
• Argument validation and business-rule checks
• Human approval interrupts for high-risk actions
• Credential broker and short-lived tokens
• Execution gateway, sandbox, or durable worker
• Output validation and result sanitization
• Context hydration and state checkpointing

Cross-Cutting Platform Services

• OpenTelemetry tracing
• Audit logging
• Cost and token governance
• Rate limits and resource budgets
• Secrets management
• Security monitoring
• Incident response and revocation

This architecture aligns naturally with an enterprise AI platform: the agent orchestrator maintains workflow state, the LLM gateway governs model access, the tool-ingestion control plane governs capabilities, the tool gateway enforces runtime execution, and enterprise systems of record remain authoritative business data rather than agent memory.

Production Deployment Checklist

• Maintain an approved source list for MCP servers and external tool providers.
• Record provenance, owner, version, and definition hash for every tool.
• Normalize heterogeneous schemas into a canonical internal contract.
• Reject unknown properties and enforce explicit field constraints.
• Separate omitted fields from explicit nulls in mutation APIs.
• Evaluate identity, tenant, scope, resource, and environment at runtime.
• Do not treat tool visibility as authorization.
• Classify tools by read/write/destructive/financial/privileged/external effects.
• Require human approval for irreversible or high-impact actions.
• Keep secrets and service credentials out of prompts and model context.
• Use short-lived credentials and least-privilege scopes where possible.
• Validate and sanitize tool results before adding them to model context.
• Bound tool-result size and project only fields required for reasoning.
• Define idempotency and retry behavior for every mutating tool.
• Use durable queues or task systems only where execution semantics require them.
• Run schema diff, contract, authorization, security, and failure tests before publication.
• Re-review tools when schemas or descriptions change unexpectedly.
• Trace every invocation with agent, user/tenant, tool version, policy, and execution identifiers.
• Set per-tool timeouts, rate limits, concurrency limits, and cost budgets.
• Provide an emergency revocation path for compromised tools or credentials.

Parallel Tool Calls, Dependency Graphs, and Concurrency Control

Agents increasingly issue multiple tool calls in the same reasoning step. Parallelism can reduce end-to-end latency when operations are independent, but it also changes the failure model. Two read-only searches can usually execute concurrently. Two writes against the same customer, account, file, or infrastructure resource may race, produce inconsistent state, or invalidate each other’s assumptions.

The orchestrator should therefore model tool calls as a dependency graph rather than simply dispatching every emitted call at once:

Agent Plan
   ├─ Search customer profile ─────┐
   ├─ Retrieve open invoices ──────┤  parallel reads
   └─ Check account permissions ───┘
                    ↓
              Decision Barrier
                    ↓
           Issue account credit
                    ↓
             Update CRM note

Concurrency policy should consider the tool’s side effects, target resource, transaction domain, idempotency, and whether a later action depends on an earlier result. Useful controls include per-tool concurrency limits, per-tenant quotas, resource locks, optimistic version checks, serialization keys, and fan-out ceilings. The goal is not to eliminate parallel execution but to reserve it for operations whose semantics permit it.

Parallel results also create context pressure. If ten tools each return large payloads, the model may receive far more data than it can use. A result aggregator can project, summarize, deduplicate, or rank parallel outputs before rehydrating the agent state. This is another reason to separate raw execution results from model-visible results.

Policy-as-Code for Tool Exposure and Execution

As tool inventories grow, authorization rules spread quickly across teams, environments, and resource types. Hardcoded if statements inside every tool adapter become difficult to audit and nearly impossible to reason about globally. Policy-as-code provides a deterministic decision layer between model intent and execution.

A policy decision can evaluate attributes such as:

• authenticated user and agent identity
• tenant and organization
• tool ID and approved version
• risk class and side-effect classification
• environment such as development, staging, or production
• target resource or account
• time, geography, or network zone where appropriate
• requested amount or other business threshold
• whether the triggering context came from an untrusted external source
• whether a human approval token is present

Agent Tool Intent
       ↓
Schema Validation
       ↓
Policy Decision Point
       │
       ├─ ALLOW
       ├─ DENY
       └─ REQUIRE_APPROVAL
                ↓
        Execution Gateway

This keeps security decisions independent of model persuasion. Even if indirect prompt injection convinces the model that an operation is “urgent” or “pre-approved,” the policy engine evaluates machine-verifiable facts. For sensitive tools, the approval artifact should be issued by a trusted workflow outside the LLM context and bound to the execution ID, user identity, tool, target resource, and expiration time.

Performance and Cost Engineering for Tool-Heavy Agents

Tool-enabled agents introduce latency outside model inference. A request can spend time retrieving tool definitions, evaluating policy, waiting for approval, opening network connections, executing downstream services, validating results, and performing another model turn. Optimizing only tokens per second misses much of the user-visible latency.

A useful latency budget separates the stages:

Stage	Example Metric	Optimization Lever
Tool retrieval	P95 registry/search latency	Catalog caching, metadata indexes, prefiltered toolsets
Model decision	Time to tool call	Smaller relevant tool set, prompt-prefix reuse
Policy	Decision latency	Local policy cache, low-cardinality attributes
Execution	Tool P50/P95/P99 latency	Connection pooling, regional routing, async execution
Result processing	Validation/sanitization latency	Bounded output, streaming parsers, field projection
Continuation	Incremental prefill + decode	Prefix reuse, bounded results, stable schemas

Tool catalogs should also be optimized for token economics. Long prose descriptions, redundant examples, and giant response schemas can consume context without improving tool selection. The registry can retain rich operational metadata while exposing a smaller model-facing contract. In other words, the canonical internal representation and the LLM-visible representation do not need to be identical.

Cost governance should correlate model tokens with downstream service costs. Some tools may invoke paid APIs, cloud jobs, searches, or GPU workloads that cost more than the model turn itself. A resource-budget controller can therefore enforce per-session limits across both token usage and tool execution.

Failure Containment, Circuit Breakers, and Graceful Degradation

External systems fail. APIs rate-limit, databases become unavailable, MCP servers return malformed content, and SaaS providers exceed latency targets. The agent should not translate every infrastructure failure into an uncontrolled retry loop.

The execution gateway should normalize failures into stable categories such as validation failure, authorization denial, dependency timeout, rate limit, transient server error, permanent business rejection, unsafe result, and unknown outcome. The orchestrator can then route deterministically:

Tool Failure
    ↓
Classify Error
    ├─ Validation → Ask model/user for corrected input
    ├─ Authorization → Deny / request approved escalation
    ├─ Rate Limit → Backoff within budget
    ├─ Transient Read Failure → Safe retry
    ├─ Ambiguous Write Failure → Reconcile before retry
    ├─ Circuit Open → Use fallback tool or degrade gracefully
    └─ Unsafe Result → Quarantine and stop propagation

Circuit breakers are especially useful when a failing dependency would otherwise trigger repeated model/tool cycles. Once an error threshold is crossed, the platform can temporarily remove the tool from selection, route to an approved fallback, or inform the model that the capability is unavailable. This is preferable to letting the model repeatedly “reason” about an infrastructure failure it cannot solve.

Multi-Agent Tool Boundaries and Delegated Capabilities

In a multi-agent system, each agent should have its own bounded capability set. A research agent may need web and document tools but no production write access. A billing agent may require financial APIs but not infrastructure controls. A deployment agent may need cluster operations but no customer-data export tool.

This principle limits blast radius and reduces confused-deputy risk:

Supervisor Agent
      │
      ├─ Research Agent → Search / Docs / Read-Only APIs
      ├─ CRM Agent → Customer Read + Approved CRM Writes
      ├─ Billing Agent → Invoice / Refund Tools with Approval
      └─ Ops Agent → Infrastructure Tools in Isolated Runtime

The supervisor should not automatically inherit every child agent’s privileges. Delegation should pass a constrained task and capability grant, not a global credential bundle. This aligns tool ingestion with the same bounded-autonomy principles used in multi-agent orchestration frameworks and stateful graph execution.

Frequently Asked Questions

What is tool ingestion engineering for LLMs?

Tool ingestion engineering is the lifecycle that transforms external capabilities such as MCP tools, REST APIs, database operations, and internal functions into validated, versioned, policy-bound tools that an LLM-based agent can discover and use through a controlled runtime.

How is tool ingestion different from function calling?

Function calling is the mechanism by which a model expresses a structured request to invoke a function. Tool ingestion covers discovery, schema acquisition, normalization, trust validation, policy binding, versioning, selection, execution controls, result validation, and observability around that request.

Should every enterprise tool be placed in the LLM context?

No. Large catalogs should generally be filtered and retrieved dynamically based on task relevance, identity, tenant, policy, risk, and context budget. Injecting every schema increases token overhead, tool-selection ambiguity, and attack surface.

Does strict JSON Schema prevent hallucinated or unsafe tool calls?

No. Strict schemas improve syntactic and structural reliability, but a schema-valid request can still be semantically wrong, unauthorized, destructive, or based on malicious context. Schema validation must be combined with business rules, authorization, risk policy, and runtime enforcement.

Are MCP tool annotations safe to trust for authorization decisions?

No. MCP documentation explicitly treats annotations as hints and warns clients not to make sensitive decisions based on untrusted servers. Enterprises should verify tool behavior and maintain their own approved risk and policy metadata.

Why must tool outputs be validated?

Tool outputs can be malformed, over-sized, expose sensitive data, cross tenant boundaries, or contain indirect prompt injections. Result validation, projection, sanitization, and content-risk controls should run before tool data reaches the model.

Should tool calls always use a message queue?

No. Synchronous calls are appropriate for many low-latency operations. Durable queues, workflow engines, or task systems are most useful for long-running jobs, burst handling, retryable workflows, and work that must survive process failure.

Why is idempotency important for agent tools?

Agents can retry after timeouts or ambiguous failures. Without idempotency or reconciliation, a second call may duplicate a payment, email, ticket, database mutation, or other side effect. Mutating tools should declare safe retry semantics and use execution or idempotency identifiers when supported.

Where should tool credentials live?

Credentials should live in the execution infrastructure—ideally in a secret manager, workload identity system, or credential broker—not in the LLM prompt, tool schema, conversation history, memory, or model-visible results.

Conclusion: Tool Calling Is the Runtime Mechanism; Tool Ingestion Is the Capability Supply Chain

Production agent infrastructure should not treat external tools as a collection of JSON descriptions pasted into a prompt. A tool is an executable software dependency with identity, provenance, schema, version, risk, permissions, credentials, side effects, failure modes, and observable runtime behavior.

The strongest architecture therefore separates the capability control plane from the execution runtime. The control plane discovers tools, verifies their source, normalizes schemas, classifies risk, binds policy, tests changes, and publishes approved versions. The runtime plane retrieves only relevant capabilities, validates arguments, authorizes the caller, interrupts for approval when required, brokers credentials, executes through a gateway or sandbox, validates results, and returns bounded data to the agent.

This distinction becomes increasingly important as organizations connect stateful agents to MCP servers, SaaS platforms, databases, infrastructure APIs, and multi-agent workflows. The more capable the agent becomes, the less acceptable it is to depend on prompt instructions as the final control.

Tool calling is the model-facing mechanism. Tool ingestion engineering is the enterprise capability supply chain that makes autonomous execution governable.

Automated Execution Safety: Moving your access vectors past simple read-only queries into dynamic write capabilities requires a programmatic abstraction layer. Review our systems engineering blueprint on designing secure tool ingestion engineering llm workflows to safely sandbox agentic read/write permissions via isolated API environments.