Your AI Agents Are Running Queries You Can't See — The Database Observability Gap

Half of all enterprises are piloting agentic AI right now. Only 24% have it in production. And the number one reason pilots stall isn’t model quality, cost, or even hallucinations — it’s that nobody can see what the agents are actually doing.

This week, three things happened that make the database observability gap impossible to ignore:

1. Codenotary launched AgentMon (March 31) — the first enterprise-grade monitoring platform built specifically for AI agent networks, tracking tool calls, data access patterns, and policy violations in real time.
2. Microsoft published an AI observability framework (March 18) focused on proactive risk detection in AI systems, emphasizing that traditional APM tools miss the autonomous decision-making layer entirely.
3. Oracle announced its Unified Memory Core and AI Database Private Agent Factory (March 24) — embedding persistent agent memory and pre-built data agents directly inside the database engine.

Three different companies, three different approaches, one shared conclusion: when AI agents access your data, you need to see every query, every tool call, every decision — or you’re flying blind.

The Problem: Agents Don’t Follow Scripts

Here’s the fundamental shift that breaks traditional monitoring. A human developer writes a query, ships it in a PR, and that query runs the same way every time. You can trace it, profile it, alert on it. Your Datadog dashboard knows exactly what’s happening.

An AI agent doesn’t work that way. Give an agent access to your database through MCP or a REST API, and it will:

• Decide which tables to query based on the user’s natural language request
• Choose how to filter, join, and aggregate based on its own reasoning
• Chain multiple queries together, using the output of one to inform the next
• Retry with different parameters if the first attempt doesn’t return useful results

Every interaction is different. There’s no static query to trace. There’s no predefined execution path to monitor. The agent is making autonomous decisions about your data access patterns in real time.

This is why traditional Application Performance Monitoring (APM) tools have a blind spot. They’re built to monitor known code paths. They can tell you that your /api/users endpoint is slow. They can’t tell you that an AI agent just ran a full table scan on your transactions table because it misunderstood a prompt about “recent activity” and decided to pull every row from the last five years.

What AgentMon Gets Right

Codenotary’s AgentMon represents the first serious attempt to solve this at the infrastructure level. Rather than monitoring individual API endpoints or database queries, AgentMon monitors the agent itself — tracking its behavior as a first-class entity.

The platform provides:

• Communication path mapping between agents and services — which agents are talking to which databases, through which tools
• Token usage and model selection tracking — what LLM is driving each agent’s decisions, and how much each interaction costs
• File access and secrets handling monitoring — detecting when an agent accesses data it shouldn’t, or handles credentials in ways that violate policy
• Data access pattern analysis — identifying anomalous query patterns that might indicate prompt injection, data exfiltration, or simple misconfiguration

That last point is the one that matters most for database access. When an agent is autonomously generating queries against your production data, you need something watching the pattern of access, not just the individual queries. Is this agent suddenly reading from tables it’s never touched before? Did its query volume spike 10x in the last hour? Is it accessing PII columns that its role shouldn’t permit?

BCG projects the AI agents market will grow at a 45% CAGR over the next five years. As adoption scales, the number of autonomous database interactions will grow exponentially. Without observability at the agent layer, every one of those interactions is a black box.

Microsoft’s Framework: Observability as a Security Requirement

Microsoft’s AI observability blog post takes a different angle — treating observability not as an operations concern but as a security imperative. Their framework centers on two capabilities that traditional monitoring doesn’t provide:

Evaluation: Measuring whether agent outputs are grounded in source material and whether tools are used correctly. For database access, this means validating that the data an agent returns actually matches the query it generated — catching hallucinated results before they propagate.

Governance: The ability to measure, verify, and enforce acceptable system behavior using observable evidence. This goes beyond access control. It’s about proving, after the fact, that an agent’s database interactions complied with policy. Audit logs for AI.

The governance angle is critical for regulated industries. If your AI agent pulls customer financial data to answer a support question, your compliance team needs to see exactly what was accessed, why, and whether the agent had authorization. A 200 OK from your API isn’t enough. You need the full chain: prompt → reasoning → tool selection → query generation → data access → response.

Oracle’s Bet: Put the Agent Inside the Database

Oracle’s approach is the most radical of the three. Instead of monitoring agents from the outside, Oracle is embedding agent capabilities directly into the database engine with its AI Database 26ai release.

The Unified Memory Core gives AI agents persistent, stateful memory inside the database — so agents can maintain context across sessions without external state management. The AI Database Private Agent Factory is a no-code platform for deploying data-centric agents as portable containers, with pre-built agents for database knowledge, structured data analysis, and deep data research.

Oracle’s thesis is straightforward: if the agent lives inside the database, you get observability for free. Every agent action is a database operation. Every database operation is logged, traced, and auditable through Oracle’s existing enterprise tooling.

It’s an elegant solution if you’re already on Oracle. But it also represents a form of vendor lock-in that should make any architect pause. Your agent observability shouldn’t depend on your database vendor. And for the majority of teams running PostgreSQL, MySQL, SQL Server, or multiple databases simultaneously, Oracle’s approach isn’t an option.

The API Layer: Your Natural Observability Plane

Here’s what all three approaches miss — or at least undervalue: the API layer between your agents and your databases is the single best place to implement observability.

Think about it. Every tool call an MCP-connected agent makes to access your database passes through an API. That API is the chokepoint where you can:

1. Log every request with full context — which agent, which user, which tool, what parameters
2. Enforce access control before the query hits the database — not after
3. Rate limit aggressive agents that generate too many queries
4. Validate query patterns against expected behavior
5. Capture the full audit trail that compliance teams need

This isn’t theoretical. If you expose your database through a structured REST API with proper authentication and logging, you already have 80% of what AgentMon, Microsoft, and Oracle are trying to build — because every agent interaction is an HTTP request with a traceable identity, timestamp, and payload.

The problem is that most teams don’t have a structured API layer in front of their databases. They connect agents directly to databases through raw SQL tools, or they build one-off API endpoints that don’t have consistent logging, auth, or rate limiting. That’s how you end up with the observability gap.

Building Observable Agent-Database Access with Faucet

Faucet generates a complete REST API from your database schema in seconds, with built-in features that directly address the agent observability gap:

Structured, Predictable Endpoints

Every table gets a consistent set of CRUD endpoints with typed parameters:

# Install Faucet
curl -fsSL https://get.faucet.dev | sh

# Generate APIs from your database
faucet serve --db "postgres://localhost:5432/myapp"

Now every agent interaction follows a predictable pattern:

GET  /api/customers?age=gt.30&state=eq.CA    # Filtered read
GET  /api/orders?customer_id=eq.123           # Relationship traversal
POST /api/support_tickets                      # Create operation

No ad-hoc SQL generation. No full table scans because an agent got creative. Every request maps to a documented endpoint with validated parameters.

Request-Level Audit Logging

Every API call Faucet handles includes:

• Timestamp and request ID
• Authenticated identity (which agent or user made the request)
• Full request parameters (which filters, which fields, which table)
• Response metadata (row count, execution time)

Feed this into your existing log aggregation (ELK, Datadog, Grafana) and you have a complete picture of what your agents are doing with your data.

RBAC That Agents Can’t Bypass

Faucet’s role-based access control operates at the API layer, before any query reaches your database:

# Create a read-only role for AI agents
faucet config set roles.ai_reader.tables.customers.operations '["GET"]'
faucet config set roles.ai_reader.tables.customers.fields '["name", "email", "plan"]'

# Sensitive fields are never exposed
# SSN, payment info, internal notes — invisible to the agent

This is fundamentally different from database-level permissions. Database roles control which tables a connection can access. API-level RBAC controls which fields, which operations, and which filter patterns a specific agent identity can use. It’s the granularity that agent observability requires.

MCP Server with Tool Annotations

Faucet’s built-in MCP server exposes your database tools with the safety annotations that agents and monitoring systems need:

faucet serve --db "postgres://localhost:5432/myapp" --mcp

Every tool includes MCP tool annotations — readOnlyHint, destructiveHint, idempotentHint — that tell the agent (and any monitoring layer like AgentMon) exactly what each operation does before it executes. A monitoring system can flag any agent that attempts to call a destructiveHint: true tool without explicit user approval.

OpenAPI 3.1 Spec for Agent Discovery

Faucet auto-generates an OpenAPI 3.1 specification for every database it serves:

curl http://localhost:8080/api/_openapi.json

This spec becomes the contract that observability tools can validate against. If an agent tries to access an endpoint that isn’t in the spec, something is wrong. If an agent’s query patterns deviate from what the spec defines as valid parameters, that’s an anomaly worth investigating.

The Observability Stack for Agent-Database Access

Here’s what a production-ready observability stack looks like when AI agents are accessing your databases:

Layer	What It Monitors	Example Tools
Agent Layer	Agent behavior, reasoning chains, tool selection	AgentMon, LangSmith, Braintrust
API Layer	Request patterns, auth, rate limits, audit trail	Faucet logs, API gateway metrics
Database Layer	Query performance, connection pools, locks	pg_stat_statements, Datadog DBM
Infrastructure	CPU, memory, network, container health	Prometheus, Grafana, CloudWatch

Most teams have layers 3 and 4 covered. Layer 1 is what Codenotary, Microsoft, and others are building now. But layer 2 — the API layer — is the one most teams are missing entirely, and it’s the one that ties everything together.

Without a structured API layer, your agent monitoring tool sees tool calls but can’t map them to specific database operations. Your database monitoring sees queries but can’t attribute them to specific agents. The API layer is the join key between agent identity and database activity.

The 40% Failure Rate

Here’s the stat that should concern every team building with AI agents: over 40% of enterprise AI agent projects fail. The top reasons cited are governance gaps, unclear ROI, and inadequate integration infrastructure.

Observability addresses all three:

• Governance gaps close when every agent-database interaction is logged, attributed, and auditable
• ROI becomes measurable when you can track exactly how many queries agents run, how much time they save, and what their error rate is
• Integration infrastructure solidifies when agents access databases through a standardized API layer rather than bespoke connections

The companies that will successfully scale AI agents in production — the ones that join Pinterest in that 24% club — are the ones investing in observability now, before they need it.

What Comes Next

The convergence of AgentMon, Microsoft’s observability framework, and Oracle’s database-embedded agents signals a market that’s maturing fast. Within the next 12 months, expect:

• Agent observability to become a procurement requirement — CISOs will demand it before approving production agent deployments
• MCP tool annotations to expand — the current readOnlyHint/destructiveHint annotations are just the beginning; expect cost hints, latency hints, and data classification hints
• API layers to become the standard for agent-database access — raw database connections will be treated like SSH access: technically possible, operationally unacceptable

The teams that build observable, auditable, access-controlled database APIs today won’t have to retrofit them tomorrow.

Getting Started

Deploy a fully observable database API in under a minute:

# Install Faucet
curl -fsSL https://get.faucet.dev | sh

# Serve your database with API logging and MCP support
faucet serve --db "postgres://user:pass@localhost:5432/mydb" --mcp

# Your agents now have:
# - REST API with request logging at :8080/api/
# - MCP server with tool annotations at :8080/mcp
# - OpenAPI 3.1 spec at :8080/api/_openapi.json
# - RBAC configuration at ~/.faucet/config.yaml

Every query your AI agents run against your database is now a logged, authenticated, rate-limited API call. That’s not just observability — that’s the foundation for running agents in production.