Supabase Real-World Testing: How MCP+Skills Enable AI Agents to Safely Operate Databases

AI Agents Are Smart Enough, But Lack Proper Guidance

At the recent AI Engineer conference, Pedro, an AI Tools Engineer at Supabase, shared their hands-on experience developing Agent Skills. He highlighted a key insight: the current bottleneck for AI agents isn't capability—it's guidance.

Agents are already intelligent enough to handle routine tasks, but when it comes to product-specific security standards and best practices, they often make critical mistakes. The Supabase team found that agents frequently overlook Row Level Security (RLS) settings when operating databases, rely on outdated training data, and are "extremely lazy, unwilling to admit they don't know something."

Row Level Security (RLS) is a fine-grained access control mechanism provided by PostgreSQL that allows database administrators to define policies controlling which users can view or modify which rows in a table. In traditional permission models, access control granularity only goes down to the table level—either you can access the entire table or you can't at all. RLS refines this to the row level, for example allowing rules like "users can only see orders they created." For BaaS (Backend as a Service) platforms like Supabase, RLS is especially critical because clients interact directly with the database without a traditional backend server as an intermediary to filter data. If RLS configuration is missing or bypassed, it means any authenticated user could potentially access other users' sensitive data.

Experimental Validation: MCP+Skill vs. MCP-Only Security Comparison

A Typical Security Vulnerability Case

The Supabase team designed a precise comparative experiment: having the same model (Claude Sonnet 4.6) perform the same task—creating an SQL view on a table with RLS enabled.

In PostgreSQL, if you create a view on an RLS-enabled table without explicitly setting the security_invoker = true flag, the view will bypass RLS policies, leading to data leakage. This is a classic security trap.

To understand the severity of this trap, you need to understand PostgreSQL's view permission execution mechanism. Views execute queries with the view creator's permissions by default—this is called "security definer" semantics. This means that even if the underlying table has RLS policies enabled, accessing data through the view bypasses RLS policies because the system considers the view's owner (typically a superuser or high-privilege role) to be executing the query. PostgreSQL 15 introduced the security_invoker attribute, which when set to true, causes the view to execute with the caller's (i.e., the actual user querying the view) permissions, allowing RLS policies to function properly. This subtle but critical configuration difference is exactly the kind of "implicit knowledge" that AI agents easily miss—it doesn't cause syntax errors or execution failures, but silently creates security vulnerabilities.

The experimental results were very clear:

• Agent using MCP tools only: Did not set the security flag, creating a view with data leakage risk
• Agent with MCP+Skill: Correctly identified and implemented the security configuration

Large-Scale Multi-Model Evaluation Results

The team further designed 6 Supabase-specific scenarios, running comparative tests across 4 different models (Claude Code/Opus 4.6, Sonnet 4.6, GPT 5.4, GPT 5.4 Mini) under three conditions:

1. Baseline: No MCP, no Skill
2. MCP only: Tool-calling capability only
3. MCP + Skill: Tools plus guidance

Here it's important to understand the technical background of MCP (Model Context Protocol). MCP is a protocol standard open-sourced by Anthropic in late 2024, designed to provide AI models with a unified interface for interacting with external tools and data sources. Before MCP, every AI application needed custom integration code for each external service, creating an N×M integration complexity problem. MCP defines a standardized client-server architecture that allows AI agents to invoke database queries, file operations, API requests, and other tools through a unified protocol. MCP servers expose Tools, Resources, and Prompts, while AI agents consume these capabilities as MCP clients. However, as this experiment reveals, tool-calling capability alone isn't sufficient—agents know "what they can do" but don't necessarily know "how they should do it."

The results were consistent across all models: The Skill + MCP combination outperformed other conditions on every model. This proves that tools alone aren't enough—proper usage guidance is the key.

Three Principles for Writing High-Quality Agent Skills

Principle 1: Don't Repeat Information—Point to a Single Source of Truth

Pedro emphasized that Skills shouldn't duplicate existing documentation content but should guide agents to find the latest docs. You need to be "very stubborn" in requiring the model to search the web or consult documentation, providing lookup paths and methods.

Supabase made an innovative attempt for this: exposing documentation via SSH. This lets agents navigate docs like browsing a file system, since agents are very familiar with file system operations and Linux tools. This interface design reduces friction for agents to access up-to-date information.

Principle 2: Anything That Can Be Skipped Will Be Skipped

This is the most practical insight Pedro shared. Agents behave "extremely lazily" when loading Skills:

• Tool calls and fetching information online are "expensive" for agents—they default to using training data
• Even when Skills contain reference files, agents rarely load them proactively
• If a problem requires information from multiple reference files, agents almost never load more than two files

Practical recommendation: Put absolutely essential information directly in the skill.md main file. Supabase's security checklist was initially placed in reference files, and agents frequently ignored it; after moving it to the main file, the problem was resolved.

Principle 3: Be Opinionated—Define the Best Workflow

You know your product best—don't be afraid to define clear operational procedures for agents. Supabase defined this best workflow for database schema management:

1. Freely execute DDL operations on the development/staging database
2. Once satisfied, run Supabase's advisor to check for security and performance issues
3. Only after fixing issues, create the migration file

This involves an important concept in database development. DDL (Data Definition Language) is a set of SQL statements used to define and modify database structures, including CREATE TABLE, ALTER TABLE, DROP INDEX, and other operations. In modern software development, database schema changes are typically managed through Migration files—each Migration file records one structural change that can be applied or rolled back in sequence, similar to version control for code. However, if an agent generates a Migration file for every exploratory modification, it results in numerous redundant intermediate states being recorded. Supabase's recommended workflow is to experiment freely in the development environment first, then generate a single clean Migration file after confirming the final state—consistent with the actual work habits of experienced database engineers.

This workflow avoids the inefficient behavior of agents generating migration files for every schema modification.

Technical Structure of Agent Skills

A Skill is essentially a folder containing instructions, scripts, and resources that agents progressively discover. The structure includes:

• Front Matter: Contains name and description, which agents use to decide when to load the Skill
• skill.md: The main instruction file containing core information
• Optional resources: Scripts (for executing operations) and reference files (supplementary information)

Agent Skills represent a new knowledge organization paradigm that is fundamentally different from traditional API docs, READMEs, or tutorials. Traditional documentation is designed for human readers and relies on the reader's judgment to decide when to apply which information. Skills are structured instruction sets designed for AI agents, taking into account agents' cognitive characteristics—including limited context windows, tendency to take shortcuts, and proficiency with file system operations. The progressive discovery mechanism means agents don't load all information at once but gradually acquire relevant instructions as needed by the task. This design both conserves precious context space and simulates how human experts "consult manuals on demand."

One detail worth noting: Skills are agent-agnostic, and an increasing number of agents are adopting this open standard.

Challenges and Current Solutions for Skill Distribution

The biggest challenge facing the Skill ecosystem today is the distribution system. Pedro candidly admitted this problem hasn't been fully solved:

• Vercel has launched a skills package solution
• Various plugin packaging formats (.clot, .cursor, etc.) are model-specific
• Supabase's current approach is packaging Skills in code repositories, distributed via open source or skills packages

Skill/plugin distribution in the current AI agent ecosystem faces fragmentation problems similar to early package managers. Different AI coding assistants (such as Cursor, Claude Code, Windsurf, etc.) each define their own configuration formats and plugin mechanisms—.cursorrules, CLAUDE.md, .windsurfrules, etc.—which are mutually incompatible. Vercel's skills package solution attempts to provide a cross-agent standardized distribution mechanism, similar to npm's role in the JavaScript ecosystem. This lack of standardization means tool providers need to maintain instruction files in different formats for multiple platforms, increasing maintenance costs. The industry is evolving toward agent-agnostic open standards, but full unification will still take time.

Users can now install Supabase's Agent Skill with a single command and start using it immediately.

Conclusion: Product "Agent-Friendliness" Becomes a New Competitive Dimension

This presentation revealed an important trend: in the age of AI agents, a product's "agent-friendliness" is becoming a new competitive dimension. Supabase's practice demonstrates that carefully written Skill documentation can significantly improve the safety and efficiency of agent-operated products. For any company providing developer tools, writing high-quality Agent Skills is shifting from "nice to have" to "must have."

Key Takeaways

• The MCP+Skill combination significantly outperformed MCP-only solutions across all tested models, proving that tools + guidance is the optimal approach
• Three principles for writing Skills: point to a single source of truth to avoid repetition, put critical information in the main file to prevent skipping, and be opinionated in defining best workflows
• AI agents easily overlook security settings like RLS when operating databases—Skills can effectively address this gap
• Supabase innovatively exposes documentation via SSH, letting agents access the latest information as if browsing a file system
• The Skill distribution system is not yet standardized, currently relying primarily on code repositories and open-source distribution