Datasette Agent 0.2a0: A New Mechanism for AI Agents to Ask Users Questions During Tool Execution

Simon Willison has released the latest alpha version 0.2a0 of datasette-agent, introducing two noteworthy features: the ability to ask users questions during tool execution, and a built-in SQL query saving tool. These improvements make the collaboration between AI agents and humans more flexible and secure.

Core Feature: Human-Agent Interaction During Tool Execution

The most important feature in this update is the ask_user() mechanism. In traditional AI agent workflows, once a tool is invoked it runs to completion with no opportunity for user interaction along the way. Datasette-agent 0.2a0 breaks this limitation—tools can now pause mid-execution, pose a question to the user, wait for a response, and then continue.

In modern AI agent architectures, tool calling (also known as Function Calling) is the core mechanism through which LLMs interact with the outside world. Model providers like OpenAI and Anthropic allow developers to define a set of tool schemas, and during a conversation the model can decide which tools to call and what parameters to pass. After tool execution completes, the results are returned to the model for further reasoning. This pattern is widely adopted in frameworks like LangChain, CrewAI, and AutoGen, but in traditional implementations tool calls are atomic—they either complete and return a result, or fail and terminate, with no room for pausing or interaction in between. The ask_user() in datasette-agent is precisely a breakthrough against this limitation.

In terms of implementation, tools that declare a context parameter receive a ToolContext object, and can initiate three types of questions via await context.ask_user(...):

• Yes/No questions: Simple binary confirmations
• Multiple choice: Providing a list of options via the options=[...] parameter
• Free text: Allowing users to input any content via free_text=True

The engineering details of this design are quite thoughtful. When a question goes unanswered, the agent's current turn is suspended, the question is rendered as a form in the chat UI, and simultaneously persisted to an internal database. This means even if the server restarts, suspended conversations won't be lost. After the user answers, the tool re-executes from the beginning, with previously stored answers automatically replayed.

This suspend-replay pattern draws from Event Sourcing concepts. In the distributed workflow engine space, Temporal (formerly Cadence) is a typical representative of this pattern—it allows long-running workflows to suspend while waiting for external signals, then replay completed steps upon resumption. However, this pattern also introduces idempotency challenges: during replay, operations that have already been executed should not produce duplicate side effects.

Interestingly, the official documentation specifically reminds developers to call ask_user() before executing side effects. This is due to the tool re-execution mechanism—if you perform database writes before asking a question, those operations will be repeated during re-execution. This constraint is a concrete manifestation of the idempotency problem inherent in the suspend-replay pattern.

Built-in save_query Tool: Human Approval as a Safety Guarantee

The second new feature is the built-in save_query tool. SQL queries written by the AI agent while interacting with the database can now be saved as Datasette stored queries (canned queries) for later reuse.

Some context on Datasette's Canned Queries mechanism: Datasette was created by Simon Willison in 2017, originally aimed at making it easy for anyone to publish and explore SQLite databases. It follows the "data as API" philosophy, automatically transforming SQLite files into browsable, queryable web applications accessible via REST APIs. The canned queries feature allows administrators to predefine SQL queries and expose them to users as named URLs—a secure data access pattern where end users don't need to write SQL directly. The datasette-agent save_query tool brings AI-generated queries into this governance framework.

This reflects an important design philosophy: save operations always require human approval. The agent displays the complete SQL statement, suggested query name, target database, and visibility settings, and storage only happens after the user clicks to confirm. This is a textbook application of the ask_user() mechanism—seeking human consent before executing irreversible operations.

This Human-in-the-Loop (HITL) design is particularly important in today's AI agent ecosystem. HITL is one of the core design principles in AI safety. As AI agents increasingly gain autonomous action capabilities, ensuring humans maintain control at critical decision points is essential. This aligns with the "Governability" principle emphasized in the NIST AI Risk Management Framework. In practice, the granularity of HITL implementation is a difficult problem: overly frequent confirmations reduce efficiency (so-called "confirmation fatigue"), while being too permissive may allow AI to execute unauthorized operations. Datasette-agent builds HITL into the tool layer, letting developers decide which operations require human approval—a balanced approach. As AI agents gain access to ever more tool-calling capabilities, finding the right balance between automation efficiency and safe controllability is a challenge every developer must face.

Technical Background: LLM Alpha and Claude-Assisted Development

According to Simon Willison, the implementation of the ask_user() feature relies on a new LLM alpha version he built the previous day with Claude's assistance. This creates an interesting recursive scenario: using AI-assisted programming tools to build new features for an AI agent framework.

Simon Willison is an active practitioner and advocate of AI-assisted programming, consistently documenting his experiences using Claude, ChatGPT, and other tools in development. His LLM project is a Python command-line tool and library that provides a unified interface for accessing multiple large language models. Using AI-assisted tools to build AI infrastructure—this recursive pattern is becoming increasingly common in the developer community. From the Cursor editor to GitHub Copilot, AI is accelerating the construction of its own ecosystem. This also sparks discussions about the auditability and reliability of "AI writing AI."

Datasette, as a lightweight data exploration and publishing tool, has been actively embracing AI capabilities. Datasette-agent combines LLM natural language understanding with Datasette's data querying capabilities, allowing users to explore databases through conversation. The human-agent interaction improvements in version 0.2a0 make this process more controllable and practical.

Implications for AI Agent Framework Developers

The ask_user() design pattern is worth adopting by other AI agent frameworks. It solves a common problem: AI agents executing complex tasks often need to obtain human judgment or confirmation at certain critical junctures. Building this capability into the tool-calling protocol is far more elegant than wrapping an approval workflow around the outside.

At the same time, the persistence design for suspended states is highly practical. In production environments, users may not immediately answer an agent's questions, services may restart or migrate, and conversation state persistence ensures reliability in these scenarios.

Datasette-agent is still in alpha (0.2a0), and the API may continue to evolve, but its design direction—letting AI agents "stop and ask" at critical moments—is undoubtedly the right one.