Building a Sound Source Localization System with Claude Code to Find a Mysterious Beeping Sound

A device in the room keeps intermittently beeping, but the intervals are so long that standing in the room you simply can't tell where the sound is coming from. This is exactly the situation Simon Willison found himself in—in a storage room piled with unopened moving boxes, something was beeping intermittently. He didn't want to dig through every box, so he had Claude Code build a system called Beep Locator: using multiple devices as microphones, it pinpoints the sound source through trilateration.

The final system locked the sound location to within about 40 centimeters. Following that position, he searched through the boxes and found an old carbon monoxide detector badly in need of a battery replacement.

How the System Works

At its core, Beep Locator is a Python web application server. The user starts the server on a computer, and the main interface displays a QR code and a join link. Other devices that scan the code or open the link become "listeners." The author used five devices: two iPads, a phone, a MacBook Pro, and a Surface Ultra.

The listener browser requests microphone permission and continuously records a roughly 60-second audio buffer, while running a simple beep detection algorithm locally (detector.js) that uses a fast Fourier transform plus a high-pass filter to identify suspected beeps, then reports events to the server.

The key challenge here is time synchronization. When a listener reports hearing a beep, the server has the device with the lowest round-trip latency play a short chirp, then asks all listeners to upload audio clips from about 0.6 seconds before the beep to about 0.8 seconds after the chirp. This chirp lets the recordings from each device be aligned with one another—the foundation of localization accuracy.

Early in the project, Claude retrieved the 2007 paper BeepBeep: A High-Accuracy Acoustic-Based System for Ranging and Localization Using COTS Devices. That approach uses only the most basic commercial hardware—speakers, microphones, and inter-device communication—and achieves high-precision ranging through three techniques: two-way sensing, self-recording, and sample counting. It fit the requirements perfectly.

Trilateration: What the article calls "triangulation" is usually called trilateration in acoustics—using the distances from a sound source to multiple receivers at known positions to determine the source's coordinates. Sound travels through air at about 340 m/s, so a 1-millisecond arrival-time error corresponds to about 34 centimeters of distance error—which is precisely why the article repeatedly emphasizes time synchronization. In a real system, each listener's distance to the source is derived indirectly through time difference of arrival (TDOA), then equations are solved simultaneously in two or three dimensions to find the intersection. The more devices there are and the more spatially distributed their geometry (avoiding collinearity or coplanarity), the more robust the solution—this is also the theoretical basis for the "suggested position" feature the author added at the end.

Chirp and audio alignment: A chirp is a short signal whose frequency changes linearly or exponentially over time; its autocorrelation function is very sharp, so cross-correlation computation can yield extremely high temporal resolution. The system has each device record the same chirp, then uses cross-correlation to find the chirp's peak position in each recording, aligning the timelines of different devices to the same reference frame. This is more reliable than relying on device system clock synchronization, because clock precision and network latency in browser environments are uncontrollable. The BeepBeep paper uses exactly this kind of "self-recording" idea: a device records the sound it emits itself and calibrates using the inherent speaker-to-microphone delay.

WebAudio and FFT beep detection: The browser-side WebAudio API provides AnalyserNode, which can perform real-time spectral analysis on the microphone stream; what getByteFrequencyData returns is the frequency-domain energy distribution after a fast Fourier transform. Carbon monoxide detectors commonly sound an alarm as a high-frequency pulse around 3–4 kHz, so the detector first applies a high-pass filter to remove human voices and low-frequency ambient noise, then looks for energy spikes in the target band. Frequency-domain thresholds alone are prone to false positives (e.g., keyboard typing), so the system ultimately cross-validates using simultaneous reports from multiple devices and has the server do a second confirmation using the aligned audio clips.

From Rough Localization to Trustworthy Results

Mutual sound-based ranging between devices didn't work well, so the author added "Rough mode": manually drag device icons to approximate positions in the room, and the system then corrects based on the sound data.

He also added a clapping mode for verification—clapping at different locations in the room shows localization results as diamond markers on the map. This gave him confidence in the system's ability to localize sounds. Beeps are shown as X markers and written to the "Sounds located" log.

After confirming the target frequency, the system listens specifically for that frequency, measures the beep intervals, and uses the already-trilaterated positions to converge on the single most likely point on the map. The interface also has a "hot/cold" panel showing the loudness each device hears—the material's claim is that roughly 6 dB higher loudness means roughly half the distance, providing an intuitive basis for judging how far away the source is.

The system also retains a skepticism mechanism: when the arrival times across devices are physically impossible, it flags "this may be an echo or another sound."

Development Process: Letting Claude Iterate on Its Own

The project started on Claude Code Cloud on an iPhone, using Opus 5.5's Extra thinking effort mode. The first prompt simply asked it to write a problem.md describing the problem. The author then had Claude add a mechanism to push new code to listeners for automatic execution—so he could set up the devices and then let Claude keep iterating while he did other things.

This hot-reload mechanism proved to be the key to efficiency. Some devices re-request microphone permission on page refresh, requiring manual interaction; by pushing new code over WebSocket, there's no need to interact with each device manually.

The project later switched to Fable 5.2 to continue. The author's involvement became sitting at the computer discussing the current state with the model and occasionally making requests. Features he proposed included: drag-based rough localization, clap testing with distinct symbols, map symbols with timestamps, sound playback with a play button (temporarily disabling microphone recording during playback), labeling a recording as "this is that mysterious beep," and a "Reposition the devices" button—repositioning without restarting devices to improve accuracy through finer geometry. Finally, a suggested-position feature was added: based on existing data, it suggests on the map where placing one more listener would be most useful.

Giving the Coding Agent "Ears"

The author considers the most interesting part of this project to be that it proves you can give Claude Code ears. Claude itself has no native audio input, but it can run JavaScript and Python code to analyze audio captured by microphones. Having Claude start a server with WebSocket, run WebAudio microphones on devices, and allow new code to be pushed and executed over the connection—this combination lets it repeatedly query audio logs and reconfigure its own "hearing" to make progress on the problem.

He mentioned hearing Martin Woodward talk about a similar approach at GitHub Universe: connecting a GitHub Badge computer to a computer via USB-C and pointing a camera at its screen, so the coding agent can tell whether an update has taken effect. Creating channels for agents to perceive the real world is itself a fascinating direction to explore.

Key Takeaways

• With multiple ordinary devices + a Python server + browser WebAudio, you can build a usable sound-source trilateration system without specialized hardware
• The core of localization accuracy is time alignment: having one device play a chirp lets the audio clips from each listener be aligned with one another
• Audio analysis code can be hot-pushed to listeners, avoiding the need to operate each device due to microphone permission popups
• Perception channels such as vision and hearing can be "bolted on" to coding agents, significantly expanding their ability to solve problems autonomously
• The combination of rough localization + clap verification + sound playback makes the system both usable and trustworthy

One-line Summary

Using multiple ordinary devices for sound-source trilateration to help Claude Code find an intermittently beeping carbon monoxide detector in a storage room.

Paragraph Summary

Faced with a device in a storage room that beeped intermittently with no way to tell where it was coming from, Simon Willison had Claude Code build Beep Locator: a sound-source localization system consisting of a Python web server plus multiple browser-based listeners. The listeners use WebAudio to record continuously and run local FFT beep detection; after receiving a report, the server has a low-latency device play a chirp, and cross-correlation of the recordings aligns the devices' timelines. The system ultimately locked the source to within about 40 centimeters, and he actually found a carbon monoxide detector that needed a new battery. The system also added rough localization, clap verification, sound playback, and suggested device placement to build trust. During development, code was hot-pushed to listeners over WebSocket so Claude could iterate autonomously, leading the author to propose the broader idea of "giving coding agents ears."