SpectralCanary: How Low-Frequency Signals Audit T2I Dataset Misuse

Introduction: The Challenge of Dataset Auditing

Text-to-image (T2I) models are astonishingly capable, but the provenance of their training data is often opaque. A core question is: can we determine whether a given model has been fine-tuned on a specific image collection? Existing auditing methods often trade off verifiability, fidelity, robustness, and ease of deployment. A recent paper proposes SpectralCanary, an "embarrassingly simple" solution—using low-frequency signals as a dataset watermark to enable efficient auditing.

Core Method: Low-Frequency Mask-Mixed Canary Images

The Active Auditing Approach

Active dataset auditing typically involves two steps: implanting a shared signal (called a "canary") into a dataset before release, and then detecting whether that signal appears in the outputs of a suspicious model. Existing methods often require auxiliary generation or per-image optimization, which adds complexity and computational cost. SpectralCanary's simplicity lies in using a smooth low-pass mask to blend a fixed canary image into every collection image while keeping captions unchanged, with no additional generation or optimization required.

Dataset-Use Auditing is a technique in model governance aimed at determining whether a model has used a specific dataset for training or fine-tuning. Depending on when the signal is implanted, it can be divided into passive and active auditing. Passive auditing relies on inherent traces in the data itself (such as copyright watermarks), but such traces are easily destroyed by data preprocessing; active auditing, on the other hand, requires actively implanting a detectable signal (the "canary") before dataset release, so that dataset usage can be traced in model outputs. The core challenge of active auditing is that the implanted signal must be sufficiently subtle (not affecting data usability) while also being sufficiently robust (still detectable after training). SpectralCanary belongs to the active auditing category, where its "canary" is a fixed low-frequency image blended into every collection image.

Technical Details

Specifically, SpectralCanary applies a low-pass filter (e.g., Gaussian blur) to each image, then overlays the canary image at a certain weight. Because low-frequency components dominate in images and have little impact on human vision, this blending is nearly imperceptible visually, yet effectively transmits the signal into the fine-tuned model. A lightweight verifier then detects the signal in model outputs to perform auditing.

A low-pass filter is an image processing operation that allows low-frequency signals to pass while suppressing high-frequency signals. In the spatial domain, it manifests as a weighted average of neighboring pixels (e.g., Gaussian blur), smoothing the image; in the frequency domain, it preserves the overall brightness and color distribution of the image while attenuating details such as edges and textures. Since the energy of natural images is mainly concentrated in low-frequency parts, low-pass filtered images change little visually and are difficult for humans to perceive. SpectralCanary exploits this property by first applying a low-pass filter (e.g., Gaussian blur) to each image, then overlaying the canary image at a weight. In this way, the canary's low-frequency components naturally fuse with the image's original low-frequency components, preserving the overall appearance of the image while embedding the signal into a frequency band that the model can easily learn.

TPR@1% FPR is a commonly used evaluation metric in detection and auditing. TPR (True Positive Rate) refers to the proportion of correctly detecting the presence of a signal; FPR (False Positive Rate) refers to the proportion of no-signal samples misclassified as having a signal. TPR@1% FPR represents the highest detection rate achievable when the false positive rate is strictly controlled at 1%; a higher value indicates that the detector is more sensitive under low false alarm conditions. In dataset auditing scenarios, a low FPR is crucial because falsely accusing a model of using a dataset can bring legal or reputational risks. SpectralCanary reports a TPR@1% FPR of over 91%, indicating that it can reliably detect the canary signal even under extremely low false positive rates.

Experimental Results: High Detection Rate and Low Visual Impact

The paper conducts extensive evaluations on multiple datasets and model architectures. On the Pokemon dataset, across three real data transformations and two architecture families, SpectralCanary achieves the highest average TPR@1% FPR (91.1%) per generation, and with 30 generations, the model-level accuracy at a fixed threshold reaches 94.8%.

On five datasets covering art, medical, satellite, and fashion domains using the SD3.5 model, SpectralCanary achieves an average TPR@1% FPR of 92.4% while maintaining the visual quality of released images (SSIM 0.989, LPIPS 0.022), with FID change near zero (average ΔFID 0.05). This means that implanting the signal barely affects the original distribution of the dataset.

Why Is Low Frequency More Effective?

The researchers conducted matched-energy ablation experiments on Pokemon and ROCOv2, comparing the effects of low-frequency, mid-frequency, and high-frequency placement. The results show that low-frequency placement transmits signals more reliably. This may be because low-frequency components are easier for the model to learn and generalize, while high-frequency components are easily discarded or distorted during fine-tuning. This finding supports the design choice and reveals the potential of low-frequency visual signals as collection-level auditing traces.

Matched-energy ablation experiments are a controlled-variable experimental design used to fairly compare the effects of signals in different frequency bands. In image frequency analysis, different frequency bands (low, mid, high) contain different amounts of energy (i.e., signal strength); direct comparison may lead to biased results due to energy differences. Matched-energy ablation experiments adjust the amplitude of signals in each frequency band so that their total energy at embedding is kept consistent, thereby eliminating the interference of energy factors and examining the impact of frequency band position on signal transmission reliability in isolation. SpectralCanary conducted such experiments on the Pokemon and ROCOv2 datasets and found that even with equal energy, low-frequency placed signals are still easier to detect after fine-tuning than mid- and high-frequency ones, supporting the superiority of low-frequency signals as auditing traces.

Robustness: Performance Under Adversarial Attacks

The paper evaluates robustness under adaptive attacks. The most effective removal methods require aggressive modifications and lead to a ΔFID increase of over 25 points, meaning that if an attacker wants to remove the signal, they must significantly degrade image quality, thereby sacrificing the model's utility. This provides SpectralCanary with robustness guarantees in practical scenarios.

Significance and Outlook

SpectralCanary provides a simple, efficient, and low-cost T2I dataset auditing solution. It does not require complex optimization or auxiliary generation; low-frequency blending alone achieves high detection rates while having minimal impact on image quality. This method is expected to become a standard practice for dataset release, helping creators and researchers track data usage. Future work can further explore adaptability across different model architectures and attack scenarios, as well as how to combine low-frequency signals with other auditing methods.

In summary, SpectralCanary demonstrates that low-frequency visual signals are an effective carrier of collection-level T2I auditing traces, opening new avenues for dataset misuse auditing.

One-line Summary

SpectralCanary uses low-frequency mask blending of canary images to achieve simple and efficient T2I dataset auditing.

Paragraph Summary

SpectralCanary proposes an embarrassingly simple text-to-image dataset auditing method: before dataset release, a low-pass mask blends a fixed canary image into every image, with no auxiliary generation or per-image optimization required. The fine-tuned model retains this low-frequency signal in its outputs, which a lightweight verifier can detect. On the Pokemon dataset, TPR@1% FPR reaches 91.1%; on five domain datasets, it reaches 92.4% while maintaining image quality (SSIM 0.989, ΔFID 0.05). Matched-energy ablation experiments show that low-frequency signals are more reliable than mid- and high-frequency ones. Adversarial attacks require aggressive modifications (ΔFID >25) to remove the signal, indicating practical robustness. This work provides a new avenue for dataset misuse auditing.