Membership Inference Attack

Common methods for detecting memorization include: constructing prompts from training set excerpts and measuring the similarity between the output and the original audio, analyzing the model's likelihood anomalies on training samples, and membership inference attacks.

The pretraining corpora of the vast majority of models are closed-source, forcing membership inference research to rely on heuristic methods to construct members and non-members, which may introduce systematic bias.

The authors cancel out the confounding effects of fluency and text quality by reading the same sentence with two different models.

When a sentence is repeated more than approximately one thousand times, membership inference traces become significant.

Highly repeated sentences tend to be famous quotes repeatedly cited on the internet, and the two corpora are highly consistent in which sentences are highly repeated, making it impossible to distinguish training exposure from fame.

The core logic of Membership Inference Attacks (MIA) is: when a language model assigns an extremely low perplexity to a sentence, it is inferred that the sentence was in the training data.

When the control sentences were replaced with sentences of a different register from the member sentences, the AUC of the MIA detector soared from 0.83 to 0.94.

High-AUC MIA detectors are actually detecting stylistic and register differences between member and non-member sentences, rather than model memorization.

Great caution should be exercised when using membership inference attack results as legal or policy evidence, as high AUC may simply be an artifact of stylistic differences.

In word-substitution experiments, regardless of whether the original sentence appeared once or a hundred times in the corpus, the model's preference gap for the original sentence was the same, indicating that the author's phrasing rather than memorization is being rewarded.