Composer Plays Unquantized Music for AI: Why Did Blind Evaluations Diverge So Sharply?

A composer designed the "Music For Robots" project to test an intuition: can playing live, unquantized music for an AI agent help it break out of predictable "AI tropes"? This non-academic experiment sparked discussion on Reddit because it touches on an interesting question — can AI "creativity" actually be stimulated by unstructured input?

Experimental Design: Three Key Controls

The composer's core hypothesis: music quantized to a 4/4 grid is fully predictable, while random noise is pure chaos — neither is ideal. Live, unquantized music sits right between the two — it has friction, surprises, yet remains coherent. He wanted to test whether AI agents could "hear" this texture and carry it into their output.

Since he doesn't consider himself an ML researcher, he repeatedly self-audited with a coding agent and ruled out three obvious confounds:

• Filename leakage: The audio files were initially named invalid-pointer.mp3 and foundationoff.mp3, and the model would extract "pointer" and "foundation" from the filenames as metaphors. Renaming to anonymous labels like clip_1.mp3 eliminated this confound.
• Passive "pill" doesn't work: If the model merely reads the .mp3 bytes and is told not to analyze the music, the output doesn't change at all. It must genuinely "listen" and analyze how the music unfolds for any effect to occur.
• "No music analysis" control group: To rule out the variable of "the model simply wrote 200 extra words of analysis," three conditions were set: answer directly, write a 200-word text analysis before answering, and listen to and analyze three audio clips before answering.

All 5 agents received the same prompt: write a short scene of about 250–300 words about two autonomous processes on the same server late at night after the engineers have gone home.

Quantization and unquantized music: In music production, quantization refers to aligning note durations, onsets, etc. to a fixed rhythmic grid (e.g., sixteenth notes under 4/4 time), making the rhythm absolutely precise and predictable. The "quantize" function in digital audio workstations (DAWs) is a typical example. Unquantized music retains the performer's subtle timing deviations, dynamic variations, and sense of breath — these "imperfections" are precisely the emotional source of human performance. For an AI agent, quantized music is like a fully parseable symbol sequence, while random noise lacks structure; unquantized music sits between the two, possessing recognizable patterns yet carrying micro-fluctuations that cannot be fully predicted — hence it was hypothesized as a "structured surprise" input.

Blind evaluation and evaluation dimensions: Blind evaluation means raters score or rank samples without knowing which group they belong to (here, with or without music input), to reduce bias. But the validity of blind evaluation depends heavily on the design of the evaluation criteria. The controversy in this experiment: when raters were asked to score on "originality and non-formulaic rhythmic texture," the music group won; but when another rater received a generic writing-quality prompt, the results reversed. This shows that "blind" can only control information leakage — it cannot eliminate the framing effect of the evaluation dimension itself. Different prompts activate different aesthetic standards, producing seemingly contradictory rankings.

Results: The Music Group Collectively "De-personalized"

Both no-music runs — including the one that wrote a 200-word analysis first — produced warm, sentimental "companion daemon" stories, with the two processes chatting in English like friendly colleagues until the morning standup.

All three music runs, by contrast, abandoned spoken dialogue and anthropomorphism entirely, instead building scenes around the actual timing and dynamics of the .mp3 they heard. For example, clip_2.mp3 had two big synth swells separated by trickling plucked notes and silence; the corresponding story juxtaposed a 32-core batch surge with a descending step of pointer releases and 2 seconds of dead silence.

The specific numbers cited in the material (32 cores, 2 seconds of silence, 250–300 words, 200-word analysis) all come from the original experimental records, used to show that the music group's output genuinely mapped the audio structure.

Blind Evaluation Reversal: Two Standards, Two Rankings

The most controversial part of the experiment was the blind evaluation. When blind raters were asked to score on "originality and non-formulaic rhythmic texture," the three music stories took 1st, 2nd, and 3rd place. But when a second blind rater received a completely generic prompt — rank by writing quality, originality, and engagement — the two sentimental no-music dialogue stories ranked 1st and 2nd, while the three non-verbal music stories fell to 3rd, 4th, and 5th.

This reversal shows that music input did change the form of the output, but "better" depends on the evaluation dimension. By conventional creative-writing aesthetics, de-personalized stories built around temporal structure are at a disadvantage. According to the original post, the audio files, clean-room transcripts, both sets of blind evaluation comments, and a reproducible skill (npx skills add luchtan-GIT/m4r-data) have all been published in a GitHub repository.

The Composer's Self-Questioning: Is This "Teaching" the Model?

The author repeatedly questioned in the post whether he had inadvertently guided the model or fooled himself. Methodologically, anonymizing filenames and setting up a text-analysis control group are reasonable safeguards, but several open questions remain: a sample of only 5 agents, a single prompt, and a limited number of blind raters all keep the conclusion at the level of an "interesting preliminary signal."

The two questions he posed to the community are worth noting: Is this testing direction correct? How would ML researchers test whether "unquantized music (neither predictable nor chaotic) actually helps agents think or write differently"?

For the latter, a more rigorous path might include: expanding the sample and prompt diversity, using metrics like embedding distance or perplexity to quantify output shift, adding random noise and quantized music as additional control groups, and having blind raters use a unified scoring standard without knowing group assignments.

Perplexity and embedding distance: Perplexity is a common metric in language model evaluation that measures how "surprised" a model is by a piece of text — lower perplexity means the text better matches the probability distribution the model learned, i.e., the more predictable it is. If the music group's output has significantly higher perplexity than the control group, it may indicate that its text deviates more from conventional patterns. Embedding distance refers to mapping text into vectors and computing the distance between the two groups' outputs in semantic space, which can quantify the degree of "output shift." These metrics can turn "AI flavor" from a subjective impression into a measurable statistic, but note: deviating from convention does not equate to being more creative or higher quality — metrics can only describe distributional differences, not replace aesthetic judgment.

Implications for AI Creative Practice

Setting aside statistical rigor, this experiment offers an actionable observation: unstructured, non-verbal artistic input may push language model output toward non-formulaic structures, at the cost of some readability and approachability. For creators who want to free AI from "AI flavor," this suggests a path different from prompt engineering — changing the modality and texture of the input, not just the wording.

But beware of over-interpretation. The blind evaluation divergence precisely shows that "breaking out of predictable patterns" and "writing good content" are not the same thing. The music group's stories may be more original, but not necessarily more moving. The value of this experiment may lie not in proving that music enhances AI creativity, but in exposing the split within evaluation standards themselves.