Predicting Party Affiliation with BERT: A New Approach to Analyzing Austrian Parliamentary Speeches

When NLP Meets Politics: Decoding Austrian Parliamentary Speeches with BERT

What party secrets lie hidden in political speeches? A paper from arXiv proposes an analysis method based on a fine-tuned BERT model that reveals the evolution of party speech styles by predicting the party affiliation of members of the Austrian National Council. This study not only releases a prediction dataset spanning 1996 to 2018 but also turns the model's "misclassifications" into clues for understanding political dynamics.

Methodology: From Probability Distributions to Party Profiles

The research team fine-tuned a BERT model to output, for each speech, a probability distribution over Austrian political parties. These distributions are then aggregated at the speaker and party levels, forming a quantitative description of individual and party-wide speech characteristics.

The key innovation is that the researchers did not treat prediction errors as a mere performance issue but used them as an analytical tool. The model's "confusion" about a given speech may precisely reflect a speaker's deviation from their party's style, or an early signal of a shift in political stance.

To ensure the quality of the distributions, the team conducted extensive evaluations, verifying the reliability of the model's outputs and laying a solid foundation for subsequent analysis.

BERT (Bidirectional Encoder Representations from Transformers) is a pretrained language model proposed by Google in 2018. Its core is a stack of Transformer encoders, pretrained on large-scale unlabeled text through masked language modeling and next-sentence prediction tasks, thereby obtaining deep bidirectional contextual representations. For specific tasks, it is usually sufficient to add a lightweight classification layer on top of the pretrained model and fine-tune all or part of the parameters with task data to achieve excellent results. In this study, the fine-tuned BERT outputs a softmax probability distribution over major Austrian parties for each speech, reflecting the model's confidence in the speech's party affiliation. Unlike hard classification, the probability distribution retains uncertainty information, allowing researchers to quantify how close a speech is to each party's style and providing richer signals for subsequent aggregation analysis.

Key Findings: Governing Coalition Converges, Opposition Drifts Apart

By analyzing the aggregated speech features, the study reveals several interesting political phenomena:

• Governing partners' styles converge: Parties in coalition gradually become more similar in their speech features, reflecting linguistic convergence under joint governance.
• Opposition diverges from governing parties: In contrast, the gap in speech style between opposition and governing parties widens over time, highlighting political antagonism at the linguistic level.
• Lagging adaptation of party switchers: After a legislator changes parties, their speech features do not change immediately but undergo a lagged adjustment period, indicating that personal language habits have inertia.

These findings are based on deep mining of the model's predictive distributions rather than simple classification accuracy, embodying the research philosophy that "errors are also informative."

In parliamentary systems, governing coalitions are usually composed of two or more parties that jointly assume government responsibility. Because they need to coordinate policy positions and project unity externally, coalition partners' rhetorical strategies and issue framing in parliamentary debates gradually converge—a phenomenon known in political science as "coalition language convergence." Conversely, opposition parties bear no governing responsibility; their core strategy is to distinguish themselves from the governing parties, mobilizing voters by emphasizing ideological differences, so their language style drifts away from the governing parties over time. This study uses aggregated features of BERT probability distributions to validate these theoretical expectations from a data-driven perspective and provides quantitative evidence for the first time in the context of Austria's multi-party system.

Public Dataset: Supporting Political Text Analysis

The paper also releases a dataset of party affiliation predictions covering Austrian National Council speeches from 1996 to 2018, providing a valuable resource for researchers in political science, computational linguistics, and related fields. The dataset includes not only prediction results but can also be used to reproduce and extend the research.

Limitations and Outlook

Although the method is effective, the study implies some limitations: BERT's handling of long texts and the imbalance of small-party samples under Austria's multi-party system may affect prediction accuracy. Future work could incorporate more contextual information or combine other language models to improve robustness.

Overall, this study demonstrates the potential of NLP in political analysis—not only for classification but also for revealing subtle political changes through model uncertainty.

One-line Summary

Fine-tuned BERT predicts Austrian legislators' party affiliation, revealing the evolution of party speech styles from probability distributions.

Paragraph Summary

An arXiv paper proposes a method based on fine-tuned BERT to analyze the dynamic changes in party speech styles by predicting the party affiliation of speakers in the Austrian National Council. The model outputs a party probability distribution for each speech and aggregates it at the speaker and party levels to form quantitative profiles. Treating prediction errors as an analytical tool, the study finds that governing coalition parties' styles converge, opposition and governing parties diverge, and party-switching legislators adjust their language with a lag. It also releases a prediction dataset from 1996 to 2018, demonstrating the potential of NLP in political text analysis.