IntLawNER
IntLawNER is the first named entity recognition dataset and benchmark for international law, covering ICJ judgments, UN Security Council resolutions, and ECtHR rulings. It contains 2,987 manually annotated sentences and 8,094 entity spans, using a seven-category annotation scheme based on institutional function.
Timeline (last 90 days)
IntLawNER is the first token-level named entity recognition (NER) dataset and benchmark for written international law sources in the field of international law.
The corpus of IntLawNER comes from three major legal sources: International Court of Justice (ICJ) judgments, United Nations Security Council resolutions, and European Court of Human Rights (ECtHR) rulings.
IntLawNER contains 2,987 manually gold-annotated sentences and 8,094 entity spans, divided into seven entity types based on institutional characteristics.
The research team compressed the original 468,000-sentence source text into a streamlined annotation collection.
The IntLawNER annotation pipeline consists of three stages: candidate retrieval, large language model-based review, and human verification.
In the final gold annotations, 89.6% of entity spans were accepted directly from the silver layer (machine preliminary annotation results) without modification.
Cohen's kappa reaches 0.964 on boundary-matched spans, but drops sharply to a macro F1 of 0.753 after incorporating missed entities, boundary errors, and label corrections.
The zero-shot span-based GLiNER model almost completely fails on entity types that depend on institutional function, achieving a micro F1 of only 0.243.
When few-shot examples demonstrating label contrasts are included in the prompt, all large language models outperform zero-shot prompting.
In the IntLawNER benchmark, Claude Opus 4.6 achieves the best result with a micro F1 of 0.873.
2 more timeline events
All Facts (12)
IntLawNER is the first token-level named entity recognition (NER) dataset and benchmark for written international law sources in the field of international law.
50%UnverifiedThe corpus of IntLawNER comes from three major legal sources: International Court of Justice (ICJ) judgments, United Nations Security Council resolutions, and European Court of Human Rights (ECtHR) rulings.
50%UnverifiedIntLawNER contains 2,987 manually gold-annotated sentences and 8,094 entity spans, divided into seven entity types based on institutional characteristics.
50%UnverifiedThe research team compressed the original 468,000-sentence source text into a streamlined annotation collection.
50%UnverifiedThe IntLawNER annotation pipeline consists of three stages: candidate retrieval, large language model-based review, and human verification.
50%UnverifiedIn the final gold annotations, 89.6% of entity spans were accepted directly from the silver layer (machine preliminary annotation results) without modification.
50%UnverifiedCohen's kappa reaches 0.964 on boundary-matched spans, but drops sharply to a macro F1 of 0.753 after incorporating missed entities, boundary errors, and label corrections.
50%UnverifiedThe zero-shot span-based GLiNER model almost completely fails on entity types that depend on institutional function, achieving a micro F1 of only 0.243.
50%UnverifiedWhen few-shot examples demonstrating label contrasts are included in the prompt, all large language models outperform zero-shot prompting.
50%UnverifiedIn the IntLawNER benchmark, Claude Opus 4.6 achieves the best result with a micro F1 of 0.873.
50%UnverifiedInternational legal texts contain a large number of institutional-function-dependent entities, whose entity types depend on the role played in a specific legal proceeding rather than on the literal wording.
50%UnverifiedIntLawNER is publicly released as a reusable resource, primarily for extracting citations and entities from international legal texts.
50%