Strengths and Weaknesses of Corpus Linguistics in Legal Analysis: A - - PowerPoint PPT Presentation

Strengths and Weaknesses of Corpus Linguistics in Legal Analysis: A Case Study of the Law and Language at the European Court of Justice Project

Karen McAuliffe University of Birmingham k.mcauliffe@bham.ac.uk @dr_KMcA

Goldfarb, Neal, ‘Corpus LinguisFcs in Legal InterpretaFon: When Is It (In)appropriate?’ (Paper presented at the Law & Corpus LinguisFcs Conference, BYU Law School, 5 February 2019). Available at SSRN: hZps://ssrn.com/abstract=3333512 Solan, Lawrence, ‘Legal InterpretaFon by Big Data: Promises and Perils’ (Keynote Lecture at the JurilinguisFca II Conference, Seville, 24 October 2018).

Outline

• 1. The LLECJ Project: brief overview
• 2. Focus on CL methods used in one subproject
• 3. Consideration of the value of the CL analysis for

legal research

• 4. Takeaways: a note of caution

Law and Language at the ECJ

Producing a multilingual jurisprudence Development of ‘precedent’ in ECJ judgments The changing role of the AG

www.llecj.karenmcauliffe.com

28 Member States 24 Official Languages

28 Member States 24 Official Languages

TranslaFon at the ECJ

BG ES CS DA DE ET

EL

EN FR GA HR IT LV LT HU MT NL PL PT RO SK SL FI SV BG ES CS DA DE ET

EL

EN FR GA HR IT LV LT HU MT NL PL PT RO SK SL FI SV

JUDGMENT WORKING LANGUAGE: FRENCH

Processing a case through the CJEU

Case brought before CJEU Allocated to judge rapporteur (and AG where relevant) Documents translated into French Report of the judge rapporteur prepared by référendaire (in French) Where relevant, AG and référendaires prepare opinion (in pivot languages) First version of judgment drajed by référendaire (in French) Secret deliberaFons (in French) Final judgment drajed (in French) Judgment translated into language of the case (authenFc version of judgment and version signed by judges) and all other official languages

AG Opinions

• Non-binding but Court must ‘take account’ of opinion

before delivering judgments (in relevant cases)

• Not Fed to maZers raised in a parFcular case
• Historically wriZen in the style of an academic paper
• The Court’s ‘sparring partner’ – creaFng a dialogue in

which principles of EU law are developed

• The most crea&ve point in the system of producFon of

the ECJ’s jurisprudence

ECJ Judgments

• WriZen in a new hybrid legal language
• Built up like Lego building blocks – formulaic style, mulF-

layered authorship

McAuliffe, K (2011) “Hybrid Texts and Uniform Law? The producFon of a mulFlingual jurisprudence by the Court of JusFce of the European Union” Interna&onal Journal for the Semio&cs of Law 24(1), 97-115 McAuliffe, K (2013) “The LimitaFons of a MulFlingual Legal System” Interna&onal Journal for the Semio&cs of Law 26(4) 861-882

Pivot TranslaFon

PIVOT LANGUAGES

• EN
• DE
• IT
• ES

Since 2004 (Convention)

Opinions drajed in pivot languages

EN DE ES FR IT (PL)

BG ES CS DA DE ET EL EN FR GA HR IT LV LT HU MT NL PL PT RO SK SL FI SV

Pre-2004: AGs always drajing in their mother tongue Post-2004: AGs drajing in one of the pivot languages of the Court (=EN/someFmes FR)

What implica4ons might changes in the linguis7c aspect of the AG’s role have for the construc4on and consolida4on of ECJ jurisprudence?

• 1. To what extent has language use had an impact on the

usefulness of opinions?

• 2. Are AGs’ opinions becoming more syntheFc in construcFon

and their arguments more constrained by language as a result of the fact that they no longer draj in their mother tongue?

• 3. Do the opinions of permanent AGs differ from those drajing

in pivot languages?

LinguisFc Analysis

• 1. To what extent has language use had an impact on the

usefulness of opinions?

• 2. Are AGs’ opinions becoming more syntheFc in construcFon

and their arguments more constrained by language as a result of the fact that they no longer draj in their mother tongue?

• 3. Do the opinions of permanent AGs differ from those drajing

in pivot languages?

• 1. Judgments are stylis4cally simpler and less ‘fluent’

than opinions

• 2. Ajer the 2004 language regime change opinions

become more similar (linguisFcally) to judgments

• 3. Opinions drajed ajer 2004 by non-permanent AGs

(i.e. AGs drajing in a language other than their mother tongue) are stylisFcally simpler and less ‘fluent’ than the ones drajed by AGs (drajing in their mother tongues) prior to 2004.

Main objecFves

• 1. Determining the stylisFc complexity and the

fluency of the judgments and the opinions before and ajer the linguisFc reform

• 2. Comparing the results

between judgments/opinions between the opinions drajed before/ ajer the 2004 linguisFc reform

Research quesFons

• Which linguisFc features do represent the fluency

and the stylisFc simplicity/complexity of a text?

• How can such features be idenFfied in the

considered opinions and judgments?

• Do the judgments present less features

represenFng stylisFc complexity and fluency than the opinions?

• Do the opinions become more similar to the

judgments ajer the 2004 linguisFc reform?

• Do the opinions drajed before 2004 present

more features represenFng stylisFc complexity and fluency than the ones drajed ajer 2004?

DeterminaFon

• f stylisFc

complexity and fluency Comparison

• f the results

Analysed archive of corpora

EN FR Judgments……………………………………………………………………………………………….. Opinions

drajed before 2004……….……………………………….…. drajed ajer 2004 from naFve AGs……….... drajed ajer 2004 from non-naFve AGs…..

Judgments ………………………………………………………........................................

….

Opinions

drajed before 2004…………………………………………… drajed ajer 2004 from naFve AGs…………. drajed ajer 2004 from naFve AGs………....

(2208 texts, > 10 million words) (341 texts, 2 million words) (276 texts, > 2,5 million words) (367 texts, > 3 million words) (2357 texts, > 10 million words) (410 texts, > 2,5 million words) (339 texts, > 3,5 million words) (345 texts, > 3,5 million words)

Methodology

• 1. DeterminaFon of the features related to

fluency and stylisFc simplicity/complexity

• 2. Search of each one of the determined

features in each corpus

• 3. Comparison of the obtained between the

four sets of texts analyzed for each language

• 4. Extra exams suggested by the obtained

results

Results

Judgments are stylisFcally simpler and less fluent than the opinions

• Lexical variety à More repeFFons
• Lexical density
• Sentence length
• Lower presence of hypotacFc structures

In English In French

Results

Opinions drajed ajer 2004 by non naFve authors are stylisFcally simpler and less fluent than the ones drajed before 2004 in a naFve language

• Lexical variety
• Lexical density
• Sentence length
• Lower presence of hypotacFc structures

Opinions become more and more similar (linguisFcally) to judgments

Results

LEXICAL VARIETY CreaFon of a WORD LIST 𝑈𝑈𝑆=​𝑢𝑧𝑞𝑓𝑡/ 𝑢𝑝𝑙𝑓𝑜𝑡 ×100

0.468 0.468 1.146 0.86 0.849 0.2 0.4 0.6 0.8 1 1.2 1.4 before 2004 ajer 2004

English corpus lexical variety

Judgments NaFve opinions Non NaFve opinions 0.43 0.43 1.103 0.893 0.788 0.2 0.4 0.6 0.8 1 1.2 before 2004 ajer 2004

French corpus lexical variety

Judgments NaFve opinions Non NaFve opinions

Results

Sentence length

52.01 55.07 33.63 45.01 36.22 10 20 30 40 50 60 ajer 2004 before 2004

Mean sentence length of English corpus

Judgments NaFve opinions Non naFve opinions 57.32 64.13 33.92 46.24 36.09 10 20 30 40 50 60 70 ajer 2004 before 2004

Mean sentence length of French corpus

Judgments NaFve opinions Non naFve opinions

Results

HYPOTACTIC STRUCTURES

• Choice of an exhausFve list of

subordinate conjuncFons

(e.g. when, than, because, etc.)

• IdenFficaFon of each conjuncFon within

the wordlist and of its frequency

• Sum of the frequency of each

subordinate conjuncFon included in the considered list

2882 + 2767 + ……. = In the French corpus:

Judgments vs. Sta4s4cal significance Minimal LL value = 6,63, the greater the LL value, the greater the difference Opinions drajed before 2004 1192,59 NaFve opinions drajed ajer 2004 134,28 Non-naFve opinions drajed ajer 2004 0,19

Results

HYPOTACTIC STRUCTURES In the English corpus: Opinions drajed before 2004 NaFve opinions drajed ajer 2004 Non naFve opinions drajed ajer 2004 vs. 965.49 978.25

Results

Lexical density: no fix paZern

0.86 0.88 0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 English Judgments Opinions before 2004 NaFve opinions ajer 2004 Non naFve opinions ajer 2004 0.88 0.9 0.92 0.94 0.96 0.98 1 1.02 French Judgments Opinions before 2004 NaFve opinions ajer 2004 Non naFve opinions ajer 2004

Resuming….

Opinions become gradually more similar to judgments (i.e. stylisFcally simpler and less fluent) as for:

• A decreasing lexical variety
• An increasing quanFty of repeFFons
• An increasing sentence length
• A decreasing quanFty of hypotacFc

structures

Extra exam

IdenFficaFon of idenFcal fragments between judgments and opinions

English corpora 100% matching fragments Opinions Before 2004 Ajer 2004 NATIVE Ajer 2004 NON NATIVE All matching fragments 10,832 100% 4,145 100% 29,218 100% ≥ 10 words 331 3% 1,972 23.17% 2,028 27.81% ≥ 30 words 85 0,7% 1,000 11.75% 897 12.30% ≥ 100 words 5 0.04% 45 0.52% 41 0.56% French corpora 100% matching fragments Opinions Before 2004 Ajer 2004 NATIVE Ajer 2004 NON NATIVE All matching fragments 5,407 100% 7,155 100% 6,139 100% ≥ 10 words 389 7.19% 1,438 20.09% 1,269 20.08% ≥ 30 words 115 2.12% 559 7.81% 421 6.66% ≥ 100 words 8 0.14% 24 0.33% 21 0.33%

Conclusions?

• Results show judgments to be stylis4cally

simpler and less ‘fluent’ than Advocates Generals’ opinions

Conclusions?

• Results show Judgments to be stylis4cally simpler and

less fluent than Advocates Generals’ opinions

• The introduc4on of the linguis4c reform in 2004 did have

an influence on the style and the fluency of the opinions Opinions become more similar to judgments Opinions become stylisFcally simpler and less ‘fluent’ than previously So what?

Conclusions?

• Results show judgments to be stylis4cally simpler and

less fluent than Advocates Generals’ opinions

• The introduc4on of the linguis4c reform in 2004 did have

an influence on the style and the fluency of the opinions Opinions become more similar to judgments Opinions became stylisFcally simpler and less fluent than previously Opinions are the same as judgments so lose their value

Mixed-Method Empirical Research

Literature Reviews: Value of AGs’ opinions is in persuasiveness and language and writing style is a fundamental part

• f the ability to persuade

Interview & Observational Data: Opinions are multi-layered/multi-authored texts Tensions between law, language, and translation 2004 language changes – ‘no impact’ on language and style of opinions

Conclusions?

• Results show judgments to be stylis4cally simpler and

less fluent than Advocates Generals’ opinions

• The introduc4on of the linguis4c reform in 2004 did have

an influence on the style and the fluency of the opinions Opinions become more similar to judgments Opinions became stylisFcally simpler and less fluent than previously Opinions lose their original eloquence Do they then begin to lose their value?

What is the purpose of AGs’ Opinions?

• To compensate for lack of individual judgments/

dissenting judgments?

• To persuade the ECJ in a particular case?
• To force the ECJ to (re)evaluate principles of EU law?
• To provide context for wider EU scholarship?

What might all this mean for the development of EU Law?

Takeaways

• 1. Research design is important
• 2. You need legal expertise in your research team
• 3. Resist the temptation to publish at the stage of

achieving results that are interesting from a CL point of view alone – take a deeper dive

• 4. CL can be very valuable in legal research, but
• nly as part of a wider methodology