Linguistic Discrimination in Writing Assessments: How Raters React - - PDF document

Linguistic Discrimination in Writing Assessments: How Raters React to African American Errors, ESL Errors, and Standard English Errors on State-Wide Tests by David Johnson and Lewis VanBrackle (2011)

Dangerous Minds example: “stupid” and “idiot”. David and Johnson and Lewis VanBrackle tackle the issue of how African-Americans and ESL students are evaluated on standardized writing tests compared to speakers

• f Standard English. Looking at how certain aspects of African American dialects

have lexical items that are considered to be “errors” in Standard English writing. After conducting research and a series of studies, Johnson and VanBrackle were able to successfully examine how there are in face bias opinions when rating essays that were written by African American and ESL student writers compared to those who use Standard English.

A Breakdown

• f Terms
• AAE (African

American English)

• ESL (English as a

Second Language)

• SAE (Standard

American English)

• Dialects: Varities of

a language that differ lexically, phonologically, and grammatically from the standard

• dialect. (Johnson &

VanBrackle, 38)

2

Purpose

3

• Raters of student writing react strongly to writing errors and those reactions range

from irritation and exasperation to amusement and sympathy. We will argue that reactions to errors are a significant factor in the assessment of student writing on a standardized writing exam that is graded holistically. Most significant, this study will demonstrate that raters react differently to “errors” typical of African American English (AAE) writers, of errors of English as a Second Language (ESL) writers, and errors of Standard American English (SAE) writers. The end result is imbalanced assessment

• f essays that are identical except for eight different kinds of errors.
• For this study, we limited our exploration of this linguistic discrimination on the part of

raters to an examination of which kinds of errors raters consider more egregious; thus, we did not consider the other higher level writing features. As the data collection procedures will show, we were able to isolate raters’ reactions to errors exclusively and not consider reactions to other writing issues such as coherence, originally, or style.

• Linguists note that what mainstream writing teachers regard as errors might be

reclassified as features. The African American dialect has many non-standard features (as does any dialect of English) which should be considered a feature from a descriptive linguistic standpoint. These contrast from ESL errors which result from a lack of complete language acquisition.

• We acknowledge this classification of features by non-linguistics as errors, but at the

same time our focus is not on a justification of a group’s right to their own dialect. Our focus is on how different kinds of features (or “errors”) influence writing assessors. Thus, all non-standard elements in the students essays will henceforth simply be called errors.

“Most raters are not fully aware of the linguistic motivations of these errors, and we believe most raters succumb to a societal view that regards non-standard dialect features in writing as simply substandard and careless writing.” (Johnson & VanBrackle, 38)

4

Academic and Anecdotal

“We have heard many raters condemn non-standard dialect features (both AAE and southern American English) as “lazy” or “careless” mistakes and comment that they should result in a failing grade not because of the quantity but because of the severity ad linguistic saliency of them.” (Johnson & VanBrackle, 38)

History

5

• AAE became a dialect due to its unique history and it remains a distinctive dialect like

all non-standard dialects, due to sociopolitical pressure which motivates speakers to maintain it for identity purposes (Milroy, 1992). As with all dialects, there is a variation within the dialect, ubt there are sufficient common linguistic features to justify using a single designation. It is equally important to note not all African Americans use this

• dialect. As with all dialects, there is a continuum of usage by speakers of that dialect.

AAE remains a highly stigmatized dialect of American English.

“

The present study is built upon the notions of surface writing errors. We define surface errors as those non-standard writing (marked) features that do not interfere with communication but would be noted by ost raters as non-conformity to the conventions of standard

• English. Surface errors include spelling, wrong verb forms,

punctuation, and syntactic problems that do not obscure meaning (Connors & Lunsford, 1988) (Johnson & VanBrackle, 39)

6

Research Questions

▹ Do essay raters penalize essays with AAE errors more than essays with SAE errors on a holistically scored writing assessment? ▹ Do essay raters penalize with ESL errors more than essays with SAE errors on a holistically scored writing assessment? ▹ Do essay raters penalize essays with ESL errors more than essays with AAE errors on a holistically scored writing assessment?

7

Three essay topics

The Study

8

• In order to investigate these questions, we used the following design. 1st, we used

collected three sample student-written essays from previous years. These were actual essays written by university students from previous years and graded holistically by exam raters. We used an essay that had a failed, an essay that had passed, and an essay that had a high pass. We then corred all the surface errors in the three exams, but left everything else unchanged. Subsequently, we created an AAE version, an ESL version, and a SAE version of each of the three essays (for a total of nine essays). Topic 1: Failing essay: Do you believe it is the responsibility of the young to provide financial security for the aged? Why or why not? Topic 2: Passing essay: What are most important factors in personal success? Topic 3: High pass essay: Is increased life expectancy a blessing or a curse?

9

The three versions of each of the three essays differed only by the insertion of eight surface errors into each essay. The eight surface errors were errors typical of either AAE, ESL, or SAE writers.

10

11

12

13

Tables 1-3 include samples of the errors that were inserted into the three essays.

14

15

It is important to note that each version of the three essays had exactly eight errors. One or two single errors in the versions would not be a reliable method for noting differences in rater reactions. However, eight was a number we determined (based on a previous pilot study) that caused readers to note errors were of three kinds: AAE, ESL, or SAE. Certainly, many of the errors could cross boundaries. A comma splice and an inappropriate preposition are not, in and of themselves, indicative of one language group. However, we were careful to select linguistic features and a predominance of errors that indicated a pattern that was consistent with AAE, ESL, or SAE.

Limitations

1st Limitation “A simple particular error taken in isolation (e.g. punctuation) might not be considered as serious as another single particular error (e.g. missing preposition or use of hisself) on a timed, holistically-scored writing test.” (Johnson&VanBrackle, 45) 2nd Limitation “While we did not have hundreds of ratings, we consider having “ecological sound” or naturalistic data to answer to this criticism. We had actual essays with actual rates and so despite what some may consider a small sample size, the study employed naturalistic data collection.” (Johnson&VanBrackle, 45)

16

1st-The emphasis in the research design was not to investigate reactions to single errors, but to investigate rater reaction to types of errors taken collectively as they represent a non-standard dialect speaker, a non-native speaker, and a standard English speaker. 2nd-Sample size

Results

17

• The results of this study confirmed that the percentage of failures and low ratings

amongst African-American and ESL students compared to Standard English students had a bias when it came to “errors” in their writing. Even though the students of Standard English did have errors, they were not considered as great as the ones of ESL and African American writers. They say: “This discrimination against AAE (African-American English) features may explain, at least in part, the achievement gap on these standardized tests. Many AAE speakers score well below their SAE (Standard American English) speaker counterparts particularly on assessments that have free responses and essay components…” (Johnson and VanBrackle, 46).

“

“Those who have acquired additional codes because their local language differs significantly from the language of the national culture may actually be in a better position to gain access to the global culture than “mainstream” Americans who, as Martha says, “only know one way to talk.” Rather than think of these diverse students as problems, we can view them instead as resources who can help all of us learn how to become citizens of global community.” Lisa Delpit

18

Connecting this to Lisa Delpit, who is a scholar and author of “Other People’s Children: Cultural Conflict in the Classroom”

Why is this important? (AAE) Sonja Lanehart Alice Lee Lisa Delpit

19

• Lanehart: Wrote Language and Identity. “Schools cna broaden the spectrum of

tolerance and acceptability for different languages as well as linguistic differences in the English language by teaching primary-school children a foreign language using an immersion moden (especially since they would more easily be able to become fluent in the language at such a young age) and teaching middle and high school students the history of the English language. By teaching children a foreign language by immersion, the school is acknowledging the importance of other cultures. Students in American schools would thereby more away from thinking that they’re the center of the universe and realize that we all have something important to contribute in part because of our differences. Hence language differences would be more likely to be seen as something good or as something simply different rather than as something inferior or something to be feared or shunned” (Lanehart, 329)

• Lee: Scholar Alice Lee wrote this article called Why “Correcting” African American

Language Speakers is Counterproductive. “What we learn from all these theories is that language is something that is innate in all of us-no group of people has a capacity for “more” or “better” language than another” (Lee, 31). Going back to Johnson and VanBrackle, when it comes to the raters that evaluated the essays, I questioned who determines if a student is “right” or “wrong”. One cannot assess an essay because of what they consider to be “errors” while the writer considers those “errors” to be a source of familiarity and identity.

• Delpit: “One aspect of language diversity in the classroom, form (the code of a

language, its phonology, grammar, inflections, sentence structure, and written symbols). Usually received the most attention from educators, as manifested in their concern about the “non-standardness” of the code their students speak” (Delpit, 48)

Educators feel the need to simply let the students know that their language or dialect is just not acceptable in a classroom or professional setting. Instead of trying to learn from the student, more than not the teacher dismisses something that is part of their identity.

Why is this important? (ESL)

20

Peter Elbow Paul Kei Matsuda

Muriel Harris and Tony Silva

• Elbow: Peter Elbow wrote Voice in Writing Again: Embracing Contraries. “When

readers hear a voice in a piece of writing, they are often more drawn to read it, and that audible voice often makes the words easier to understand” (Elbow, 176). Allowing the freedom to write without the anxiety and pressure of trying to sound in a way that is unnatural to them, I believe the raters would realize that this student does not lack anything but has a unique voice that can be shaped or molded only to excel even further in their studies.

• Matsuda: Who wrote the article Teaching Composition in the Multilingual World:

Second Language Writing in Composition Studies. “In other words, the question is no longer limited to how to prepare students from around the world to write like traditional students from North America; it is time to start thinking more seriously about how to prepare monolingual students to write like the rest of the world.” -Paul Kei Matsuda (50)

• Harris and Silva: Educators and authors who the article Tutoring ESL Students:

Issues and Options. “Understanding and accommodating cultural differences is, to a great extent, what ESL instruction is all about. This is especially true when working with students who are very new to and not very cognizant of the workings of American

• culture. (Harris and Silva, 528)

More on Matsuda and ESL

• Another main issue that second language writers have is that they have limited

exposure to what is considered the correct use of the English language and formal written English, which means that it is harder for them to develop their writing proficiency in U.S. English compared to people who grew up learning the U.S. formal

• English. Although having these lack of resources is one of the cons that second

language writers have, they also have a pro. “Others suggest that second language writers may have expanded their intellectual capacity as a result of the constant demand of working with a broader range of linguistic and discursive resources” (Matsuda, 40).

• An interesting point that Matsuda brought up that I never thought about before is the

level of difficulty bringing this issue to the classroom is because of the teachers. The teacher should have a balanced knowledge of English and second language writers, which many teachers don’t have.

Conclusion

“The odds ratio table and the percentage failure raters above clearly indicate a bias against errors that contain AAE errors…” (Johnson & VanBrackle, 46) “This discrimination against AAE features may explain, at least in part, the achievement gap on these standardized tests…” (Johnson and VanBrackle, 46) “A more positive pedagogical interpretation is that raters may be simply trying to prepare students for the ‘real-world’... ” (Johnson and VanBrackle, 46)

22

1st: And they equally indicate that raters are not as critical of ESL errors. There are several possible interpretations as to why AAE errors, ESL errors, and SAE errors are viewed different by raters. One interpretation is that AAE speakers are native speakers of English and thus the errors are viewed simply as carelessness and raters are more “annoyed” by them. 2nd: Where AAE errors will be less tolerated by potential employers, so raters fail them now in the hopes that the students will address these errors. This brings into question the pedagogical wisdom of giving students a “right” to their dialect. The “real-world” of standardized writing tests and job applications will most likely continue to penalize AAE features more harshly. 3rd: Many AAE speakers score well below their SAE speaker counterparts particularly

• n assessments that have free responses and essay components. A bias against

AAE could be a partial explanation.

“

23

“Our results indicate and confirm a long-standing sociolinguistic principle: stigmatized varieties of a language suffer in formal educational settings. This empirical study demonstrates that AAE errors are viewed quite differently from ESL errors…” (Johnson and VanBrackle, 46-47)

• Education and dissemination of information about ESL issues have affected raters in

regard to ESL writing errors. While there has been more discussion about AAE in recent years, it remains more controversial and discriminated against than ESL writing issues in university settings.

• It is possible that a better understanding of AAE on the part of raters would change

the results of this study. This perplexing pedagogical question of how to teach SAE without denigrating non-standard dialects remains an open issue prescriptive attitudes toward errors are a hallmark of rater practices.

Questions and Discussion

• 1. How do we teach SAE without denigrating

non-standard dialects?

• 2. Do you think only SAE should be allowed on

standardized writing exams or do you think writers should be allowed to write “freely” in their dialect because of (language barriers, being timed, etc.)

24

Credits

❖

Delpit, Lisa D. Other People's Children: Cultural Conflict in the Classroom. New York: New Press, 2006. Print.

❖

Elbow, Peter, "Voice in Writing Again: Embracing Contraries" (2007). College English. 7. Retrieved from https://scholarworks.umass.edu/eng_faculty_pubs/

❖

Harris, Muriel, and Silva, Tony “Tutoring ESL Students: Issues and Options.” College Composition and Communication, vol. 44, no. 4, 1993, p. 525., doi:10.2307/358388.

❖

Lanehart, S. L. (1996). The Language of Identity. Journal of English Linguistics, 24(4), 322–331. https://doi.org/10.1177/007542429602400407

❖

Johnson, David, and Lewis VanBrackle. "Linguistic Discrimination in Writing Assessment: How Raters React to African American “errors,” ESL Errors, and Standard English Errors on a tate-Mandated Writing Exam." Assessing Writing 17.1 (2011): 35-54. Print.

❖

Milroy,J. (1992) Linguistic variation and change. Oxford: Blackwell

❖

Lee, Alice (2017) "Why “Correcting” African American Language Speakers Counterproductive," Language Arts Journal of Michigan: Vol. 32: Iss. 2, Article 6. Print.

❖

Matsuda, P. (2012). Teaching composition in the multilingual world: Second language writing in composition

• studies. In Exploring Composition Studies: Sites, Issues, Perspectives (pp. 36-51). Utah State University Press.

25