The CEA Forum
Summer/Fall 2008: 37.2
Book Review:
Elliot, Norbert. On a Scale: A Social History of Writing Assessment in America. New York: Peter Lang, 2005.
by Carole Pelttari, The College at Brockport, State University of New York
A purpose for Elliot's book On a Scale is set forth in the acknowledgments section, stating the book "traces the ways that student writing has been viewed in educational settings" (vii). Would that it did so in a more straightforward manner. Extended chapter introductions and conclusions hide summaries of writing assessment within military and political history. The main body of each chapter is often more coherent, but the reader is still repeatedly pulled up short to wonder about details such as a date or an organization or a particular correlation.
Tons of expository material is stuffed into a prologue, seven chapters, and an epilogue. The prologue and epilogue narratives cleverly utilize two accounts of classroom teachers named Katherine, one set in 1913, the other in 2004. Both women, former elementary school teachers, are instructing high school juniors. Both women are new graduate students determined to use the latest products available to prepare their high school students to complete writing samples that might open the doors to higher education.
Chapters 1-5 are arranged chronologically, beginning in 1874, the year Harvard's president removed the oral recitation exam for admittance and replaced it with a written exam. If the reader will persist through paragraphs muddled with distracting details, a plethora of assessment fundamentals will appear. The point of Chapter 1 seems to be that nineteenth-century Americans' fascination with science led to a desire for standards in college admissions, which led to the formation of the College Admissions Examination Board and the College Entrance Exam. The exam itself, examined and critiqued by Edward L. Thorndike, was scientifically studied in an attempt to create a measurement for written composition "as definite as an inch or an ohm or kilogram" (39).
Chapter 2 covers the years from the beginning of World War I to harbingers of World War II. Intelligence testing became the order of the times. Illiterate military recruits led researchers to develop literacy tests. Field-testing conducted by Carl Campbell Brigham led to large-scale testing. A nine-section academic aptitude test was created, becoming the first Scholastic Aptitude Test (SAT). Brigham, who by 1930 had become associate secretary of the College Board, suggested the time had arrived to separate the science of testing from psychology. At the end of the period, the SAT was the major source of income for the College Board, but research had not gained the position Brigham desired; in 1933 he lamented the Board "had developed a system of readers rather than of examiners" (85).
Chapter 3 discusses the testing situation from 1937 through 1947, focusing on the College Board's Comprehensive Examination in English. Brigham had recruited John M. Stalnaker who declared validity the most important criterion of testing. The entire decade is summed up as a search for reliability and validity, especially in regard to essay exams in the field of English. The chapter rounds out as the College Board delegates the technical aspects of testing to the newly formed Educational Testing Service (ETS) at the beginning of 1948.
Chapter 4 delineates additional searches for reliable measurement of essay exams. Edith Huddleston's research into the nature of writing led her to characterize writing ability as "no more than verbal ability" (141) which could be assessed more efficiently through objective tests. The struggles of ETS under the guidance of Earle G. Eley to score essays for the five qualities of "mechanics, style, organization, reasoning, and content" seemed successful until critiqued by Richard Pearson for failing to reach either inter-reader reliability or validity. Fred I. Godschalk and Frances Swineford offered a paradigm shift: consider "English Composition as a skill rather than a body of knowledge" (161) and assess it holistically. Acceptable reader reliability scores allowed Godschalk, Swineford, and William Coffman to claim the essay as an acceptable means of evaluating a student's writing.
Chapter 5 brings writing assessment to the present. The 1975 conference held by the National Council of Teachers of English (NCTE) introduced new questions regarding writing instruction and assessment. Those ideas dealt with writing as process and product as well as considering the writer as an individual. Portfolios and student involvement in evaluation gained a place at the table. Holistic scoring systems were developed and found to be valid and reliable, especially when human readers were correlated against computerized grading systems. Standardized tests were repeatedly questioned during the 1966-2005 period, only to have the period end with the tests thoroughly entrenched as accountability measures. Learning to write was deemed critically important but a neglected component of school curricula.
Chapter 6 analyzes the history presented, focusing on the emphasis placed on reliability and validity. In the end, Elliot declares the link between the power of writing assessment and students' rights to literacy. He states, "Instructive evaluation would extend the traditional boundaries of formative and summative evaluation and provide ways for writers to become participants in the evaluation process" (293).
Chapter 7 provides Elliot's "lagniappe," which he defines as "a little something ... to use, or not, as you please" (311). He offers his reasons for including and excluding material. Unlike the purpose in the acknowledgement section, here he relates his desire "to document those historic occasions in which communities met to investigate writing ability" (315). That task is managed through the use of a twenty-page chart listing particular assessments (the act), the sponsor or author (the agent), the motives (purpose), the means (agency). He allows that writing assessment will undoubtedly continue to develop because "the drive for accountability leads to an efficiently designed assessment that, in turn, leads to a construct of literacy that is reified from the design–a solipsistic nightmare" (352).
Conclusion
On a Scale: A Social History of Writing Assessment in America provides important information that would be more accessible with clearer writing and better organization. For instance, numerous grading scales are mentioned but none are shown. In addition, the chart in chapter 7 could be given more prominent space earlier to assist the reader in following often convoluted explanations of people, places, and events. Finally, every exhaustive work will miss something, but how can the story of writing assessment be told without mention of Donald M. Murray, Mina Shaughnessy (except in two end-notes), or the concept of writing-to-learn? Nevertheless, in this extensive history, test makers may find guidance, professors may find details to share with students, and academicians will discover many areas for continued research.
Return to Table of Contents