By Jennifer Roland
Essays have long been considered the gold standard for measuring students' understanding of a subject. But because multiple-choice tests have been graded by machines, making them easy and relatively inexpensive to administer, these sub-standard assessments are primarily what schools use for standardized test.
But is it possible to combine the more nuanced understanding that essays give us with the ease of automated grading? The Hewlett Foundation would like to find out. Last month, the organization announced a $100,000 prize that will go to the software designer who can best create an automated essay grader. The goal, they say, is to find an efficient tool that will encourage schools to use essays more regularly for assessments.
"[High cost and slow turnaround] typically mean that many school systems exclude essays in favor of multiple-choice questions, which are less able to assess students’ critical reasoning and writing skills," Hewlett Foundation wrote in its announcement. “Rapid and accurate automated essay scoring will encourage states to include more writing in their state assessments. And the more we can use essays to assess what students have learned, the greater the likelihood they’ll master important academic content, critical thinking, and effective communication,” said Barbara Chow in the release.
But there's valid skepticism around robo-graders. Les Perelman, director of the Writing Across the Curriculum Program at MIT, who's a noted critic of these types of tools, believes automated essay graders are incapable of measuring anything but superficial elements of an essay -- and they do a bad job of that, too.
Automated essay graders prefer a standard five-paragraph essay format. They assess the maturity of vocabulary by counting the number of letters in the words and the number of words in each sentence. They look for the use of quotations, especially in the concluding paragraph, as a marker of an effective argument. And they check grammar and spelling. These elements are not the hallmark of good writing, Perelman says. In fact, most college writing instructors use their first-year writing courses to “deprogram students from writing five-paragraph essays,” he says.