Hands-On Science Exams Reveal Students' Skills

By Lillian Mongeau

To get a better understanding of how well students can solve complex problems and apply science to real-life scenarios, the National Assessment for Education Progress recently used hands-on experiments as a way to test 4th, 8th, and 12th grade students, and found that this kind of assessment gives a much more accurate reflection of student comprehension.

Results from a 2009 round of testing called The Nation's Report Card Science in Action: Hands-On and Interactive Computer Task, examined 6,000 students—2,000 at each grade level—from across the country. Students performed tasks like testing water samples (12th grade) and assembling electric circuits (4th grade). They also participated in interactive computer tasks that simulated longer term experiments, like observing plant growth. In both scenarios, students were evaluated on their ability to perform the tasks, observe the results and draw conclusions.

“The bottom line is, we learned so much more that we couldn’t have learned from those paper and pencil tests,” said Jack Buckley, commissioner at the National Center for Education Statistics, which creates the annual “Nation’s Report Card” based on the results of tests like this one administered by the National Assessment for Educational Progress (NAEP).

But what they learned was a mixed bag.

A majority of students at all grade levels (76 percent) were able to perform the simpler experiments correctly and accurately observe the results. However, when experiments involved more complicated data sets, students’ ability to execute and observe fell sharply -- only 36 percent of students tested across grade levels were able to complete the tasks under these conditions.

The test also revealed a disconnect between observation and explanation. Even though a majority of students (71 percent) were able to draw the correct conclusions from the results of their experiments, less than a third (30 percent) were able to explain their results.

For example, one of the hands-on tasks for 12th grade students was to determine the best location for a new town based on water quality. The students were expected to test various water samples for specific pollutants and then compare those levels to a chart put out by the Environmental Protection Agency. A whopping 75 percent of students were able to do this accurately. But when it came time to make a recommendation for where the new town should be built, only 11 percent of students were able to explain their recommendation using the data they’d collected.

The conclusion? "[Students] can conduct science investigations using limited data sets, but many students lack the ability to explain results. The report shows that students were challenged by parts of investigations requiring more variables to manipulate, strategic decision-making in collecting data, and the explanation of why a certain result was the correct conclusion," the report states.

For the most part, student performance broke down as it usually does along ethnic and economic lines. Low-income students performed worse than their wealthier peers and black and Latino students performed worse than their white and Asian counterparts. However, there were a few notable exceptions.

On some parts of some tests, black and Latino students did as well or nearly as well as white students. For example, on a computer task that required 4th grade students to observe plant growth, 80 percent of students came to the correct conclusion. Eighty-one percent of white students got the right answer, 79 percent of black students did and 74 percent of Hispanic students did. (Eighty-six percent of Asian and Pacific Islander students got that one right.)

What's more, though male students generally outperform female students on the national science assessment, female students beat male students on the hands-on tasks.

Alan Friedman, a physicist and the chair of the committee in charge of developing national assessments, said that as a scientist he was relieved that students did well on the first section of the test. “There’s no way for them to memorize for this test. You really had to think on your feet,” he said.

Still, Friedman said, he wasn’t shocked that students struggled to explain their results. “Unfortunately, that’s not surprising,” he said.

Though hands-on standardized tests aren’t brand new, they have historically been too expensive and complicated to use on a wide scale. And the technology needed for interactive computer tasks has not been up to snuff until recent years.

Officials at NAEP said tests like these are more accurate and provide far more detailed results.  Buckley said they must become the norm to keep up with new curriculum standards meant to keep pace with the changing world of science and technology.

“We’re in a really good position to provide models for assessment,” Buckley said, that can “provide information on what students can know and do that’s called for in the new standards.”