More Progressive Ways to Measure Deeper Levels of Learning

How do we measure learning beyond knowledge of content? Finding that winning combination of criteria can prove to be a complicated and sometimes difficult process. Schools that are pushing boundaries are learning that it takes time, a lot of conversation, and a willingness to let students participate in that evaluation.

“Most schools and most of our learning stops at knowing and we need to move that and broaden it to the doing and the reflecting,” said Bob Lenz, co-founder & chief executive officer of Envision Schools while participating in a Deeper Learning MOOC panel. The charter network’s teachers follow three steps for assessment: know, do, reflect. Skills like critical thinking, problem-solving, and collaboration require practice, Lenz said. Students have to do them constantly and be observed throughout the process for a true assessment.

“The real power comes in the reflective process, both individually and with peers,” Lenz said. “Any of the deeper learning outcomes, the reflection is really where the power is and it puts the onus back on the student, instead of the teacher standing in judgment.” Most projects at Envision schools culminate in an exhibition of work at which students reflect on how they could have done things differently or improved on their work. All four years of high school at Envision are a cycle of performance frameworks feeding into a portfolio and culminating in a defense of four years of learning at which students show what they have learned by demonstrating their knowledge and skill, as well as the ability to learn how to learn.

RUBRICS

Teaching rubrics are a common tool in any classroom, but they can easily become a disguised checklist of tasks, instead of a living document designed to structure learning towards a desired skill or outcome. Setting clear goals about what students should know and be able to do when they graduate high school is a good way to start.

“We find it effective to start with the types of student outcomes that we’re after and have all the staff make sense of that, and commit to the common ‘why,’ and then have the instructional practices to reach those outcomes,” said Megan Pacheco, senior director of school design and implementation at the New Tech Network.

Having a rubric doesn’t mean students aren’t engaged in the assessment process or that there’s no room for surprise or creativity. “If those performance tasks are really open ended then students go about them in very unique ways,” Pacheco said. Asking students to dig into the rubric themselves, unpack it, compare themselves against it and reflect on that experience is a great way to get them to understand their own learning.

“Helping students not to think about assessment as just for a grade or the endpoint of learning, but really as that continuing path of development towards all the skills we know they need,” Pacheco said. Students could even be involved in creating the rubric.

“[Students] often don’t like using rubrics that I bring to them,” said Cady Staff, an eighth-grade teacher High Tech Middle Chula Vista. “So when I do use rubrics it would be a co-designed rubric.” Her students reject outside assessments that haven’t been personalized to their classroom and agreed upon by one another. The process of tinkering with the rubric can be on-going. For example, some students at the neighboring high school became interested in gun-violence and started a Kickstarter campaign to raise money to make a documentary about gun violence in their lives. As they worked through their project, they continually changed what elements should be included in the rubric, as they experimented with what would entice people to give to their cause.

ASSESSING AT THE RIGHT MOMENT

In addition to making it clear to students what learning goals will be covered and required of them through the rubric or performance framework, students need constructive feedback in order to improve. But feedback isn’t always appropriate; there are times when it’s very effective and other times when it can be a waste of precious energy.

“Assessment is really time consuming and exhausting,” Staff said. “So when I do it, and give a lot of meaningful feedback, I want to do it at a time that it will help them to improve.” Students don’t want to know how they could have done better after they’ve already turned in the project. Peer assessment is another way for students to gain valuable input on how they can iterate on an idea or project, helping to push towards another, better version.

Assessment can happen as students work to improve their projects. Teachers don’t have to wait until students turn in a final product to know if they are understanding the content, demonstrating their knowledge, working well together, thinking critically through problems that arise and reflecting on their own work. In fact, that’s the only time many of those qualities can actively be assessed since learning happens over time and can’t be capture in just a snapshot. “Strive to help students assess their own deeper learning,” Staff said. When students can reflect on their own learning process to the point of assessing themselves, teachers know they’re learning deeply.

USING PISA AT SCHOOL LEVEL

One standardized test that has been praised for its ability to measure critical thinking and creativity is the Programme for International Student Assessment (PISA), an international test that ranks countries in reading, math and science. While those scores give leaders a sense of U.S. students’ international competitiveness in the global market, they don’t give school-based data. A group of educators is trying to capitalize on what is considered to be a fairly effective test, bringing it down to the school level.

“It gives a school a sense of how they are doing on these deeper learning, critical thinking, problem solving skills,” said Peter Kannam, a managing partner with America Achieves tasked with coordinating the OECD Test for Schools. Schools have used the test to identify weaknesses and change course if necessary. One school found that while its students were reading more, they weren’t reading deeply. The school built in more emphasis on reading for enjoyment to engage students.

The drawback with the OECD Test For Schools, as with any assessment that tries to authentically measure soft skills, is that it’s expensive. It costs $11,000 to administer and score the OECD test in each school. “At the end of the day it costs money to grade,” Kannam said. That is typically the drawback with authentic assessment at scale. “We don’t need to necessarily do more assessments, we need to do better assessment,” Kannam said. He pointed out that many assessments currently in use for accountability purposes don’t give educators useful, actionable data at the classroom level.

SCALING AUTHENTIC ASSESSMENTS

Some educators are hoping that new assessments meant to test Common Core State Standards will provide more authentic assessment of some deeper learning skills. The Hewlett Foundation has been supporting work to grow the Deeper Learning Movement and believes that assessment will not only drive teacher practices, but is ultimately the only way to know if schools are succeeding in their efforts.

“We don’t know if we are getting any better unless we can document it,” said Marc Chun, program officer at the Hewlett Foundation. The foundation commissioned a National Center for Research, Evaluation, Standards and Student Testing (CRESST) study to evaluate the Smarter Balanced and PARCC tests for various aspects of deeper learning. The study found that the test actually does measure content and critical thinking better than any other test and it does a reasonable job at measuring written communication.

“Ultimately what we want students to be able to do is solve problems they’ve never seen before,” Chun said. That’s why he thinks even nationally administered tests like those being used to evaluate Common Core standards can be effective. If a classroom is doing incredible project-based work or sending students on interesting and dynamic internships the ultimate test of how much students learned is their ability to transfer that knowledge to a new setting.

While the new tests don’t measure effective oral communication, collaboration, learning how to learn or develop academic mindsets -- the other three and a half parts of Hewlett’s definition of deeper learning -- there is still a lot of work to be done to effectively scale up well-rounded assessments of deeper learning. But Chun believes the new tests will be a step in the right direction.