Credit: Dana Nelson
Credit: Dana Nelson

The “marshmallow test” invented by Stanford psychologist Walter Mischel and colleagues in the 1960s is famously known as a measure of willpower. The experiment gave preschoolers the option of either eating one mini-marshmallow right away or waiting 15 minutes to get two mini-marshmallows. Decades later, those who were better at delaying gratification, and resisted immediately snarfing the treat, ended up with stronger SAT scores, higher educational achievement and greater self-esteem and capacity to cope with stress in adulthood.

Now other psychology researchers have come up with a test that challenges the willpower of schoolkids to resist the brain-candy of today’s digital distractions — the YouTube videos, Instagram and mobile gaming apps like Angry Birds. Some people are calling it a “digital marshmallow test,” although it’s tailored for an educational context and doesn’t involve any sweets or near-term rewards.

Officially known as the “academic diligence task,” the new computer-based test offers students a choice between doing math or watching videos or playing a video game. The test was created by postdoctoral research fellow Brian Galla and associate psychology professor Angela Duckworth of the University of Pennsylvania, with Sidney D’Mello of the University of Notre Dame, as a better (and free) research tool for measuring self-control. The researchers hope this new tool will advance their studies of ways to improve academic perseverance in students.

A report recently published online by the team documents the test’s reliability and validity and shows that performance on the task predicts academic achievement — including whether high school seniors graduate on time and enroll in college.

“It’s a really creative and interesting approach to measuring an aspect of self-control,” said Smith College psychologist Philip Peake, who has worked with Mischel on the longitudinal follow-up of participants in the Stanford “delay of gratification” studies. The new diligence task is quite different from the marshmallow experiment, so they can’t be equated, he said, but both are research tools that can contribute to our understanding of the processes that underlie self-control. And both have the advantage of measuring how people actually behave, not just what they say or think they do.

“It’s not just people filling out a questionnaire that says, ‘Oh, I tend to persist in things’ or ‘I work hard at things,’ ” Peake said. It’s assessing real behavior. “And you can see that that behavior has some consequential relations to real-life outcomes.”

Studying Self-Control in the Face of Digital Distractions

The recent work by the Penn and Notre Dame psychologists is part of their ongoing national study of the role that non-cognitive factors such as “grit” and self-control play in students’ persistence in school. That endeavor, which is funded by the Gates Foundation, is following about 1,800 high school seniors over six years to track who enrolls in and finishes college.

Duckworth is known for her work on grit, which she defines as a tendency to pursue long-term challenging goals with passion and perseverance. Self-control or “self-regulation,” on the other hand, is more about the short-term exercising of self-discipline in the face of momentary diversions, an ability that also feeds into perseverance. The research team needed a standardized way to assess self-control, but most of the existing measurement tools were self-report questionnaires, which can give biased results.

So they devised a task that uses behavioral responses to measure academic diligence, which they define as “working assiduously on academic tasks which are beneficial in the long run but tedious in the moment, especially in comparison to more enjoyable, less effortful diversions.”

The rationale behind the test is that with many subject areas or skills, such as mathematics, the basic process of building fluency and mastery involves a lot of practice. It requires “hard work that is perceived as tedious, even though people know it’s immensely important,” D’Mello said. “But that’s just the reality.”

With math, for instance, that means “studying your multiplication tables, solving equations, again and again and again,” which is essential for building more complex numerical knowledge. However, “in the digital age, it’s so hard to focus,” D’Mello said. In psychology-speak, students are faced with having to “regulate” their emotions and impulses to overcome boredom and concentrate on homework instead of something more fun.

To measure this skill in a scenario simulating real life, D’Mello, who is an assistant professor of computer science and psychology, designed the diligence task with a split computer-screen interface (click here for a demo). On the left side, students can choose to do a series of boring skill-building math problems — simple, single-digit subtraction. On the right side, they can play Tetris or watch short, entertaining YouTube video clips of movie trailers or sports highlights. The test is delivered online.

Road-Testing the Test

Galla, Duckworth and their colleagues took the diligence task into two large, ethnically diverse public high schools in Philadelphia, where they enlisted 921 seniors in early 2013. The students were instructed to answer as many math problems as they wanted, as fast as they could, in five consecutive four-minute sessions. They could take a break at any time to watch videos or play the game. The instructions informed them that practicing basic math skills could improve their problem-solving abilities for the future.

In total, the teens spent about half the time on math skill-building, answering an average of 244 problems, D’Mello said. Overall performance on the task consistently correlated with individual differences in conscientiousness, self-control and grit (which were also assessed in the students through questionnaires), just as the psychologists had theorized.

The kids who solved more math problems tended to have higher senior-year GPAs, better scores on standardized math and reading tests, and were more likely to graduate on time; they were also more likely to be enrolled in college at the end of the following fall semester, almost a year later. About 98 percent of the pupils who spent more than 17 minutes doing the math problems successfully graduated, compared with 95 percent of students who spent four minutes or less on math. While that’s a small difference, it was interesting to see it even after the researchers adjusted for other factors, including intelligence, gender, ethnicity, interest in math and whether the kids were in the free lunch program, Galla said.

Demo of the "academic diligence task" created by Sidney D'Mello.
Demo of the “academic diligence task” created by Sidney D’Mello.

The diligence test “was able to pick up a signal in college enrollment, and this was above and beyond things like cognitive ability, socioeconomic status — things that we know tend to correlate with or predict later college success,” he said. So it isn’t just IQ or braininess that matters for academic achievement, but self-control as well. Yet, unlike IQ, the researchers believe self-control in schoolwork is a skill that can be taught and developed.

The correlation between performance on the diligence task and academic achievement is modest, but it is significant and important, commented Peake of Smith College. Whether the task predicts long-term consequences, and whether those are limited just to academics or apply more broadly to other aspects of self-control, are interesting questions to further explore, he said.

Staying on Task

The results held some surprises. “I was really shocked that some people actually stayed entirely on the math problems the whole time,” D’Mello said. “It’s a really difficult task. I can’t do it myself, frankly.” One super-diligent student did 966 math problems; a few kids did none at all.

The teens used a range of tactics to resist the distractions. “Some students would turn it into a game for themselves,” Galla said. “So they want to just see how many problems that they can solve in the four-minute task blocks.” Others did math until they needed a break, then switched to the fun stuff, which is “very healthy behavior,” D’Mello said. “We all know there’s advantages to taking a break. … It’s just that if you do that too much, then you get into trouble.”

The research team will track the students through the six-year national college persistence study, and is updating their diligence task to include verbal and spatial reasoning problems. “The hope is that by giving a good measure, you could really inspire a lot of science,” said D’Mello. At this point, the researchers say they don’t envision the test as something teachers would routinely use to assess students in the classroom; it isn’t designed or validated for that purpose.

Different Takes on Willpower and Grit

Not everyone believes the growing focus on individual students’ responsibility to demonstrate self-control or grit is the best way to support academic achievement. Some progressive education experts worry low-income or minority kids who are struggling in school might be blamed for lacking grit as the primary reason for underperformance. Those critics point out that in many schools, poverty and an inequitable lack of resources are much bigger roadblocks to teaching and learning that need to be addressed.

This viewpoint identifies an economic, social and racial overtone to the notion that if students “would just put their nose to the grindstone harder and work harder, and be more diligent and more resilient, that they will do better,” said Grant Lichtman, education consultant and author of #EdJourney: A Roadmap to the Future of Education, who moderated an impromptu heated discussion on this issue on his blog. “It sort of plays into the mythology of the American dream. It sounds good, but it may be more relevant to some folks than to others,” Lichtman said.

It’s absolutely necessary to structurally address the income inequalities in society, said psychologist David Yeager of the University of Texas in Austin. “Increasing diligence is no replacement for that.” But he believes cultivating skills like diligence and grit in students can still be valuable. Yeager points to high-performing urban charter high schools in the poorest areas that boosted their graduation and college enrollment rates with substantial financial investment, yet still find that 60 to 70 percent of their graduates drop out of college. At several such schools where Yeager worked with Duckworth in studying low-income students in their senior year, kids’ performance on the academic diligence task predicted their likelihood of dropping out of college a year later.

“Diligence still matters when [students] make transitions to the next setting,” Yeager said. And the unfortunate and unfair reality is that it matters more for pupils who are disadvantaged than for those with ample resources, he added. When rich students fail at self-control and make poor choices, they can fall back on family support or finances to keep them pushing through school. In contrast, “disadvantaged kids simply have fewer opportunities to make up for poor decisions,” Yeager said. But a “sense-of-purpose” mindset intervention and other strategies that boost self-control and academic perseverance might help to narrow the inequality gap in education, he said. And, he points out, teachers and schools could start these interventions tomorrow.

A Debate Over Drudgery

Meanwhile, the research on self-control and the diligence task also raise broader questions about drudgery and the definition of success in education. “If you had done this study with the metaphorical Bill Gates in his senior year in high school,” Lichtman said of the diligence task, “he would have gone to the other side of the screen” — skipping the math problems — “or he would have tried to start hacking into the computer.”

Screenshot of Marshmallow Test video. (FloodSanDiego/YouTube)
Screenshot of Marshmallow Test video. (FloodSanDiego/YouTube)

Lichtman is one of many progressive educators who think schools need to teach content in ways that are more engaging and relevant to students’ lives, rather than just drilling them with monotonous math practice sets. More and more teachers, parents and students believe that academic success should be measured not by repetitive regurgitation of facts, high test scores or even a college degree, he said, but rather by whether kids learn skills like collaboration, creativity, communication, empathy, and, yes, persistence and resilience, too. Instead of trying to make the assembly-line education system work better by turning up kids’ self-control, Lichtman says it’s time to focus on alternatives such as the deep, project-based learning experiences offered at Expeditionary Learning schools, among others.

“We all support a forward-looking view of education and are excited to see the future of learning,” said D’Mello in response. The research on the diligence task is an attempt “to study how learning occurs today for better or for worse,” he said. Not everything taught in schools can be fun and easy, he said — that’s why he and his colleagues are focusing on “boring but important” skill-building tasks — and a lot of research has demonstrated the merits of impasse-driven learning, desirable difficulties and productive failure in promoting deep learning. “I’m in favor of doing what it takes to make learning more engaging and intrinsically motivating when appropriate, and fortifying kids with the appropriate mindsets, emotion regulation strategies and cognitive strategies when things get difficult and tedious,” D’Mello said.

Regardless of one’s educational philosophy, there’s no question that diligence is universally necessary for anyone to accomplish something important in life that they really want to do. Any job, career or project, no matter how inspiring, will at times require the self-discipline to resist distractions and plod through some drudgery — whether it’s proofreading a book chapter for the umpteenth time or hand-pipetting hundreds of samples of reagents for a molecular biology experiment.

While diligence may be an independent factor that contributes to academic success, “it’s really important to know it’s just one contributor,” Peake said. “And it’s not going to determine by itself whether or not kids do well in college — there are so many other factors that are playing into this.” The relationships between self-control and positive outcomes are correlations, “not determinative kinds of things.”

That was certainly true for the pre-schoolers who couldn’t wait to gobble the marshmallow in the Stanford experiments, Peake said: “There are many, many kids who didn’t wait, who by all the standards that you put out there do perfectly well in life.”

Measuring Students’ Self-Control: A ‘Marshmallow Test’ for the Digital Age 15 September,2014Ingfei Chen

  • Colette

    Thank you for the thoughtful discussion over a very interesting focus on grit and
    perseverance.

    If the limits of such assessments to glean authentic measures
    were kept always in mind, and the findings were not considered a static pronouncement such a test could be
    extremely useful. As a teacher of elementary students, or now grad students preparing to be teachers, I could use such a test to teach, to
    initiate a dialogue with students over what their penchant to play games or to solve problems means to them, what they think it could
    indicate about a particular learning experience, and if and whether it could illuminate anything about their other approaches to learning. Unfortunately, from Binet on, tests are used on rather than with students and teachers and we lose out on meaningful opportunity to learn from their findings.

    Thank you Inga!

  • LadyPack
  • Grant Lichtman

    I wanted to make clear my core question regarding the work described in this excellent article. I wonder if the students were not given a choice that is largely, if not completely, framed by an industrial age model of learning that is receiving increased scrutiny as an
    ineffective preparation for the lives our students will face? Here is what I mean: in the study the students are given a choice of doing something they find boring and largely irrelevant even though their teachers tell them it is important; or playing a fun video game/watching a movie. This is a false choice. What if, instead, the students took classes where they were deeply immersed in a learning ecosystem constructed around their interests and passions in ways that made the material engaging, and therefore
    “necessary” to the students? Some might say this is a pipe dream…except that many of us see it happening in many schools, public and private, rich and poor. In those cases the students are willing, in fact eager, to wade through what others see as drudgery, to understand what they need to know. They CHOOSE not to be distracted. So while this study may purport to link “grit” with success, I ask the question: what if students did not have to make this false choice when we KNOW there is a better pathway? The real
    link to success is the one that John Dewey uncovered for us more than a century
    ago: we learn more and better when we undergo an experience that leads to personal passion and engagement. I would encourage these talented researchers in their next study to construct a study based on the best we have to offer our students, what might be if we are willing to be great educators, not the least common denominator of a model
    suited for a past that no longer exists.

    • emma

      Yes!!!

    • WaldorfWorks

      Thank you for the additional clarifications, Mr. Lichtman – I appreciate your points very much, particularly “What if, instead, the students took classes where they were deeply immersed in a learning ecosystem constructed around their interests and passions in ways that made the material engaging, and therefore “necessary” to the students?”
      If you are unfamiliar with Waldorf Education, may I encourage you to look into it as another viable alternative learning model?

      • Grant Lichtman

        Of course I am aware of the wonderful work at Waldorf Schools and others that focus on student engagement; bravo for your work!

  • Tynaqua

    Actually, I was thinking the math tasks were not perceived in the same way for everyone. That is, those who got higher GPA, test scores, etc., might have experienced the task as easier than those who didn’t. Ye olde ancient motivation theory tells us that perceived task difficulty can have a big effect on motivation. Basically though, I’m arguing ecological validity. The alleged academic task was not “the same” for everyone except the researchers.

    • Name

      Hi Tynaqua,
      The math task was actually a series of very simple single-digit subtraction problems. The kids tested were old enough to do these problems quickly and easily. If anything, the problems might have been TOO easy.

    • Guest

      The researchers also controlled for cognitive ability (which includes IQ) when predicting outcomes so they were able to take into account student ability and still found effects.

  • Pingback: Measuring Students' Self-Control: A 'Marshmallow Test' for the Digital Age - Alltheanswers()

  • x

    So the Puritan/Calvinist mindset survives, albeit craftily cloaked in the robes of “science”. What a crockThis is a test of obedience, not self-control. If the poor kids in the research pool believed the shit that these ivory tower losers tell them, they score “better”. I’d be worried about the critical thinking of all the students; no one should ever take instructions from an academic psychologist.

  • Grant Lichtman

    Thanks, LadyPack for link to Alfie Khan article http://www.edweek.org/ew/articles/2014/09/09/03kohn.h34.html That article states more clearly and with citations what I was trying to say…of course he did a better job of it!

  • Pingback: Measuring Students’ Self-Control: A ‘Marshmallow Test’ for the Digital Age | MindShi ft | So. Consider()

  • Pingback: Something to add to our reading « FYS 100: Going Viral()

  • revel911

    Grant, I agree. There is no way this test proves self-control because it the outcome is irrelevant.

    The better way to show self-control is have the students solve problems on a calculator or their phone, and see at what point their phone stops being a tool and starts being a distraction.

    • Grant Lichtman

      Interesting test!

  • If the author had done even cursory research on the Marshmallow Test (like, say Wikipedia,) she would have found there is more to it than simply whether a child has the ability to defer gratification or not.

    First of all, the initial study was about natural human development of this ability, and not really about outcomes. Second of all, a later study showed that the initial Marshmallow test did not factor in the child’s belief that the reward would actually be delivered – so, a child’s trust of adults is a huge factor in the Marshmallow test.

    Any child living in an atmosphere where they don’t trust adults is probably not going to do well later in life – but that has nothing to do with their internal “strength,” or “boredom” and needs to be addressed in a totally different way than by making learning “fun.” (not to say that learning should be boring, just that it isn’t really related to the problem.)

  • tc75480

    Given the math test, I would have solved most of the problems. Given a crossword puzzle, I would have done none.

  • Maureen E. Russell Messier

    The fact that “grit” leads to success is not a ground-breaking conclusion. What I would like to see is how to instill “grit” (read self-discipline) in children in spite of distractions.

  • Kim Hatfield

    Some people just don’t like math. Some of us would read historical documents, diagram sentences, or do a crossword, but wouldn’t spend much time on the math. This doesn’t mean you’re not diligent in other areas or aren’t engaged in learning.

    • PLK

      Kim, this is the first task of many that are currently being developed for other subjects. This is a start!

  • Emily Sojourn

    “… The new diligence task is quite different from the marshmallow experiment, so they can’t be equated…” So why are you equating them in the headline? Oh, I know. Click bait.

  • flaker

    That dastardly ‘science’ and those lame ‘ivory tower’ informed scientists… Much better to guess based on gut instinct.

  • Pingback: The Modernized Mischel Marshmallow Test | Internal Assessment()

  • Pingback: Forbes Top Relocation Spots & Psych Study on Kids and Tech | MommyNoire()

  • Clorinda Mason

    What a great idea to track delayed gratification.

  • Pingback: Measuring Students’ Self-Control: A ‘Marshmallow Test’ for the Digital Age()

  • CarolineSanchez

    Just wondering if there is any way that undergraduate students or their supervisors can access this test for dissertation purposes? Have read all the articles related to the test and made numerous far-fetched Google searches but no trace of a published version is evident! Thanks in advance.

  • Kayla Burrow

    I enjoyed discussing The Marshmallow Test with my students and helping them apply it to their own lives. Here’s how it went for me: http://avidcollegeready.org/college-career-readiness/2012/9/21/what-can-students-learn-from-marshmallows.html

  • Pingback: Loren Soeiro, Ph.D. ABPP | Mind/Shift blog: Measuring Students’ Self-Control()

  • Pingback: Imperial Tobacco Posts Rise in Full-Year Profit - ELiquid()

  • Pingback: Bringing life to this time of the semester()

  • Pingback: Measuring Students’ Self-Control: A ‘Marshmallow Test’ for the Digital Age | MindShift | Learning Curve()

  • Pingback: Why I Hate the Marshmallow Test | Open Doors Literacy()

  • Great your article. kitchen chairs

Author

Ingfei Chen

Ingfei Chen is a freelance writer in Northern California whose work has appeared in Scientific American, the New York Times and Smithsonian.

Sponsored by

Become a KQED sponsor