Education has become a hot industry, both for the technology community and for investors hoping to back the next big success. It’s also in the spotlight nationally as new Common Core State Standards role out in most U.S states. Many of the new products and teaching methodologies claim that research backs up the product or claim, but how can interested educators separate self-promotional claims from effective tools?
Many products tap into education trends and buzz words while using “experts” to bolster claims. “We’re asked to believe a claim because someone who appears to be in a position of authority makes that claim,” said Daniel Willingham, a psychologist and author of When Can You Trust the Experts? How to Tell Good Science from Bad in Education in a recent Future of Education conversation with Steve Hargadon. By authority figures, he means people with advanced degrees, or who have written books, or appeared on television. “These things are designed to make us think, other people think this person is smart,” and by extension the consumer should trust the claim, Willingham said.
People are most susceptible to these kinds of claims when there is little other information available to make an informed decision. Some professions have an organization dedicated to vetting claims like these in order to protect consumers. A good example is the U.S. medical board, which licenses doctors who demonstrate acceptable knowledge and skill levels. “In the case of education, there’s no vetting body at all,” Willingham said. “It’s up to me to judge the criteria. And the criteria that constitute an expert are very much up for grabs.”
Awareness that companies are explicitly playing on the consumer psyche is a good first step to using a critical eye on any new claim. It can help a consumer identify what’s called the “sunk cost fallacy,” the idea that because a lot of money has already been spent on an ineffective course of action the only choice is to continue throwing good money after bad. In this case, once the consumer is alerted to the ineffective method, they're likely to switch tactics because it helps them. They won’t have to keep paying for something ineffective.
But there are other forms of psychological manipulation that aren’t easy to spot or are convenient for the consumer, like “confirmation bias,” a common tendency to twist new evidence to corroborate a viewpoint already held. Willingham has developed a few shortcuts to help the layperson approach research claims critically.
- STRIP IT: When evaluating if a new intervention makes sense, strip it down to the essential parts, making sure to understand clearly what the change is, what outcomes it is supposed to create and the probability that if taken up those outcomes can be achieved. Looking at these concrete elements will help strip away the emotional part of the pitch.
- TRACE IT: Find out where the product originated and who's supporting it. Just because a famous speaker supports it does not mean it’s a reliable claim. Being clear about how supporters are connected to the venture helps determine whether they're trustworthy.
- USE EXPERIENCE TO EVALUATE: “You can’t pretend that you’ve never had any experience or that it’s completely unreliable,” Willingham said. Many educators will know if a claim sounds plausible, or if it could work at their site.
- ACT LIKE A SCIENTIST: Once the decision has been made to try a new software product or curriculum, write down what is expected to happen, what indicators will show that it has happened, the timeframe in which it should happen, and some thoughts on what might happen next if the intervention does or doesn’t work. Physically documenting these things at the start of the project can prevent confirmation bias from springing up when it comes time to evaluate.
“This is a piece that I don’t see happening in many educational situations,” Willingham said. He’d like to see more of the scientific process applied to educational interventions to ascertain whether they actually work. He also warns that educators shouldn’t be naïve about how businesses will operate in the education space. No one should expect altruism, he said. Take, for example, publishing companies. Most will sell expensive and ineffective products if it makes them money, Willingham said, and no one should expect anything else.