Despite the potential of freeing teachers to incorporate inventive projects to class time, the SRI report reveals that teachers didn’t drastically alter their teaching style when using these online tools. What the videos and quizzes have done is help with tracking data, assigning practice exercises, and supplementing instruction into existing classroom structures. Most teachers did not allow students to explore independently with the Khan platform, and instead assigned specific work. Researchers found a positive correlation with increased test scores, but little else to show that Khan Academy tools helped transform classrooms.
STUDENT AND TEACHER RESPONSES
Overall, 71 percent of students reported that they liked using Khan Academy and 32 percent said they "liked math better" after using the tools. Eight in 10 teachers reported they thought students were at least moderately engaged when using Khan Academy, although usage varied widely depending on the school or classroom. Some students said Khan Academy encouraged them to learn independently, with 45 percent reporting they were able to learn math on their own without the help of a teacher.
All but one of the pilot schools kept a direct instruction model, indicating to the researchers that teachers want to remain in charge of instruction. They also noticed that most students were not accustomed to independently learning and needed teachers to help them navigate that new freedom. Still, the SRI report found a positive association with the use of Khan Academy materials and test scores. Perhaps even more importantly, the report also notes that using Khan Academy helped students in three non-academic areas: it reduced math anxiety, improved their belief they could do math and improved academic efficacy -- students’ belief in their ability to succeed academically.
DRAWBACKS
Despite the positives, teachers had a lot of suggestions for how Khan Academy could serve their needs better. One big complaint was that Khan Academy had significant gaps in state-mandated curriculum, especially in geometry. Fifth- and sixth-grade teachers thought Khan’s practice problems were too hard for students.
During the two years since the study started, Khan Academy content has been adapted to address many of these problems. The content is now mapped to Common Core State Standards and videos are searchable by grade level and standard. There are tutorials and playlists that teachers can modify, and teachers can now recommend specific videos and problem sets to students, and students can set goals for themselves within the program. Additionally, all the data is accessible to teachers, allowing them to filter everything by class, or individually down to the problem set or time period.
But teachers like the tool overall, with 91 percent saying it supported them in introducing new concepts and 82 percent reporting the tool helped them identify students who were ahead and expose them to more advanced materials. Eight out of 10 teachers said the tools helped them monitor student progress and intervene when needed. Researchers also found a statistically significant positive relationship between increased use of Khan Academy and performance on the California Standards Test (CST).
The only site that had more advantaged students and high test scores used Khan Academy primarily to vary instruction. Teachers at this site were much more likely to allow students to poke around on the site, finding topics of interest to explore, than teachers at schools concerned about bringing students up to grade level. Could Khan Academy be falling into the same trap as other tech innovations that best serve a better educated and affluent population?
DIFFERENT APPROACHES
Teachers in the study used three main approaches to implementing Khan Academy in class, each dictated by the resources and goals of the school and teacher. While Khan Academy videos are often associated with the flipped classroom model in which teachers assign instructional videos to be watched at home, none of the teachers in this study used that model.
And though some educators have used Khan Academy videos as a launching point for project-based learning models, none of the teachers in this study reported using that model.
ROTATIONS
Perhaps the most common way to use Khan Academy in younger grades, according to the report, is through a rotational model. Teachers break the class into groups, often based on ability, and students rotate through different stations: one practicing problems on Khan Academy, one with direct instruction from the teacher and one with more practice on worksheets. This model is often used when teachers share devices between classes and don’t have enough for every student.
At a middle school serving mostly low-income, Latino students, teachers liked using Khan Academy this way because it allowed more advanced students to move at their own pace and struggling students could use the hint features to work their way through difficult problems. Teachers knew immediately when kids were struggling, which helped facilitate small group instruction time. And students said they liked getting immediate feedback on their progress, instead of waiting for teachers to grade worksheets.
REMEDIATION
Another common way schools use Khan Academy is to help students who are behind to catch up. At a high school serving 80 percent students on free or reduced price lunch and 75 percent English Language Learners, Khan became a remedial tool. The school’s main goal was to show students the connection between hard work and success.
Khan Academy did help them do this, although it didn’t change the instructional approach at all. Teachers still provided direct, in-class instruction, supplemented by online practice with Khan Academy problems. Teachers using this model found the data Khan provides helpful for motivating students. They could demonstrate to students that when they did the work, they improved. It also helped teachers pinpoint when students had really mastered a topic. Using Khan Academy at this school was associated with higher test scores, although the uptick can’t be attributed entirely to the use of Khan Academy.
SELF-PACED
The least traditional use of Khan Academy was at a high school using the online tool as a primary way to deliver self-paced instruction. The school’s mission focuses on helping students set goals for themselves, hold themselves accountable to those goals, and assess themselves along the way to make sure they stayed on target. Their approach to Khan Academy tools is similarly independent. Students move at their own pace through playlists compiled by teachers.
At this school, math time is two hours per day with 200 kids in a big room, each with access to his or her own computer. For the first 10 minutes of class, students look at their playlists and set a plan of action for their class time. During class, they work their way through Khan videos and problem sets, with teachers walking around offering individual help or giving smaller tutoring sessions in side rooms. When a student feels he's mastered a concept he takes a school-generated quiz and if he gets four out of five questions correct, he can move on to the next unit.
At the end of every two-hour work period students spent 10 minutes reflecting on their work, including writing exercises, surveys and self-evaluations. If a student falls too far behind, she's asked to make a step-by-step plan for how she would get back on track. This model puts the responsibility to learn on students and doesn’t work well for those who struggle to work independently. At the end of the study’s timeframe, the school was in the process of trying to modifying its approach to provide more face time to those students and to find a management tool that would give instructors even more real-time feedback on how well students were grasping topics.
The fact that the school has a one-to-one laptop program allowed students to use the entire two-hour time period to work at their own pace and finish assigned work. The design of the school also facilitated 200 students studying different topics in the same room, working together and discussing the math they were learning.
