Research Highlights Article
July 13, 2022
eLearn Classrooms
An experiment in Pakistan shows the power of giving teachers expert-led videos to use as aids in the classroom.
Source: Oguzhan Dursun
An educated populace is critical to today’s advanced economies. But many countries face significant obstacles to providing quality schooling for all students.
For those countries, making the most of limited resources is essential to improving education.
A paper in the American Economic Journal: Economic Policy highlights an inexpensive way to boost student achievement by integrating technology into classrooms. But the authors, Sabrin Beg, Waqas Halim, Adrienne M. Lucas, and Umar Saif, also demonstrate the pitfalls of trying to bypass teachers with technology.
“I think what's really great about this intervention is that it didn't really add extra personnel,” Lucas told the AEA in an interview. “It took the personnel that were already there who were already interested in students' learning and just made their effort more effective.”
The findings come from two randomized controlled trials conducted in Pakistani Middle schools.
I think what's really great about this intervention is that it didn't really add extra personnel. It took the personnel that were already there who were already interested in students' learning and just made their effort more effective.
Adrienne Lucas
Like many low-income countries, Pakistan struggles to consistently staff all of its schools with effective instructors. Teacher absenteeism is a common problem, and many teachers focus on rote learning over a deeper conceptual understanding of topics. As a result, student achievement in government schools is low.
Looking to update classroom techniques on a budget, the government in Punjab, Pakistan, began exploring technological solutions in partnership with the researchers. The team came up with two approaches.
One intervention, called eLearn Classrooms, gave eighth-grade teachers tablet PCs with short videos of expert instructors explaining concepts from the official curriculum, a few multiple choice review questions to use after each video, and simulations to demonstrate complex ideas. Along with the tablets, LED screens were installed in classrooms, which gave teachers the ability to display the material to students during class periods.
The other intervention, called eLearn Tablets, provided the same tablets directly to sixth-grade students and teachers, but classroom screens were not installed. Students could take the tablets home and use them in school, whether or not the teacher was present.
The eLearn Classroom and eLearn Tablet interventions were randomized across 60 and 75 Punjabi schools, respectively, affecting over six thousand students in 2016.
After several months, the researchers tested the students' understanding of math and science and observed their scores on standardized exams. They found that the two interventions had dramatically opposite effects.
Breaking down effects
The chart below shows how the effects of the two interventions on student achievement is the sum of direct effects plus any changes in student or teacher behavior caused by the interventions. Both interventions exposed students to expert content. But the eLearn Classrooms intervention had classroom screens, while the eLearn Tablets intervention provided tablets directly to students.
On a combined measure of the researchers’ and state’s tests, the eLearn Classroom students saw a 60 percent improvement in achievement compared to classrooms that didn’t introduce any changes. This gain in learning would cost just $9 per student at scale.
In contrast, students in the eLearn Tablets treatment saw a 95 percent drop in performance compared to control classrooms, even though this program would cost $131 per student at scale—more than ten times the cost of eLearn Classrooms.
Furthermore, teachers in eLearn Classrooms increased their attendance and self-reported effort, unlike those in the eLearn Tablets intervention.
The authors argue that this increased effort from teachers likely drove the biggest improvements in student achievement. In particular, they highlight that the videos illustrated effective teaching skills, such as how to explain ideas at a grade-appropriate level.
“This is like a sneaky, continuous professional development program,” Lucas said. “I think if we had told teachers to watch a video about how to teach, there would have been very little take-up. But when you say, ‘here are nine minutes where you can watch your students learn,’ it creates its own impetus for use. I think that's a really promising avenue to improve teaching.”
The authors found that the schools likely to have the teachers with the most room for improvement saw some of the largest gains in test scores after the eLearn Classroom intervention. Similarly, students with the lowest baseline scores—those most likely to struggle without help from a teacher—also saw the most improvement.
However, when independent learning through eLearn Tablets was encouraged—effectively circumventing teachers—there was actually a decrease in learning.
The findings illustrate that although there is significant opportunity for technology to enhance education in a cost effective way, some approaches may, in fact, decrease student achievement. The author's research contributes valuable insight into the success of incorporating technology in a way that improves the existing classroom experience.
♦
“Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not” appears in the May 2022 issue of the American Economic Journal: Economic Policy.