A government programme in Ghana that encouraged school principals to act as leaders to improve classroom instruction revolutionised classroom teaching and significantly enhanced student learning. The intervention boosted student performance by 30% in one academic year with persistent achievement effects two years later.
Public service productivity and service provision in developing countries are often deficient. In particular, the government schooling sector's lack of productivity is acute as millions of students are in school but not learning, i.e. the learning crisis (World Bank, 2018). Randomised controlled trials (RCTs) have shown how learning could be improved (e.g. Banerjee et al. 2007, Banerjee et al. 2017, Duflo et al. 2022). Yet, getting existing teachers in existing systems to implement such programmes has been elusive in Ghana and India because of the educator sector's weak managerial and oversight structures, leading to implementation rates of around 5% (Banerjee et al. 2017, Duflo et al. 2022). Can public sector managers be the key to implementing reform at scale?
Unlike the private sector, civil servants cannot be compelled by the private sector management tools of pay, promotion, and termination to increase their effort levels (Bloom et al. 2013, Bloom et al. 2020, Bruhn et al. 2018). Nevertheless, the public expects, and deserves, high quality public service. Even though public sector managers do not have these tools of compulsion, they can act as leaders, eliciting others to follow. Getting existing personnel in existing systems to implement effective reforms is where truly transformative progress happens.
In Beg et al. (2023), we show through a 210 school RCT in partnership with the Ghanaian government that existing public sector managers can act as leaders to improve student learning. School principals signaled the importance and efficacy of a pedagogy reform through their own costly effort—and classroom practices and student learning increased. Additional management training did not further augment classroom practices nor student learning.
Broadly, how to successfully implement programmes at scale is relevant to policymakers, practitioners, and researchers across any publicly provided service. Specifically in education, Differentiated Instruction (DI), our implemented pedagogy, also known as Differentiated Learning (DL), Targeted Instruction (TI), or Teaching at the Right Level (TaRL) in other contexts, was part of education policy in sixteen countries across Africa and South Asia at the end of 2023. Implementing it with fidelity is key to its success.
Implementing effective pedagogy
One of the most rigorously tested and promising solutions to the learning crisis is Differentiated Instruction (DI), a student-centred pedagogy where students are taught at their learning level instead of their grade level for part of the day (Banerjee et al. 2007, Banerjee et al. 2017, Evans and Mendez Acosta 2020, Duflo et al. 2022). When implemented, DI replaces the status quo of grade-level, teacher-centered, lecture and rote memorisation based passive learning with engaging lessons, tailored to students’ learning levels. In our version, this meant dividing students across grades 4 through 6 by learning level instead of grade level by part of each school day. Despite its promise, the method only improves learning when it is implemented—how can existing personnel and system structures encourage implementation of innovative pedagogy that improves student learning?
We partnered with the Ghanaian Ministry of Education and UNICEF to test whether harnessing school principals as leaders in the production of student learning improved classroom practices and student achievement. Formally, we conducted a 210 school RCT during the 2018-2019 school year and continued to track students until 2021.
Our three-armed RCT demonstrates the power of managers acting as leaders to increase teacher effort and student learning. The Teacher Training + Management Effort Treatment (Treatment 1) trained teachers in and provided materials for Differentiated Instruction (DI) and gave managers (i.e. school principals and circuit supervisors, supervisors of groups of about 10 geographically proximate schools) costly effort tasks (primarily conducting a classroom observation and related form). These tasks credibly signaled to teachers that the new DI classroom activities were important, valued, and would increase student success, like a leader uses an effort signal to elicit change from followers and demonstrate organisational values. The Teacher Training + Management Effort + People Management Treatment (Treatment 2) included all the elements of Treatment 1 plus an additional training for school managers on People Management – improving interpersonal relationships through mentoring and support. The final group was the control group that continued with business as usual. All material design and distribution and training was implemented by existing civil servants under the Ghanaian Ministry of Education umbrella.
We use a conceptual framework to show that credible signals that DI increases student learning would induce teachers to exert effort on DI, thus improving student achievement. Both Treatments 1 and 2 included costly and visible management effort—managers attended the initial teacher training and conducted classroom observations, completing an observation form that highlighted the key aspects of DI. The additional People Management training in Treatment 2 would only further increase student learning if it led to additional teacher effort.
Our study has four main findings
First, both interventions worked as designed. Management capacity improved across a wide range of measures. Both interventions equally increased aspects of management related to classroom practices (“instructional management”). Aspects of management related to teacher support and relationships (“people management”) only occurred in the arm that included additional people management training, Treatment 2. Both interventions also increased other aspects of management not directly covered in the intervention (“other management”), with larger increases in the more intensive management training arm.
Second, teachers implemented the new pedagogy. This is not a trivial achievement—previous trainings in Ghana without management engagement led to implementation rates among teachers of about 5% (Duflo et al. 2023). The treatments increased the likelihood that teachers were in their classrooms by 11 percentage points, using active learning methods by 0.4 standard deviations, and had divided their students by learning levels instead of grade levels by 60 percentage points. These improvements were equal across the two treatment groups.
Third, the interventions increased student learning in both math and English by an additional one-third of a year of schooling over the span of a school year (0.11 SD). These learning gains occurred throughout the distribution of student ability; all students learned approximately the same amount with larger percentage gains among the students who started with lower test scores.
Fourth, both the achievement and management improvements persisted. In the year after the intervention, no additional materials were provided. One teacher from each treatment school was invited for a brief refresher training. Management quality, especially the measure of teacher support and relationships, remained above the control group in the year after the intervention and teacher retention, another common measure of management, increased by over 18%. Teachers continued to implement the programme in their classrooms. In the 2021 school year, students who had been in treatment schools had test scores that were higher than students from control schools. Relative to what control students had learned over this period, they were about 20% of a year of learning (0.06 and 0.08SDs) higher. Learning gains two years after the end of an intervention are remarkable, and these occurred despite Covid-related school closures and students graduating from primary to junior high schools.
Key takeaways
Overall, our findings show that existing teachers can and will implement a more effective pedagogy if their managers act as leaders, credibly signaling the success and importance of the programme. The programme is a blueprint for scaling other successful NGO or researcher-initiated programmes in the public sector within existing systems to reach students at scale, where truly transformative change occurs. Our results also show that not all management is equally important in the production of learning, rectifying contradictory findings in the literature. Intensive management training or consulting programs based on “management best practices” will not be effective at increasing student learning unless they change what happens in the classroom. Starting in 2022, the Teacher Training + Management Effort intervention was being scaled to 10,000 schools in Ghana.
References
Banerjee, A V, S Cole, E Duflo, and L Linden (2007), "Remedying education: Evidence from two randomized experiments in India," Quarterly Journal of Economics, 122(3): 1235-1264.
Banerjee, A, R Banerji, J Berry, E Duflo, H Kannan, S Mukerji, and M Walton (2017), "From proof of concept to scalable policies: Challenges and solutions, with an application," Journal of Economic Perspectives, 31(4): 73-102.
Beg, S A, A E Fitzpatrick, and A Lucas (2023), "Managing to Learn," National Bureau of Economic Research, No. w31757.
Bloom, N, B Eifert, A Mahajan, D McKenzie, and J Roberts (2013), "Does management matter? Evidence from India," Quarterly Journal of Economics, 128(1): 1-51.
Bloom, N, A Mahajan, D McKenzie, and J Roberts (2020), "Do management interventions last? Evidence from India," American Economic Journal: Applied Economics, 12(2): 198-219.
Bruhn, M, D Karlan, and A Schoar (2018), "The impact of consulting services on small and medium enterprises: Evidence from a randomized trial in Mexico," Journal of Political Economy, 126(2): 635-687.
Duflo, A, J Kiessel, and A Lucas (2022), "Experimental evidence on alternative policies to increase learning at scale," National Bureau of Economic Research, No. w27298.
Evans, D K, and A M Acosta (2023), "How to recruit teachers for hard-to-staff schools: A systematic review of evidence from low-and middle-income countries," Economics of Education Review, 95: 102430.
World Bank (2017), "World Development Report: Learning to Realize Education's Promise," World Bank.