Duration of project: 2014 - 2015
Location of project: Pinetown District, Kwazulu-Natal
Evaluator: University of the Witwatersrand: Led by Prof. Brahm Fleisch, Dr Stephen Taylor and Dr Thabo Mabogoane.
The Gauteng Department of Education (GDE) implemented the Reading Catch-up Programme in primary schools that were part of the Gauteng Primary Literacy and Mathematics Strategy (GPLMS) in 2011. These are schools that performed below 40% at the 2008 Systemic Evaluation for Grade 3 literacy. The Reading Catch-Up Programme’s was a 11-week programme whose key objective was to remediate the inadequate levels of English competence in underperforming Grade 4 classrooms that did English as a First Addition Language. The key features of the programme was the provision of teaching and learning materials, teacher training and coaching. The schools that were part of the Programme suspended normal teaching of the curriculum for the term (the duration of the Programme).
Reports from the internal evaluation of the project and the Annual National Assessments suggested that there was improved outcomes because of the project. However, it was difficult to conclusively link the improved outcome to the Reading Catch-Up Programme as it was implemented together with the GPLMS and a rigorous external evaluation had not taken place.
As such, in 2014, the Zenex Foundation commissioned a research study to evaluate the impact of the Reading Catch-Up Programme on English (FAL) performance of Grade 4 learners in Pinetown District.
The Randomised Control Trial design was employed for this study. The sample consisted of 100 schools with similar characteristics, of which 60 were control schools used as a basis for comparing the intervention’s impact, and 40 schools were randomly assigned to the intervention.
Researchers collected data from 40 intervention schools and 57 control schools. The district manager advised that 3 control schools be excluded from the study. 2,543 learners wrote the pre- and post-test. Learners that only wrote the pre-test and not the post-test and vice-versa were excluded from the analysis. Furthermore, interviews and observations were conducted in the intervention schools to establish how the programme was being implemented.
The study showed an overall improvement from both control and intervention schools, with intervention schools showing a slightly better performance particularly in specific areas of reading (spelling and grammar). Furthermore, learners who achieved 50% and more in their pre-test scores benefited more from the programme than learners who achieved less than 20% on the pre-test.
Finding 1: Marked improved time-on-task in the intervention schools
The research found that teachers appreciated the structure, routines and standardised methodologies and content introduced by the project, despite minor difficulties with the pacing. There was a marked improvement on teacher time-on-task and work rate which was evidenced by the higher proportion of the curriculum covered which was higher than what teachers tend to cover. Participating teachers covered the phonics lessons more than the shared reading and comprehension lessons. The researchers suggest that this is due to the fact that teaching phonics (which is rule bound) is far easier than teaching shared reading and comprehension which requires learners to have stronger vocabularies, understand the structure of language and engage with higher cognitive activity.
Finding 2: Teacher competence a challenge
The teachers’ level of competence to teach English was identified as a challenge. It was found that teachers gave poor instruction and the verbal and written feedback to learners on their written work was unsatisfactory. Classroom resources were not managed and used in a meaningful way to reinforce learning and the use of unplanned code switching was quite common.
Finding 3: Pre-test learner performance was low
The intervention and control schools both had very poor pre-test results despite the fact that tests were pitched at Grade 3 level and included some items from Grade 1 and 2 level. Schools in both categories showed similar performance for English FAL.
Finding 4: Evidence of improved learner performance from Intervention and control schools
There was evidence of slight improvements in post-test English performance for both intervention and control schools. The intervention schools performed slightly better than the control group. There was a 4% increase on the post-intervention scores of the schools that participated in the intervention and only a 2% increase on control schools performance. The changes in control school is explained using the Hawthorne Effect, which occurs when individuals improved an aspect of their practice because they are aware that they are under observation.
Finding 5: Intervention improves Spelling and Grammar
There was no major marked difference in performance on English FAL comprehension and writing between intervention and control schools. However, intervention schools showed significant improvement in spelling and grammar in comparison to the control group. Researchers explained that “spelling and grammar” easily transferred to learners in a short space of time.
Finding 6: Intervention works best for learners with better pre-test scores
Learners who performed better on the baseline test benefited the most from the intervention. Researchers concluded that learners with some basic knowledge and skills benefitted more from the intervention.
In conclusion, the project did not lead to improved learner performance in English Literacy to enable them to handle the demands of the Grade 4 curriculum, with an average performance of just 24% for English at the pre-test level and 26% at the post-test level. However, there were gains in spelling and grammar skills, particularly for learners that had basic literacy knowledge and skills.