Duration of Project: 2005 – 2008
Where is the Project based: North West
Name of Evaluation Service Provider: JET/Centre for Education Policy Development
The Education Support Development Centres (EDSC) project was a large-scale initiative implemented by the North West Department of Education, NGO-partners and funded by the Royal Netherlands Embassy. The primary objective of the project was to establish resource centres in rural areas which would act as decentralised delivery points for a range of educational (and social) services.
The Zenex Foundation became a partner of the project and funded the Foundation Phase Teacher (FPT) component which was designed as a two-year specialised project adjunct to the main EDSC programme.
The target group of educators in the selected primary schools included under qualified and unqualified teachers.
The FPT set out to address the lack of adequately qualified Mathematics and Language educators in the North West province by providing 80 Foundation Phase educators with an accredited training course to cover a two-year period. On completion of the course successful candidates would have obtained 48 credits on Level 5 of the National Qualifications Framework. The credits would be recognised should the successful candidates subsequently register for an Advanced Certificate in Education or a degree in Primary Education.
Three teacher training NGOs delivered the teacher development workshops, residential, as well as onsite support for participating teachers.
The inclusion of far-flung, small rural schools in the project (in direct opposition to the model proposed in the initial proposal) was mooted by the Department of Education officials who felt it important that teachers in schools that had historically not received much support from either the Department of Education or donor-funded programmes should benefit from the project.
Teacher performance in the programme (the 48 credit course)
The vast majority of students (96%) achieved a pass in the assessed modules. Of the 80 educators who started, 75 finally wrote exams and five did not complete for various reasons (including resignation and death).
The Education Development and Support Centres (EDSCs)
The EDSCs were envisaged as resource hubs where teachers could make use of materials stored at the Centre, and even borrow learning and teaching support materials (with the Centre operating as some kind of lending library). When asked if they had ever used EDSC, 56% of participating teachers reported that they had used the resource centre. It was a concern that more than 40% of teachers responded that they never used the resource centre, with some of the feedback being that the centre was always closed when one tried to visit, and that the centre was too far from the teacher’s school.
Project management and co-ordination
The baseline study revealed that the project implementation was somewhat complicated with no clear indications of the roles and responsibilities of the various role-players. The project went through some major challenges during its implementation, stemming partly from issues of commitment from the participants (teachers), the role of the department (of education) and various partners in the partnership. During the second year, the project activities were suspended, pending a resolution of challenges facing the project. It was only once the issues were resolved, that a fresh implementation plan was developed.
The data suggest that some improvements were achieved by the project in the teaching practice component of the intervention (learning and teaching approaches in the classroom).
High levels of grade appropriateness of the lessons that were delivered
The evaluation found improvements regarding grade appropriateness of the lessons for the grade. Overall, the study suggests that while the lessons presented were organised in terms of focus on the main skills and subsidiary skills, there were a few areas in which lessons were at a lower level than the grade in which it was delivered.
Movement away from asking learners questions that require basic recall of facts towards contextual questions.
The baseline evaluation report indicated that to a large extent, the nature of questions that were asked by teachers neither required nor encouraged learners to engage critically. The majority of questions asked for both Literacy and Numeracy, according to the baseline evaluation report, were factual questions, which merely required learners to recall certain facts about various aspects of the lesson.
The learner performance results are those of the Grade 3 Numeracy test and the Grade 3 Literacy test, which were administered to Grade 3 learners in a sample of 27 schools in the follow-up study in September 2008.
Notable improvements were recorded in both the project and control groups in relation to the overall performance on the Numeracy test. The skills where the biggest gains were made by project schools were addition (which experienced a 10.1% gain) and multiplication (which improved by 11.5 percentage points). In control schools, the biggest improvements were recorded in division and multiplication, improving by 7.7 and 8.7 percentage points respectively.
Although similar gains were recorded across the skills by both groups, project schools showed bigger gains than control schools in addition and multiplication while the reverse trend was noticed with counting and division. The gains made by both groups in subtraction were virtually identical, but not statistically relevant.
The overall performance on the Literacy test was 55.4% for project schools and 54.7% for control schools in the follow-up study.
In addition, as with the baseline, although the Literacy levels achieved by Grade 3 learners in both project and control schools were higher than the overall mean percentage achieved by the same learners in the Numeracy test, overall means of the Literacy test were still below expected levels of at least 50%.
Education Development Resource Centres: This was a noble idea, but it did not make a difference to the project as a value-add resource.
School selection: The selection of far flung schools and the partly fulfilled commitment by the Department of Education to provide transport shows that the Zenex Foundation should have been more involved in school selection.
Teachers: The focus on unqualified and under qualified educators at Foundation Phase level assisted in addressing some of the education challenges faced at Foundation Phase on language and numeracy. Evidence also shows that the training and support given to educators by the Service providers has positively impacted both the academic achievement (formal study) of educators as well as in improving classroom practice of educators.
Partners: The role of the project manager was pivotal in planning, coordination, managing stakeholders and ensuring delivery takes place. The project manager, however, experienced a number of challenges (i.e. teacher commitment, role clarity) faced by the project which could have been addressed earlier.
Learner Gains: There were significant gains in learner achievement of project schools from baseline to the summative assessment. However, control schools also improved and while there are slight differences in improvement between project and control schools, they were not statistically significant. It is only the rate of change from baseline to summative in one area of literacy where project schools outperformed control schools.
For this reason, it cannot be claimed that the project impacted positively on learner results. Also, the learners were operating at low levels (under 50%) in numeracy. Of importance is that research has shown that the lag time between teacher training and measuring impact on learner performance require consideration when planning evaluations. Therefore, post intervention tracking of progress would be desirable in interventions of this nature.
Accreditation: The fact that participation did not lead to a full qualification, but some credits towards a qualification, requires the Foundation to explore:
- whether there will be benefits for educators for the credits to be linked to the Occupation Specific Dispensation for the credits to be recognised in the category of short courses
- developing the model of intervention in a way that the objectives include a process for completion of a full qualification within the project cycle.