Evaluation of the Limpopo B.Ed. programme: summary of findings
Duration of Project: 3 years (2005 – 2007)
Location of the project: Limpopo Province
Name of Evaluation Service Provider: Benita van Wyk
The project set out to address the lack of adequately qualified Mathematics educators in Limpopo by providing 20 Further Education and Training (FET) Mathematics educators with a three year B.Ed. course.
The course was offered by a Mathematics-focused NGO in partnership with the Nelson Mandela Metropolitan University (NMMU).
The objectives of the project included:
- increasing the number of qualified teachers who teach Mathematics by gaining an appropriate B.Ed. degree;
- assisting these teachers to implement the content learned in the B.Ed.;
- coaching and mentoring teachers in the Mathematics curriculum content knowledge not covered in the B.Ed. course;
- coaching and mentoring teachers to develop their pedagogical skills through classroom support to enable the efficient delivery of quality mathematical curriculum;
- providing teachers with the necessary resources in order to effectively implement acquired skills and knowledge; and
- improving learner performance in Mathematics.
The project evaluation aimed to achieve the following:
- provide conclusive evidence that indicated whether the project achieved the set outcomes and impact;
- assess whether this specific model of educator development was more effective than other methods of teacher development; and
- provide summative information at key points in the programme in order to allow for adjustments of the programme and facilitate decision making by the Foundation.
The evaluation compared a group of project teachers to a control group of teachers from Limpopo who had registered with NNMU but did not have the same NGO service provider intervention.
When exam marks were assessed, the indications were the experimental group did not necessarily outperform the control group, but rather they were able to write and complete their exams more effectively than the control group. Both groups obtained equivalent distinction rates and there was no evidence to suggest that the additional support provided by the NGO contributed to the actual performance of the experiment group except that the project teachers completed all the modules in the course.
Classroom observation measured performance of educators in terms of long term planning, lesson planning, classroom management, lesson presentation, understanding of mathematical concepts and assessment. Statistical analysis indicated that there had been significant change in classroom practice of the educators in the project schools compared to the first observation scores of November 2006. This can be attributed to the Mathematics-focused NGO intervention and the academic training. No control group educators were observed.
A pre-test was conducted for Grade 10 learners in March 2006 for two groups of learners i.e. the experimental and the control groups. The assessment instrument consisted of 25 questions that aimed to assess a number of learning outcomes and assessment standards. The content of the test was designed to be comfortable for Grade 10 learners to be able to complete at the end of Grade 10. The same test was repeated with the same learners in October 2006 so as to assess whether the learners in experimental schools gained more over a one-year period compared to learners in control schools.
The evaluation results suggested that by the end of 2006, one could not yet say that as a group, those learners taught by teachers in the intervention programme would improve their marks more than any other learner. This also meant that the extra support provided through this project had not yet delivered results in terms of learner performance.
The analysis of results of individual schools give a clearer picture in that they indicated that only 40% of the control group schools showed a significant increase between March and October, while 69% of the experimental schools showed a statistical significant increase.
In March of 2007, a second round of testing was done and the overall average test scores for learners increased from 19% to 24%. However, the post results were disappointing as they show better performance from the control group than the experimental group. The analysis of individual school results indicated a decline in scores of some schools in the experimental group.
The results showed that the higher the academic scores of the educator in Mathematics and the higher the classroom observation scores, the higher the learner scores. It would therefore seem that teachers who performed well academically and scored better on classroom observation are more likely to transfer knowledge and skills to the learners.
In summary, the findings were that there was no real evidence that the support given to educators by the Mathematics-focused NGO had positively impacted the academic achievement of educators in the programme as a control group who had no intervention appeared to perform just as well academically.
It can, however, be concluded that NGO support did play a significant role in improving classroom practice of educators. It is, however, disappointing to note that the intervention was not able to impact positively on learner achievement.
It must be noted that the educators had just completed their academic studies and evaluators on other similar projects (Professor Paul Hobden and Dr. Howard Summers) have cautioned on expecting massive transfers to learners at early stages. They indicate that in the beginning stages, educators focused on their academic study and that transfer of learning comes at a later stage.
The Zenex Foundation had a meeting with the evaluators late in 2007 to discuss the project model and its viability as an intervention model for teacher training. In the discussion it was agreed the model did not seem to work as a strategy to improving learner results, but rather contributed to improving educator qualifications.
The tendency is for educators in the programme to focus on their academic programme and thus use the Mathematics-focused NGO as facilitators to support their academic work rather than for improvement of Mathematics teaching in the classroom.
It is therefore recommended that an alternative model for educator training for Mathematics teaching (skills, content and attitude) improvement should be considered.