Evaluation of Inkanyezi Learners in Public Schools Programme
Name of Project: Inkanyezi – Learners in Public Schools
Duration of Project: 2009-2013 (extended until 2016)
Where is the Project based: KwaZulu-Natal
Name of Evaluation Service Provider: Quality Projects in Education (Paul Hobden) and Schaffer & Associates (Angela Schaffer)
The Zenex Foundation supported a project aimed at increasing the number of Black (African, Coloured and Indian) school-leavers qualifying for university entrance in fields requiring Mathematics and Science. This project involved the provision of significant support to selected learners with potential in Mathematics and Science to enhance their performance in English, Mathematics and Science.
The Inkanyezi Project (2009 to 2013) targeted public schools with a history of providing quality Mathematics and Science education. In 2009, eleven Inkanyezi Project schools were awarded a Zenex Foundation grant for a period of five years to improve the number and quality of its university exemption passes in National Senior Certificate Mathematics and Science.
The grants were intended to cover extra English, Mathematics and Science tuition, and mentoring a select number of economically disadvantaged Black learners entering Grade 8 in 2009. In addition, ten learners in Grade 10 were to be selected for financial and academic support to improve their core Mathematics and Physical Science results in 2009, 2010 and 2011.
The Zenex Foundation also considered requests for funds to cover school improvement and staffing that sought to increase the school's capacity to deliver more effective teaching programmes in English, Mathematics and Science.
These included academic enrichment and career-related support in the form of career assessments and feedback, support for applications at university and bursary applications; the supply of learning and teaching materials; the renovation and refurbishment of science, computer and library facilities; as well as the payment of salaries for additional staff for Mathematics teaching, learner tutorials, counselling support and project administration.
Among the 11 schools, the target of the Foundation was to have 405 quality passes in matric by the end of the project period in 2013. A qualified is defined by the Foundation as a Level C (60% pass) in Mathematics, Science and English with a Bachelor pass. A total of 345 quality passes was achieved by the end of 2013.
The key evaluation questions to be addressed were:
- Has the overall participation and performance rate in Mathematics and Science at the project schools improved?
- Has the performance of the participating learners improved in Mathematics and Science?
- Has the capacity of the Mathematics and Science Departments of the project schools improved?
- What lessons can be learnt in terms of expanding the project to more schools and advancing Mathematics and Science teaching and learning in schools?
Grade 8 learners
The Grade 8 learners selected for the Inkanyezi Project achieved a wide range of scores on the Zenex administered Mathematics entrance tests, from the cut-off minimum of 40% up to scores above 90%. The mean was 63%.
Baseline Mathematics tests were written by all the Inkanyezi learners and their classmates in August 2009. The mean score overall was 67%. The Inkanyezi learners as a group, performed better than their classmates at each of the grade levels of the test, in each learning outcome, and overall. This indicates that the Inkanyezi learners are not at an academic disadvantage in their Mathematics classes as their scores were high at the start of the programme.
The Natural Science baseline test indicated a wide range of basic skills, with a mean of 65% and scores ranging from marks in the 40s to marks above 80%. The Inkanyezi learners scored better on average, than their classmates in all components of the test in all but three schools.
Grade 10 learners
Grade 10 Mathematics and Physical Science baseline tests were written by all the Inkanyezi learners and their classmates in August 2009.
The mean Mathematics score was 67%. The Inkanyezi learners scored significantly better on average than their classmates at five schools, and in the two instances where the mean for the classmates was higher, the difference was not statistically significant. It was thus inferred that the Inkanyezi grade 10 cohort did not enter the project at an academic disadvantage in their Mathematics class.
In the Physical Science test, the Inkanyezi learners were as a group the higher achievers in the class across all areas of the baseline test. However, some of the cohort scored below 50% on the basics skills test and did not have sufficient skills and knowledge required of a targeted intervention.
A key advantage of the placement of Inkanyezi learners in the selected schools was that it was assumed that they would receive quality teaching as a matter of course.
The learners at several schools had separate Zenex Interventions in the form of extra classes. These ran in parallel with the school classwork in a "double dose" model of support. The normal school work was done again by another teacher, sometimes before, sometimes after the work was done in class.
On the whole the Inkanyezi learners were pleased with the extra tuition provided to them. Nevertheless, some tuition programme coordinators had underestimated the degree of forethought and preparation required to meet the specific needs of the learners.
Most of the Inkanyezi learners were grateful for the opportunity they had received, and for the easing of the financial pressure on their families. They did feel under pressure to do well in order to retain the bursary, and had to cope with jealousy from their classmates.
Teacher Development and Networking
The 2009 teacher development programme was not well attended. The most successful aspect of the programme was the professional sharing and networking which took place between Project teachers. However, service providers had not paid sufficient attention to the particular learning and teaching circumstances at each school.
The evaluation notes that there was only a slight improvement in Mathematics and English when comparing the results of the entrance test to the NSC results. It also notes that there was a slight decline in the Science NSC results when compared to the entrance test. This is explained by the way learners were selected and the type of academic support provided. There were a number of different approaches to supporting learners in the different schools which could impact on the learner outcomes.
The recommendations show that the project selected high performing learners who would have done well despite the project. As such, the Foundation worked with the project partners to review the selection processes. In particular, the new selection process ensured that economically and educationally disadvantaged learners from poorly resourced schools with poor background in instructional delivery were selected.
It is important to note the overall achievement of the learners on the project. The Inkanyezi Project produced 345 Bachelor passes, 311 quality passes in Mathematics, 291 in Science and 416 in English by the end of 2013 out of a total of 450 selected learners.
While the evaluation found that the project did not significantly increase Mathematics and Science participation, from the Zenex Foundation’s perspective, the data showed significant increases in participation from 2009 to 2013.
The Inkanyezi summative evaluation corroborates findings from related projects that there was no difference between learners that came into the project from Grade 8 to Grade 12 for five years and those that were selected from within or outside the school for three years. The evaluation findings posits that the only time five years would benefit learners is when they come from poorly resourced school backgrounds and need the extra years to improve their language skills, fill the foundational gaps and adapt to the more effective schooling environments.
In relation to the schools’ ability to manage funding and produce project reports, the evaluators recommended that there was a need for capacity building of schools. Various capacity building workshops were held at project level on financial matters including reporting, tracking expenditure etc.
The evaluation recommended that targets should be negotiated and agreed to before the start of the project to prevent concerns about “changing the goalposts” from participants. The evaluation also recommended that donors should not expect the same learner outcomes from schools with limited resources and staffing capacity as from well-resourced schools.
Service Provider Capacity
The evaluation recommended that there should be service provider project orientation meetings and possible professional development where gaps exist in their capacity to deliver. The Zenex Foundation accordingly funded leadership training and facilitated capacity building training for some of the service providers it partnered with. In addition, the Zenex Programme Manager negotiated with Professor Paul Hobden of QPiE to train the service providers on the differentiated approach in order for them to introduce and train teachers in the Inkanyezi schools that are not part of the differentiated research project.
Mentorship support for learners
The evaluation highlighted the importance of identifying appropriate mentors, as well as the need for assigning clear roles, providing training as well as structured support. The mentorship component of the Inkanyezi project was developed and strengthened in various ways at schools. No specific model was presented to schools and as such mentorship approaches evolved. The project management team requested schools that showed creativity and innovation to support other schools to launch effective programmes. This worked well as schools cooperated well with each other in this regard.