Story Powered Schools - external evaluation (randomised controlled trial)
The United States Agency for International Development (USAID) commissioned NORC at the University of Chicago to assess the causal impact of Nal’ibali’s Story Powered Schools programme on reading outcomes of primary school children. The evaluation is also examining teachers’ attitudes towards reading, own reading behaviour and classroom practices; children’s attitudes towards reading and reading behaviour; and access to and use of reading materials in schools and homes. The methodology for this evaluation is a randomized-controlled trial (RCT), with communities of schools randomly assigned to treatment and control groups. This data will be complemented with qualitative data from classroom observations in a sub-sample of treatment and control schools, and with in-depth case studies in four treatment schools. It is taking place over three years (2017-2019).
Nal’ibali supplement: external evaluation
In 2018, JET Education Services undertook a formative evaluation of the Nal’ibali bilingual reading-for-enjoyment supplement. The mixed-methods evaluation employed telephonic and face-to-face surveys, interviews, focus groups, and a cost-effectiveness analysis. It sought to understand the effectiveness of the supplement, with a focus on targeting, use, appropriateness, quality, messaging and cost-effectiveness.
FUNda Leader survey
Nal’ibali is a national reading for enjoyment campaign. It is powered by FUNda Leaders, a volunteer network of literacy activists who run reading clubs, read aloud and tell stories to children, and promote a culture of reading in their communities. By mid-2018, Nal’ibali had 14 582 FUNda Leaders across the country. We called a random sample of 337 network members to better understand who they are, what they are doing, and how we can better support them.
Story Powered Schools - internal evaluation
In 2017, the first year of Nal’ibali’s USAID-funded Story Powered Schools project, we sought to grow a culture of reading in 240 rural schools in KwaZulu-Natal and the Eastern Cape. Two members of Nal’ibali’s internal research and innovation team visited 10 schools from the 2017 (year 1) cohort to investigate the degree and nature of project take-up: four schools in September 2017, and six schools in March 2018.
Nal'ibali data analysis reports
Every six months, Nal’ibali produces an internal data analysis report that examines key outcomes and outputs in relation to Nal’ibali’s theory of change. It includes an overview of key growth metrics, including flagship event reach, people trained and reading clubs registered, and situates these in the context of the national population. It examines key quality indicators, such as number of children per reading club, number of adult volunteers per reading club, and trends in adult-to-child ratio. It analyses reading club sustainability and turnover over time, and begins to interrogate whether long-running reading clubs exhibit different characteristics. And it provides an overview of reading material targeting, reading clubs’ access to reading material, and reading material distribution in relation to language groups in the population at large.
Nal’ibali print reading material pilots: external evaluation
In 2017 and 2018, Nal’ibali piloted new print material formats and distribution channels, in partnership with Pick n Pay, Free4All and NAPTOSA. Ikapadata conduct a mixed-methods evaluation of these pilots in 2018. The review employed face-to-face surveys and focus groups to understand target audience access, perceptions, use, preferences and feedback.
Nal’ibali review: behavioural science for design
In 2018, Nal’ibali commissioned GMT+ to review its operations and programmes through a behavioural science lens, and identify opportunities to strengthen Nal’ibali’s work and impact by applying behavioural science insights.
Nal’ibali review: use of isiXhosa in the Nal’ibali supplement
In the external evaluation of the Nal’ibali supplement, feedback on language was almost entirely positive. However, 12% of isiXhosa-speaking participants felt that the isiXhosa used was not easy to understand, and 14% noted it is not similar to the language they use every day. Nal’ibali commissioned a review of isiXhosa translation and use in the Nal’ibali supplement.
The review, conducted by translator, lecturer and doctoral student Xolisa Guzula, found the isiXhosa used to be “appropriate, rich and well written”, and the translation “of high quality and standard”. The review also highlights a number of factors to consider in evaluating the quality of language use and translation, which provide a useful framework for similar reviews.
Story Powered Homes pilot: formative evaluation
Review of Nal’ibali training programmes
In 2017, PRAESA (Project for the Research of Alternative Education in South Africa) conducted a formative evaluation of Nal’ibali’s training programme and how training translates into reading club practice. The qualitative study employed training and reading club observations, interviews with training participants (both immediately after training, and one year after training), and key informant interviews.