Guest blogger this week is Benjamin Zeitlyn, Lecturer in International Education in the Department of Education at Sussex, who reflects on his recent fieldwork in Ethiopia part of the evaluation of the Speed Schools Project.
In the 1990s a new trend swept through development, led by people such as Robert Chambers from the Institute of Development Studies, the radical new idea was that development practitioners might actually ask the people they purported to be helping what they thought, and even more radically, listen to them. Participatory approaches to development rapidly became the orthodoxy and now almost everything in development is badged as ‘participatory’ even while the poorest and most marginalised are still rarely consulted or listened to by development agencies with any sincerity. Their voices, what they say to researchers are not evidence in the same way that data sets produced by elites, for elites and only accessible to elites are.
In the 2000s another innovation swept through development: ‘evidence based policy’. After 50 years of development, someone had the radical idea that perhaps it would be a good idea to find out whether development interventions actually work. The resulting rush for impact evaluation is the topic of much discussion in development circles (see this fascinating debate on Duncan Green’s Blog). There is a forthcoming conference on ‘The Politics of Evidence’ organised by the ‘Big Push Forward’ initiative, again at the Institute for Development Studies.
It seems strange that no one thought of asking poor people what they wanted before the idea of participatory development came along. It is also strange that no one apparently measured what development projects actually achieved before someone invented ‘evidence based policy’ the ‘results agenda’ and ‘value for money’. I think it is possible do both these important things, by reconsidering the politics and hierarchy of evidence and valuing qualitative research with children and other normally ignored groups.
I am currently working on a research project that is an impact evaluation as part of a team from the Centre for International Education. We are evaluating the impact of an innovative project in Ethiopia called the ‘Speed Schools Project’. The project aims to re-integrate children who have dropped out of school in Ethiopia back into the mainstream education system. To do this the children, who have all dropped out of school in grades 1, 2 and 3 are given a year long accelerated learning programme, which covers the curriculum of three years in one intense year. At the end of the year they take a placement exam and can re-enter school. In addition, the project seeks to help the parents of the children to earn a bit of money through linked microfinance initiative of ‘self help groups’ so that they can support their children’s education after the project has ended. It also prepares the schools where children will be re-integrated for their arrival through a capacity building element working with teachers from these linked schools.
Kwame Akyeampong, Ricardo Sabates and I along with Ethiopian colleagues are evaluating the project’s impact using a mixed methods approach. We are using a quantitative quasi-experimental approach to measure the learning of the children in the schools compared to equivalent children in local government schools. These attainment tests are linked to a household survey to help understand the influence of other factors such as poverty. We have tracked the children through an academic year and will see how many of them have re-integrated into the government school.
I have been leading qualitative research that has evaluated the project’s work through classroom observations of the project’s classes and interviews with project staff, teachers, parents and children. I want to provide a subjective counterpoint, drawing on the opinions and experiences of the children, to the quantitative, ‘objective’ evaluation experiment that we are carrying out.
Research with children often goes to great lengths to listen to points of view that are ignored, and tries to understand the complex impacts of adult ideas and actions on children. I interviewed 14 children in April 2012, midway through the project’s accelerated learning programme and managed to find 8 of them this January to see how they were getting on in the government schools they now attended. I also found a few children who had not made the successful transition back into school to ask them about their experiences. I want to discuss briefly here the methodological challenges of research with children in this context and the empirical insights I have nonetheless learned from the process.
Due to the work I am occasionally required to do at the University of Sussex, I have spent rather less time in ‘the field’ than I would have liked. My research approach has not followed most of the good advice I have been given, read and indeed written about on the best practices for qualitative research with children. Due to the restrictions on my time and the nature of the project, I descend on schools, villages and the homes of children in a four-wheel drive, dripping with gadgets and sweat and launch headlong into interviews with children. Nearly all the attempts to break down the power relations between the researcher and researched that are so important for researchers of childhoods fail, are too time consuming or are dismissed out of hand by hard bitten children, adults and NGO workers.
I work with a translator who is a kind, intelligent, local student; but the two of us, urban, educated men interviewing a poor rural child who has dropped out of school produce a palpable sense in the children we meet that they should say the right thing to us. The fact that we are evaluating the project that provides them with an education and their parents with some income generating opportunities makes it even more difficult to get a relaxed conversation going with them. Their impression of us seems to be that we are from the donor agency and are checking on the local implementation of the project. So the children and their families are not only afraid of us, but of the local NGO’s response if they tell us the ‘wrong’ things.
We are not alone with the children, but usually surrounded by an audience of curious friends and family. This is sometimes reassuring, as everyone can see and hear that we really mean no harm and are just interested in finding out what the children think of their schools. In other cases I felt that the presence of parents, friends and local NGO workers had an undue influence on proceedings. In some cases they even answered questions for the children. Some of the children were obviously shy or worried about saying the wrong thing, while others were outspoken and confident, surprising us with their insights and willingness to speak truth to power.
Interview, translator, audience
Still, despite the methodological challenges, the experience of speaking to children about an intervention designed to help them has led to some valuable insights, which we would not have learned any other way. The children’s perspectives on the project do not contradict the quantitative data, but they provide some useful nuances and explanations.
When I sat in their classes to observe lessons in the speed schools, I sometimes found the teaching methods a little bit dull and repetitive. Why did the teachers keep repeating things? Without prompting, in discussions of their classes many of the children explicitly mentioned that they liked how the teachers in the speed schools repeated, summarised and revised lessons at the start and end of the day. They appreciated the care teachers in their classes took to repeat the key points from a lesson until everyone in the class understood. In government schools they said, teachers worked to rule, running through a lesson plan, writing the material up on the board and then stopping as soon as it was time for the next class, whether they had finished the material or not. The children unanimously said that they thought the teachers in the project schools were better and nicer than in the government schools. They couldn’t tell that the project’s teachers were not qualified teachers but they really noticed and appreciated the increased care and attention that they got in the project’s schools.
Children reported that bullying and conflict between children in Speed Schools is much less than in government schools, partly due to the small class sizes and available desk / chair space. In government schools four squeezed children routinely use the benches and tables designed for two. In the speed school every child has a chair; pens and books are provided free by the project – which almost every child noted enthusiastically – this helps to stop teasing, bullying and exclusion of children without materials and is important to the identities of children as learners.
Inside a government school classroom
Children reported being excluded by other children if their clothes were shabby or dirty in the government schools. They reported not going to school if they did not have sufficient clothes to wear. They reported that peers became annoyed with them for constantly asking to borrow pens. They reported being insulted and punished by teachers for not having pens in class. The lack of seats, clothes, pens and exercise books affect the children’s self-esteem and relationships with their peers. Like the tensions among children that overcrowded desks and benches caused, sharing pens between children caused tension and the exclusion of poor children from classes.
Clothes were a source of shame and exclusion from peer groups for poor children, especially among girls. For some children a lack of clothes was a reason why they missed school. For others their clothes differentiated poor children who did not have money to buy nice clothes from others.
The selective and privileged nature of Speed Schools, the free resources and feeling of being included in something special are powerful motivating factors for children who are otherwise the ones at the bottom of the list, back of the class, with no shoes, no pen, no book and no hope. In the Speed School they are included, encouraged and given attention that they had not had before. They quickly saw the results, many of them commented in interviews (midway through the academic year) that they were cleverer now, or had learned a lot, or could not read before but could now. Our attainment test results corroborate their sense that, as one boy said: “my mind has been opened”.
None of these insights were possible from the survey. The detailed data we have from the household survey and attainment tests will provide us with some great analysis and will be more widely respected as ‘evidence’ than the voices of the children I spoke to. My point is that these short, badly executed interviews provide a crucial perspective for our research. It is important to ask the people who are affected by development interventions what they think and listen to their views. Even if the methods of my research were not very participatory or ‘child friendly’ I think they were successful in understanding the project and its impact in a more nuanced way. I don’t think you can evaluate an intervention aimed at children without asking them what they think of it. Personally, I think we could have learned much more about the impact of the project with more hanging around and chatting to children and less surveying and data crunching. It depends what sort of knowledge you trust and whether you are willing to listen to what children say.
circyatsussex.wordpress.com 