A rights based approach to evaluation: a tool for social change in interventions with children and young people.
Valentina Terra
Three months ago the United Nations Association of Chile (ACHNU in Spanish), published an Evaluation Model under a Human Rights Approach (MEED[1]). ACHNU is a Chilean Non-Governmental Organization focused on the promoting and defending children´s rights. The aim of this particular publication was to encourage organizations working with children and young people to embed human and child’s rights approaches in their work.
ACHNU has a long history of working with children and young people. Since 1991 it has developed different initiatives and social projects in areas such as child abuse, street children, young offenders and working children among others. I have worked there for the last seven years, specifically in ACHNU’s Research and Evaluation Department. Within this department, my colleague and I (we are both sociologists) have been developing and implementing evaluation processes to support the work of the project teams. We also try to generate knowledge based on their project work, for example through developing research, analyzing cases of study, writing academic papers, and also using an methodological approach that is widely applied in Latin America, called systematization of experience[2].
Working in this context we started to realize the need for an evaluation tool that was convergent with both a human rights approach[3] and the particular experiences of social intervention within each project. So, in 2009 my colleague and I created the rough draft of an evaluation toolkit. This was based on the work developed by a civil engineering student with the support and guidance of ACHNU staff.
The following year we collected information in order to refine the dimensions, variables and indicators of the evaluation model. This included a literature review, application of questionnaires and interviews, and workshops with professional teams. To do this we visited each ACHNU project in order to both meet the staff, the children and young people involved, and to get to know the socio-cultural context within which the project was taking place. For instance, we went several times to the street-children project in Recoleta, a commercial area of the city with an active informal economy. Ten practitioners work at this project with professional backgrounds in anthropology, social work, psychology, social pedagogy, etc. They work with young people that live on the banks of Mapocho river, or children who do not live in the street but spend almost the entire day roaming or engaged in the informal economy or/and illegal activities.
This phase of the research was very significant as it enabled us not only to gain a better understanding of the logic underlying each intervention, but also to understand the main difficulties and challenges faced by the professionals in their everyday practice. Then, during 2011 we refined and completed the draft giving way to the Evaluation model itself, with all its components.
In 2012 the model was validated through experts’ assessment, and then we incorporated all the suggested modifications. Thus, after five years of intense and collective work we finally produced the final version of the MEED. Here are some of its main features, contributions and challenges.
Main features:
The MEED aims to foster and incorporate human rights approaches in work with children and young people in the following ways: i) evaluating the implementation of human rights approaches in social interventions addressed to children and young people; ii) supporting professional teams as the intervention develops.
This model promotes a comprehensive perspective on evaluation as it:
- Incorporates the views of different stakeholders: professional teams, children and young people, families, and representatives of institutions and the community.
- Combines quantitative and qualitative methodology, methods and techniques.
- Understands evaluation as an inseparable part of the intervention and planning process.
- Conceives evaluation both as a collective learning process and a space for reflecting on social action rather than as a mechanism of social control.
- Acknowledges and values the work developed by practitioners.
In addition, the MEED provides a toolkit ready and easy to use due to its flexibility, adaptability and relevancy to work with children and young people. It also includes a methodological guide aimed at facilitating its implementation. This guide explains each of step of the evaluation, and makes recommendations to professional teams and managers.
Main components:
a) Conceptual framework. The MEED is based on the United Nation Convention on the Rights of the Child; the theory of change[4]; the approach “Evaluar para la transformación” (evaluation for transformation) developed by three Argentinian sociologists[5], and a social intervention perspective promoted by the Chilean social worker, Teresa Matus, emphasizing subjectivity, socio-cultural context, positive language, and generation of knowledge.
b) Matrix of evaluation under a human rights approach. The model is grounded in a human rights approach. The matrix of evaluation is divided into three dimensions: Social intervention, Influence in public policy, and Knowledge generation. Each of these dimensions comprises sub-dimensions (as shown in the scheme below), and each sub-dimension comprises variables, indicators and evaluative questions.
c) Instruments are both quantitative and qualitative, including questionnaires and comparison guidelines, evaluation workshops with professional teams, and group games with children and young people. For instance, some workshops with teams consist of a board game, in which each professional is represented as a game piece. There are also several cards containing general questions or real-life situations related to particular dimensions of the project, such as: which strategies have been more effective to promote children and young people participation in the project’s activities? What are the main obstacles of children and young people participation and how have you faced them? Each professional has to choose a card, answer the question and then the rest of the group can give their opinions. In order to make the activity more fun, we added some ‘joker cards’ containing a different task such as telling a joke, creating a poem or, choosing the soundtrack of the project, etc.
d) Methodological phases. Each evaluative moment (base-line, process and outcomes) involves the following eight phases: field-work: the use and implementation of instruments; analysis of information; preliminary evaluative report; the “return” of the preliminary report; final evaluative report; the presentation and sharing of the main evaluation’s outcomes; analysis of recommendations; and decision making.
Contributions and challenges
Social evaluation has become increasingly significant within organizational management. It is a methodological tool that not only facilitates the decision-making process, but also promotes generation of knowledge and learning. These features encourage the development of more intelligent, responsible and effective organizations. In addition, evaluation is a key element in enhancing social intervention and improving public policies.
The MEED can make an important contribution in this direction. It can be used both to support the management and practice of organizations that work with children and young people. Firstly, it can help to consolidate an institutional evaluative culture within these organizations. Secondly, the model would allow for compiling information on the achievement level of the goals proposed, and analyzing information transversally and longitudinally. Thirdly, it would provide recommendations leading to the embodiment of the human rights approach in interventions aimed at children and young people.
In addition, we hope that this model will put important issues on the agenda in order to contribute to enforcing children’s rights and improving public policies in this area. Its utilization will thus help not only to enhance transparency and accountability within civil society, but also to build more equitable societies that are more respectful of children and young people as subjects of rights.
At the moment, and after five years of work, we can say that even though the model is finished, our journey is just beginning; or more exactly we are about embarking on a new stage of this journey: its implementation. We will now see how this model is going to work in the context of the many different ACHNU’s projects.
One of the main challenges is to respect and put into practice the participatory nature of the MEED, mainly regarding children and young people’s participation: to what extent can the MEED consider both the diversity and different realities of children’s lives? How can we effectively and genuinely ensure their involvement? Which strategies can we develop in order to deepen their participation in the whole evaluative process?
Another significant challenge relates to the capacity of implementing this model in a reflexive way, avoiding routinisation of practices. This requires a constant exercise of Meta evaluation to determine the extent to which the MEED is able to assess the incorporation of the human rights approach in our interventions and institutions? Are the variables, indicators and instruments relevant? Are the recommendations arising from the evaluation useful? Have they led to any changes?
Finally, we will have to draw up a work agenda in order to not only disseminate this model, but also to encourage its use in other organizations working with children and young people. Therefore, we will have to address new tasks, inquiries and challenges. Some are known; others cannot be predicted, and will arise along the way. We look forward to continuing our journey and encouraging others to join us on our way.
[1] MEED = Modelo de Evaluación con Enfoque de Derechos – Evaluation Model under a Human Rights Approach
[2]. In this link you can find some useful definitions and resources of this methodology:
[3] Based mostly in the Child Rights Programming (Save the Children, 2006):
[4] In this link you can find different resources: http://www.theoryofchange.org/
[5] Olga Nirenberg, Violeta Ruiz and Josette Brawerman: http://www.libreriapaidos.com/resultados.aspx?c=Nirenberg,+Olga&por=AutorEstricto&aut=29784
circyatsussex.wordpress.com 
It’s an excellent article. Congratulations. Hope every NGO tat work with children and youth could use it to validate it.
LikeLike
Hi Valentina – Excellent paper on Children’s Rights showing a deep understanding and knowledge of this critical issue – at least in third world countries. This notes is a valuable saga of your studies on Chile regarding the subject. Very well done — I wish you a sound success in your studies at Sussex’s
Patricio Guzman
LikeLike