Strengthening Evaluation Capacity through Empowerment Evaluation in a Pandemic
Although the world is facing the global COVID-19 pandemic, that does not mean evaluation needs to stop. Programs are changing in response to COVID-19, and evaluations of those programs are needed to make necessary and timely decisions. Despite the need for ongoing evaluation, there are new restrictions preventing evaluators from traveling to conduct fieldwork as accustomed. This critical time provides an opportunity to strengthen local evaluation capacity and consider empowerment evaluation as an appropriate approach for long-term success with communities.
Collaborative evaluation actively involves key stakeholders in the evaluation process and seeks their input during all parts of the evaluation (O’Sullivan, 2004). Participatory evaluation takes collaboration one step further, inviting stakeholders to work together, often in-person, with evaluators. During a global pandemic, however, I believe we need to shift the ownership of the process to allow stakeholders to lead the evaluation through transformative empowerment evaluation with the evaluator working as a coach or facilitator to be a resource for a community.
Transformative empowerment evaluation is an approach to evaluation in which those involved learn how to take more control of their circumstances and the resources around them. There is an emphasis on finding new ways of doing things instead of following the pre-determined and conventional approaches (Fetterman, 2017). This approach is particularly useful during a global pandemic because top down approaches may not be the most appropriate in an ever-changing situation. Further, program beneficiaries or implementing partners have the best community knowledge needed to support organizational learning and the goals of the evaluation.
In empowerment evaluation, the dynamic between the evaluator and others “at the table” changes. Instead of joining stakeholders to determine how to approach the evaluation as in participatory evaluation, in empowerment evaluation, the stakeholders are leading the evaluation with support and guidance from the evaluator. There may be different voices included in the evaluation through using this framework. The cartoon by Chris Lysy illustrates the differences between the different evaluation methods.
Equitable Empowerment in a Global Pandemic
Each of the ten principles of empowerment evaluation is relevant to evaluation and evaluative thinking in a global pandemic. I highlight four principles that are especially relevant (capacity building, community ownership, inclusion, and social justice) and suggest additional considerations to frame empowerment evaluation in an equitable empowerment lens.
Capacity Building & Community Ownership
Capacity building should be the first step of any evaluation, as the stakeholders are growing their skills to conduct an evaluation and improve their program leading to improved organizational learning. As evaluators adapt how they conduct evaluations in the context of COVID-19, new methods are being considered, increasing the emphasis on the role of the stakeholders. Remote data collection has been a popular solution. Most respondents in a survey from the USAID and IDEAL reported they were using this option followed by phone/SMS surveys (USAID & IDEAL, 2020). However, caution must be given, as the Independent Evaluation Group from the World Bank warns, to exclusionary factors (Chelsky & Kelly, 2020). While this recommendation is specific to mobile surveying and technology, I believe it should be extended further. Do the methods selected increase community ownership and promote empowerment, or are the stakeholders who are collecting and transmitting data only a means to an end?
If stakeholders’ involvement is not a means to an end but is the end itself, as is the goal with empowerment evaluation, then capacity building must accompany and build community ownership throughout the evaluation process. The “use and sustainability are dependent on a sense of ownership” (Fetterman, 2017). Capacity building must align with the goals for the evaluation and the stakeholders and their starting point. A tailored approach could include the evaluator guiding the community through evaluation and evaluative thinking at an individual, institutional, and systemic level. Community ownership is necessary, perhaps now more than ever, as evaluators are less able to conduct site visits, and the success of an evaluation relies on leadership and support from the community.
Inclusion, Equity, and Social Justice
We also need to consider which stakeholders are involved in the evaluation. Inclusion is important as stakeholders from diverse backgrounds will support a rich, informative, useful evaluation. As an evaluator, I work to build capacity from my experiences, education, and training. However, I do not have the same community knowledge as members of the community if I am not a member of that community. Their expertise and voices are essential to conducting an equitable evaluation, especially when the ability to connect in person has been reduced or restricted. If these voices are not welcomed in the evaluation, a disservice has been done.
Further, “evaluation and evaluative work should be in service of equity” (Equitable Evaluation Initiative). Through a social justice lens, we can work to address social inequities we uncover through the evaluative process. Applying the equitable evaluation framework to empowerment evaluation and focusing on the underlying drivers of systemic inequity will guide not only the evaluation but will set the precedent for future evaluative thinking within the community. Equity should be guiding this work by examining the data and methods and considering a theory of equity to accompany a theory of change.
Innovative Approaches
Equitable empowerment evaluation in a global pandemic should be appropriate to the situation and the evaluand. I suggest considering innovative community approaches and including the youth perspective. IPDET hosted the first Evaluation Hackathon in July 2020 to propose solutions to crowd-sourced questions facing the evaluation field. You can read about my experience in the Hackathon in this post. Approximately half of the participants were young and emerging evaluators (IPDET, 2020). The winning team ideated a platform, EvalHub, to facilitate remote data collection using an app where community members can be trained as data collection facilitators and receive payment for their work (Das et al., 2020). This concept not only centralizes the training, work, and payment, but empowers stakeholders to become involved in evaluation long-term and has the potential to shift the power dynamics in fieldwork. Organizational learning and improvement should create space for young voices to be heard through evaluation.
As of 2017, empowerment evaluation was being used in 16 countries worldwide (Fetterman, 2017), but in a global pandemic, there is an opportunity to see the adoption of empowerment evaluation increase internationally. There are aspects of empowerment evaluation in evaluative work that is done in various contexts internationally, but not at the intentional level that empowerment evaluation requires. Intentional consideration should be given to the role of stakeholders in evaluation, especially if beneficiaries may be asked to directly provide data about the program in which they are participating. A decision should be made if the role of these key stakeholders in evaluations is one of collaboration, participation, or empowerment. For long-term evaluation success, I suggest strengthening local evaluation capacity through the stages of empowerment evaluation.
References:
Chelsky, J., & Kelly, L. (2020, April 1). Bowling in the dark: Monitoring and evaluation during COVID-19 (Coronavirus). Retrieved July 26, 2020, from https://ieg.worldbankgroup.org/blog/mande-covid19
Das, A., Reginald, C., Bednarova, E., Otikal, K., Aabye, M., & Morozan, O. (2020, July 13). EvalHub. Retrieved July 26, 2020, from https://evalhack.org/project/10
Equitable Evaluation Initiative. (n.d.). Equitable Evaluation Framework™. Retrieved July 19, 2020, from https://www.equitableeval.org/ee-framework
Fetterman, D. (2017). Empowerment Evaluation. BetterEvaluation. Retrieved from https://www.betterevaluation.org/en/plan/approach/empowerment_evaluation
IPDET. (2020, July 9). Workshop Day 3. Retrieved from https://ipdet.org/wp-content/uploads/2020/07/EvalHack-Powerpoint-Day-3.pdf
O'Sullivan, R. G. (2004). Collaborative evaluation as an evaluation approach. In O'Sullivan, R. G. Practicing evaluation (pp. 24-40). Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412985468
USAID & IDEAL. (2020, April 1). Discussion on Challenges and Strategies for M&E in the Time of COVID-19. Retrieved from https://www.fsnnetwork.org/sites/default/files/Discussion%20on%20Challenges%20and%20Strategies%20for%20M%26E%20in%20the%20Time%20of%20COVID-19.pdf