To Aggregate or Disaggregate? That is the Question.
I often talk about how important it is to disaggregate data to uncover the meaning within the data. When we disaggregate data, we learn about the hidden nuances that help us understand programs and progress at a deeper level. However, like all things in life and evaluation – intentionality matters. As important as it is to disaggregate data, there are also times when it is necessary to aggregate data to understand the whole situation to inform a data-driven decision.
The Sustainable Development Goals (SDGs) are a great example of this. There are 17 SDGs, and each SDG has its own targets and measurable indicators. These targets are measured for each country, and within each country, they are measured at various levels depending on the context. This provides data for each goal every year, resulting in over 1.3 million data points as of 2020 (United Nations). While a reason for disaggregating data is to better understand the data to lead to more informed decisions, I also suggest that in some cases, we need to reaggregate the data and present overall results.
You may wonder why we should bother to disaggregate the data just to reaggregate it. It is an important question because, without disaggregating data, we can miss the story or message within the data. Before aggregating the data and looking at overall summary numbers, it is a good practice to determine what subgroups of the data should be considered. The SDG data is often considered by the eight regions, and additional disaggregation is determined by the target. If we only looked at the overall numbers without considering the context provided by disaggregating data, we might miss a key finding, but when reporting progress and results, striking a balance is crucial.
The 2020 Sustainable Development Goals Report does a great job of both disaggregating and aggregating data to tell a compelling narrative. Goal 1, End poverty in all its forms everywhere was not on track to be met by 2030, and it is now farther behind.
This overview page provides a lot of great information and is easy to understand the status of the goal as part of the 2020 report, but I was left with questions about some of the information provided. Later in the report, additional information was shared, such as the following graph with a proportion of people living below the $1.90 a day, the international standard for extreme poverty, including a nowcast and forecast before and after COVID-19.
This additional information is helpful to provide context for the statistics represented. However, if this level of detail were represented for each figure presented, it would be overwhelming for the reader to process and comprehend it all. Clear intentionality is important to decide how to present data, which also comes from reflection about what is the single most important message for the stakeholders to remember.
Find the Core of the Message
In Made to Stick by Chip Heath and Dan Heath, they suggested finding the core of the message and determining the single most important thing as a method to make an idea stick (Heath & Heath, 2010). While the book was not specific to data or data visualization, I believe this translates well to how we choose to present data and its story to our stakeholders. If we focus on too many things, it becomes confusing to the audience and they will end up not being sure what the main point is. If there is one clear message, such as “the world was off track to end poverty by 2030,” there is a story the audience can follow. The rest of the information supports that narrative with evidence and additional context, such as the first increase in global poverty in decades due to COVID-19. The message is clear.
Determining the single most important thing requires intention. What is the single most important thing to the stakeholders? Is the most important message found within disaggregated data or aggregated data? This will depend on the context, and that will guide how to present the data.
Because of the implications of the SDGs on countries across the world and the various responses in reporting data, I was looking for more insight on the progress of individual countries towards each goal. While it was not included in this report, there are additional regional reports, such as the Latin America and Caribbean Report, which includes country rankings as well as detailed statuses for each target. Comparing the regional report and main 2020 report provides a direct contrast for disaggregated and aggregated data. Each provide value for their intended purpose.
When deciding how to present data, it is important that we also consider the purpose, main message, and how the story around the data will impact outcomes of our work. Understanding the context of both the program or project and the stakeholders will help you determine the most important message in your data and whether disaggregate data or summary data will best support a data-driven story.
References:
Heath, C., & Heath, D. (2010). Made to stick: Why some ideas survive and others die. New York: Random House.
United Nations. (n.d.). SDG Indicators. Retrieved August 08, 2020, from https://unstats.un.org/sdgs/indicators/database/