Understanding context and the limitations of data

“The successes of responding to the pandemic as one data community illustrate the necessity and strength of this kind of collaboration.” - Rabah Kamal

Recently we asked Amanda Makulec, Rabah Kamal, Francis X. Campion, members of the Tableau Advisory Board, what they felt were the biggest data lessons they’ve learned since the beginning of the pandemic. The learnings were both practical and provocative; from the necessity for trust, to the power of multi-disciplinary collaboration, to addressing the limitations of data that lead to misinformation and inequality. Understanding context and the limitations of data is part two of a three-part series: Data in the time of COVID-19: What have we learned?

____________________

“We need to dive deeper into data limitations,” according to Rabah Kamal. “There was a global scramble last year more than ever before (in my lifetime, at least) to collect and analyze real-time data, and that process necessarily required transparency, since people immediately saw the need to compare different reporting and projections of COVID-19 cases and deaths.” According to Rabah, there is always interest in finding new sources of data, “but last year highlighted how important it is to consider the limitations of the data we already have. It has never been enough to just identify data limitations. It’s important to also explore the roots of those limitations and scrutinize the reasons certain data are not available or are not robust enough – why are certain data not collected, not reported, or inconsistent across data sources? Who was involved in collecting data, and who was not? How might data collection and reporting itself be perpetuating inequities? And - very importantly in a time of so much widespread misinformation - how do we explore these issues without discounting the useful and credible information we do have?” More from Rabah:

Social media has in some ways opened a door to more data accountability in recent years, since people can and have been publicly asking these questions of those who are collecting or presenting data. I hope this will lead to a more common and standardized documentation of this kind of context wherever data are presented. For example, last year, data on the pandemic’s disproportionate impact on Native American, Black, and Hispanic communities made it clearer than ever that we have not collectively addressed long-standing health inequities people of color face due to structural racism in our health system. The path ahead is to more widely consider why our data discussions largely did not highlight these issues and to center the expert voices from underrepresented communities who have spoken about these issues for a long time.

Interdisciplinary data collaboration is powerful. We can and should pursue more interdisciplinary data collaborations. Public health crises like a pandemic offer a chance for people in different fields to come together to source, process, and scrutinize data and compare notes on how data are best reported and shared. We’ve seen this happen with our COVID-19 Tableau Advisory Board, for example. Even within research organizations, people with expertise in different areas have needed to collaborate more so than before, since the pandemic affects pretty much every aspect of health care and health outcomes. I think it’s great, this immediate springing to collaboration across disciplines and areas of expertise. I’m a big fan of interdisciplinary problem-solving, and of breaking out of our communication silos to compare notes on multidisciplinary efforts to address a shared challenge. The successes of responding to the pandemic as one data community illustrate the necessity and strength of this kind of collaboration.

Context is critical for data visualizations. Data can be used to misinform, and we have recently seen the very real dangers of misinformation. This raises the question: What is the responsibility of an organization presenting information when it comes to how the information can be misconstrued? People can’t always easily see the “Why” of data, no matter how great a chart looks or how cool an interactive is, and people also process information in different ways -- especially when it’s as complex as most health policy info. At KFF, we see our role as not just collecting and analyzing data, but also contextualizing it, and last year exemplified why it’s important to do so. Anyone presenting data should consider: What background or explanation could help people approach the data objectively? Include that context.

Context also includes information on data collection and analysis methods. Last year also highlighted how important data transparency is when people need to apply data analysis results to daily decisions, particularly given all the missing or unreliable data bound to arise in tracking trends that are changing from day to day, if not hour to hour. This is an important part of fighting misinformation, because so many people have access to public datasets nowadays, and if you’re working with some of that data, someone else likely is, too. So it’s important to anticipate how methods may impact findings, to be able to articulate what a chart or table does and does not show, and to make that clear upfront for audiences coming across so many snippets of data that might seem contradictory at first glance. It has always been the case that analytical methods need to be well-documented and accessible, and I’d say last year was a reminder of why.

____________________

This is part one of a three-part series. Find the other blogs in the series here, and visit the Tableau COVID-19 Data Hub for more data insights, resources, and thought leadership to help you navigate the pandemic.