Reflections from a summer research student: Researching the use of online deliberation in health policy
“Just keep experimenting”. From May 2021 to December 2021, this became my mantra for navigating my first self-directed research project: Assessing good practice in the online public sphere: a descriptive evaluation of virtual deliberation in the COVID-19 era. Supervised by Dr. Julia Abelson and supported by a Bachelor of Health Sciences summer research scholarship, I explored the question: How might COVID-19 and the shift to a digital space transform approaches to public deliberation in health policy? To address this question, I collected cases of online deliberation, then developed my own rubric to assess them. What made this project especially challenging was my unfamiliarity with the methodology. While I had learned about systematic reviews, randomized controlled trials, and observational studies, I had never conducted a project that involved developing an evaluation rubric - in my case, a rubric for assessing the quality of online public engagement. Up until this summer, I didn’t even know a research project could take this form! Throughout this blog post, I will detail my process, highlight my findings, and share some of my key takeaways.
Starting a research project is the equivalent of ‘staring at a blank slate’ - where to start? Thankfully, Dr. Abelson is an expert in the field of public engagement and health policy. She provided me with articles to gain a foundational understanding of the public engagement field and prompted me to think about what I was interested in: was it methodology? particular populations? certain challenges to public engagement? As I read, I took notes and maintained a list of questions which could guide my research. Given that I had started the project in the midst of the pandemic, I thought the subject of COVID-19 and its impact on public engagement to be particularly timely. When Dr. Abelson mentioned that health policymakers might benefit from research to inform them on how and when to employ online public engagement in the pandemic and even post-pandemic context, a light bulb went off in my head. I narrowed my focus to online deliberation and established my research question: How might COVID-19 and the shift to a digital space transform approaches to public deliberation in health policy?
Next, I needed to design a methodology to help me answer my research question. The recency of the pandemic meant that a large amount of peer-reviewed literature wasn’t available, so I would also need to search grey literature, which includes technical or research reports from government agencies, reports from scientific research groups, conference proceedings, and other non-peer-reviewed sources. Dr. Abelson pointed me to some of the most well-known public engagement organizations and interdisciplinary journals. The more I read, the more I gained a sense of the field’s methodological approaches. It just so happened that around this time, our broader research team was working on a review of cases of public engagement in health policy across Canada, which gave me the idea to conduct a parallel review of cases of online deliberation. The only remaining issue - how to assess the cases to generate guidance for future deliberative engagement in the virtual online space? I identified a number of rubrics for assessing the quality of deliberation, but none were specifically focused on the online environment. After numerous discussions with the team, I came to a conclusion: if there was no existing rubric, I would create my own! I adapted a framework for evaluating deliberation that was originally developed by Goold et al. (2019) and incorporated elements from studies of online deliberation. By scanning well-known databases on public engagement such as Participedia and websites including Bang the Table and Involve, I came up with a set of 12 cases to analyze. From there, I applied the rubric to each of my cases to assess 3 broad categories of deliberation: the structure (how it was organized), the process (how it occurred) and the outcomes (what the impact was).
The analysis was by far the most difficult part of the project. The immense amount of data intimidated me and sorting through and separating out my results from my interpretation of the results was challenging. Other members of the team prompted me to think about what patterns I was observing across case studies. For instance, which cases met the different criteria? Was the criterion something fundamental to deliberation? Was the criterion easy to achieve? These questions helped me categorize the results across the cases, and eventually identify three major trends: 1) practitioners used an extensive variety of digital platforms to facilitate online engagement, 2) many cases placed emphasis on including a diverse range of information sources to supplement deliberation, clearly articulating methods of recruitment with rationales and limitations considered, and promoting a respectful and trusting environment, and 3) few cases placed emphasis on the design of accessible engagement processes (e.g. providing laptops to those without access at home) and other key features unique to online engagement such as confidentiality, security, and content moderation.
This self-directed project was like navigating through a snowstorm. I couldn’t see the end, but I could see one step ahead, so I learned to just follow my instinct and keep moving. I learned how to confront confusion, and make progress despite uncertainty. Each time I hit a mental roadblock, I would ask myself: can this be solved by reflecting and thinking? Can this be solved by exploring different resources on the Internet? Or would I benefit from guidance from my supervisor? In this way, I learned the appropriate balance between asking for help and relying on my own intuition and resourcefulness to solve novel challenges. Research skills cannot simply be “taught”, they must be experienced. Thank you to Dr. Abelson and her research team for teaching me this important lesson and for supporting my self-directed research; I could not have done this without them!
Grace Kuang is a 4th Year Bachelor of Health Sciences Student that has been working as a research assistant under Dr. Julia Abelson since the Fall of 2020. In the summer of 2021, she received a Bachelor of Health Sciences Honours Scholarship to pursue a self-directed research project.
This essay was prepared by members of the Public Engagement in Health Policy team, which is supported by the Future of Canada Project at McMaster University. Please visit www.engagementinhealthpolicy.ca for further research outputs and resources.
