top of page
  • adwaalsaleh

Navigating the Complexities of Innovative Research: from RCTs to Qualitative Insights

Updated: Nov 11, 2023

Education is the cornerstone of empowerment and transformation. Yet, in many parts of the world, access to quality education remains a challenge, especially for children in poverty-dense areas. During my internship at Busara, I had the incredible opportunity to work on a project that aimed to bridge this educational gap by assessing the impact of Educational Technology (EdTech) solutions on children's human capital outcomes. The project was a collaboration with the EdTech Hub, and it involved conducting a randomized control trial (RCT) in Kenya.

The Pursuit of Knowledge: Blending Theory and Hands-on Experience

Our journey began with the curiosity to understand how EdTech solutions could influence children's literacy, social-emotional learning, and gender attitudes. We knew that innovative technology had the potential to transform education and provide children with the tools they needed to thrive, irrespective of their socio-economic backgrounds.

The project aimed to generate evidence that would be crucial for policymakers to improve human capital outcomes in marginalized communities. It was about turning our curiosity into actionable insights.

Challenges in the RCT: Balancing Competing Objectives

In the project's embryonic stages, meticulous planning was the order of the day. We opted for an RCT, recognizing its potential to yield the most policy-relevant results. Yet, this decision necessitated a tightrope walk along four competing objectives: representativeness, high statistical power, low risk of spillovers, and fiscal restraint.

. The Complexities of Reality

At this stage, the project seemed poised for smooth implementation, with a well-defined plan that included selecting a mix of six eligible counties in Kenya. However, as many experienced researchers will attest, research and reality often diverge.

An article by Jasmin Baier, "Why Randomized Controlled Trials Look Different in the Field than They Do in Class," eloquently underscores the disparity between the conceptualization and execution of RCTs. This article elucidates the challenges we faced in balancing the competing objectives of this project.

From RCT Pivot Points to Qualitative Inquiry: An Unexpected Turn

The path to executing our RCT was fraught with unexpected challenges and pivotal adaptations. However, I'll spare you the intricate details of these initial RCT pivot points and instead focus on the turning point that became the focus of my internship – the qualitative study.

While the RCT would provide a yes or no answer about the impact of EdTech, it was only through the qualitative study that we could delve into the "why" and "how" of this impact.

Crafting the Qualitative Study: Seeking Nuanced Insights

The purpose of the qualitative study was clear: to unearth the hidden dimensions of the educational impact. We aimed to understand why certain elements of the educational program resonated with children and why neurodivergent children might have different experiences. We realized that educational television could be a game-changer in low-income countries, offering cost-effective, wide-reaching educational solutions. Still, there were many unanswered questions about its impact, especially in the context of neurodivergent children.

Our qualitative study aimed to dive into these questions. Specifically, the objective was to explore the qualitative aspects of how children engaged with an educational television program called "Nuzo and Namia" and its impact on literacy, social-emotional learning, and gender attitudes.

Designing Innovative Research Tools: A Bridge to Understanding

My role in this qualitative endeavor was to design research tools that could capture these elusive insights. Designing the qualitative study was a unique challenge, mainly because it involved working with children. We needed to ensure that each child could express themselves in a way that respected their diverse learning capabilities. Standard interviewing methods weren't sufficient because they relied heavily on verbal communication, which might not capture the nuances of children's experiences.

Our qualitative study employed three distinct research methods:

1. Static Observations: This method involved observing children during the TV show using tools like the Stallings Snapshot Tool and the Viewing Observation Tool. These tools allowed us to capture not just what children were saying but also their emotions and interactions during specific segments of the program. 2. Play-Based Observations: After watching the TV show, children engaged in constructing responses using LEGO blocks. This hands-on approach helped us understand how children, both neurodivergent and neurotypical, grasped and applied the play-based learning methods. 3. Caregiver In-Depth Interviews: We conducted interviews with caregivers to gain insights into how they perceived the program's impact on their children. This helped us understand the broader context of the viewing experience.

From Design to Execution: The Challenging Yet Rewarding Journey

Designing the tools marked just the beginning. The subsequent phase involved rigorous training for enumerators, ensuring they could proficiently use these innovative tools. It demanded extensive preparation and close collaboration with the data collection team.

We launched a pivotal pilot phase, which proved instrumental in refining our approach. During this stage, we encountered unexpected issues, such as challenges with the programming for enumerators to input responses on tablets. This compelled us to pivot, opting for guided note-taking templates on paper. Additionally, one child experienced a convulsive episode during an interview, underscoring the importance of comprehending and accommodating neurodivergent traits in our study.

Into the Field: Uncovering Valuable Insights

Equipped with the lessons derived from the pilot phase, we ventured into the field. This phase encompassed household visits, assuring that enumerators adhered meticulously to the research process, and fostering coordination with caregivers and children to gather invaluable insights. The path was not without challenges, but each obstacle was a stepping stone in our learning journey.

Key Findings That Matter

Our research uncovered a wealth of insights. We identified the pedagogical elements that made the TV program engaging for children, such as its introduction of skimming skills and the show's ability to prompt conversations among children. Moreover, we found that children exhibited diverse emotional cues, reflecting their varied emotional responses while watching the program.

The play-based observations using LEGO blocks revealed how children, both neurodivergent and neurotypical, had different preferences and prior experiences with the blocks. This hands-on approach facilitated creativity, spatial reasoning, and an understanding of spatial relationships.

We also uncovered the mechanisms that influenced children's viewership, including the presence of active television subscriptions, caregiver support, and a comfortable viewing environment.

Towards a Brighter Future

The impact of our research goes far beyond the confines of a project. It has the potential to influence education policies and make educational technology more accessible to children worldwide. By understanding how EdTech, like educational television, affects children, we can create more inclusive, effective learning experiences for all.

My journey at Busara has been a transformative one. It's not just about collecting data; it's about making a difference. This project taught me that the path to understanding is often filled with unexpected twists and turns, but it's these challenges that lead to meaningful discoveries.

As I move forward in my academic and professional journey, I carry with me the lessons learned at Busara—lessons of adaptability, creativity, and the unwavering pursuit of knowledge. I'm excited to continue working on projects that have the potential to change lives and empower future generations through education. Together, we can create a brighter, more equitable future for all children.



bottom of page