MLC Event: Monitoring, Evaluation and Adaptive Learning for Scaling Innovations
MLC Event | September 14th 2022
Innovators are continuously seeking for new ways to design and scale up impactful social innovations. Monitoring and evaluation strategies are essential components of this process. However, traditional approaches to monitoring and evaluation can limit the flexibility and adaptability to iterate along the scaling journey of an innovation. This is where the concept of adaptive learning comes in. Results for Development defines adaptive learning as a flexible approach to program implementation and research in which data collection, reflection, and iteration happen throughout a program’s lifecycle, with the goal of improving the program’s effectiveness. But how is this different from traditional monitoring and evaluation strategies? What do adaptive learning approaches look like in practice? And how does adaptive learning support the scaling up of innovations?
To address these questions the Million Lives Collective co-hosted a learning and networking event with MLC Member Farm Radio International on the 14th of September 2022. A panel of speakers from Results for Development (R4D), MLC members Farm Radio International and Gram Vaani shared their knowledge and experience on adaptive learning, exploring its application to measure and monitor scaling progress. The session, chaired by Megan Grace Kennedy-Chouane, Head of the Evaluation Unit of the Development Co-operation Directorate at the OECD, also provided attendees a chance to exchange learnings on the topic.
Katie Bowman, Associate Director of R4D's Monitoring, Evaluation, and Adaptive Learning team, introduced the concept of adaptive learning and emphasised two important distinctions with traditional monitoring and evaluation (M&E). Firstly, adaptive learning allows innovators to continuously recalibrate approaches along scaling journeys and entails working closely with the implementers of the solution, while traditional M&E typically requires innovators to maintain a fixed plan and most of the time includes an endline external evaluation. Secondly, adaptive learning emphasises aligning methods with the innovation to be tested and its development stage. For example, levels of uncertainty experienced in the launch of a mobile app may lead an innovator to employ interviews and market analysis to determine the optimal market penetration approach. However, as the invention develops with greater certainty and rigour, adaptive learning methods may shift toward evaluating different types of app messaging channels
“Adaptive learning is about working closely with the doers, allowing them to drive the key research questions to answer the biggest unknowns”
Katie concluded by reminding the audience that, while traditional M&E is still crucial for innovators and funders, adaptive learning is becoming increasingly important in specific circumstances, namely:
To test new innovations brought into complex systems
To test multiple program options to innovate from idea to scale
To assess an intervention’s likelihood to accelerate impact , rule out non starters and change course if needed
Bernard Pelletier illustrated some of Katie’s points by showcasing how Farm Radio International combines traditional M&E strategies with adaptive learning approaches. Bernard shared some of the traditional indicators Farm Radio uses, such as calculating the potential audience and listenership rate through secondary sources as well as evaluating changes in knowledge, attitudes and practices through surveys and qualitative methods. He also emphasised how Farm Radio is increasingly using interactive radio tools as a method for adaptive learning to inform their analysis and real-time impact. Their Uliza Application, for example, has an interactive voice response system that enables listeners to communicate and exchange information with their radio station quickly, easily and free of charge. In turn, this contributes to Farm Radio’s real time data gathering.
The interactiveness of the Uliza Application brings four unique benefits to the organisation:
Interactivity: It’s a two way communication tool, enabling listeners to ask questions, provide feedback and amplify their voices;
Dynamism: It allows broadcasters and other stakeholders to adjust the content of radio programs on an ongoing basis;
Data: It creates real time data and knowledge to assess how attitudes and practices of listeners change over time.
“We are increasingly working towards developing adaptive learning approaches because we think that they provide a rich way to address issues we are trying to tackle working in very complex and diverse environments”
Vijay Sai Pratap continued the conversation by presenting an alternative approach to adaptive learning. Gram Vaani’s innovation, Mobile Vaani provides curated and user-generated audio content to inform listeners from low-literacy populations of local news and events, provide a support system to ensure the delivery of public services, and connect rural communities to partner products and services. The most distinctive feature of Gram Vaani’s model is their focus on co-creating content with the communities. In doing so, Gram Vaani identifies volunteers who engage with communities and ensures mobilisation through voicing and responding to questions on the same platform. This generates what Gram Vaani calls “ground truth” which is then used to inform critical evidence required for program evaluation, impact and policy advocacy.
Like Farm Radio, Vijay highlighted the importance of continuously recalibrating Gram Vaani’s scaling process. Mobile Vaani allows the collection of real time data by gathering feedback from beneficiaries and identifying access and awareness among individuals and their communities. Data collection allows Mobile Vaani to better understand biases, barriers to change, and institutional weaknesses. Responding to one of Katie’s points, Vijay explained how community engagement allows Gram Vaani to improve their program design through iterations of their delivery model, enhance scale and sustainability, and test different financial incentive structures for Mobile Vaani volunteers. This constant monitoring allowed for greater adaptability during the COVID-19 pandemic as it allowed for the analysis of real time data to better support their users. They also published a report analysing how the COVID-19 pandemic disproportionately affected poor and vulnerable groups in India during its first 100 days.
“It is imperative and our responsibility to ensure that monitoring is not and end of a program process but part of the whole design and implementation”
After a very insightful first half of the discussion and an initial round of questions and answers, participants were divided into six breakout rooms which allowed them to pose additional questions to the speakers and network amongst themselves by sharing their experiences, challenges and knowledge.
We look forward to learning how the new connections from the networking session develop! If you want to get in touch with any of the speakers, please do so by filling in this form. You can also watch the session recording here on our YouTube channel. Thank you to all of our speakers and attendees - Stay tuned for our next MLC event!
Event Speakers:
