SIMLab's framework for monitoring and evaluating inclusive technologies in social change projects
3 minute read
SIMLab staff workshop indicators to evaluate
The use of inclusive technologies in development and social change is maturing, and should move beyond pilots and prototypes to longer-term interventions, grounded in existing learning and best practice, with more rigorous evaluations that specifically review the contribution that inclusive technologies make to our work. To date, robust evidence of the contribution that inclusive technologies make has emanated more from research than from project-level M&E.
Pressure is growing to bring this evidence to bear, and to move beyond continually re-committing the same mistakes. We have seen a new effort to consolidate best practice, establish measures of impact for programming that include the contributions of newer kinds of technology components, and focus energy on new boundaries of scale and effectiveness. This can be seen in the Principles for Digital Development, as well as sector-specific efforts like DFID’s Conflict, Crime and Violence Results Initiative, which offers a series of papers and guides on good practice in security and justice issues (see Bibliography for more information on these resources).
Accordingly, more emphasis needs to be given to: how well a particular channel, tool, or platform works in a given scenario; how it contributes to development goals in combination with other channels and tools; how the team selected and deployed it; and whether it is a better choice than not using technology or using a different sort of technology. This doesn’t mean that M&E tools, guidelines, and systems that are used for other scenarios are not relevant to inclusive technologies. Rather, there are additional considerations, and particularly relevant approaches, that we propose might be helpful to consider in constructing an M&E plan.
SIMLab’s Framework for monitoring and evaluating inclusive technologies in social change projects aims to guide SIMLab staff in measuring our work and determining, to the degree possible, the contribution of inclusive technology to the outcomes and impact of our implementation projects. As with all our learning and best practice resources, the Framework is shared publicly for others to read, refer to and use as they find helpful; for external review, and as a contribution to the thinking of the wider sector.
This project is a work in progress that will be publicly available under an open license, and regularly updated and improved with support from the ICT4D, aid and development communities in the hopes that it might serve as a resource for others who are working with inclusive technology. We gratefully recognize that this work was made possible by support from the UK Department of International Development and the Hewlett Foundation.
Review the Google Doc here.
Social Impact Lab (SIMLab) helps people and organizations to use inclusive technologies to build systems and services that are accessible, responsive, and resilient. Until December 2014, SIMLab was the home of the FrontlineSMS project, a suite of software that helps organizations build services with text messages. FrontlineSMS has now spun out as a separate, for-profit social enterprise, and SIMLab continues to focus on solving many of the challenges of implementing projects using inclusive technologies. They support implementation, the sharing of learning and synthesis of best practice, and advocate to decision-makers and donors for policy-level change.
SIMLab defines inclusive technologies as those which embody values critical to truly scalable, locally-owned impact; accessibility, ease of use, interoperability, and sustainability. Mobile is a key example—SMS and voice telephony reach all of the world’s 3.6 billion mobile subscribers—as is radio, a critical technology for broad reach at relatively low cost. We also embrace both ends of the spectrum of inclusive tech—the increasing availability and affordability of cheap web-enabled phones and mobile data make them more accessible for relatively disconnected communities, and more analogue communications technologies, such as public criers, noticeboards and human networks, like religious structures and community leadership, reach into even the most remote and disconnected communities.