The PREval (Pilot Research Evaluation) Framework: Evaluating Pilot Projects in Information Communication Technology for Development (ICTD)

Abstract

Information and communication technology for development (ICTD or ICT4D) is a burgeoning field that has attracted increasing interest from researchers, sponsors and policymakers in the last decade. Much of the work being carried out in this area is at the pilot stage, where researchers explore potential technology solutions to challenges in developing communities across the globe. Although ICTD projects are now widespread, there is still little in the way of theory or standards upon which this body of work is based. In terms of project outcomes, Heeks posits that "Most of the ICT4D research being produced is…descriptive not analytical." While there is a need for standards in many aspects of this field, there is a particularly evident lack of structure concerning how such projects are appraised. Therefore, this document was created to offer a systemic approach to evaluating pilot-stage field projects in ICTD. Most currently employed evaluation methods in ICTD are borrowed from economic development and information systems projects and programs. Although these approaches are theoretically sound they do not cater to the unique aspects of ICTD, which combines development endeavors with efforts in technology innovation and adaptation. Furthermore, these methods typically focus on summative evaluations that examine end results, and do not conduct formative evaluations, which scrutinize processes that greatly influence results of a project. Thus, we developed an ICTD-centric, practical method for conducting more comprehensive pilot project evaluations. Our belief is that laying the foundation for evaluation of pilot-stage ICTD projects can benefit this emergent field of research in many ways. First, it would offer an opportunity to improve pilot studies, learn more from them and also make better decisions on how to scale them. Second, it can generate a standardized and more refined approach to reporting results of ICTD endeavors. Finally, it could improve the overall quality of work produced in the field of ICTD and thereby better serve the relevant developing communities.