The PREval (Pilot Research Evaluation) Framework: Evaluating Pilot Projects in Information Communication Technology for Development (ICTD)

Mary Beatrice Dias , M Bernardine Dias, David Dausey, and Elizabeth Casman
tech. report CMU-RI-TR-11-02, Robotics Institute, Carnegie Mellon University, February, 2011


Download
  • Adobe portable document format (pdf) (490KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Information and communication technology for development (ICTD or ICT4D) is a burgeoning field that has attracted increasing interest from researchers, sponsors and policymakers in the last decade. Much of the work being carried out in this area is at the pilot stage, where researchers explore potential technology solutions to challenges in developing communities across the globe. Although ICTD projects are now widespread, there is still little in the way of theory or standards upon which this body of work is based. In terms of project outcomes, Heeks posits that "Most of the ICT4D research being produced is…descriptive not analytical." While there is a need for standards in many aspects of this field, there is a particularly evident lack of structure concerning how such projects are appraised. Therefore, this document was created to offer a systemic approach to evaluating pilot-stage field projects in ICTD. Most currently employed evaluation methods in ICTD are borrowed from economic development and information systems projects and programs. Although these approaches are theoretically sound they do not cater to the unique aspects of ICTD, which combines development endeavors with efforts in technology innovation and adaptation. Furthermore, these methods typically focus on summative evaluations that examine end results, and do not conduct formative evaluations, which scrutinize processes that greatly influence results of a project. Thus, we developed an ICTD-centric, practical method for conducting more comprehensive pilot project evaluations. Our belief is that laying the foundation for evaluation of pilot-stage ICTD projects can benefit this emergent field of research in many ways. First, it would offer an opportunity to improve pilot studies, learn more from them and also make better decisions on how to scale them. Second, it can generate a standardized and more refined approach to reporting results of ICTD endeavors. Finally, it could improve the overall quality of work produced in the field of ICTD and thereby better serve the relevant developing communities.

Keywords
Evaluation Framework, ICTD, Pilot Projects, Technology and Development

Notes
Associated Center(s) / Consortia: Field Robotics Center
Associated Lab(s) / Group(s): TechBridgeWorld
Associated Project(s): Assistive Educational Technology

Text Reference
Mary Beatrice Dias , M Bernardine Dias, David Dausey, and Elizabeth Casman, "The PREval (Pilot Research Evaluation) Framework: Evaluating Pilot Projects in Information Communication Technology for Development (ICTD)," tech. report CMU-RI-TR-11-02, Robotics Institute, Carnegie Mellon University, February, 2011

BibTeX Reference
@techreport{Dias__2011_6794,
   author = "Mary Beatrice {Dias } and M Bernardine Dias and David Dausey and Elizabeth Casman",
   title = "The PREval (Pilot Research Evaluation) Framework: Evaluating Pilot Projects in Information Communication Technology for Development (ICTD)",
   booktitle = "",
   institution = "Robotics Institute",
   month = "February",
   year = "2011",
   number= "CMU-RI-TR-11-02",
   address= "Pittsburgh, PA",
}