Evaluability assessment for impact evaluation 1 focus this guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact. The monograph will also move beyond the early technology of evaluability assessments to models that better fit presentday correctional programming. Posted on rick on the road our team has recently begun work on an evaluability assessment of an agencys work in a particular policy area, covering many programs in many countries. Evaluability assessment, its expected outputs and the checklist, which provides a menu of specific questions to ask during an assessment and covers three separate dimensions of. Report of a study commissioned by the department for international development.
A definition evaluability assessment ea is a systematic process for describing the structure of a program and for analyzing the plausibility and feasibility of. Outline structure for terms of reference for an evaluability. An evaluability assessment ea based on guidelines and standards of wholey 1979. The evaluability assessment ea is an important first step in planning a useful, costeffective impact assessment. Evaluability assessment ea is a systematic process for describing the structure of a program and for analyzing the plausibility and feasibility of achieving objectives. An evaluability assessment ea is a qualitative investigation employed before a programme is evaluated. This checklist has been extracted from pages 1923 of the following report. At minimum, an evaluable program must have welldefined program goals, target populations, and eligibility criteria, as well as reliable and accessible performance data, and a defensible. Evaluability assessments eas are a way of thinking through whether and how to assess the impacts of policies, programmes or interventions.
An examination of guidance documents produced by eight international agencies. Chapter 3 introducing our evaluability assessment model 33. Evaluability assessment download ebook pdf, epub, tuebl. Introducing the impact evaluability assessment tool this checklist is a tool designed to help organizations assess and discuss a programs readiness to participate in a rigorous impact evaluation, particularly using a quasiexperimental or experimental. Introducing the impact evaluability assessment tool.
Evaluability assessment, also commonly known as exploratory evaluation, has assisted the field of public health to improve programs and to develop a pragmatic, practicebased research agenda. Evaluation of the potential effectiveness of a programmes objectives health or otherwise is important in demonstrating how programmes work. They are a way of weighing up the costs and benefits of an evaluation in advance so that the. What is an evaluability assessment an evaluability assessment can help newer programs prepare for evaluation and determine the extent to which evaluation will be useful and feasible. An evaluability assessment examines the extent to which a project or a programme can be evaluated. This guide from the juvenile justice evaluation center is aimed at providing juvenile justice program managers with a guide to implementing evaluability assessment in order to ensure that programs are ready for evaluation. Criteria for assessing the evaluability of a theory of change. Evaluability assessment tool of the kind provided by the dfid working paper. An evaluability assessment checklist better evaluation. The model and rationale previously described catapulted ea onto the evaluation scene in the late 1970s with great. An evaluability assessment should examine evaluability. Evaluability assessment is crucial in determining if a project is a candidate for meaningful evaluation wholey, hatry, and newcomer 2004.
Analysis cpa, the repowering americas land initiative aims to promote renewable energy re development on current and formerly contaminated lands, landfills, and mine sites cls, when such development is aligned with the communitys vision for the site. Using evaluability assessment to assess local community development health programmes. The programs documents and case files should be ready for. Questions should be developed with these categories in mind, giving relative weight to categories given ea purposes, scope, and boundaries. The ea was conducted by the evaluability assessment collaborative eac on behalf of the scottish government. Evaluability assessment ea is a means of collaboratively planning and designing evaluations, seeking to ensure they generate relevant and robust evidence that supports decisionmaking and. Introducing the impact evaluability assessment tool this checklist is a tool designed to help organizations assess and discuss a programs readiness to participate in a rigorous impact evaluation, particularly using a quasi. Evaluability assessment ea is a process that helps to clarify program theory by exploring program intent and program practice and, if necessary, helps to redesign programs to ensure. Using evaluability assessment to assess local community. Evaluabilityassessment anevaluabilityassessmentisasystematicprocessthathelpstoidentifywhetheraprogramisina. Evaluability assessment is a method for determining. One tool to further strengthen evaluation planning is to take a strategic pause. When asked about barriers to evaluation quality and use, usaid staff members often cite instances of an evaluation taking place at the wrong time, focused on the wrong questions, or.
It analyses the level of achievement of both expected and. Evaluability and why it is important for evaluators and. Any assessment of the evaluability of a programme will attempt to determine how successful any. Plan for the evaluability assessment although an ea uses fewer resources than an outcome evaluation,11,12 it can take between. Evaluability assessment for impact evaluation 1 focus this guidance note focuses on the utility of, and guidance for, evaluability assessment before undertaking an impact evaluation. A tool for program development in corrections find, read and cite.
Ea can save evaluation dollars by preventing a premature outcome evaluation that can result in. Any assessment of the evaluability of a programme will attempt to determine how. Evaluabilityassessment anevaluabilityassessmentisasystematicprocessthathelpstoidentifywhetheraprogramisina conditiontobeevaluated,andwhetheranevaluationisjustified. Evaluability assessments eas go further than merely providing. The idea of evaluability assessment was first set out in 1979, by joseph wholey, an official in the us department of education and welfare later the department of health and human services. Evaluability assessment was originally developed as a lowcost preevaluation activity to prepare better for conventional evaluations of programs, practices, and some policies. A synthesis of the literature with recommendations. Evaluability assessment ea is defined as a set of pro cedures for planning evaluations so that stakeholders interests are 2 t he c anadian j ournal of p rogram e valuation. Evaluability assessment sfus summit simon fraser university.
The authors shown below used federal funds provided by. The term evaluability is described as the extent to which an activity or project can be evaluated in a reliable and credible fashion oecddac. An evaluation is an assessment, conducted as systematically and impartially as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area or institutional performance. Program evaluability assessment checklist an evaluability assessment ea is a preevaluation tool that is used to determine whether it is appropriate and worthwhile for a program to be subject to an outcome evaluation.
It strengthens impact assessment by basing it on a solid foundation of. Program evaluability assessment checklist an evaluability assessment ea is a preevaluation tool that is used to determine whether it is appropriate and worthwhile for a program to be. That is, data collected when gathering feedback on program theory will address questions that fall into one or more of the. Evaluability assessments in peacebuilding programming. An evaluability assessment can help newer programs prepare for evaluation and determine the extent.
The overall purpose of an evaluability assessment is to decide whether an evaluation. Examining the readiness of a program for evaluation. Evaluability assessment, capacitybuilding activities and qualitative assessment show that evaluation capacitybuilding is an iterative process, requiring partnerships between funders. An evaluability assessment examines the extent to which a project or a programme can be evaluated in a reliable and credible fashion. The fact that two logically and analytically distinct evaluation assignments and processes were telescoped and combined for reasons best known to the evaluation office, means that this document contains significant limitations.
This ea was a response to an educator who created a. Using evaluability assessment and evaluation capacity. An evaluability assessment ea based on guidelines and standards of wholey. An evaluability assessment is a systematic process used to determine if a programme has a set arrangements that would make it an evaluation feasible, and if such an evaluation would yield useful information. Evaluability assessment, its expected outputs and the checklist, which provides a menu of specific questions to ask during an assessment and covers three separate dimensions of evaluability. Criteria for assessing the evaluability of a theory of change rick davies, thursday, 05 april 2012. An evaluability assessment of the toyota families in schools program prepared for the national center for family literacy janet quint with anne sweeney manpower demonstration research. Evaluability assessment ea is a systematic process that helps iden.
Evaluability assessment download ebook pdf, epub, tuebl, mobi. Using research and evaluation to support programs that. Sample interview questions for evaluability assessment. The ea process was developed by wholey 1979 upon the realization that. Evaluability assessment to improve public health policies. Many nonprofits and government agencies are looking for ways to support the needs of lowincome families by helping parents take steps toward economic security while improving their children s well.
It strengthens impact assessment by basing it on a solid foundation of accurate and relevant information. Even once an evaluability assessment has been carried out, changes to the programme design or delivery could be incorporated into iterations of the logic model and evaluation options. Support programs that promote parents economic security and childrens wellbeing by christine ross, emily sama miller, and lily roberts. Evaluation options stakeholders agreed that the evaluation should. Apr 21, 2017 an evaluability assessment does have several linear steps, however some steps need to be revisited and such a method can be seen as an iterative process. Repowering americas land evaluation scoping assessment final report april 16, 2015 prepared for. Introducing the impact evaluability assessment tool this checklist is a tool designed to help organizations assess and discuss a programs readiness to participate in a rigorous impact.
Evaluability assessment, capacitybuilding, self administered surveys and semistructured interviews were conducted from 2004 to 2007 and analyzed through transcript assessment and spss to identify trends, relationships and capacity changes over time. The term evaluability is described as the extent to which an activity or project can be evaluated in a reliable and credible fashion oecddac 2010. July 2015 1usaid evaluation management process checklist evaluation management process checklist planning taskmilestone completed date evaluability assessment conducted optional decision made to undertake an evaluation evaluation managercontracting officers representative cor named evaluation parameters defined. The assessment produced 15 individual site evaluability assessment reports located in the appendix and this crosssite final report, which synthesizes findings and themes. Evaluation options stakeholders agreed that the evaluation should include both outcome and process evaluation elements, to identify the effect of the baby box and to explore the processes and mechanisms by which effects were achieved. An evaluability assessment aims to assess to what extent, and how best, an intervention can be evaluated in a reliable and credible fashion. There is significant potential for evaluability assessments to be used at the project design stage or during implementation, as it involves a systematic assessment of the quality and logic of the. The extent to which a project or activity is ready for an evaluation the changes that are needed to increase readiness. Epa office of solid waste and emergency response center for program analysis. Evaluability assessment to improve public health policies, programs, and practices. An evaluability assessment is a systematic process used to determine if a programme has a set arrangements that would make it an evaluation feasible, and if such an evaluation would yield. The fact that two logically and analytically distinct evaluation assignments and processes were. These sample questions are intended for people interviewing stakeholders as part of an evaluability assessment. An evaluability assessment of the toyota families in.
The term evaluability is described as the extent to which an activity or project can be evaluated in a. The authors shown below used federal funds provided by the. This was accomplished by developing a program theory, usually by examination of program documentation. The assessment produced 15 individual site evaluability assessment reports located in the appendix and this crosssite final report, which synthesizes. Conducting an evaluability assessment june 2017 usaid. Even though it closely resembled a formative evaluation in some elements, the evaluability assessment of the pbea was not.
This tool was developed by anne buffardi and bronwen mcdonald for use in the methods lab. Policy scphrp agreed to undertake an evaluability assessment of the programme. Evaluability assessment template cdc pdf external examines the extent to which a project or a program can be evaluated in a reliable and credible fashion to help you decide. Pdf on jan 1, 1996, patricia van voorhis and others published evaluability assessment. Evaluability assessments in peacebuilding programming i. Developed by joseph wholey in 1979, ea is a tool that can help an. Evaluability assessment ea is a means of collaboratively planning and designing evaluations, seeking to ensure they generate relevant and robust evidence that supports decisionmaking. An evaluability assessment of the toyota families in schools. That is, data collected when gathering feedback on program theory will address questions that fall into one or more of the five categories. An evaluability assessment does have several linear steps, however some steps need to be revisited and such a method can be seen as an iterative process. What works scotland wws aims to improve the way local areas in scotland use evidence to make decisions about public service development and reform. It is envisioned that these reports will be used to inform current and future. Intended users the primary audience for this guidance note is evaluators conducting an evaluability assessment for impact evaluation.
Seven steps for conducting an evaluability assessment. Evaluability assessment of the cost of the school day. Evaluability assessment template cdcpdf external examines the extent to which a project or a program can be evaluated in a reliable and credible fashion to help you decide whether an evaluation is worthwhile in terms of its likely benefits, consequences, and costs. An evaluation is an assessment, conducted as systematically and impartially as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area or. Leviton,1 laura kettel khan,2 debra rog,3 nicola dawkins,4 and david cotton4 1the robert wood johnson foundation, princeton, new jersey 085432316. Evaluability assessment, capacitybuilding activities and qualitative assessment show that evaluation capacitybuilding is an iterative process, requiring partnerships between funders, evaluators and grantees over time to finetune strategies in an audiencetargeted manner because one size does not fit all.