Case Study: Greater recognition of the need to document and analyze the process of program implementation has been an important development in evaluation research. The purpose of formative evaluation...

Case Study:

Greater recognition of the need to document and analyze the process of program implementation has been an important development in evaluation research. The purpose of formative evaluation as being to collect information that can be used primarily for ongoing program development and improvement, and the purpose of summative evaluation as being to make an overall judgment about the effectiveness of the program
Formative evaluation is an ongoing process that is integrated into the development and implementation of a research project. It provides assessment information within a feedback loop. This assessment identifies the strengths and weaknesses of the project as it progresses. Data obtained from evaluations may be used to modify and redevelop the measurement instruments, the research design and the intervention program during the course of implementmg a project.
Dehar M, Casswell S, Duignan P. Formative and Process Evaluation of Health Promotion and Disease Prevention Programs. Evaluation Review. 1993;17(2):204-220.


Process Evaluation
Two important uses of process evaluation information are assisting in interpretation of program outcomes, and informing future efforts in similar areas. The insight into program implementation that process evaluation provides can be seen to be equally important, whether a program successfully achieves its objectives or not. Without knowledge of program implementation it is impossible to judge whether a program that fails to show impacts reflects a failure of program design, or a failure to implement the program as originally specified
The process evaluation strategy recommended involved two major aspects (a) monitoring daily tasks (e.g., Are service obligations being met? Are staff members working where/when they should? Are staff adequately trained? Is the program administratively sound?), and (b) assessing program activities (e.g., Are the target groups being served? How well are the activities being run? Feedback from clients? Does the program have a good image?)

Dehar M, Casswell S, Duignan P. Formative and Process Evaluation of Health Promotion and Disease Prevention Programs. Evaluation Review. 1993;17(2):204-220.


Important issues that must be addressed when developing a process-evaluation plan include (a) understanding the health promotion program and how it is supposed to work, (b) defining the purposes for the process evaluation, and (c) considering program characteristics and context and how these may affect implementation.
Saunders R, Evans M, Joshi P. Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide. Health Promotion Practice. 2005;6(2):134-147.


Impact/outcome Evaluation

Impact measurement should also ensure that results can be generalised to the individuals and settings for which the health promotion program is intended. In some cases, impact measurement may involve measuring all individuals or organisations participating in the program.
Victoria’s Primary Care Partnerships (PCPs) [Internet]. Health.vic.gov.au. 2017 [cited 7 June 2017]. Available from: http://www.health.vic.gov.au/pcps/evaluation/index.htm


This type of evaluation is used to measure short- and medium-term effects (impacts) and longer-term effects (outcomes) of the program. It is also used to check whether programs are having an impact on populations facing the greatest inequalities.
Prevention and Population Health Branch 2010, Evaluation framework for health promotion and disease prevention programs, Melbourne, Victorian Government Department of Health.


Given the social and political nature of public health, an appraisal of evidence should determine whether the outcome variables cover the interests of all the important stakeholders, not just those who conduct or appraise evaluative research.60 important stakeholders include those with responsibility for implementation decisions as well as those affected by the intervention. Some of the latter may be in disenfranchised groups, and it is not always clear whose interests have been (or should be) considered in evaluative research.