Phase Four: Decide What to Measure, and How (Study Design and Indicators)
Key Questions
Which PYD constructs do I want to measure?
Which PYD measurement sources (e.g. survey tools) do I use to measure my constructs of interest?
What research design best addresses the evaluation needs of my project?
After you complete the logic model and clarify the underlying assumptions of how your program features lead to your outcomes, you need to decide exactly what to measure and how you will measure it. There are two steps in this phase: deciding on indicators that pertain to the constructs of interest, and selecting the right sources of those indicators and data collection strategies.
Selecting indicators and measurement sources
Indicators show how and whether a program, project or activity is meeting the stated program objectives. Indicators will inform you if what you are trying to change is improving or not. The indicators you select for your PYD program should follow these guidelines.
How to Use the PYD Framework to Select Indicators
Indicators for PYD Program should:
- Reflect the PYD constructs central to the goals and objectives of your project.
- Use sources appropriate to that indicator.
- Reflect the appropriate level or stage of the program.
- Be the right type of indicator
- Pertain to the targeted beneficiaries of the program.
Monitoring PYD Programs
Traditional program monitoring focuses on systematically documenting recruitment and retention of program beneficiaries to learn from experiences, have internal and external accountability of program inputs and results, and improve current and future program activities. For instance, if the program is designed to change perceptions of self or others, or to change attitudes or norms, these could be assessed with just a few questions several times during implementation. If there are multiple sessions, classes, meetings, activities, etc., just asking a few questions about these PYD outcomes after each session could help pinpoint what the active ingredients of the program are. Because these measurement tools are for monitoring purposes it is not as important that they be established reliable and validated measures. Measures used in outcome evaluations that measure evidence of program efficacy, however, must be reliable and valid and administered before any program exposure (baseline) and after the program is complete (endline).
Qualitative measures such as notes from debriefing meetings with teachers/facilitators might also be useful for monitoring program effects on PYD constructs. What are their impressions of how many youth are experiencing change in the PYD constructs of interest? What is the nature of that change? If a program is long – over a full school year for example – doing a few focus groups or interviews in the middle might be used for this purpose as well.
Do you have more questions about the toolkit? See our Frequently Asked Questions (FAQ) section or visit the PYD glossary.
