Short answer
Write each training aim as an outcome you can count or observe, connect it to program activities with a one-page logic model, track a small set of measures every year (enrollment, progress, completion, time to degree, first positions, trainee publications), and follow alumni for at least five years after they leave.
What is program evaluation for a training program?
Program evaluation is the routine, planned collection of evidence about whether a training program is meeting its stated aims, used to improve the program and to report results to funders. It isn't research for publication, although good evaluation data sometimes becomes that.
Evaluation was a recurring theme at the TWD directors' meetings; the 2019 meeting ran several sessions on aligning objectives with measures of success.
How do we turn aims into something measurable?
Write aims as outcomes, not activities. "Offer a career development seminar" is an activity. "At least 80 percent of trainees will complete an individual development plan in their first year and review it annually with their mentor" is an outcome you can check.
A quick test for each aim: could someone outside the program tell, from data you hold, whether you met it?
What does a simple logic model look like?
| Inputs | Activities | Outputs | Short-term outcomes | Long-term outcomes |
|---|---|---|---|---|
| Training grant funds, faculty time, coordinator | Summer research, rigor course, mentor training, IDP meetings | Number of trainees placed, course completions, mentors trained | Research self-efficacy, skills, conference presentations | PhD completion, time to degree, research careers |
One page is enough. The value is in forcing each activity to point at an outcome.
Which measures should every program track?
- Applications, admissions and enrollment, by year
- Progress milestones (qualifying exam, candidacy, committee meetings)
- Completion and time to degree
- Publications and presentations with trainees as authors
- First position after the program and position five years later
- Trainee experience, through a short annual survey with a few stable questions
Keep the survey short and the same from year to year. Trend data from a ten-question survey is worth more than a long survey you change every cycle.
How do we report it?
Report against your aims, in the order they appear in the application. Show trends, not just one year. Say plainly where you fell short and what you changed; reviewers trust programs that show they act on their own data.
Questions people ask
Do we need an external evaluator?
Not always. Many training programs handle evaluation with a part-time coordinator and an institutional research office. An external evaluator helps for large programs, new interventions you want to test, or when reviewers questioned your evaluation before.
How do we track alumni who stop answering email?
Use several sources: professional profiles, publication databases, NIH RePORTER for later grants, department newsletters and a short annual survey. Ask trainees at exit for a permanent email and permission to contact them.