When it comes to training your healthcare team, the meaning of “success” isn’t always the same. Budget cuts, changes in workforce and differing work environments can mean that a format that worked in the past may not work now.
That’s why a regular evaluation of a community health worker training program is a smart move. Scheduled assessments mean your organization stands to earn the highest return on investment. Plus, paying attention to how well people are learning will boost your staff’s involvement in the overall educational program. For many health departments and health systems, the biggest question when it comes to evaluating a professional development program is when does it make sense to move to online training.
A helpful methodology for weighing the success of any learning process is the Kirkpatrick Four Levels Evaluation model first developed by an educator named Donald Kirkpatrick in 1959, and then published as part of a book in 1994 called Evaluating Training Programs. The model moves through four levels that cover the essentials of training programs. Evaluations with this model build–use the data gathered from each successive level to measure the effectiveness of the whole program.
Carefully examining how your in-person program stacks up against each level will help you move to an online model more smoothly. Here’s what each evaluation leve looks like in depth.
Level 1: Reaction
The first step is to find out how well the participants liked the training. In short, if they hate it, it isn’t working. Look carefully at the evaluations you offer at the end of training, and also listen to anecdotal evidence. When you’re considering migrating to an online training program, make sure to find out if time away from the office and the expense of an in-person session factor in. Listening to how community health workers’ reactions to a program, in particular negative reactions, can have an effect on learning (level 2).
Level 2: Learning
Next, evaluate how well your CHW staff learned the material. Some of the best indicators for knowledge gain is pre-tests, post-tests and self-assessments, which should all align with the learning objectives you developed in advance. Ask participants the same questions before the training as after, and you’ll be able to see an increase or plateau of knowledge. If increase isn’t significant enough, it’s time to change something.
Level 3: Behavior
A better, but more involved, way to assess how well training is working is to look at behavior. Examining behavior will reveal if those newly acquired skills are being demonstrated on the job. Health worker supervisors can help determine if staff is demonstrating knowledge-gain, but look beyond supervisors to other indicators. For example offer surveys to clients to see how well they’re being served by CHWs, asking specific questions about new skills.
Level 4: Results
Finally, look at tangible results to see how well your program is working. Are CHWs serving more clients? Are they navigating through barriers to care? Are more clients attending health screenings? Is the an in-person program costing too much? Pulling together the data to examine results is more time consuming than the other levels, but numbers don’t lie.
Once you have pulled together data for the Four Levels Evaluation, you can more easily determine if it makes sense for your organization to move to an online format for training community health workers.