If you need one helicopter, order two.
That’s a lesson we learned at Talance when creating an online course on Incident Command System. ICS is a language that all emergency responders know, including the police, firefighters, and Coast Guard. That way, when there’s a disaster, everyone knows what to do, no matter who they work for.
The lesson is to be prepared — really prepared — for when it counts. What if something goes wrong with the first helicopter? You have another at the ready.
As a learning management system, we always order a second helicopter by putting extra energy into planning, testing, and building in redundancies. We want the courses we write and the LMS we use to have the best chance of success.
Planning to launch a virtual training program? Consider Virtual Training: Top 10 Questions From Program Managers and Directors.
What Is a Pilot Program for Online Training?
A pilot program is simply a small-scale version of your online training that’s launched before the program officially launches. Your pilot program can shed light on issues like user experience, content relevance, and formats. It can also give you insights into how the process goes. Is the course engaging? How are instructors doing? Is the difficulty level appropriate for participants?
The Benefits of Running a Pilot Program for Online Learning
Saves Costs and Time
The first benefit of running a pilot program for your online learning is that done early, it can save you development costs by avoiding learning gaps and fixing issues early in the process.
Programs that subscribe to our courses also pilot their choices. That way, they know if they’re making the right selection for the right audience.
Provides Real Feedback
When we’re looking at things too closely, we tend to miss issues. That’s why it’s always a good idea to have another set of eyes on your work.
A pilot program gives participants the opportunity to engage with your online learning the same way a “real” user would do it, allowing them to discover any inconsistencies, difficulties with tech, or other potential problems.
Participants can also share feedback about aspects like the format and the value of the content that you may be too caught up to notice.
Sparks New Opportunities
A fresh perspective on your pilot program can be the thing you need to elevate your online learning from good to amazing. Participants can offer suggestions to fill existing gaps you may not have noticed on your own because (ideally) they’ll be looking at your course from the perspective of learners — not administrators.
Validates Your Initial Assumptions
Most importantly, your pilot program can confirm whether the online training you designed actually achieves the goals you intended to achieve. If not, it’s time to get back to the drawing board and restructure your course before rolling it out.
How to Assemble a Pilot Program for Your Online Learning
Selecting Pilot Program Participants
The first step is often the hardest. Choosing who to involve in the pilot planning process can be complex, and it’s tempting to involve as many stakeholders as possible. This is a mistake. When programs invite dozens of people, especially decision-makers, subject-matter experts, coworkers and community partners, they’ll likely receive a different opinion from each person.
That’s too many people, and they probably don’t necessarily have the right opinions. If you’re a program manager in this position, ask yourself: “Will my boss be taking this course when it’s done?” If the answer is “no,” then they’re not the most important person to involve in a pilot phase.
We advise recruiting a selection of people from three groups:
- learners who are similarly motivated as the intended audience but have no experience with the material or with online training;
- learners who have some experience with the program’s previous training modules;
- one or two subject-matter experts.
This mix gives program administrators the best chance to understand what learners need to know and identify gaps in the training material. It’s helpful to invite some people who have absolutely no experience with online training. They can help identify issues that everyone else ignores. Common problems include defective menus, missing instructions that would make completing the training easier, or usability issues.
Beware of Inviting Coworkers and Friends to a Pilot Program
Sometimes Talance’s clients suggest this as a way to involve stakeholders in the course development process. This is a problem because …
- They’re not the intended audience, so they can’t give the correct feedback;
- If they’re a stakeholder who should have been guiding the project, they should be involved at the beginning, not at the end. This can lead to missed deadlines and an extra expense of rewrites when the writing phase is over.
Assembling the Wrong Pilot Group Can Give You False Information
One of our clients had such an experience prior to working with us. The program managers did organize a pilot (bonus points!), but they made two mistakes:
- Their pilot group was too small — only about five people.
- Although the training program was meant for people new to the material, they invited people who were veterans in their role with up to 25 years of experience.
When they reviewed feedback from the participants, they all complained that the information was too basic. This feedback dominated the evaluation forms because they only had a handful of participants.
The Perfect Pilot Group Size
A too-small group is as much of a problem as a too-big group. It makes sense to look for a group that’s similar to the audience you intend to be taking the course when it’s complete. You can also include a mix of experts.
We aim for a group of around 20 to 30 people because that tends to be the final class size for most of the training courses we develop.
Choosing the Ideal Pilot Group Participants
It’s best to begin any pilot program by clarifying who the intended audience is. It can be helpful to create one or two personas for recruiting. Every person who participates in the pilot program should match the persona.
The learner persona can include how much experience they have and how much time they need to dedicate to testing. It’s also helpful to include what their motivations are for completing the training. The subject-matter expert persona can include familiarity with online vs. traditional learning and particular areas of expertise.
Once you define who your ideal learner is and set your group size, you’ll be in a great position to find out if your training program is working as intended. From this step, making tweaks is much easier and less costly. A little preparation, careful planning—and even an extra helicopter–means better projects that accomplish your program training goals the way they were intended.
A Pilot Program Can Determine the Success of Your Online Learning Initiatives
At Talance, one of the first things we do when we begin planning for a new project with a client is stress the importance of testing the course with a pilot group before officially launching it. This is a process that many e-learning projects skip, or don’t follow completely. The risk with creating online training is that the people writing the courses are rarely the same people taking the course. That means the course can be irrelevant or miss critical information that learners need.
There’s no substitute for gathering feedback from a pilot program for your online training. We typically show examples from our past work of how piloting has helped create better courses in ways no one could have guessed.
