Executive Summary
In my 35 years of experience in learning and development, I have noticed that newly acquired knowledge and skills tend to fade away as soon as students complete assessment tests, receive their passing grades and walk out of the corporate classroom.
In order for learning to have a lasting impact, my firm recommendation is to expose the learners to Deep Learning.
In my previous series of articles I call the “Blind Spot” series, I listed a number of areas in learning design and implementation that escapes the notice of learning managers and developers alike, resulting in substantial waste in time and money without the measurement of the the effects of learning to the real business bottom line.
In this article, I wish to bring to attention another key component – whether or not learning is actually tied to the company’s key business KPI’s – by focusing on one of the most important elements of learning – that is – finding out the effectiveness of learning through accurate measurement of learning outcomes!
But this is easier said than done and the challenge remains – How do you measure? What do you measure? When do you measure? What KPI do you use?
There are two ways learning outcomes are generally measured:
(a) By measuring the learner’s ‘before’ and ‘after’ on-the-job performance, mainly done by supervisors, and (b) By measuring, monitoring and analyzing (directly or indirectly) the learners’ behavior, actions, interest, motivation, learning pattern, curiosity, involvement, participation, etc.
In this article I present my views on the benefits of measuring learning outcome by the second method that leverages modern LMS technology and data collection improvements through re-designing the learning program, and analyzing LMS reports, to ensure Deep Learning! (You can read my previously published article on ‘Deep Learning’ here)
Introduction
In the corporate world, your success as an organization depends on the success of your employees. The talent you are hiring must demonstrate their ability to meet or exceed the skill and intellectual requirements stated in the job description. If they don’t, then training is supposed to help employees close this gap. Unless your employees are i) really interested in learning, ii) are sufficiently engaged in your training programs, and iii) are motivated to bring themselves up to the level of their job requirements, your training dollars will, unfortunately, be wasted and your company isn’t going to scale in the desired manner. That is precisely why organizations need to measure the effectiveness of their training irrespective of the industry they operate in. New technology is being developed daily that gives you the tools to monitor, measure and analyze the effectiveness of training programs. Better software, new market trends and innovative new practices are pouring into the market, giving you new capabilities to boost your organization’s learning effectiveness significantly.
One of the common statistics regarding learners’ knowledge retention is that “40 percent of material learned is forgotten in 20 minutes, 77 percent of material learned is completely wiped from learners memory in only 7 days and learners forget 90 percent of what they’ve learned after one month.”
Clearly, these statistics are cause for concern among learning and development leaders who’ve invested heavily in training courses and want to see tangible long-lasting results.
So, in order to discuss learning effectiveness measurement strategies, we must first look at what makes the student learn, and how you can design a learning program that your students will enjoy taking. Furthermore, and most importantly, can you measure the effectiveness of your learning program to understand who is learning and who is not? I would argue that from this “effectiveness” data, you can improve the learners’ learning experience, thereby improving the learning effectiveness.
There are many variables that affect learning and in order to identify these variables, here are some of the questions we can ask:
- Is there any motivation to learn?
- Is prior knowledge of the content necessary to learn the material presented?
- How closely is the content related to the work they do?
- What are the learning methods and modalities used?
- How interactive and/or immersive is the learning experience?
- What is the length of time the learning needs to be (1) delivered and (2) retained?
- Does the design of the course lead to contextual cues to the students’ personal experiences that help them see and remember scenarios from a personal perspective?
The good news is that all of these attributes can be measured, monitored and intervened (directly and/or indirectly) – so learners absorb and retain most of the material presented to them, and eventually be capable of transferring most of the new knowledge learned to apply to new challenges, new environments and new applications over time.
This is what Deep Learning is all about!
How to measure Training Effectiveness?
The training programs organized by companies normally help new and experienced employees get the hang of new business processes, new technologies, latest trends, practices and regulations. All of your organizations spend a significant amount of money on these training programs.
Naturally, it is important to measure the effectiveness of these programs. Afterall, how else would you be able to determine the success of the training programs?! Here are a few ways to measure training effectiveness:
1. “Before State” – The participants in the program should provide a response to a “Present State” questionnaire that reveals exactly where they are with respect to the new material being presented at the training session.
2. “Prior Knowledge” – In situations, where prior specific knowledge is required before he/she can enroll in the new training session, participants must be able to demonstrate the particular knowledge necessary to attend the new session. For instance, if a participant learns about electrical safety, he/she must demonstrate that he/she has worked with similar electrical system before, and should now be in a position to learn new knowledge in the new training session.
3. “Engagement” – For each presentation (regardless of whether it’s instructor led, or self-paced e-learning), the students’ interaction, collaboration and engagement with the presentation is absolutely critical to the impact of training on the learner. (Read my previously published article on ‘Learner Engagement‘ here)
4. “After State” – Each training program should be followed up by a skill assessment. This should be made mandatory for all learners. A thorough assessment, evaluation of the skills of participants would tell you how effective the program was or wasn’t.
5. Analysis of reports – Since all the above data (for each participant) is available in the LMS, the LMS reports and analytics should show how the training program has helped the student hone his or her skills. Learner feedback (also available in the LMS) at the end of the training session, will tell us a lot about presentation, engagement, student preferences and performances.
Learning effectiveness measurement with an LMS
In the last century, it used to be almost impossible to measure the effectiveness of training programs with any degree of confidence or accuracy.
Thankfully today, we have the technology and have developed a new generation of customizable Learning Management Systems which can monitor all student activities, interactions and behavior from the beginning to the end of the learning journey. Not only that, many are industry specific and easily customizable to the idiosyncrasies of the modern learning environment. The entire training session is under complete surveillance. Everything that is covered in the program will be monitored, along with the participants’ interpretation and reaction towards it.
Naturally, this tech makes it a whole lot easier to analyze the effectiveness of training and if you’re pouring money into learning programs, you should get the right tools to ensure that it doesn’t go to waste. The core tool in your arsenal is your learning management system and so care must be taken in selecting and setting one up. Here is a typical LMS functionality diagram that can be easily customized to individual client needs and learning environment.
Learning Measurement allows us to make inferences about what students know, understand, are excited to learn about, and can transfer the knowledge in an entirely new situation – removed from the learning environment. The instructional design must connect the learning objectives to the KPI(s) and the L&D designers and implementers must determine ways to measure all of the KPI outcomes, creating assessments that can be integrated seamlessly with learning.
What’s often missing in most corporate L&D departments is the will and a systematic plan to do so.
Conclusion
I want to leave you with this final thought – the role of Instructors and Instructional Designers to make learning “Fun” and effective in today’s learning environment is paramount to learning success.
That’s because today’s classrooms are very different from what we know as the traditional classrooms” of the past. In 2022 and beyond, “classrooms” are not made of brick and mortar and restricted within the boundary of four walls. Today’s learners learn on their own terms, at their own time and at their own speed. The new digital classroom must focus on student questions and interests building on what they already know. To make learning effective in tomorrow’s world, designers must focus on engaging, interactive learning that is student-centered. Teachers must have a close interaction with students to help them construct their own knowledge, perhaps rooted in storytelling, collaboration and encouraging group activities. The 21st century classroom often has teachers creating small groups – collaborative, interactive, open and immersive. Courses must be designed for the millennial and gen z generations to be short, engaging and ‘fun’.
However, this does not mean today’s learning should not have structure – it means that the curriculum must offer choices. Some students need highly structured and organized learning environments to thrive, but others may prefer to learn in a more laid-back environment. The important thing to note is – today we must help students engage in their own learning.
We have to keep in mind that assessment test results are often misleading. Today’s learning environments may place more value on student progress – clearly identified in the LMS analytics. This Deep Learning approach can lead to students not just memorizing the content but comprehending in a personal manner that may stay with him/her throughout his/her professional career.
If you are hoping to address factors I have listed above to meet today’s learning environment – this is the perfect time to do it right because all of us are trying to adjust to distance learning that has completely consumed the L&D community by surprise with no choice to go back to old ways of power points and bulleted text driven boring classrooms. It is crucial to getting on the right path.
I will be more than happy to get in a conversation with anyone looking to adjust to this new norm.
