Is it time to kill a sacred cow – or at least put Kirkpatrick out to pasture?

I was surrounded by some of the best Learning and Development (L&D) professionals in the U.S when I recently participated in two L&D conferences.

There is a strong feeling that now is the most exciting yet challenging time to be in the talent development profession. Employment rates are high. Great talent is difficult to find and recruit. Retention requires true investment in the individual. Multiple generations are in the workforce. Succession planning is critical.

Perhaps just saying that “Our people are our greatest asset” is non-optional.

At the first conference, I posited a need for creating a people development strategy that accounts for the hard-to-define “Future of Work.” Historically, L&D functions have focused on developing competencies — definable sub-elements of performance such as skills, knowledge, acumen and methodologies. Much has been achieved by using competencies to assist in interviewing, selection, training, coaching and performance management.

But what happens when there is uncertainty about the future nature of work? Are we aligning our required competencies to the ever-evolving needs of the business? How do we know that we are investing in the right initiatives?

Which brings me to the second conference.

“Innovation” was the theme for discussion. We were treated to a series of presentations from expert peers who are experimenting and achieving success using innovative approaches to develop their workforce. We heard about virtual reality to train operators for unsafe environments; high-tech micro-skill delivery vehicles to drive adoption and sustainment of learning; providing employees with access to a vast range of just-in-time curated and not curated learning solution options; and an innovative approach to demonstrating ROI.

This last presentation that particularly caught my attention: Is the L&D profession handicapped by our internal customer?

So now I am (grandly) feeling like Joseph Lister who, exactly 150 years ago, presented to an august group of surgeons at a conference in Birmingham and suggested that despite their best efforts, his colleagues were “killing their patients!”

Lister was laughed out of that conference and derided for several years before there was widespread acceptance of his research into bacterial contamination, sources of sepsis, and the need to create a sterile environment for surgery, not forgetting the “donation” of his name to the Listerine brand.

So how is L&D “killing the patient”? What is the missing link? Is there a silver bullet?

The problem

Let’s look critically at Kirkpatrick’s model, a staple for measurement, first developed over 65 years ago.

• Level 1 Reaction—aka “Happy Sheets”—It is important to get good scores, and Level 1 is helpful for program feedback, but it does not validate a huge investment regarding time, money or resources.
• Level 2 Learning—Consistent pre/post assessments (which take time, money, resources) can help measure how much individuals might have learned, but immediately creates the discussion about the knowing-vs-doing gap. “Just because they know, doesn’t mean they can or will.” Has the right “return” environment been created, and will they be given the opportunity to apply what they learned?
• Level 3 Behavior—Has there been a behavioral change? Who observes it? Is all change in behavior good? Who determines the quality? Behavior change is different for everyone and takes time, money and resources to evaluate, collate and assess.
• Level 4 Results—ROI. Organizational results—the “Holy Grail” for business and L&D leaders alike. The ultimate lagging indicator. If we can prove ROI then the training dollars are viewed as an investment and NOT a cost!

I have felt for some time that attempting to use business results to determine whether the investment is “correct” does not make practical sense. Too much time, external factors and other complications can occur between the “I” and the “R” when determining ROI.

Business leaders would obviously like to see ROI from development initiatives and L&D professionals would like to deliver it. This generates some very bad behavior—the “4 Fs” of focusing on the need to demonstrate ROI:

• Faith: “This will work, I promise; it will be good for the company; it will be good for the people; trust me!”
• Fear: “We have to do this! If you think education is expensive, try ignorance!”
• Fiction: The invention or exaggeration of results that loosely connect to the learning intervention.
• Failure: Fail to invest in the initiative this time or fail to demonstrate ROI and create a problem for the next time.

The solution

I propose the following approach to reduce patient mortality:

• Let us stop chasing a difficult or even nonsensical measure.
• Let us stop attempting to manage people by over-focusing on results.
• Let us instead use the critical leading indicators, the foundation for achieving results, required behaviors. Let us develop and measure Kirkpatrick “Level 3.5.”

This could fundamentally alter the way we think about measuring Learning and Development. We could start to focus on the learning versus the ROI and accomplish BOTH!

I would like to suggest some steps that could support L&D professions navigate these exciting yet challenging times to help prepare their organizations and workforce for the Future of Work:

1. L&D should work with their internal client, the business leaders, to define roles that are felt to be critical to the successful execution of the business strategy. Many of these roles could/should be novel and might not currently exist.
2. Each role should be dissected so as to identify the critical behaviors: What do you need people in these roles to DO? Definitions should be specific, observable, developable and coachable.
3. ONLY now it is possible to determine the competencies necessary to execute the behaviors.
4. Define the attitudes necessary in the workforce to motivate the behaviors.
5. Define and create the supportive environment (such as culture, measures, incentives, managerial behaviors, tools, systems, infrastructure).
6. Implement
7. Observe and coach the critical behaviors, the leading indicators for success.
8. Continuously improve (rinse and repeat).

So perhaps “standard” Kirkpatrick has run its course. Maybe the aged sacred cow should take a well-deserved retirement. Let us stop chasing after the ethereal “Holy Grail” and instead devote resources to defining, developing, observing, coaching and measuring behaviors. People development is a marathon, so perhaps we will begin to see bumper stickers exclaiming “3.5” alongside the more-usual “13.1” and “26.2”.

At UNC Executive Development, we feel privileged to partner with organizations to develop their leaders so they can thrive in the business environment of today and adapt to the Future of Work of tomorrow.

By Dave Roberts, president, UNC Executive Development