The TrainingCheck Approach

At the heart of the TrainingCheck approach is the latest Kirkpatrick ‘Return On Expectations’ (ROE) model of training evaluation (see below). However, because it can be informative to use a range of ways to evaluate the contribution and impact of training, we have also included elements (e.g. example evaluation questions) taken from other training evaluation models, such as Bersin’s ‘Learning Impact Measurement Framework’, Brinkerhoff’s ‘Success Case Method’, Phillips’ ‘Return On Investment’ approach and the ‘Balanced Scorecard for Skills’.

There is no one-size-fits-all solution to training evaluation. The individual circumstances and priorities of each organisation will define what is possible, practical and useful. Put very simply, sometimes all you may need to do is measure, say, compliance, while in other cases you will want to drill down into the bottom-line impact of the training.

For this reason TrainingCheck offers both a wide choice of training evaluation content and flexible authoring capabilities, which together mean you can define and carry out the kind of training evaluation which is most appropriate for and meaningful to your organisation.

The Original Kirkpatrick 'Four Level' Model

The Kirkpatrick Return On Expectations model of training evaluation (also sometimes known as 'the Kirkpatrick Business Partnership Model') evolved out of Donald Kirkpatrick’s famous ‘Four Level Model’. This model was originally developed in the 1950s and was for many years regarded as the industry standard in training evaluation across training and Human Resources communities.

The four levels referred to in the model are:

• Level 1: Participant Reaction
• Level 2: Learning
• Level 3: Job Impact
• Level 4: Business Impact

The following table below summarises this simple but influential model:

By progressing through each of the levels training evaluators can build a ‘chain of evidence’ which links learning directly with organisational performance. Having the right conditions for learning (level 1) enables learning to take place (level 2); the learning taking place lays the foundation for the transfer of the learning to the workplace (level 3); and the application of learning in the workplace enables the desired changes in organisational performance to happen (level 4).

Similarly, if the outcomes of a learning programme are not as expected, the model provides the evaluator with a diagnostic framework. By checking back through responses to the evaluation at each of the levels the evaluator should be able to identify any key barriers to training effectiveness and impact.

The Kirkpatrick ROE Model

Although the four levels still remain very much at the centre of the Kirkpatrick model, in recent years the model has evolved to become both more sophisticated and more powerful.

The focus of the updated model, which is often referred to as the Kirkpatrick ‘Return on Expectations’ or ‘ROE’ model, is on ensuring that training objectives are strategically relevant to the organisation in the first place, clearly defining the expectations of the outcomes of the training, and then evaluating against both the training objectives and the agreed expectations.

The main principles of the ROE model can be summarised as follows:

• Business/organisational objectives are seen as a starting point - To ensure training processes deliver real value to the organisation it is critical from the outset to work with key stakeholders to establish what the organisation’s strategic objectives are and precisely how the training should contribute to the achievement of these. Training objectives, content and methods should then all be tightly focused on meeting these strategic objectives. Stakeholders will be able to identify which business performance metrics are most meaningful in relation to the strategic objectives, and measurement of changes to these should form a core part of the evaluation.
• 'Return on Expectations' (ROE) is key – it is essential to negotiate and clarify with stakeholders what the expectations are regarding training outcomes. Clear and precise success indicators (eg percent changes in performance metrics) should be developed in line with these expectations. The training evaluation should then assess and report on both the actual changes that have come about as a result of the training and the extent to which these changes have met stakeholders’ expectations, ie whether there has been a ‘Return on Expectations’.
• Collective efforts are needed throughout an organisation to achieve success – For training to achieve a positive ROE, supervisors, managers and others in an organisation all need to be involved in supporting the training process before, during and after training begins. They can do this by, for example, helping prepare participants for training; providing support/coaching during the training; and, crucially, providing ‘consistent and deliberate’ opportunities to apply and reinforce learning in the workplace after the formal training has ended.
  
  Those responsible for training need to widen their role and expand their expertise and their influence within their organisations to support managers to do these things, and, more broadly, to ensure that both learning and the application of learning continue into the workplace.

The flowchart below summarises the key steps involved in the Kirkpatrick ROE evaluation process:

Kirkpatrick ROE - Evaluation Process Summary:

(Source: ‘The Kirkpatrick Four Levels: A Fresh Look After 50 Years’ (2009), Paper by Jim Kirkpatrick, PhD and Wendy Kayser Kirkpatrick. See www.kirkpatrickpartners.com)

How the Kirkpatrick ROE model has been incorporated within TrainingCheck

We have taken the core principles of the Kirkpatrick ROE model and applied them within TrainingCheck by providing:

• Detailed guidance on planning, creating, carrying out and reporting on training evaluations in line with the ROE model.
• Web-based, automated processes for designing, carrying out, analysing and reporting on evaluations at each of the Kirkpatrick levels.
• A comprehensive Question Library subdivided into the Kirkpatrick levels (for ease of use we call these ‘topic areas’ within the Library).
• Ready-made, customisable example evaluations at each of the Kirkpatrick levels – ie Participant Reaction, Learning, Job Impact (both participant and manager views), and Business Impact.
• A range of other evaluation tools to support wider aspects of the Kirkpatrick approach, including example training needs analyses, core competency assessments and 360 degree/multi-rater evaluations.

Beyond Kirkpatrick

To expand the opportunities for evaluators to gain meaningful feedback and actionable data on training impact, in addition to the core Kirkpatrick ROE model TrainingCheck also provides further training evaluation features and elements, including:

• A 'Return on Training Investment (ROTI) Calculator', which can be used to carry out ROTI%, cost-benefit ratio and 'Payback Period' analyses.
• A wide variety of supplementary training evaluation questions within the example evaluations and the Question Library, including Bersin, Brinkerhoff, Phillips ROI and Balanced Scorecard type questions (Note: These questions have been subsumed within the Kirkpatrick topic areas within the Question Library).

You can find out more about the TrainingCheck Approach and how to implement it in the following guidance:

• Planning Your Training Evaluation
• Creating Evaluations at the Different 'Levels'
• Calculating Return on Training Investment (ROTI)

Back

Print

Article Quick Jump :

The TrainingCheck Approach How to Evaluate Training Effectiveness Using TrainingCheck – At a Glance Guide The TrainingCheck Approach Planning Your Training Evaluation Creating Evaluations at the Different 'Levels' Calculating Return on Training Investment (ROTI) References and Resources