What data collection methods should I use?

The table below summarises the main traning feedback and impact data needed for each of Kirkpatrick’s four evaluation levels, the target groups that will be able to supply the data, and the data collection methods you might consider using. The table is followed by a brief description of each of the methods and how TrainingCheck can be used to implement these.

Evaluation Level	Main impact data needed	Possible Target Group(s)	Possible data collection methods
Level 1 Participant Reaction	reactions, insights, opinions, attitudes and preferences of learners	Learners	self-complete questionnaires interviews focus group
Level 2 Learning	pre and post-learning assessment, test and/or qualification data reasons for results barriers to achievement	trainer learning and development officer union learning representative participants (via assessments)	self-complete questionnaires desk research, eg data from learning management system assessments/ tests
Level 3 Job Impact	learners’ self-evaluation of changes in job performance managers’ observations of changes in learners’ job performance data on changes to job performance before and after learning reasons for and barriers to change	learners’ managers line-managers supervisors team leaders learners	self-complete questionnaires interviews focus group desk research, eg data from management information systems
Level 4 Business Impact*	manager observations of changes in business performance data on changes to key performance indicators data on changes to other organisational performance metrics financial data on programme benefits and costs reasons for and barriers to changes	learners' managers other departmental managers other experts in the organisation, eg quality control, finance, operations and human resources departments	self-complete questionnaires interviews focus group desk research, eg data from management information systems

 

* You may find our example data collection template helpful when collecting Business Impact data. You can download it by clicking on the link below:

• Performance Improvement Data Collection Template (Word)

 

Note: When it comes to gathering quantitative data such as competency ratings or performance indicator figures, you may need to ask stakeholders to provide estimates. You may also need to ask stakeholders to estimate the proportion of influence that the training programme has had in relation to other influences. While estimates are not ideal, most often they are all that is needed, especially when time and resources for evaluation are limited. However, you will need to bear in mind that the estimates will only be credible if they come from stakeholders who are in a good position to make them.

The following sections provide brief guidance on using the possible data collection methods.

• Self-complete questionnaires
• Interviews
• Focus groups
• Desk research

Back