What evaluation data should I collect, and who should I collect it from?

The table below summarises the main data needed for each ‘level’ of evaluation, the target groups that will be able to supply the data, and the data collection methods you might consider using.

Evaluation Level

Main impact data needed

Possible Target Group(s)

Possible data collection methods

Level 1

Participant Reaction

reactions, insights, opinions, attitudes and preferences of learners

Learners

self-complete questionnaires

interviews

focus group

Level 2

Learning

pre and post-learning assessment, test and/or qualification data

reasons for results

barriers to achievement

trainer

learning and development officer

union learning representative

participants (via assessments)

self-complete questionnaires

desk research, eg data from learning management system

assessments/ tests

Level 3

Job Impact

learners’ self-evaluation of changes in job performance

managers’ observations of changes in learners’ job performance

data on changes to job performance before and after learning

reasons for and barriers to change

learners’ managers

line-managers

supervisors

team leaders

learners

self-complete questionnaires

interviews

focus group

desk research, eg data from management information systems

Level 4

Business Impact*

manager observations of changes in business performance

data on changes to key performance indicators

data on changes to other organizational performance metrics

financial data on programme benefits and costs

reasons for and barriers to changes

learners' managers

other departmental managers

other experts in the organization, eg quality control, finance, operations and human resources departments

self-complete questionnaires

interviews

focus group

desk research, eg data from management information systems

 

* You may find our example data collection template helpful when collecting Business Impact data. You can download it by clicking on the link below:

 

NOTE: When it comes to gathering quantitative data such as competency ratings or business performance metrics, you may need to ask stakeholders to provide estimates. You may also need to ask stakeholders to estimate the proportion of influence that the learning programme has had in relation to other influences. While estimations are not ideal, most often they are all that is needed, especially when time and resources for evaluation are limited. You will need to bear in mind that the estimates will only be credible if they come from stakeholders who are in a good position to make them.

Click on the link below for further guidance on data collection methods:

 

Back

Article Quick Jump :