Training and Assessing Workbook - Train the Trainer manual
Section 5: Assessment, evaluation and reporting
- 5.1 What is assessment?
- 5.2 Should all competency be assessed?
- 5.3 Should assessments be conducted at regular intervals?
- 5.4 What should be recorded in an assessment?
- 5.5 Assessment methods
- 5.6 Is performance at the worksite the best way to assess competence?
- 5.7 Should assessment always be done at the supported employee's worksite?
- 5.8 Should there be more than one assessor?
- 5.9 Are assessments like exams?
- 5.10 What is evaluation?
- 5.11 What does evaluation of training involve?
- 5.12 Should assessment tools and practices be evaluated?
- 5.13 Reasons for evaluating training
- 5.14 How can the training delivered be evaluated?
- 5.15 Recording training and assessment outcomes
To allocate work appropriately to meet daily, weekly or monthly output targets you will need to be aware of the supported employees' capacity to undertake particular tasks. You also need to know how the work team's skills and knowledge combine and work together to form a functional team. Assessing helps you recognise where supported employees are competent and where training is needed. It is a routine part of most supervisors' daily work.
Assessment measures if a supported employee has the skills or knowledge required to perform their work. A very simple example is provided below.
- Training needs analysis
Georgia cannot make a salad sandwich.
- Training designed and delivered
Georgia is trained to make a salad sandwich.
- Competency is assessed
Georgia can/cannot make a salad sandwich.
- If Georgia can make the sandwich to the standard required, she is competent
If she can't, her supervisor needs to return to Step 1, analysing the parts of the task she is having difficulty with, or if there are other issues apart from training.
As a supervisor you may measure the effectiveness of training by assessing a supported employee's performance and contributions in areas such as:
- reduction in poor quality products
- reduction in injuries or lost work time
- increased output
- improved workplace behaviours.
Assessments do not have to be conducted under examination conditions and preferably are part of the everyday practices of the Disability Employment Service. The essential element in assessing a supported employee is that you, can measure the supported employee's task competency. To do this you need to have measurable data to check against.
Some supported employees retain skills and knowledge well, others are less able. For example, a supported employee may be away from work for three days and be unable to recall how to complete their tasks when they return. As a supervisor it will be important for you to know the various components of the task in order to assess where the gap is occurring and then train appropriately.
Choose an activity that you will need to assess. Apart from you and the supported employee, who else will be involved in the assessment? What are the roles of the other people?
All competency should be assessed. The reasons include making sure the supported employee can perform the task:
- to the standard required by the business
- to meet legislative requirements.
If all competency is assessed it is important to ask yourself certain questions.
- What is being assessed?
- Where should the assessment take place?
- When does the assessment need to take place?
- Who is going to conduct the assessment?
- How should the assessment be done?
- Where does the assessment 'fit' in the supported employee's workplace?
- Do the supported employees need to be assessed at their worksite?
- What are the options for conducting the assessment? Is there only one way it can be done or are there alternatives?
- Are there any workplace policies and practices that should be considered?
- Are there any occupational health and safety matters to be addressed?
Assessment should be continuous and ideally take place within the normal day-to-day activities of the Disability Employment Service. There may occasionally be times when assessment will need to be undertaken off-site but most should occur at the worksite and in the daily routine.
Most Disability Employment Services will have an assessment record sheet. These sheets may form part of a supported employee's personal development plan. Talk to your workplace manager or human resources manager to clarify your business' requirements. An example of a personal development plan is provided in the trainer's toolkit. A simple example of content is provided below.
|Supported employee's name||J. Bloggs|
|Date of assessment||21 November 2006|
|Reason for the assessment||Safety audit|
|Tools/equipment needed||Personal Protective Equipment|
|Support employees' needs||Time, questions|
|Standard/benchmark to be used||Workplace safety checklist|
|Location for assessment||Workplace|
Once you have determined the purpose of the assessment you are able to choose an appropriate assessment method that addresses the purpose.
It may be preferable that the assessment is not seen by the supported employee, as a test. Try to make the assessment part of the normal work flow wherever possible. If you need to 'test' it will be important to talk to the supported employee about why this is being done and how it can help them do their work.
Other things to think about when assessing will include:
- evidence or examples the workplace needs for its records
- particular needs of the supported employee Assessing
- equipment or adjustments needed to undertake the assessment.
Some assessment methods that may be useful for you in the workplace are set out in the following table.
|Observation||Part of the supervisor's daily role.
Offers quality evidence.
Supported employees recognise this as part of their daily routine.
Provides an opportunity for continuous assessment.
Offers opportunity for timely gathering of evidence.
|Opportunity for the supported employees to demonstrate complete set of skills may be limited by the tasks being performed at the time of observation.|
|Written test||Written tests can be quite simple, eg tick the correct answer, draw lines to match the correct answers.
Useful when it is difficult to apply other methods because of health or safety factors.
Can be delivered as a game.
Useful as a supplementary assessment tool rather than as the primary assessment tool.
|Usually not applicable at the real worksite. |
Can worry some supported employees if they perceive it to be a 'test'.
Can be time-consuming to prepare and check.
Supported employees may have the knowledge to complete the written assessment but be unable to apply that knowledge at the worksite.
|Work samples||Excellent as an ongoing assessment method.
Provides opportunity to track skill development.
Can readily assess performance against quality standards.
|Need to note any unusual circumstances that may have influenced the quality of a sample.|
|Questioning||Simple to apply.
Useful as a supplementary assessment tool.
Useful for gaining an understanding of the supported employee's knowledge.
|Requires good understanding of questioning techniques to be effective. |
Response might not be indicative of actual performance at the worksite.
|Simulation activity||Useful when it is difficult to apply other methods because of health or safety factors.
Useful as a supplementary assessment tool rather than as the primary assessment tool.
|Can worry some supported employees if they perceive it to be a 'test'. |
Can be time consuming to prepare.
Supported employees may have the knowledge to complete the simulation activity assessment but be unable to apply that knowledge at the worksite.
Observing the supported employee at their worksite will always be the primary source of evidence that they are competent in performing a task. If this is combined with work samples and questions you will have a good assessment methodology in place.
It is not always possible to observe the required performance at the worksite and you may need to develop alternative ways to assess task competence. The information you gather from off-site assessments will not be as valuable as that gathered from the worksite. It will be part of an assessment process, used in conjunction with other information you may have gathered from other sources such as discussions with a previous supervisor, previous performance plans and assessments.
The number of people involved in assessing whether a supported employee is able to complete a task or not will depend on the individual workplace and the tasks being assessed. In most situations there will be one assessor. Performance in the workplace as observed by previous supervisors and gathered through performance plans may be valuable. For example, you may speak to a supported employee's previous supervisor to discuss the employee's ability to perform particular tasks.
Having many people watching, asking questions and taking notes is difficult for many supported employees. Try to avoid having more than one assessor but recognise it may be necessary for particular tasks. If necessary, discuss options with your human resources manager.
Assessments should not be exams. Assessments should be part of the daily routine and be completed in a comfortable and realistic workplace setting.
Jenny put on her white coat and gathered up her clipboard. It was her first day on the job at Resistant Paint Supplies and Products and she was looking forward to assessing the numeracy skills of the small paint pots production team. She had decided that an unobtrusive approach was best. The supported employees had all greeted her when she walked past and she had said hello and indicated that she wouldn't disturb their work. She had then moved to a nearby alcove where she was partially hidden from the team's view. Tom, the team leader, looked a bit perplexed but continued his work with the team.
Jenny thought all appeared to be going well. The supported employees worked quickly and quietly, barely looking up. They appeared to be concentrating very well and few errors occurred during the five minutes that Jenny observed. Later, Adam, Jenny's manager, asked her about the assessment. She described the procedure and the results in detail to Adam.
What feedback do you think Adam should give Jenny?
What recommendations would you make for completing a similar assessment with another team?
- Always explain the purpose of an assessment activity to the supported employee.
- Don't write notes during an assessment unless note taking is a usual part of your daily work with the supported employees you are assessing. Note taking can be perceived as threatening and may worry some supported employees.
- Assess by doing the task with the supported employee where possible. This will help with a compare and contrast training activity. For example, 'What have I done wrong here?', 'How should I fix this?', 'Which one looks the best?'
- Consider making an incorrect sample yourself and ask the supported employee to show you how to make a good product or explain where yours needs to be fixed.
- Remember we all get it wrong sometimes.
Ingrid was being assessed to determine whether she would be able to join the gardening team. She was very anxious about the assessment and really wanted to get everything right in the last safety check so she could start with the team the following Monday.
Ruben, Ingrid's supervisor, had talked to the assessor about Ingrid and arranged for the assessment to take place just before lunch on Wednesday. During the morning Ingrid had asked him questions about what would happen during the assessment and he had answered all her questions. He had tried to reassure her that she knew all the information and that everything would be fine.
The assessment was a disaster. Ingrid became confused and unable to respond to the assessor in a way that indicated she could use the equipment safely. The assessor was concerned that Ingrid would lose her confidence and would not want to proceed with the new job.
How can the assessor help Ingrid?
- Whenever possible, provide a number of opportunities for assessment to take place.
- If an assessment situation is likely to worry or upset a supported employee, consider breaking the assessment into smaller parts.
- Think about having the supported employee accompanied by someone they feel comfortable with, for example their current supervisor or another employee or team member.
- Check that the assessment activity is really essential.
Evaluation is a way of checking that training returns value to the business. From both the business and employee perspectives it is important to evaluate if training results in:
- increased staff knowledge and skills
- appropriate application of the skills and knowledge in the workplace
- clearer capacity to identify training gaps and future training needs
- information to assist training planning
- opportunity to ensure there is continuous improvement in training provision.
Having a clear set of objectives for the training is required before any type of useful evaluation can be completed.
Training should be evaluated from the supported employee's perspective as well as the trainer's. Supported employees should have the opportunity to comment on the training in terms of its usefulness and their reactions to it. Trainers need to review the training in terms of how well it met the objectives and how its delivery could be improved.
There are a number steps involved in evaluating training.
|1||Decide if the selected training methods will help the participant gain the skills, knowledge and behaviours required||Review of prior experience training the group or individual, including learning styles and training preferences Discussion with participants or other team leaders|
|2||Conduct the training and gauge the participant's reaction to the training, find out how the participant felt about the training.||Informal interview, structured conversation or completion of an evaluation sheet|
|3||Discover the learning the participant gathered from the training||Informal interview, structured conversation or completion of an evaluation sheet|
|4||Determine the skills and behaviours the participant gained from the training||Workplace observation following training to determine the transfer of new skills and behaviours to the workplace|
|5||Review the effectiveness and results of training through the application of new skills, knowledge and behaviours||Workplace observation of the application of the new skills, knowledge and behaviours over a sustained period of time Examples could include increased job satisfaction, fewer defective products, increased output|
Assessment tools and practices should be regularly reviewed and evaluated to ensure they are suitable. They should be modified if necessary to ensure they are measuring for the purpose required and that they recognise supported employees' needs as well as business requirements.
Evaluation is a systematic way of gathering, reviewing and then judging the value of the information.
Your business service will need training to be evaluated to assist with planning and budgeting. It will also need to evaluate training to meet funding and legislative standards.
Most importantly, you should evaluate yourself. Through self-evaluation you can develop your skills, build your strengths and develop additional skills in training needs analysis, training design, delivery and assessment.
Does your business service have an evaluation policy? Does the policy discuss the role of supervisors and team leaders in evaluating supported employees?
What are the privacy and security requirements for the business service's evaluation policy and procedure?
Reflection is a good place to start. Think about the training you have provided, the assessment you have completed with the supported employees and evaluate yourself.
There are other people who can provide information on evaluation techniques and processes to help you with future training. These people could include:
- the supported employees
- your manager
- the training manager at your business service
- other team leaders.
Each of these people can provide a different perspective on your workplace training skills. The next part of this resource, the Train the Trainer toolkit, has some examples of evaluation forms.
Each Disability Employment Service will have record keeping requirements in relation to the training needed by the supported employees and the training provided. In addition, funding from FaCSIA is dependent on Disability Employment Services providing training in:
- workplace health and safety
- the 12 Standards
- work ethics and behaviour.
This means that records must be kept. It also means that rules will apply to the type of records, information to be included in the records and how the records are to be maintained.
Maintaining training records for each supported employee is useful. These detail the specifics of a training session and may initially be in a notebook kept by the team leader. These notes are important as they keep track of where a supported employee is proficient and can independently perform a task and where additional training is needed. The records help determine the learning requirements for supported employees.
An example is provided below
|1 October||Troi||10 minutes||How to interact with other team members||Will need to follow up regularly|
|1 October||Kylie||15 minutes||Using a cutting blade correctly||Check Kylie continues to wear safety gloves|
|1 October||Trent||5 minutes||Cleaning glue brush||Trent can now do this well|
|1 October||Jayde||5 minutes||Staying within safety walkways||Consider partnering Jayde with an employee with good safety practices|
If supported employees at your Disability Employment Service are completing accredited training there will be additional records to be maintained. The requirements for these would be provided as part of the accredited training.
Some businesses have personal development plans for supported employees. Each supported employee has a plan that sets out the areas of responsibility, the expected outcomes for each area of responsibility and where additional training may be required.
The additional training is identified in two ways:
- to improve current work performance
- to meet new goals identified as part of a career planning process or new work opportunities.
An example of an employee training plan is included in the Train the Trainer toolkit (refer page 16 of the Toolkit).
Assessment is part of good training practice. Knowing what to assess and how to effectively assess will help you supervise supported employees. Assessment records are integral to Disability Employment Services' ability to provide planned and individualised personal and professional development to supported employees.
Good recordkeeping practices and systems assist services provide supported employees with timely, effective training.
Consistent evaluation practices help trainers continually review their performance. They will also help trainers to plan future sessions.