1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184 
Rumb ings at the roadside
Page 28
Page 29
Page 30
Page 31
If you've noticed an error in this article please click here to report it so we can fix it.
Few would dispute that the Vehicle and Operator Services Agency (Vosa) plays a crucial role in maintaining road safety. But has the need to demonstrate its efficiency in targeting led to a scheme that distorts both the measurement and practice of enforcement activity?
Commercial Motor has received information and calls from many Vosa staff and customers since our examination of the outsourcing consultation last year — and their experience suggests that this is a question that bears closer examination. Everyone we spoke to was concerned about road safety, passionate about the work that Vosa does and felt strongly that any flaws in the current system which could detract from road safety should be highlighted so that they may be corrected.We have protected their identities with pseudonyms upon request.
Traffic light system
Vosa's intelligence-led targeting revolves around the Operator Compliance Risk Score, (OCRS), and the traffic light system that arises from it, labelling operators green, amber or red. Green suggests a good record and red are those targeted for roadside attention. As James Hookham, director of policy at the Freight Transport Association, says: If Vosa are using OCRS to target the had guys and take them off the roads I don't think any of us would have a problem with that.
But is it that simple? Those who contacted CM were concerned about another aspect of the targeted enforcement — the performance gain (PG) points targets that every examiner and region is tasked with hitting. Vosa says the PG points are collected not only by issuing prohibitions, but also by education and deterrent. However, the individual examiners we spoke to believed their points came from prohibitions, with a national target divided between the 21 regions depending on their numbers of staff. In response to a freedom of information request in 2005, Vosa said that in 2001/2 it "moved away from measuring effectiveness in terms of hitting hard numerical targets (eg the number of vehicles inspected or weighed) to focusing on the outcome of what we do. PG has three broad objectives: • To encourage more effective targeting; • To reduce the burden of enforcement on the compliant; and • To improve the understanding and awareness of operators and drivers through education and advice."
But Kevin Warden, secretary of the Vehicle Inspectorate chapter of the trade union Prospect, says: "PG was supposed to take us away from number crunching to results; ie away from the number of vehicles tested to the number of poor trucks off the road. But what has effectively been done is to take away one numerical target only to replace it with another numerical target for PG points."
Employee views
A feedback report by consultancy ORC Intemational,sent to CM, following a Vosa staff survey made the following comments: "Staff believe that Stephen [Tetlow, Vosa CEO] was brought in._ to increase efficiency which has led to a focus on targets and job cuts. In the eyes of many, these changes have overshadowed Vosa's remit as a road safety organisation."
The targets are "considered unrealistic by many" and it is not just staff but managers who suffer: "Level 8 and 9 managers complain of wasting time on 'irrelevant report writing and target chasing' and expressed the need to make targets 'meaningful and key'."
"It pushes us down a particular path," says Warden. This path is one of the chief concerns of the Vosa staff who have contacted us. It is their belief that Vosa is staffed by good men and women whose attempts at a good job are frustrated by management policy, and that PG targets at times penalise good operators and at others require a blind eye to the bad.
As a vehicle tester. Teter' often sees prohibitions which have been awarded and has spent time on roadside enforcement. He says: "I saw a prohibition on a bus brake drum that was highly dubious. [I believe] the inspector had been given his target to meet and he may have invented something to meet it... If you don't meet your target you're pulled up in front of your manager and roasted."
Peter believes that having to meet targets can distort examiners' performance: "Vosa talks about targeting the non-compliant but in fact good operators are penalised unfairly. I never agreed with bonuses as a mechanic because it makes people do a sloppy job. Equally targets are wrong because they incentivise people to dig and dig to find something despite the fact that it may not be detrimental to road safety."
Staff behaviour
Vosa itself admits that targets can in some circumstances alter the behaviour of staff. In explaining to us why numerical targets are not set for the number of vehicles that should pass or fail in order to achieve consistency across test stations, it says: "Our primary interest is recording the right result and not setting targets that could mean examiners passing vehicles which should fail."
Vehicle examiner 'Paul' fears that some roadside checks are overly influenced by the points tally: "Some people will go down the route of making a prohibition S-marked [indicating a serious lapse in maintenance] when it isn't necessary because we know it carries more points. It's a very small proportion of people. We're pressured in that direction. I myself resist it but I'm aware that it happens."
'Bill',a senior vehicle examiner,believes some examiners are over-zealous in their pursuit of the points: -One or two examiners tend to overestimate things,or push it because they know they will get more points if they make a drama out of it. But it just takes good management.You can make someone sit down and write out their justification... that tends to stop them."
But Paul is not convinced that such people are dealt with: "They don't take aside 'overachievers', which is a shame because we have to be seen to be fair and there are some loose cannons out there. I'd like to think the quality control with us now will see the suitability of prohibitions being assessed."
PG points are measured in terms of the defect's effect on road safety; the more serious the defect, the higher the points tariff. Vosa says: "Each outcome is given a value relative to its estimated contribution to road safety and environmental standards.The points value ranges from one point to a vehicle check to 100 points for an immediate prohibition with a safety-critical steering, tyre or brake defect."
Although all examiners know of the points schedule and those we spoke to had all seen it, CM was told that most staff do not possess a copy. "It's not something we [refer to] every day," says Bill.
Not only can the PG targets present a problem for staff when they have to be met, there are fears that they can work against road safety once they have been reached: "We've hit our LGV target for the 2006 financial year now so we're no longer looking at lorries at all," says Bill. "We've working on our PSV targets. It's a cause for concern because if the guys see an LGV that compromises safety, they may be discouraged from looking at it."
The OCRS system targets 'non-compliant' operators rated red and amber, but the system also offers the chance of redemption — spot checks that reveal sound vehicles could help boost a firm back into the green. But Paul says it doesn't always work that way: "[Vosa management] would rather I do two vehicles and find prohibitions than I do 10 that are sound and clean. It's how they measure our efficiency. If we only see sound vehicles we would be judged to be inefficient, whereas if I find fault 1111■ with two out of two Fm 1(10% efficient."
As a result, he says, examiners sometimes merely glance at a vehicle and, if it looks good. send it on its way without properly examining it. This protects their own efficiency rating but does not help the operator's coding under OCRS:-If we do a spot check on a vehicle and it has no defects, we register it on the system and the operator gets a green light. But what some examiners do is walk around the vehicle without tapping anything, have a quick look and then pass them out if they look sound. That's a tool we use to stop the good vehicles registering on our slats.
Bill confirms that this can happen. although again he says good management can help to curb it: "Not recording clear vehicles distorts the data. Unfortunately it can take a lot of management time to drill down into the data to spot possible discrepancies like this. For instance, you might see a vehicle turn up in the traffic examiner's notebook but not the vehicle examiner's."
The Vosa mission
Vosa's mission statement commits it to leading the way on road safety, relying on deterrents, education and sanctions in that order. For existing and familiar road safety laws it is understandable that Vosa should be motivated to emphasise the sanction and allow this to operate as a deterrent for others. As Bill says: "When someone has deliberately disconnected a brake on a 44-tonner or has a 19in TV in his windscreen, they deserve what they get."
However, evidence that managers favour sanctions over education in the enforcement of new laws would be more worrying. CM has been shown an e-mail sent by an area manager to his team under the heading "Pegs galore". It briefs them on the next phase of speedlimiter regulations and exhorts them to pursue enforcement action if they have properly identified the vehicle. But it adds: -Once you become familiar with the new requirements it will be like shelling peas when it comes to the number of prohibitions issued.
The e-mail tells Vosa staff about the new law, but does not mention Vosa's role in educating operators despite Vosa's stated policy of "improving opera tor and driver understanding of the rules and regulations so as tostop offences being committed through driver ignorance or misunderstanding".
We asked Vosa if this e-mail reflected some managers' interpretation of the efficiency drive as directing resources towards accumulating prohibitions rather than operator education. It replied: "Vosa will he applying an educational approach to speed-limiter enforcement with operators of vehicles now falling into scope.
"Enforcement action will only be taken where it is clear that previous advice has not been taken or where there is clear evidence of abuse for example, tampering with systems after they have been correctly installed and set. It is expected that this approach will remain until January 2008 when the full requirements of the changes to Construction and Use [regulations] come into effect."
The fact remains that taking time to stop and educate drivers of CVs who do not yet comply with the law without taking enforcement action could place a strain on the resources of examiners with prohibition targets to meet.
Vosa's recent employee survey showed that its staff care deeply about their role in maintaining road safety and feel their work is vital; 71% say they are clear about the purpose of their job, and 78% understand how their work contributes to Vosa's overall objectives.
Work in progress Everyone CM has spoken to confirms that view. But the system they are asked to work with might need revision. Bill says: "I'm confident that we have an 80% prohibition rate because we're very good at targeting. But this whole scoring system is in its infancy so there's a lot more thrashing out to be done.Overall it's probably better than what we had in the past, a step in the right direction. But you need to manage it well."
Paul is less confident about the effects of the targets: "The general view of colleagues now is that we've lost sight of road safety and we just focus on prohibitions because they get us points. If I've spent all day at a roadside check and haven't issued any prohibitions I'll think to myself 'I've not been very productive today' because that's how they [my managers] will perceive it.That's what they want from us," •