I have recently been thinking about the sorts of performance measures that get used in the housing sector and the way that IT systems are used to generate management information. I have come to the conclusion that much of that management information probably at best represents something different to what most senior managers and board members believe it means, and at worst is virtually meaningless.
What is an appointment?
For example, I am not convinced that the traditional measures around appointments add any real value. In order to make these measurements meaningful, we need to agree clear definitions of exactly what we mean by an appointment, by a visit and what is or is not an appointable repair. Then having reached that agreement, we mustn’t allow things like league tables to convince us that we should soften or change those definitions in order to get into the top quartile of some external benchmarking exercise.
Those of us who have worked in IT for a while are probably more used to the idea of having clear definitions for entities and attributes. I have had long conversations with groups of users who believe that they are all agreed that the organisation needs to store information on appointments, for example. It is only when we start to map out the data structures and build some sort of data model does it become clear that everyone in the room has a slightly different definition of what an appointment is.
To illustrate this point, I have produced four short definitions of entities that will probably be familiar to most people involved in producing management statistics for a housing repairs service. I’m sure that all of you will disagree to some extent with one or more of these definitions:
Appointment
This is an agreed time slot when some work will take place. The agreement can be with a tenant, leaseholder or member of staff. The time slot can be of any duration; it can be two hours, eight hours, one week or even a month. For internal jobs where a tenant is waiting at home, then a two, four or eight hour appointment slot would probably be appropriate. But for external jobs such as repairing a broken light in a communal area or fixing a broken paving stone then a week or even a month may be an acceptable timeslot. An appointment does not relate to a specific employee. And provided that a suitably-skilled person arrives at a property within the agreed timeslot then the appointment is considered to have been kept.
Visit
This is a member of staff from your organisation arriving at a property within the pre-agreed time slot. The duration of a visit will be dependent on what the staff member finds when he or she arrives on site. If the tenant had originally been told that the work would take two hours and it quickly becomes clear that it will take much more or less time than expected, then a change to the planned duration of the visit should be agreed with the tenant. It does not necessarily mean that a second visit or appointment has to be made. If the staff member leaves the site for any reason (for example to go for lunch, to borrow a specialist tool from the back of a colleague’s van or pick-up materials from a store), then provided that the customer is informed why they are leaving and when they will be back, that would be considered to be one visit, even if that visit is the next working day.
Job
This is all the work carried out during the visit, with some of the work having been defined in advance; for example, a plumber may be sent to a property to fix a leaking tap in the bathroom. However, if the plumber was also asked to look at a leaking U-bend in the kitchen while they were on site or if he/she spotted a problem with the toilet cistern and fixed it, then that would also be considered to be part of the same job. If the plumber spotted a problem that he/she couldn’t fix either because they didn’t have the necessary skills (for example, a problem with the electrical circuit of the central heating system) or because it needed a non-stock item like a new bath tub, then this would probably be considered a different job and a different visit.
Appointable Job
This is a job for which it is appropriate to agree with the customer a time slot in which the work will take place. Given that the customer can be a tenant, leaseholder or another member of staff and that the duration can be anything from two hours to a month, then I would expect that almost any job could be considered an appointable job. The job could be reactive, for example reported by a tenant or a member of staff in your estates management team, or it could be cyclical such as gas servicing or lift maintenance. In either case, it is appropriate to agree with someone a timeslot when the work is planned to take place and so according to my definitions, almost all (if not all) jobs are appointable.
The reason I am sure that most of the readers will disagree to some extent with my definitions is that if we use these definitions then many of the traditional measures we have used in the housing sector become almost meaningless.
For example, appointments made as a percentage of appointable repairs; I think this measure adds very little value for two reasons. First, if you run this report at the end of every month, it’s too late to do anything about it. It would be much better to run a report every morning on the appointments missed the previous day. That way, there may still be time to get back to the tenant or customer and try to quickly sort out the problem. If we run a report for February and then sit down to discuss it at a meeting in March, it could already be five weeks after the tenant had their appointment missed.
My second reason for disliking this measure is that it is often used to present an unrealistically positive vision of a repairs service. For example, a repairs service may report that of all the repairs where it was appropriate or possible to book an appointment, they made and kept 99.5 per cent of appointments. But we don’t know how many repairs somebody decided it was not appropriate to make an appointment for. So we don’t really know how many of our customers are actually being offered a mutually-agreed appointment slot.
Another measure that I dislike is the percentage of total repairs completed within a target period. For me, the key thing is that you have agreed with the customer when you will do the work and that you keep your promise. This statistic is based on the view that different types of jobs are given different priorities. A dripping hot tap may have a 24-hour priority but a dripping cold tap may have a seven-day priority. In many cases, this leads to unhelpful behaviour. For example, just-in-time scheduling where, when someone reports a problem with a dripping cold tap, the work is scheduled in to happen in six days even if there is a plumber free the next day. The logic often used is that we need to leave some spare slots free tomorrow in case an emergency job comes in. So the cold tap goes unrepaired for six days just in case a hot tap somewhere starts to drip.
I think that every repair should be done at the first mutually-convenient time and it should be as simple as that. I am a big fan of Wrekin Housing Trust who treats all of its day-to-day repairs as emergencies. WHT’s website says, “We try to get out to you within two hours, and our standard is that we will complete the repair the same day, wherever it is possible to do so. We usually complete around 85 per cent of repairs the same day.”
Of course, IT professionals don’t just think about data in terms of entity definitions, attribute definitions and relationships. We also think about things such as entity life histories and this can also be a useful mental discipline to bring to discussions about management information. For example, let’s take the ‘number of dwellings without a valid gas safety certificate’ statistic. This is a helpful enough report and gas safety is very important. But the weakness of this report is that it is a snap-shot in time.
If the report is run on 30th January, it may well say that 100 per cent of properties have a valid gas certificate. But we don’t know if the same is true if we had run the same report on 9th January, 16th January or 23rd January. A more useful report would be one that listed the properties that had not had a valid gas certificate during the previous month and the number of days for which they did not have a valid gas certificate.
During any calendar month, I would expect the numbers to be low and that the number of days that a property was without a valid certificate to be very few. But, if we never see any properties that don’t have a valid gas certificate then that might suggest we are wasting money by doing gas safety tests too often. So perhaps once a year, it might be helpful to have a report that shows the average amount of time between gas certificates being issued and the longest and shortest period that a gas certificate was active. In an ideal world, all gas certificates would be active for exactly 365 days. But that isn’t practically possible, so we might see not only that the average gas certificate is active for 310 days before it is replaced by a newer certificate, but also that the shortest period a certificate was active for was just 20 days and the longest was active for 370 days.
This is a rather specific example, but the general point I am making is that sometimes a traditional analysis technique like thinking about entity life cycles can bring real value to an organisation when we look at management information and KPIs. IT professionals bring skills, experience and a unique perspective to high-level discussions about performance measurement and all housing providers would be well advised to consider involving them in discussions about what will be measured and how.
Chris Deery is head of ICT for Solihull Community Housing.
