OB35 Paper 06 KPI Report

Transcription

1 OB35 Paper 06 KPI Report KPI and SLA regime: January 2016 performance summary Ref Nov 15 Dec 15 Jan 16 Target Description KPI A 99.90% 100% 99.87% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A % 99.39% 99.32% 97.5% green Engineer visits completed as scheduled with viewer 95.0% amber KPI A % 98.74% 97.01% 90% green Engineer visits completed within 3 working days 50% amber KPI A % 98.92% 97.39% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A % 100% 100% Communal engineer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 100% 100% 100% 98% green 94% amber Reactive filters issued within 3 working days [KPI A10] SLA B1 100% 100% 100% 100% green 99% amber [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation Complaints SLA C1 0.42% 0.41% 0.41% <1% green <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 case of interference Qualitative and quantitative reports provided. No specific Measure of the number of reported cases that occur targets. outside of mailed area SLA D3 Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) Page 1 of 9

2 KPIs January 2016: KPI and SLA performance summary, 10 February 2016 KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 99.87%; green Of the 767 confirmed 4G cases in January 2016, 766 were resolved within 10 working days. One cases breached this deadline, taking 14 working days to resolve Result: 99.87% In the breach case, the viewer called and a visit was booked. Though the engineer attended the job as scheduled, adverse weather coupled with hazardous on-site conditions meant that the job needed to be rescheduled. Two further visits were booked and attended, but both had to be rescheduled owing to adverse weather. A further visit was booked, and this was completed successfully, 14 working days after the viewer s initial call. The viewer is happy with the reason for the delay and appreciates at800 s ongoing contact. Although the case began in December, the 10 working day limit was hit in early January. KPI A1: Engineer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.32%; green Of the 1,902 engineer visits originally scheduled for January 2016: o A total of 1,773 visits were undertaken and closed on schedule o Thirteen visits were cancelled or rearranged by at800 o A further 116 visits were cancelled by the viewer Total scheduled visits not missed by engineer / total visits scheduled (1, ) / 1,902 Result: 99.32% KPI A2: Engineer visits completed within 3 working days. 90% (green); 50% (amber) Result: 97.01%; green Of the 1,902 scheduled engineer visits detailed above, there were 1,786 visits scheduled where the viewer did not cancel Of these 1,786 visits: o A total of 1,617 visits were completed within three working days o A further 117 were delayed beyond three working days at the viewer s request o Thirteen visits were cancelled or rearranged by at800 o A total of 39 visits were completed outside of the three working day target (1, ) / 1,786 Result: 97.01% KPIs Page 2 of 9

3 January 2016: KPI and SLA performance summary, 10 February 2016 KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 97.39%; green In January 2016, there were 919 scheduled engineer visits to people marked as eligible for additional support where the viewer did not cancel Of the 919 visits to be measured: o A total of 853 visits were completed within three working days o Forty-two visits were delayed beyond the three working day window at the viewer s request o Nine visits were cancelled by the engineer o Fifteen visits were delayed beyond the three working day target ( ) / 919 Result: 97.39% KPI A6: Communal engineer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only Result: 100% There were six communal jobs scheduled for January, all of which were undertaken on time and as scheduled Result: 100%. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber) Result: 100%; green All of the 2,716 filter requests made in January 2016 were sent within one working day. Result: 100% KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter Not applicable. KPIs Page 3 of 9

4 SLA B Awareness January 2016: KPI and SLA performance summary, 10 February 2016 SLA B1: Identifiable addresses within forecast medium, high and very high-risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation. 100% (green); 99% (amber) Result: 100%; green A request was made to activate some of the masts associated with the February 2016 rollout early. The 15,626 households affected by these early activations were mailed in December and were detailed in last month s report The remaining 79,017 households eligible for mailing because of February 2016 masts were mailed during the period 4 6 January and are detailed under SLA B2 below A total of 137,466 households were identified as being eligible for mailing because of March 2016 rollout masts. These were mailed during the period January and are detailed under SLA B2 below Result: 100% SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing February 2016 masts March 2016 masts Mailing Date Very high High Medium London medium Low Grand Total 4 January 1,666 7,984 20,350 30,000 5 January 1,157 6,449 22,392 29,998 6 January 681 5,814 12,524 19,019 Not mailed , , January 2,034 10,460 17,951 30, January 1,435 8,787 19,776 29, January 1,019 7,036 21,945 30, January ,245 18,987 29, January 369 3,453 13,204 17,026 Not mailed , ,623 Total 9,126 60, , , ,398 SLA B: Awareness Page 4 of 9

5 January 2016: KPI and SLA performance summary, 10 February 2016 SLA B3: Report on households reporting interference within unmailed forecast pixels. Where more than 40% of total reports of interference are in these pixels at800 to address through operational adjustments (e.g. mailing low risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to apply In January 2016, at800 was contacted by a total of 3,717 households reporting interference that at800 believed could be caused by 4G. Engineers visited 1,739 properties, and filters were sent to a further 1,978. Of the 1,739 viewers to whom at800 sent engineers, 742 were confirmed as being impacted by 4G Of the 3,717 households, 344 (9.3%) were suppressed through the risk-based mailing. Mailing Status 4G confirmed Not 4G Job not complete Filter sent Grand Total Mailed ,705 3,258 Suppressed Not identified Total ,978 3,717 SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled. Report on reminder mailing numbers to be provided The reminder mailing associated with inactive masts from the June 2015 rollout was undertaken in December That associated with inactive masts from the July 2015 rollout will be undertaken in early February There were no reminder mailings undertaken in January. SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas During January, seven pieces of media coverage communicated at800 s messaging to a potential audience of around two million people. Three articles online and four in print were published during the month. Examples of coverage are detailed below. The Scotsman Ben Roome, chief executive of at800, said: We want to ensure that people know it s our role to fix any disruption to Freeview due to 4G at 800 MHz and that there s no charge. The Tyrone Times The Council is now reinforcing the message that there is free support and equipment available to those affected from at800, the company responsible for resolving the problem. The Falkirk Herald The at800 organisation, funded by the leading UK mobile operators, is responsible for making sure viewers can still watch Freeview through their TV aerial. SLA B: Awareness Page 5 of 9

6 SLA C Quality of Service January 2016: KPI and SLA performance summary, 10 February 2016 SLA C1: Percentage of households interacting with at800 that make a complaint. 1% (green); 2% (amber) Result: 0.41%; green at800 interacted with a total of 6,134 households in January 2016 Of these, 25 made a complaint that related to at800 s operational processes Result: 0.41% SLA C4: Report on total number and categories of complaints There were a total of 57 complaints made in January 25 related to at800 s operational processes; 32 to its policies. Below is a breakdown of the topics of these complaints. Operations-related Engineer: damage/quality: 7 Mailing and communication: 7 Dispute of non-4g diagnosis: 4 Appointment availability: 2 Engineer: late/missed appointment: 2 Engineer attitude/behaviour: 1 Filter-related: 1 Time to resolve issue: 1. Policy-related Support of satellite or cable viewers: 18 Support of communal properties: 7 4G rollout in general: 6 Support of set-top aerials: 1. SLA C5: Vulnerable viewer complaints Out of 1,522 viewers flagged as being eligible for additional support in January 2016, 16 made a complaint: 11 related to operational processes and five to policies. These break down as follows: Operations-related Engineer: damage/quality: 4 Mailing and communication: 3 Appointment availability: 2 Dispute of non-4g diagnosis: 1 Engineer: late or missed appointment: 1. Policy-related Support of communal properties: 2 Support of satellite or cable viewers: 2 4G rollout in general: 1. SLA C: Quality of Service Page 6 of 9

7 SLA D Scale January 2016: KPI and SLA performance summary, 10 February 2016 SLA D1: Measure of the numbers of mailings per report of/case of interference The table shows mailings per engineer visit and reactive filter sent, broken down by masts activated before the end of January Each case is tied to a single mast according to the rules specified in Appendix 1. Cases are associated to the month when each mast was originally scheduled and therefore drove mailing volumes, not the month in which activation occurred. Rollout month Engineer, 4G Engineer, non-4g Reactive filter Initial mailings Mailings per 4G Mailings per non-4g Mailings per reactive filter Mailings per visit or filter sent July ,315 8,176 20,479 7,205,414 2, August , September , October , November ,946 1, December ,043 1, January ,796 2,988 1, February ,943 1, March , April , May , June , July ,157 2, , August , September , , October , , November ,152 1,009 2, , December ,536 2,671 1, January , February , March , , April , , May ,001 1, June , July , , August , , September , , October , , November , , December ,052 1,652 2,134 1, January ,254 5,710 9,602 2,374 1,903 February ,943 31,648 31,648 11,868 8,631 Nothing within 2km ,450 Total 12,368 19,473 30,959 5,983, Note: Data is included for January and February 2016 masts as some of these masts were activated before the end of December 2015 but after mailing had completed. SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 742 households confirmed as experiencing 4G interference in January 2016: Cases not mailed due to risk profile 59 (8.0%) Cases never identified in modelling 28 (3.8%) Total cases outside mailed area 87 (11.7%) Total cases caught by mailing 655 (88.3%) SLA D: Scale Page 7 of 9

8 January 2016: KPI and SLA performance summary, 10 February 2016 SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action 1.80 Cumulative number of cases per active mast Jul 13 Aug 13 Sep 13 Oct 13 Nov Dec Jan Feb Mar Apr May Jun Jul 14 Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul 15 Aug Sep Monthly Cases Per Mast Range Lower Limit Range Upper Limit Poly. (Monthly Cases Per Mast) Oct 15 Nov 15 Dec 15 Jan 16 Note: The above thresholds are based on a range of between 5,000 and 50,000 cases of interference for an estimated number of total 4G at 800 MHz masts for full rollout from all MNOs. SLA D: Scale Page 8 of 9

9 January 2016: KPI and SLA performance summary, 10 February 2016 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with at800 a number of times (either requesting multiple filters or receiving many engineer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their contacts with at800. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had engineer visits. The few households that have received both count in both analyses. For households to which at800 has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For engineer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 9 of 9