OB31 Paper 07 KPI Report (September) KPI and SLA regime: September 2015 performance summary Ref Jul 15 Aug 15 Sep 15 Target Description KPI A 100% 99.87% 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A1 99.20% 98.49% 99.79% 97.5% green Engineer visits completed as scheduled with viewer 95.0% amber KPI A2 97.16% 94.64% 96.78% 90% green Engineer visits completed within 3 working days 50% amber KPI A4 97.01% 94.25% 96.34% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 100% 100% 100% Communal engineer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 98.39% 92.24% 100% 98% green Reactive filters issued within 3 working days 94% amber [KPI A10] [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters SLA B1 60.5% 86.7% 100% 100% green 99% amber Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation SLA C1 0.14% 0.21% 0.40% <1% green Complaints <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 SLA D3 Qualitative and quantitative reports provided. No specific targets. case of interference Measure of the number of reported cases that occur outside of mailed area Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) Page 1 of 10
KPIs September 2015: KPI and SLA performance summary, 12 October 2015 KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All of the 852 confirmed 4G cases in September 2015 were resolved within 10 working days Result: 100% Green. KPI A1: Engineer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.79%; green Of the 2,373 engineer visits originally scheduled for September 2015: o A total of 2,200 visits were undertaken and closed on schedule o Five visits were cancelled and rearranged by at800 to meet capacity constraints o A further 168 visits were cancelled by the viewer Total scheduled visits not missed by engineer / total visits scheduled (2,200 + 168) / 2,373 Result: 99.79% Green. KPI A2: Engineer visits completed within 3 working days. 90% (green); 50% (amber) Result: 96.78%; green Of the 2,373 scheduled engineer visits detailed above, there were 2,205 visits scheduled where the viewer did not cancel Of these 2,205 visits: o A total of 1,960 visits were completed within three working days o A further 174 were delayed beyond three working days at the viewer s request o Five visits were cancelled or rejected by the engineer o Sixty-six visits were completed outside of the three working day target (1,960 + 174) / 2,205 Result: 96.78% Green. KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 96.34%; green In September 2015, there were 1,174 scheduled engineer visits to people marked as vulnerable where the viewer did not cancel Of the 1,174 visits to be measured: o A total of 1,078 visits were completed within three working days o Fifty-three visits were delayed beyond the three working day window at the viewer s request o Twelve visits were cancelled or rejected by the engineer KPIs Page 2 of 10
September 2015: KPI and SLA performance summary, 12 October 2015 o Thirty-one visits were delayed beyond the three working day target (1,078 + 53) / 1,174 Result: 96.34% Green. KPI A6: Communal engineer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only Result: 100%; green All four of the communal jobs that were scheduled to take place in September 2015 were undertaken on time and as scheduled. Result: 100% Green. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber) Result: 100%; green There were 4,603 filter requests in September 2015, all of which were sent within one working day Result: 100% Green. KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter Not applicable. KPIs Page 3 of 10
SLA B Awareness September 2015: KPI and SLA performance summary, 12 October 2015 SLA B1: Identifiable addresses within forecast medium, high and very high-risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation. 100% (green); 99% (amber) Result: 100%; green A total of 205,511 households were identified as being eligible for mailing because of October 2015 rollout masts. These were mailed during the period 1 11 September and are reported under SLA B2 below A risk trial missed households mailing was sent to 267,694 households during the period 14 29 September. For 125,080 of these households, masts had already been activated in their area. These households received a tailored communication to this effect, and were mailed in the first week of the mailing. The remaining 142,614 households were sent a regular postcard mailing and were mailed in the second and third weeks. These are reported under SLA B2 below Green. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing Mailing Date Very high High Medium London medium Low Grand Total 1 September 4,435 2,371 3,194 10,000 2 September 14,320 6,668 8,750 29,738 3 September 5,985 8,390 15,625 30,000 7 September 8,319 6,958 14,721 29,998 October 8 September 7,561 6,869 15,570 30,000 2015 9 September 3,386 7,063 19,551 30,000 10 September 8,588 10,079 11,330 29,997 11 September 3,731 3,642 8,405 15,778 Not mailed 765 169,251 170,016 Risk trial 14 September 151 9,818 15,031 25,000 missed 15 September 2 7,673 17,325 25,000 households 16 September 65 6,650 18,285 25,000 special 17 September 59 8,121 16,819 24,999 mailing* 18 September 214 13,356 11,511 25,081 Risk trial 21 September 304 10,365 19,603 30,272 missed 22 September 111 5,971 24,223 30,305 households 23 September 650 4,298 25,051 29,999 rollout 28 September 155 8,890 21,728 30,773 mailing* 29 September 135 3,186 17,944 21,265 Total 58,171 130,368 284,666 765 169,251 643,221 * By definition, there are no unmailed risk trial households. SLA B: Awareness Page 4 of 10
September 2015: KPI and SLA performance summary, 12 October 2015 SLA B3: Report on households reporting interference within unmailed forecast pixels. Where more than 40% of total reports of interference are in these pixels at800 to address through operational adjustments (e.g. mailing low risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to apply In September 2015, at800 was contacted by a total of 5,143 households reporting interference that at800 believed could be caused by 4G. Engineers visited 2,172 properties, and filters were sent to a further 2,971. Of the 2,172 viewers to whom at800 sent engineers, 848 were confirmed as being impacted by 4G Of the 5,143 households, 313 (6.1%) were suppressed through the risk-based mailing. Mailing Status 4G confirmed Not 4G Job not complete Filter sent Grand Total Mailed 811 1,225 15 2,684 4,735 Suppressed 34 78 0 201 313 Not identified 3 6 0 86 95 Total 848 1,309 15 2,971 5,143 SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled. Report on reminder mailing numbers to be provided A reminder mailing to 20,924 households forecast to be affected by masts originally expected to activate in February 2015 was undertaken on 1 September 2015. This is detailed in the table below A reminder mailing to 44,527 households forecast to be affected by masts originally expected to activate in March 2015 was undertaken during the period 15 17 September 2015. This is detailed in the table below. Mailing Mailing Date Very high High Medium London medium Low Grand Total Feb 2015 1 September 443 8,830 11,651 20,924 reminder Not mailed 23,473 49,374 72,847 15 September 399 5,063 9,068 14,530 March 16 September 370 6,812 7,818 15,000 2015 17 September 150 6,562 8,285 14,997 reminder Not mailed 9,718 50,392 60,110 Total 1,362 27,267 36,822 33,191 99,766 198,408 SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas The pause on proactive PR activity that continued through September meant coverage levels were significantly lower than in previous months. During the month, three online articles and one in print communicated at800 s messaging to a potential audience of around 416,000 people. Some examples of coverage are detailed below. SLA B: Awareness Page 5 of 10
September 2015: KPI and SLA performance summary, 12 October 2015 North Belfast News Bridget Mateer and Cathy Anderson are delighted [ ] The recent activation of a 4G mast in the area had caused interference with the signal, but a filter fitted by engineers of at800, a group set up to monitor Freeview disruption, means that the local women can now kick back and enjoy their favourite programmes. Business Reporter at800 has been created to ensure that UK viewers that rely on Freeview for TV can still receive it if 4G signals cause problems. To support these viewers, at800 can arrange for an engineer visit free of charge, or send free at800-approved filters for viewers to fit, to restore Freeview reception. at800 s engineers have already visited more than 18,000 homes and resolved more than 7,000 cases of 4G disruption. Antrim Guardian at800 are able to solve this problem by sending an engineer who will do the work for free if he can confirm that the problem is caused by 4G interference. This information may be helpful to others SLA B: Awareness Page 6 of 10
SLA C Quality of Service September 2015: KPI and SLA performance summary, 12 October 2015 SLA C1: Percentage of households interacting with at800 that make a complaint. 1% (green); 2% (amber) Result: 0.40%; green at800 interacted with a total of 8,569 households in September 2015 Of these, 34 made a complaint that related to at800 s operational processes Result: 0.40% Green. SLA C4: Report on total number and categories of complaints There were a total of 57 complaints made in September 34 related to at800 s operational processes; 23 to its policies. Below is a breakdown of the topics of these complaints. Operations-related Mailing and communication: 9 Dispute of non-4g diagnosis: 8 Engineer: damage/quality: 7 Engineer: late/missed appointment: 3 Engineer: attitude/behaviour: 2 Appointment availability: 2 Contact centre agent: 2 Other: 1. Policy-related Support of satellite or cable viewers: 10 Support of communal properties: 8 4G rollout in general: 5. SLA C5: Vulnerable viewer complaints Out of 2,197 viewers flagged as being eligible for additional support with whom at800 interacted in September 2015, 20 made a complaint, 16 of which related to at800 s operational processes and four to its policies. These are broken down as follows: Operations-related Engineer: damage/quality: 5 Mailing and communication: 4 Dispute of non-4g diagnosis: 3 Engineer: attitude/behaviour: 2 Contact centre agent: 1 Other: 1. Policy-related Support of communal properties: 2 4G rollout in general: 2. SLA C: Quality of Service Page 7 of 10
SLA D Scale September 2015: KPI and SLA performance summary, 12 October 2015 SLA D1: Measure of the numbers of mailings per report of/case of interference The following table shows the number of mailings per engineer visit (4G and non-4g) and reactive filter sent, broken down by mast activated before the end of September 2015. Each case is tied to a single mast according to the rules specified in Appendix 1. Note that the cases are tied to the month in which the mast was scheduled to be activated (and therefore the masts that drove the mailing volumes); not the month in which the mast was activated. Engineer, Engineer, Reactive Initial Mailings Mailings Mailings per Mailings per visit or filter Rollout month 4G non-4g filter mailings per 4G per non-4g reactive filter sent July 2013 5,119 7,735 19,648 7,205,414 2,403 948 373 268 August 2013 305 331 803 153,875 522 481 198 140 September 2013 312 344 751 183,179 601 547 252 172 October 2013 114 144 397 117,425 955 778 304 219 November 2013 96 102 278 129,946 1,428 1,354 489 359 December 2013 39 79 212 103,043 2,862 1,338 515 372 January 2014 139 250 541 534,796 4,178 2,305 1,061 727 February 2014 108 141 323 128,943 1,264 977 427 297 March 2014 18 29 52 5,727 382 229 133 84 April 2014 6 19 81 4,133 689 243 52 43 May 2014 7 10 35 4,606 658 461 144 110 June 2014 157 191 490 94,100 663 592 233 167 July 2014 575 937 2,509 320,204 577 360 135 98 August 2014 192 261 696 64,130 345 266 99 72 September 2014 414 618 1,768 142,452 367 260 88 66 October 2014 511 564 1,258 343,052 719 658 310 211 November 2014 851 626 1,962 556,165 709 1,049 323 247 December 2014 88 137 340 296,536 3,902 2,965 1,098 801 January 2015 166 194 393 187,594 1,312 1,187 565 383 February 2015 173 226 412 95,345 691 571 332 210 March 2015 280 297 653 172,228 772 629 345 223 April 2015 241 308 634 244,807 1,139 956 386 275 May 2015 54 73 169 116,001 3,053 2,320 686 530 June 2015 136 152 339 296,835 2,604 2,968 876 676 July 2015 185 192 560 314,797 2,315 2,385 562 455 August 2015 165 107 436 205,731 1,400 2,312 472 392 September 2015 75 59 279 288,602 11,544 10,307 1,034 940 October 2015 23 19 48 205,511 N/A N/A 4,281 4,281 Nothing within 2km 316 816 6,658 Total 8,721 15,089 39,749 12,309,666 1,411 816 310 224 Note: Data is included for October 2015 masts as some of these masts were activated before the end of September 2015 but after mailing had completed. SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 848 households confirmed as experiencing 4G interference in September 2015: Cases not mailed due to risk profile 34 (4.0%) Cases never identified in modelling 3 (0.4%) Total cases outside mailed area 37 (4.4%) Total cases caught by mailing 811 (95.6%) SLA D: Scale Page 8 of 10
September 2015: KPI and SLA performance summary, 12 October 2015 SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action 1.80 Cumulative number of cases per active mast 1.66 1.60 1.40 1.20 1.00 0.80 0.69 0.73 0.79 0.81 0.82 0.85 0.88 0.93 0.95 1.00 1.02 1.03 1.02 1.03 1.06 1.07 1.10 1.15 1.18 0.60 0.55 0.61 0.40 0.33 0.41 0.20 0.17 0.11 0.11 0.19 0.25 - Jul 13 Aug 13Sep 13Oct 13 Nov Dec 13 Jan 14 Feb 14 Mar Apr 14 May Jun 14 Jul 14 Aug 14Sep 14Oct 14 Nov Dec 14 Jan 15 Feb 15 Mar Apr 15 May Jun 15 Jul 15 Aug 15Sep 15 13 14 14 14 15 15 Monthly Cases Per Mast Range Lower Limit Range Upper Limit Poly. (Monthly Cases Per Mast) Note: The above thresholds are based on a range of between 5,000 and 50,000 cases of interference for an estimated number of total 4G at 800 MHz masts for full rollout from all MNOs. SLA D: Scale Page 9 of 10
September 2015: KPI and SLA performance summary, 12 October 2015 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with at800 a number of times (either requesting multiple filters or receiving many engineer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their contacts with at800. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had engineer visits. The few households that have received both count in both analyses. For households to which at800 has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For engineer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 10 of 10