Paper 06 OB20 KPI Report KPI and SLA regime: October 2014 performance summary Reference Outcome Result Target Description KPI A Green 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A1 Green 98.66% 97.5% green Installer visits completed as scheduled with viewer 95.0% amber KPI A2 Green 97.88% 90% green Installer visits completed within 3 working days 50% amber KPI A4 Green 97.75% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 100% Communal installer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 Green 100% 98% green Reactive filters issued within 3 working days 94% amber [KPI A10] [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters SLA B1 Red 97.19% 100% green 99% amber Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation SLA B2 Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) SLA B3 Report on households reporting interference within un-mailed forecast pixels SLA B4 Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled SLA B5 SLA B6 SLA B7 Qualitative and quantitative reports provided. No specific targets. Awareness to be maintained (reminder mailings) with those viewers at renewed risk of experiencing interference from infill masts where the addition of the mast increases the risk of interference in that pixel to very high Report on households mailed and not mailed reminders within forecast pixels (i.e. number of low risk households) Report on above the line comms and research outcomes, including awareness in rollout areas Complaints SLA C1 Green 0.37% <1% green <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / Qualitative and quantitative reports case of interference SLA D2 provided. No specific targets. Measure of the number of reported cases that occur outside of mailed area SLA D3 Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout)
KPIs KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All of the 390 confirmed 4G cases in October 2014 were resolved within 10 working days Green. KPI A1: Installer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 98.66%; green Of the 967 installer visits originally scheduled October 2014: o 883 were undertaken and closed on schedule o 13 visits were cancelled or missed by the installer o 71 visits were cancelled by the viewer Total scheduled visits not missed by installer / total visits scheduled (883 + 71) / (882 + 13 + 71) = 954 / 967 Result: 98.66% Green. KPI A2: Installer visits completed within 3 working days. 90% (green); 50% (amber) Result: 97.88%; green Of the 967 scheduled installer visits detailed above, there were 896 visits scheduled where the viewer did not cancel Of these 896 visits: o 794 were completed within three working days o 83 were delayed beyond three working days at the viewer s request o 13 visits were cancelled by the installer o Six were not completed within three working days (794 + 83) / 896 Result: 97.88% Green. KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 97.75%; green In October 2014, there were 222 scheduled installer visits to people marked as vulnerable where the viewer did not cancel Of the 222 visits to be measured: o 205 were completed within three working days o 12 were delayed beyond the three-day window at the viewer s request o Two visits were cancelled by the installer Page 2 of 10
o Three visits were delayed beyond three working days (205 + 12) / 222 Result: 97.75% Green. KPI A6: Communal installer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only. Result: 100% Of the 10 communal jobs in the month of October 2014: o Nine were completed as agreed with the customer o One was cancelled by the viewer or landlord Therefore all 10 were completed as scheduled with the viewer (or cancelled by the viewer) giving a success rate of 100%. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber). Result: 100% There were 2,219 filter requests in October 2014 All of these were sent within one working day Green. KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter. Not applicable. Page 3 of 10
SLA B Awareness SLA B1: Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation. 100% (green); 99% (amber). Result: 97.19%; red A total of 551,523 households were identified as being eligible for mailing because of November rollout masts. Of these: o 131,465 households were mailed 8 October o An operational error meant that a further 420,058 households associated with November masts were identified on 10 November as having not received a mailing. These households had already been identified in a previous rollout as being low risk, but November s rollout submission increased their risk of interference such that they became eligible for a mailing. Of these: 15,492 were associated with November rollout masts that have already been activated, and these were mailed first class on 11 November A further 29,034 households, also mailed first class on 11 November, are associated with November rollout masts that the operators are planning to activate within weeks of 10 and 17 November but that were not active by the time the postcards landed on 12 November 375,532 are associated with November rollout masts that the operators have not yet planned to activate. These were mailed between 14 and 19 November to profile the resulting inbound calls to the call centre All 296,076 households identified for mailing relating to December masts were mailed 23 October Result: 97.19% Red. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing Date Very high High Medium London Medium Low Grand Total 8 October 703 52,169 78,593 131,465 23 October 11,288 88,700 196,088 296,076 11 November 436 22,459 21,631 44,526 14 November 50,000 15 November 5,573 138,828 231,131 75,532 17 November See note below 100,000 18 November 100,000 19 November 50,000 Null 17,935 1,248,602 1,266,537 Grand Total 11,991 140,869 274,681 17,935 1,248,602 1,694,078 Page 4 of 10
Note: As detailed in SLA B1 above, a total of 375,532 households were mailed between 14 and 19 November. While the risk profile of these is shown in the table above, the risk profile of each day s mailings will not be known until after this paper has been issued. SLA B3: Report on households reporting interference within un-mailed forecast pixels. More than 40% of total reports of interference DMSL to address through operational adjustments (e.g. mailing low risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to apply. In October 2014, DMSL was contacted by a total of 2,764 households reporting interference that DMSL believed could be caused by 4G. Installers visited 860 properties, and filters were sent to a further 1,904. Of the 860 viewers to whom we sent installers, 390 were confirmed as being impacted by 4G Of the 2,764 households, 320 (11.6%) were suppressed through the risk-based mailing. Mailing Status 4G Filter Grand Not 4G confirmed sent Total Mailed 264 291 389 944 Suppressed 83 108 129 320 Not identified 43 71 1,386 1,500 Total 390 470 1,904 2,764 SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled. Report on reminder mailing numbers to be provided. A reminder mailing was undertaken on 1 October to 3,344 households forecast to be affected by masts originally expected to activate in April 2014 A further 204 households associated with a single mast in Wallingford were sent a reminder mailing on 29 October The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing Date Very High High Medium London Medium Low Grand Total 1 October 1 1,656 1,687 3,344 29 October 105 99 204 Not mailed 25,988 18,680 44,668 Total 1 1,656 1,687 25,988 28,373 48,012 SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas The proactive PR work carried out during October continued to integrate social media activity with traditional press relations work. Local influencers in the targeted areas were sent tweets from the @at800tv Twitter account, and were asked to retweet to their followers to create some local awareness before the news release was sold in over the phone to journalists. Page 5 of 10
National residential organisations were also targeted with a special press release that generated two pieces of coverage. This approach was effective in securing 12 individual pieces of media coverage in the nine areas targeted communicating at800 s key messages, with a total potential reach of 236,604 people. Date Publication Title 9 October Derby Telegraph Freeview TV signal could be disrupted due to new 4G mobile service 9 October Burton Mail New mobile masts could cause television issues in Burton 9 October East Anglian Daily Times Clacton: Warning that 4G mobile phones could disrupt TV signals 10 October Burton Mail Freeview customers may face signal disturbance 10 October East Anglian Daily Times Rick of TV disruption by 4G mobile phone masts(print) 15 October East Lindsey Target Firm drafted in as 4G goes live (print) 15 October Lichfield Mercury Fears 4G mast could disrupt Freeview TV in Lichfield - but help is at hand 19 October Skegness Standard Help at hand if new masts disrupt TV 19 October Clacton & Frinton Gazette 22 October RLA Landlord Newshub 4G mobile broadband could hit TV signals Support on offer to landlords should new 4G masts disrupt TV 22 October ARMA 4G disruption (print) 29 October Folkestone Herald 4G coming to Folkestone - but it could hurt your television signal Adverts ran on Facebook to support the direct marketing and PR work as a mast was activated in Wallingford, Oxfordshire. The adverts targeted addresses registered in the area, and ran from 30 October until 6 November. There are approximately 4,600 people in this area on Facebook. By 3 November, 54 had been spent on the adverts, which had created 3,199 opportunities to see, generating two comments, two shares and 102 click-throughs to the at800 website. Page 6 of 10
SLA C Quality of Service SLA C1: Percentage of households interacting with DMSL that make a complaint. 1% (green); 2% (amber). Result: 0.37% We interacted with a total of 6,454 households in October 2014 Of these, 24 (0.37%) made a complaint Green. SLA C4: Report on total number and categories of complaints Below is a breakdown of the 24 complaints received in October 2014: Compensation: 13 Diagnostics over the phone: 4 Communal process: 3 Rollout in general: 2 Non-4G issue that we cannot help with: 1 Installer-related: 1. [SLA C5:] Vulnerable viewer complaints Out of 926 viewers flagged as being vulnerable with which we interacted in October 2014, one made a complaint, which was related to the communal process. Page 7 of 10
SLA D Scale SLA D1: Measure of the numbers of mailings per report of / case of interference The following table shows the number of mailings per installer visit (4G and non-4g) and reactive filter sent, broken down by mast activated until the end of October 2014. Each case is tied to a single mast according to the rules specified in Appendix 1. Installer, Reactive Initial Mailings Mailings Mailings per reactive Mailings Rollout Month non-4g Installer, 4G filter mailings per non-4g per 4G filter per case Jul 2013 6,028 2,499 15,811 7,205,414 1,195 2,883 456 296 Aug 2013 219 227 554 153,875 703 678 278 154 Sep 2013 242 241 505 183,179 757 760 363 185 Oct 2013 103 77 220 117,425 1,140 1,525 534 294 Nov 2013 42 59 141 129,946 3,094 2,202 922 537 Dec 2013 43 17 115 103,043 2,396 6,061 896 589 Jan 2014 60 38 171 534,796 8,913 14,074 3,127 1,988 Feb 2014 101 80 225 128,943 1,277 1,612 573 318 Mar 2014 10 7 24 5,727 573 818 239 140 Apr 2014 7 2 44 4,133 590 2,067 94 78 May 2014 5 4 17 4,606 921 1,152 271 177 Jun 2014 17 33 62 94,100 5,535 2,852 1,518 840 Jul 2014 276 195 760 320,204 1,160 1,642 421 260 Aug 2014 43 40 116 64,130 1,491 1,603 553 322 Sep 2014 68 63 289 142,452 2,095 2,261 493 339 Oct 2014 23 11 65 343,052 14,915 31,187 5,278 3,465 Nov 2014 10 170 172 136,107 13,611 801 791 387 Dec 2014 4 7 41 296,536 74,134 42,362 7,233 5,703 Nothing within 2km 375 78 2,035 Total 7,676 3,848 21,367 9,967,668 1,299 2,590 466 303 Note: Data is included for November and December 2014 masts as some of these masts were activated out of sequence. SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 390 households identified as experiencing 4G interference in October 2014: Cases not mailed due to risk profile or low pop in pixel 1 (0.26%) Cases never identified in modelling 7 (1.79%) Total cases outside mailed area 8 (2.05%) Total cases caught by mailing 382 (97.95%) SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action. Page 8 of 10
1.80 1.60 Cumulative number of cases per activated mast 1.66 1.40 1.20 1.00 0.80 0.60 0.51 0.57 0.64 0.68 0.74 0.78 0.79 0.83 0.87 0.92 0.40 0.20 0.09 0.09 0.16 0.22 0.30 0.38 0.17 - Cumulative Cases Per Mast Range Lower Limit Range Upper Limit Linear (Cumulative Cases Per Mast) Note: The above thresholds are based on a total mast count of 50,000. Page 9 of 10
Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with us a number of times (either requesting multiple filters or receiving many installer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their interactions with us. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had installer visits. The few households that have received both count in both analyses. For households to which DMSL has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For installer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), then the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 10 of 10