KPI and SLA regime: November 2016 performance summary

Transcription

1 KPI and SLA regime: November 2016 performance summary OB46 Paper 06a KPI Report Ref Sep 16 Oct 16 Nov 16 Target Description KPI A 100% 100% 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A % 99.74% 99.61% 97.5% green Engineer visits completed as scheduled with viewer 95.0% amber KPI A % 95.33% 96.63% 90% green Engineer visits completed within 3 working days 50% amber KPI A % 95.97% 96.42% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 100% 100% 100% Communal engineer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 100% 100% 99.52% 98% green 94% amber Reactive filters issued within 3 working days [KPI A10] SLA B1 100% 100% 100% 100% green 99% amber [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters Previously unmailed properties in areas identified as at risk of interference above agreed thresholds from active and forecast masts, to be mailed at least once no more than six weeks ahead of forecast mast activation Complaints SLA C1 0.31% 0.27% 0.30% <1% green <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 case of interference Qualitative and quantitative reports provided. No specific Measure of the number of reported cases that occur targets. outside of mailed area SLA D3 Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) Page 1 of 10

2 KPIs November 2016: KPI and SLA performance summary, 7 December 2016 KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All 549 confirmed 4G cases in November 2016 were resolved within 10 working days Result: 100% KPI A1: Engineer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.61%; green Of the 2,057 engineer visits originally scheduled for November 2016: o A total of 1,923 visits were undertaken and closed on schedule o Eight visits were cancelled or rearranged by at800 o A further 126 visits were cancelled by the viewer Total scheduled visits not missed by engineer / total visits scheduled (1, ) / 2,057 Result: 99.61% KPI A2: Engineer visits completed within 3 working days. 90% (green); 50% (amber) Result: 96.63%; green Of the 2,057 scheduled engineer visits detailed above, there were 1,931 visits scheduled where the viewer did not cancel Of these 1,931 visits: o A total of 1,582 visits were completed within three working days o A further 284 were delayed beyond three working days at the viewer s request o Eight visit was cancelled or rearranged by at800 o A further 57 visits were completed outside of the three working day target (1, ) / 1,931 Result: 96.63% KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 96.42%; green Of the 951 scheduled engineer visits to people marked as eligible for additional support where the viewer did not cancel: o A total of 808 visits were completed within three working days o One hundred and nine visits were delayed beyond the three working day window at the viewer s request o Four visits were cancelled or rearranged by at800 o Thirty visits were delayed beyond the three working day target KPIs Page 2 of 10

3 ( ) / 951 Result: 96.42% November 2016: KPI and SLA performance summary, 7 December 2016 KPI A6: Communal engineer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only Result: 100% Of the three communal visits scheduled for November, two were undertaken on time and as scheduled. The other visit was cancelled by the viewer before the scheduled visit Result: 100%. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber) Result: 99.52%; green All of the 2,093 filter requests made in November 2016, 2,083 were sent within one working day. Ten filters requested on 15 November during an IT outage took four working days to send Result: 99.52% KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter Not applicable. KPIs Page 3 of 10

4 SLA B Awareness November 2016: KPI and SLA performance summary, 7 December 2016 SLA B1: Previously unmailed properties in areas identified as at risk of interference above agreed thresholds from active and forecast masts, to be mailed at least once no more than six weeks ahead of forecast mast activation. 99% amber, 100% green Result: 100%; green A total of 171,823 households were identified as being eligible for mailing against the 19 October mast submission. These were mailed during the period 3 15 November and are detailed under SLA B2 below. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The table below details the number of households mailed in November under the new mailing regime. Mailing Mailing Date Mailed Not mailed* 3 November 25,000 7 November 29,998 8 November 29,999 Revised 9 November 30,000 mailing #4 14 November 29, November 26,828 Not mailed 246,490 Total 171, ,490 * Not mailed: properties within 2km of a mast used in the model that have not previously received a postcard but where the predicted likelihood of receiving DTT in their pixel has not reduced by more than two percentage points on any multiplex. SLA B3: Report on households reporting interference within unmailed forecast pixels. Where more than 40% of total reports of interference are in these pixels, at800 to address through operational adjustments In November 2016, at800 was contacted by a total of 3,202 households reporting interference that at800 believed could be caused by 4G. Engineers visited 1,564 properties, and filters were sent to a further 1,638. Of the 1,564 viewers to whom at800 sent engineers, 463 were confirmed as being impacted by 4G Of the 3,202 households, 363 (11.3%) were identified in the model but suppressed, either through the risk-based mailing approach or because under the new regime, their degradation was below the threshold to trigger a mailing. Mailing Status 4G confirmed Not 4G Job not complete Filter sent Grand Total Mailed ,383 2,783 Suppressed Not identified SLA B: Awareness Page 4 of 10

5 November 2016: KPI and SLA performance summary, 7 December 2016 Total 463 1, ,638 3,202 SLA B4: Raising awareness with properties where the risk of interference due to active masts has increased by more than the agreed threshold from a point 20 weeks after it was previously mailed. Report on reminder mailing numbers to be provided. A reminder mailing was sent to 150,332 properties forecast to experience increased interference because of masts that were activated between 23 September and 19 October. This mailing excluded any properties that had been mailed within the 20 weeks up to 23 September. These mailings are detailed in the table below. Mailing Mailing Date Number of properties* 16 November 30, November 19,998 Batch R34 21 November 29,999 reminder 22 November 30, November 29, November 10,336 Total 150,332 * Number of properties: these are properties that have previously been contacted by at800 more than 20 weeks prior to 23 September where the likelihood of receiving DTT on any single multiplex has reduced by more than two percentage points solely due to recent mast activations (up to six weeks before mailing). SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas During November, 34 pieces of media coverage communicated at800 s key messages in locations where new mast activations were forecast. Thirteen print articles and 21 online were published during the month. Examples of coverage: Isle of Wight County Press Free help is at hand for TV viewers if new 4G mobile phone masts cause interference in the Seaview area. Penarth Times A new 4G service is set to be introduced in Dinas Powys which could cause problems for Freeview users, it has been warned To help those affected, a company called at800, set up under government direction, can offer free support North Wales Pioneer Free support is being offered to residents whose TV services may be affected as part of a 4G mobile switch-on. SLA B: Awareness Page 5 of 10

6 SLA C Quality of Service November 2016: KPI and SLA performance summary, 7 December 2016 SLA C1: Percentage of households interacting with at800 that make a complaint. 1% (green); 2% (amber) Result: 0.30%; green at800 interacted with a total of 6,321 households in November 2016 Of these, 19 made a complaint that related to at800 s operational processes Result: 0.30% SLA C4: Report on total number and categories of complaints There were a total of 31 complaints made in November 19 related to at800 s operational processes; 12 to its policies. Below is a breakdown of the topics of these complaints. Operations-related Engineer: damage/quality: 6 Mailing and communication: 4 Dispute of non-4g diagnosis: 4 Appointment availability: 2 Engineer late/missed appointment: 2 Contact centre: 1. Policy-related Support of satellite or cable viewers: 6 Support for communal viewers: 6. SLA C5: Vulnerable viewer complaints Out of 1,638 viewers flagged as being eligible for additional support with whom at800 interacted in November 2016, three made a complaint about at800 s operational process; and three complaints were made about its policies. These are broken down below: Operations-related Engineer: damage/quality: 4 Appointment availability: 2 Dispute of 4G diagnosis: 1. Policy-related Support of satellite or cable viewers: 1 Support for communal viewers: 1. SLA C: Quality of Service Page 6 of 10

7 SLA D Scale November 2016: KPI and SLA performance summary, 7 December 2016 SLA D1: Measure of the numbers of mailings per report of/case of interference The following table shows the number of mailings per engineer visit (4G and non-4g) and reactive filter sent, broken down by mast activated before the end of November Each case is tied to a single mast according to the rules specified in Appendix 1. Note that the cases are tied to the month in which the mast was scheduled to be activated (and therefore the masts that drove the mailing volumes); not the month in which the mast was activated. Engineer, Engineer, Reactive Initial Mailings Mailings Mailings per Mailings per visit or Rollout month 4G non-4g filter mailings per 4G per non-4g reactive filter filter sent July ,584 8,947 21,922 7,205,414 2, August , , September , October , November ,946 1, December ,043 1, January , ,796 2,247 1, February , March , April , May , June , July ,370 3, , August ,051 64, September ,203 2, , October ,042 2, , November ,322 1,457 3, , December ,536 1, January , February , March , , April ,023 1, , May , June , , July ,046 2, , August , , September , , October , , November , December , , January , , February , March , April , May , June ,635 1, July ,623 1, August ,666 1,557 1, September , Nothing within 2km 380 1,111 8,593 Total 18,132 30,131 69,462 14,246, The following table shows the equivalent information for masts submitted under the new mailing regime, based on the first submission in which they appeared. This table only includes cases where SLA D: Scale Page 7 of 10

8 November 2016: KPI and SLA performance summary, 7 December 2016 the associated mast using the logic specified in Appendix 1 is a mast first submitted under the new mailing regime. Such masts are tagged under their earliest submission date. Engineer, Engineer, Reactive Initial Mailings Mailings Mailings per Mailings per visit or Submission date 4G non-4g filter mailings per 4G per non-4g reactive filter filter sent 27 July , August , September ,126 1, October ,823 3,996 4,191 2,912 1,718 Total 631 1,005 1, , SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 463 households confirmed as experiencing 4G interference in November 2016: Cases not mailed due to risk profile 30 (6.5%) Cases never identified in modelling 3 (0.6%) Total cases outside mailed area 33 (7.1%) Total cases caught by mailing 430 (92.9%) SLA D: Scale Page 8 of 10

9 November 2016: KPI and SLA performance summary, 7 December 2016 SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action Note: The above thresholds are based on a range of between 5,000 and 50,000 cases of interference for an estimated number of total 4G at 800 MHz masts for full rollout from all MNOs. SLA D: Scale Page 9 of 10

10 November 2016: KPI and SLA performance summary, 7 December 2016 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with at800 a number of times (either requesting multiple filters or receiving many engineer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their contacts with at800. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had engineer visits. The few households that have received both count in both analyses. For households to which at800 has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For engineer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 10 of 10