KPI and SLA regime: March 2017 performance summary

Transcription

1 KPI and SLA regime: March 2017 performance summary OB49 Paper 06 KPI Report Ref Jan 17 Feb 17 Mar 17 Target Description KPI A 100% 100% 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A % 99.68% 99.81% 97.5% green Engineer visits completed as scheduled with viewer 95.0% amber KPI A % 92.71% 97.26% 90% green Engineer visits completed within 3 working days 50% amber KPI A % 92.75% 96.62% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 100% 75% 100% Communal engineer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 100% 100% 100% 98% green 94% amber Reactive filters issued within 3 working days [KPI A10] SLA B % 100% 99.98% 100% green 99% amber [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters Previously un-mailed properties in areas identified as being at risk of interference above agreed thresholds from active masts to be mailed at least once ahead of activation.* Complaints SLA C1 0.31% 0.21% 0.41% <1% green <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 case of interference Qualitative and quantitative reports provided. No specific Measure of the number of reported cases that occur targets. outside of mailed area SLA D3 Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) * Note that the definition of SLA B1 has changed from this month s report onwards. Its previous wording was as follows: Previously un-mailed properties in areas identified as at risk of interference above agreed thresholds from active and forecast masts, to be mailed at least once no more than six weeks ahead of forecast mast activation. Page 1 of 11

2 KPIs March 2017: KPI and SLA performance summary, 10 April 2017 KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All 554 confirmed 4G cases in March 2017 were resolved within 10 working days Result: 100% KPI A1: Engineer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.81%; green Of the 2,055 engineer visits originally scheduled for March 2017: o A total of 1,930 visits were undertaken and closed on schedule o Four visits were cancelled or rearranged by at800 o A further 121 visits were cancelled by the viewer Total scheduled visits not missed by engineer / total visits scheduled (1, ) / 2,055 Result: 99.81% KPI A2: Engineer visits completed within 3 working days. 90% (green); 50% (amber) Result: 97.26%; green Of the 2,055 scheduled engineer visits detailed above, there were 1,934 visits scheduled where the viewer did not cancel Of these 1,934 visits: o A total of 1,624 visits were completed within three working days o A further 257 were delayed beyond three working days at the viewer s request o Four visit was cancelled or rearranged by at800 o A further 49 visits were completed outside of the three working day target (1, ) / 1,934 Result: 97.26% KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 96.62%; green Of the 917 scheduled engineer visits to people identified as being eligible for additional support where the viewer did not cancel: o A total of 809 visits were completed within three working days o Seventy-seven visits were delayed beyond the three working day window at the request of viewers o Two visits were cancelled or rearranged by at800 o Twenty-nine visits were delayed beyond the three working day target KPIs Page 2 of 11

3 ( ) / 917 Result: 96.62% March 2017: KPI and SLA performance summary, 10 April 2017 KPI A6: Communal engineer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only Result: 100% The one communal visit scheduled for March were undertaken on time and as scheduled. Result: 100%. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber) Result: 100%; green All of the 2,103 filter requests made in March were sent within one working day Result: 100% KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter Not applicable. KPIs Page 3 of 11

4 SLA B Awareness March 2017: KPI and SLA performance summary, 10 April 2017 SLA B1: Previously un-mailed properties in areas identified as being at risk of interference above agreed thresholds from active masts to be mailed at least once ahead of activation. 99% amber, 100% green Result: 99.98%; amber A total of 51,660 households were identified as being eligible for mailing against the 8 March mast submission. These were mailed during the period March and are detailed under SLA B2 below A mast that activated on 22 March had not been submitted to at800 before its activation. Because the mast was in an infill area, only nine households were identified as not having been mailed previously. The site was locked down on 5 April and will only be reactivated once the households have been mailed 51,660 / (51, ) Result: 99.98% Amber. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The table below details the number of households mailed in March. Mailing Mailing Date Mailed Not mailed* 21 March 29,998 March March 21,662 submission Not mailed 45,398 Total 51,660 45,398 * Not mailed: properties within 2km of a mast used in the model that have not previously received a postcard but where the predicted likelihood of receiving DTT in their pixel has not reduced by more than two percentage points on any multiplex. SLA B3: Report on households reporting interference within unmailed forecast pixels. Where more than 40% of total reports of interference are in these pixels, at800 to address through operational adjustments In March 2017, at800 was contacted by a total of 3,249 households reporting interference that at800 believed could be caused by 4G. Engineers visited 1,708 properties, and filters were sent to a further 1,541. Of the 1,708 viewers to whom at800 sent engineers, 512 were confirmed as being impacted by 4G Of the 3,249 households, 297 (9.1%) were identified in the model but suppressed, either through the risk-based mailing approach or because under the new regime, their degradation was below the threshold required to trigger a mailing. 4G Filter Grand Mailing Status confirmed Not 4G sent Total Mailed 469 1,086 1,339 2,894 SLA B: Awareness Page 4 of 11

5 March 2017: KPI and SLA performance summary, 10 April 2017 Suppressed Not identified Total 512 1,196 1,541 3,249 SLA B4: Raising awareness with properties where the risk of interference due to active masts has increased by more than the agreed threshold from a point 20 weeks after it was previously mailed. Report on reminder mailing numbers to be provided. A total of 181,049 households were forecast to experience increased interference because of masts activated between 13 January and 8 February. These households excluded any properties that had been mailed during the 20 weeks up to 13 January. Of these, 1,834 households fell within the coverage area of the Selkirk TV transmitter, which was being used for a Selkirk retune event throughout March. Those households outside of the Selkirk area were mailed during the period 6 15 March and are detailed in the table below. Those within the Selkirk area will be mailed in April to avoid message confusion A total of 171,467 households were forecast to experience increased interference because of masts activated between 10 February and 8 March. These households excluded any properties that had been mailed during the 20 weeks up to 10 February. These households are scheduled to be mailed between 28 March and 10 April. Those households mailed in March are detailed in the table below. Mailing Mailing Date Number of properties* 6 March 29,999 7 March 30,000 Batch R39 8 March 30,000 reminder 13 March 29, March 29, March 29,218 Batch R40 28 March 30,000 reminder 29 March 30,000 Total 239,215 * Number of properties: these are properties that have previously been contacted by at800 more than 20 weeks prior to the cut-off date where the likelihood of receiving DTT on any single multiplex has reduced by more than two percentage points solely due to recent mast activations (up to six weeks before mailing). SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas In March, 53 pieces of media coverage communicated information about potential 4G-related TV interference and at800 s support. Forty-seven pieces were generated by proactive PR outreach and four articles were published after journalists contacted our press office. Two further pieces of coverage were generated organically. SLA B: Awareness Page 5 of 11

6 March 2017: KPI and SLA performance summary, 10 April 2017 Information about at800 appeared in 31 online articles, with 19 appearing in print alongside coverage in three radio shows. The organically generated coverage comprised two mentions of 4G-related Freeview interference on BBC Radio Somerset. Show hosts discussing local newspaper articles mentioned an article about the Freeview interference issue that had been published in the Western Gazette (Wincanton edition) following act800 s PR outreach work. Ben Roome was also interviewed on BBC Radio Shetland about Freeview interference caused by 4G and help that at800 is providing. Examples of coverage: Batley News Accredited engineers can visit homes to help resolve the problems, or free filters can be provided. Tavistock Times The company also offers extra support for people who are 75 years of age or older, are registered blind or partially sighted or receive any benefits The Paisley Daily Express Villagers have been warned that new 4G masts could cause some interference with their Freeview TV services. SLA B: Awareness Page 6 of 11

7 SLA C Quality of Service March 2017: KPI and SLA performance summary, 10 April 2017 SLA C1: Percentage of households interacting with at800 that make a complaint. 1% (green); 2% (amber) Result: 0.41%; green at800 interacted with a total of 5,064 households in March 2017 Of these, 21 made a complaint that related to at800 s operational processes Result: 0.41% SLA C4: Report on total number and categories of complaints There were a total of 29 complaints made in March 20 related to at800 s operational processes; nine to its policies. Below is a breakdown of the topics of these complaints. Operations-related Dispute of non-4g diagnosis: 9 Engineer: damage/quality: 3 Mailing and communication: 3 Late or missed engineer appointment: 2 Engineer attitude/behaviour: 1 Appointment availability: 1 Time taken to resolve issue: 1. Policy-related Support of satellite or cable viewers: 4 4G rollout in general: 2 Support for satellite/cable viewers: 2 Support for commercial viewers: 1. SLA C5: Vulnerable viewer complaints Of the 1,441 viewers flagged as being eligible for additional support with whom at800 interacted in March 2017, eight made a complaint about at800 s operational process; none complained about its policies. The complaint reasons are detailed below. Operations-related Dispute of non-4g diagnosis: 4 Engineer: damage/quality: 2 Engineer attitude/behaviour: 1 Time taken to resolve issue: 1. SLA C: Quality of Service Page 7 of 11

8 SLA D Scale March 2017: KPI and SLA performance summary, 10 April 2017 SLA D1: Measure of the numbers of mailings per report of/case of interference The following table shows the number of mailings per engineer visit (4G and non-4g) and reactive filter sent, broken down by mast activated before the end of January It only includes cases tied to masts that were submitted as part of the old mailing regime. Each case is tied to a single mast per the rules specified in Appendix 1. Note that the cases are tied to the month in which the mast was scheduled to be activated (and therefore the masts that drove the mailing volumes); not the month in which the mast was activated. Rollout month Engineer, 4G Engineer, non-4g Reactive filter Initial mailings per 4G per non-4g per reactive filter per visit or filter sent July ,827 9,627 22,734 7,205,414 1, August , , September , , October , November , December ,043 1, January , ,796 1, February , March , April , May , June , July ,465 3, , August ,090 64, September ,282 2, , October ,140 2, , November ,367 1,623 3, , December ,536 1, January , February ,041 95, March , , April ,147 1, , May , June , , July ,198 2, , August , , September , , October , , November , December ,009 1, , January , , February , March , April , May , June , July , August , September , Nothing within 2km ,738 Total 19,500 34,018 74,982 14,246, The following table shows the equivalent information for masts submitted under the new mailing regime, based on the first submission in which they appeared. This table only includes cases where SLA D: Scale Page 8 of 11

9 March 2017: KPI and SLA performance summary, 10 April 2017 the associated mast using the logic specified in Appendix 1 is a mast first submitted under the new mailing regime. Submission date Engineer, 4G Engineer, non-4g Reactive filter Initial mailings per 4G per non-4g per reactive filter per visit or filter sent 27 July , , August , September , October , November , December ,280 1, , January ,285 1,982 1,680 1, February ,354 2,445 1,210 2, March ,660 5,740 2,246 25,830 2,066 Total 2,026 3,938 4,008 1,058, SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 512 households confirmed as experiencing 4G interference in March 2017: Cases not mailed due to risk profile 40 (7.8%) Cases never identified in modelling 3 (0.6%) Total cases outside mailed area 43 (8.4%) Total cases caught by mailing 512 (91.6%) SLA D: Scale Page 9 of 11

10 March 2017: KPI and SLA performance summary, 10 April 2017 SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action Note: The above thresholds are based on a range of between 5,000 and 50,000 cases of interference for an estimated number of total 4G at 800 MHz masts for full rollout from all MNOs. SLA D: Scale Page 10 of 11

11 March 2017: KPI and SLA performance summary, 10 April 2017 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with at800 a number of times (either requesting multiple filters or receiving many engineer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their contacts with at800. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had engineer visits. The few households that have received both count in both analyses. For households to which at800 has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For engineer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900-metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500-metre radius If the above yields no masts, undertake the same process using a 2,000-metre radius If the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 11 of 11