KPI and SLA regime: June 2015 performance summary Ref Apr 15 May 15 Jun 15 Target Description KPI A 100% 100% 100% 99% green

Transcription

1 KPI and SLA regime: June 2015 performance summary Ref Apr 15 May 15 Jun 15 Target Description KPI A 100% 100% 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A % 97.49% 99.22% 97.5% green Engineer visits completed as scheduled with viewer 95.0% amber KPI A % 96.62% 98.52% 90% green Engineer visits completed within 3 working days 50% amber KPI A % 97.52% 99.10% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 100% 100% 100% Communal engineer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 100% 100% 100% 98% green Reactive filters issued within 3 working days 94% amber [KPI A10] [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters SLA B1 100% 100% 100% 100% green 99% amber Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation SLA C1 0.43% 0.20% 0.39% <1% green Complaints <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 SLA D3 Qualitative and quantitative reports provided. No specific targets. case of interference Measure of the number of reported cases that occur outside of mailed area Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) Page 1 of 10

2 KPIs June 2015: KPI and SLA performance summary, 13 July 2015 KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All of the 495 confirmed 4G cases in June 2015 were resolved within 10 working days Result: 100% Green. KPI A1: Engineer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.22%; green Of the 1,535 engineer visits originally scheduled for June 2015: o 1,404 were undertaken and closed on schedule o Twelve visits were cancelled and rearranged by at800 to meet capacity constraints o 119 visits were cancelled by the viewer Total scheduled visits not missed by engineer / total visits scheduled (1, ) / 1,535 Result: 99.22% Green. KPI A2: Engineer visits completed within 3 working days. 90% (green); 50% (amber) Result: 98.52%; green Of the 1,535 scheduled engineer visits detailed above, there were 1,416 visits scheduled where the viewer did not cancel Of these 1,416 visits: o 1,270 were completed within three working days o 125 were delayed beyond three working days at the viewer s request o Twelve visits were cancelled or rejected by the engineer o Nine were completed outside of the three working day target (1, ) / 1,391 Result: 98.52% Green. KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 99.10%; green In June 2015, there were 669 scheduled engineer visits to people marked as vulnerable where the viewer did not cancel Of the 669 visits to be measured: o 629 were completed within three working days o 34 were delayed beyond the three-day window at the viewer s request o Five visits were cancelled or rejected by the engineer o One visit was delayed beyond the three working day target KPIs Page 2 of 10

3 ( ) / 669 Result: 99.10% Green. June 2015: KPI and SLA performance summary, 13 July 2015 KPI A6: Communal engineer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only Result: 100%; green All of the ten communal jobs that were scheduled to take place in June 2015 were undertaken on time and as scheduled Result: 100% Green. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber) Result: 100%; green There were 2,105 filter requests in June 2015, all of which were sent within one working day Result: 100% Green. KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter Not applicable. KPIs Page 3 of 10

4 SLA B Awareness June 2015: KPI and SLA performance summary, 13 July 2015 SLA B1: Identifiable addresses within forecast medium, high and very high-risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation. 100% (green); 99% (amber) Result: 100%; green A total of 190,478 households were identified as being eligible for mailing because of July 2015 rollout masts. Of these, 70,051 households were mailed in May and were reported in last month s report. The remaining 120,427 were mailed in early June and are reported under SLA B2 below A total of 131,088 households were identified as being eligible for mailing because of August 2015 rollout masts. These were all mailed in June and are reported under SLA B2 below. Result: 100% Green. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing July 2015 rollout August 2015 rollout Mailing Date Very high High Medium London medium Low Grand Total 1 June 3,532 8,623 12,845 25,000 2 June 3,524 8,558 12,918 25,000 3 June 2,848 6,882 10,270 20,000 8 June 3,529 8,534 12,937 25,000 9 June 3,555 8,534 12,911 25, June Not mailed* 1, , , June 167 1,571 1,378 3, June 1,305 9,892 13,801 24, June 1,317 9,896 13,787 25, June 1,303 9,862 13,835 25, June 1,300 9,872 13,827 24, June 1,458 11,080 15,437 27,975 Not mailed , ,517 Total 23,900 93, ,162 1, ,699 1,200,930 * This includes the unmailed volume for the modelling for the entirety of the July 2015 batch. Note that from the August 2015 rollout mailing onwards, five seed addresses of people within the at800 organisation are being added to the mailing files. This is to introduce an additional operational check that mailings are being delivered. These five postcards will not be included in the numbers reported, unless the addresses are also identified as being eligible for mailings by the interference modelling. SLA B: Awareness Page 4 of 10

5 June 2015: KPI and SLA performance summary, 13 July 2015 SLA B3: Report on households reporting interference within un-mailed forecast pixels. More than 40% of total reports of interference at800 to address through operational adjustments (e.g. mailing low risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to apply In June 2015, at800 was contacted by a total of 3,032 households reporting interference that at800 believed could be caused by 4G. Engineers visited 1,441 properties, and filters were sent to a further 1,591. Of the 1,441 viewers to whom at800 sent engineers, 489 were confirmed as being impacted by 4G. Of the 3,032 households, 281 (9.2%) were suppressed through the risk-based mailing. Mailing Status 4G confirmed Not 4G Job not complete Filter sent Grand Total Mailed ,371 2,694 Suppressed Not identified Total ,591 3,032 SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled. Report on reminder mailing numbers to be provided A reminder mailing to 56,463 households forecast to be affected by masts originally expected to activate in November 2014 was undertaken during the period June These are detailed in the table below A reminder mailing to 86,833 households forecast to be affected by masts originally expected to activate in December 2014 was undertaken during the period June These are detailed in the table below. Mailing Mailing Date Very high High Medium London medium Low Grand Total November 10 June 437 7,352 17,211 25, June 534 9,131 21,798 31,463 reminder Not mailed* 150, , June 93 4,662 15,236 19,991 December 15 June 96 4,692 15,202 19, June 113 5,834 19,047 24,994 reminder 17 June 98 5,087 16,673 21,858 Not mailed 93,819 93,819 Total 1,371 36, , , ,862 * One viewer with a risk score of Medium who requested after their initial mailing not to be contacted by at800 was also suppressed from this reminder mailing, and is not included in the above numbers. SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas Fourteen pieces of media coverage appeared during June across print and online outlets. Seven online articles, six in print and one radio interview communicated at800 s messaging to a potential audience of around 1.46 million people. SLA B: Awareness Page 5 of 10

6 June 2015: KPI and SLA performance summary, 13 July 2015 Some examples of coverage are detailed below. Ulster Herald: to date, it has received 48 calls from households in Omagh, with its accredited engineers visiting eight households in the area. North Wales Pioneer: An independent organisation known as at800 will be sending out postcards to houses to raise awareness, as well as offering free support to any household that is experiencing the disruption Mansfield Chad: Mobile broadband masts are set to go live around Mansfield, and people are being warned that it could cause TV interference for those watching Freeview channels. SLA B: Awareness Page 6 of 10

7 SLA C Quality of Service June 2015: KPI and SLA performance summary, 13 July 2015 SLA C1: Percentage of households interacting with at800 that make a complaint. 1% (green); 2% (amber) Result: 0.39%; green at800 interacted with a total of 5,917 households in June 2015 Of these, 23 (0.39%) made a complaint that related to at800 s operational processes Result: 0.39% Green. SLA C4: Report on total number and categories of complaints There were a total of 40 complaints made in June 23 related to at800 s operational processes; 17 to its policies. Below is a breakdown of the topics of these complaints. The categories were changed from the April 2015 report onwards to be more specific to their root cause. Operations-related Mailing and communication: 7 Appointment availability: 4 Length of at800 journey: 4 Engineer: Damage/Quality: 3 Engineer: Late/missed appointment: 2 Contact centre agent: 1 Other: 2. Policy-related Support of satellite or cable viewers: 7 Support of communal properties: 6 4G Rollout in general: 4. SLA C5: Vulnerable viewer complaints Out of 1,183 viewers flagged as being eligible for additional support with whom at800 interacted in June 2015, eight made a complaint, six of which related to at800 s operational processes and two to its policies. These are broken down as follows: Operations-related Engineer: Damage / Quality: 2 Appointment Availability: 1 Timeliness of resolution: 1 Mailing and communication: 1 Contact centre agent: 1 Policy-related Support of communal properties: 1 Support of satellite or cable viewers: 1. SLA C: Quality of Service Page 7 of 10

8 SLA D Scale June 2015: KPI and SLA performance summary, 13 July 2015 SLA D1: Measure of the numbers of mailings per report of / case of interference The following table shows the number of mailings per engineer visit (4G and non-4g) and reactive filter sent, broken down by mast activated until the end of June Each case is tied to a single mast according to the rules specified in Appendix 1. Note that the cases are tied to the month in which the mast was scheduled to be activated (and therefore the masts that drove the mailing volumes); not the month in which the mast was activated. Mailings per non- 4G Mailings per reactive filter Mailings per visit or filter sent Engineer, Engineer, Reactive Initial Mailings Rollout month 4G non-4g filter mailings per 4G July ,951 7,418 19,019 7,205,414 2, August , September , October ,425 1, November ,946 1,511 1, December ,043 3,435 1, January ,796 5,192 2,715 1, February ,943 1,289 1, March , April , May , June , July , , August , September , , October , , November , , , December ,536 4,119 4,361 1, January ,594 1,787 1, February , March , April ,508 1, May ,825 3,739 3,324 1, June ,953 3,101 3, July ,478 31,746 10,582 2,295 1,886 Nothing within 2km ,068 Total 6,893 13,288 35,237 11,253,669 1, Note: Data is included for July 2015 masts as some of these masts were activated before the end of June SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 491 households confirmed as experiencing 4G interference in June 2015: Cases not mailed due to risk profile 35 (7.2%) Cases never identified in modelling 3 (0.6%) Total cases outside mailed area 38 (7.8%) Total cases caught by mailing 451 (92.2%) SLA D: Scale Page 8 of 10

9 June 2015: KPI and SLA performance summary, 13 July 2015 SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action Monthly 4G confirmed cases per mast activated in that month Jul-13 Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15 Mar-15 Apr-15 May-15 Jun-15 Monthly Cases Per Mast Range Lower Limit Range Upper Limit Poly. (Monthly Cases Per Mast) Note: The above thresholds are based on a total mast count of 50,000. SLA D: Scale Page 9 of 10

10 June 2015: KPI and SLA performance summary, 13 July 2015 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with at800 a number of times (either requesting multiple filters or receiving many engineer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their contacts with at800. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had engineer visits. The few households that have received both count in both analyses. For households to which at800 has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For engineer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 10 of 10