KPI and SLA regime: September 2014 performance summary Reference Outcome Result Target Description KPI A Green 100% 99% green

Transcription

1 OB19 Paper 07 KPI Report KPI and SLA regime: September 2014 performance summary Reference Outcome Result Target Description KPI A Green 100% 99% green 98% amber Service Restoration within 10 working days where household is a primary DTT user KPI A1 Green 99.24% 97.5% green Installer visits completed as scheduled with viewer 95.0% amber KPI A2 Green 99.76% 90% green Installer visits completed within 3 working days 50% amber KPI A4 Green 99.25% 90% green Vulnerable visits completed within 3 working days 70% amber KPI A6 Communal installer visits completed as scheduled with landlord where access and liability is confirmed KPI A7 Green 100% 98% green Reactive filters issued within 3 working days 94% amber [KPI A10] 100% [SUSPENDED FOR REACTIVE-ONLY TRIAL] Addresses in very high pixels mailed filters SLA B1 Red 98.70% 100% green 99% amber Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation SLA B2 Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) SLA B3 Report on households reporting interference within un-mailed forecast pixels SLA B4 Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled SLA B5 SLA B6 SLA B7 Qualitative and quantitative reports provided. No specific targets. Awareness to be maintained (reminder mailings) with those viewers at renewed risk of experiencing interference from infill masts where the addition of the mast increases the risk of interference in that pixel to very high Report on households mailed and not mailed reminders within forecast pixels (i.e. number of low risk households) Report on above the line comms and research outcomes, including awareness in rollout areas Complaints SLA C1 Green 0.30% <1% green <2% amber SLA C4 Report on total number and categories of complaints SLA C5 Report on vulnerable complaints SLA D1 Measure of the numbers of mailings per report of / SLA D2 case of interference Qualitative and quantitative reports Measure of the number of reported cases that occur provided. No specific targets. outside of mailed area SLA D3 Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout)

2 KPI A Service Restoration KPI A: Service Restoration within 10 working days where household is a primary DTT user (communal households and households where cable or satellite services are received are excluded). 99% (green); 98% (amber) Result: 100%; green All of the 320 confirmed 4G cases in September 2014 were resolved within 10 working days Green. KPI A1: Installer visits completed as scheduled with viewer. 97.5% (green); 95% (amber) Result: 99.24%; green Of the 918 installer visits originally scheduled for September 2014: o 842 were undertaken and closed on schedule o 7 visits were cancelled or missed by the installer o 69 visits were cancelled by the viewer Total scheduled visits not missed by installer / total visits scheduled ( ) / ( ) = 911 / 918 Result: 99.24% Green. KPI A2: Installer visits completed within 3 working days. 90% (green); 50% (amber) Result: 98.94%; green Of the 918 scheduled installer visits detailed above, there were 849 visits scheduled where the viewer did not cancel Of these 849 o 750 were completed within three working days o 90 were delayed beyond three working days at the viewer s request o Seven visits were cancelled by the installer o Two were delayed beyond three working days owing to operational constraints 840 / 849 Result: 98.94% Green. KPI A4: Vulnerable visits completed within 3 working days: 90% (green); 70% (amber) Result: 96.46%; green In September 2014, there were 266 scheduled installer visits to people marked as vulnerable where the viewer did not cancel Of the 266 visits to be measured o 247 were completed within three working days o 20 were delayed beyond the three-day window at the viewer s request o Two visits were cancelled by the installer Page 2 of 10

3 Result: 99.25% Green. KPI A6: Communal installer visits completed as scheduled with landlord where access and liability is confirmed. Reporting only. Result: 100% Of the 13 communal jobs in the month of September 2014: o 12 were completed as agreed with the customer o One was cancelled by the viewer Therefore all 13 were completed as scheduled with the viewer (or cancelled by the viewer) giving a success rate of 100%. KPI A7: Reactive filters issued in 3 working days. 98% (green); 94% (amber). Result: 100% There were 1,762 filter requests in September 2014 All of these were sent within one working day Green. KPI A10 [SUSPENDED]: 100% of identifiable addresses in very high pixels mailed proactive filter. Not applicable. Page 3 of 10

4 SLA B Awareness SLA B1: Identifiable addresses within forecast medium, high and very high risk pixels (high and very high risk only for London) to be mailed at least once no more than 12 weeks ahead of scheduled mast activation. 100% (green); 99% (amber). Result: 98.70%; red During September, DMSL sent a number of mailings to allow MNOs to activate masts earlier than their original schedule. In all of these cases where the MNO made a specific request, DMSL ensured postcards landed before mast activation was permitted Further, there were two special mailings undertaken to support masts that activated out of process without prior warning from the MNO. In relation to these masts, 4,642 households were identified and mailed on 24 September with 460 households mailed on 25 September. Postcards would have landed up to one week following mast activation Therefore, of all 391,257 households forecast to be affected by October masts; additional masts submitted for earlier activation; or masts activated prior to their scheduled activation month unknown to DMSL, 386,155 (98.70%) were mailed in advance of their potential or in the case of activated masts, their actual go-live date. Red. SLA B2: Report on households mailed and not mailed within forecast pixels (i.e. number of low risk households) The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing Date V_HIGH HIGH MED LON_MED LOW TOTAL 03-Sep-14 3,189 16,766 23,148 43, Sep-14 4,811 57,520 68, , Sep-14 6,007 79, , , Sep ,581 10,484 21, Sep-14 1,770 2,872 4, Sep Not Mailed 17, , ,025 TOTAL 14, , ,953 17, ,830 1,023,282 SLA B3: Report on households reporting interference within un-mailed forecast pixels. More than 40% of total reports of interference DMSL to address through operational adjustments (e.g. mailing low risk pixels in a certain area or adjusting the risk score for pixels), however, a de minimis threshold to apply. In September 2014, DMSL was contacted by a total of 2,426 households reporting interference that DMSL believed could be caused by 4G. Installers visited 846 and filters to a further 1,562. Of the 846 viewers to whom we sent installers, 320 were confirmed as being impacted by 4G Of the 2,426 households, 33 (1%) were suppressed through the risk-based mailing. Page 4 of 10

5 Mailing Status 4G Filter Grand Not 4G confirmed sent Total Mailed ,267 Suppressed Not picked up Total ,426 SLA B4: Awareness to be maintained (reminder mailings) with those viewers at continued risk of experiencing interference from masts that were not activated as scheduled. Report on reminder mailing numbers to be provided. A reminder mailing was undertaken on 1 Oct 2014 to 3,344 households forecast to be affected by masts originally expected to activate in April The following table shows the breakdown of households by risk category and those confirmed as mailed. Mailing Date V_HIGH HIGH MED LON_MED LOW Grand Total 1 Oct ,656 1,687 3,344 Not mailed 25,988 18,373 44,361 Total 1 1,656 1,687 25,988 28,373 47,705 SLA B7: Report on above the line communications and research outcomes, including awareness in rollout areas The proactive PR work we carried out during September adopted an updated approach, using Twitter to identify local voices and amplifiers in the areas we targeted. These people were sent tweets from at800 and asked to re-tweet to their followers to create some local awareness before the news release was sold-in over the telephone to journalists. This approach was particularly effective, securing 26 individual pieces of media coverage that communicated DMSL s key messages, with a total potential reach of over 1.2 million people. Examples of coverage included: The Press York: More than 800 postcards are landing on doormats in Harrogate from today [ ] the plus side to all of this is that it is free. Evening Gazette Teesside: Funded by the leading UK operators, at800 will make sure that viewers can still watch Freeview or are offered a suitable alternative. Stirling News: If at800 believes disruption to Freeview is due to 4G at 800 MHz, it will arrange for an accredited installer to visit the viewer s home, check the cause and fit an approved filter to the TV system, at no cost to the viewer. In most cases, disruption will be fixed by an at800 filter. In September we ran adverts on Facebook to support the direct marketing and PR work during the Ryder Cup, when a number of new masts were activated. Page 5 of 10

6 The adverts targeted addresses registered in Kinross, Auchterarder and Perth. There are approximately 46,000 people in these areas on Facebook. The Facebook advertising took place between Tuesday 23 and Sunday 28 September. Below is a summary of the Facebook activity. Post Area Clicks Reach Post likes Post shares comments Perth , Auchterarder 372 1, Kinross 169 2, Total 1,284 29, Background: DMSL runs above-the-line, PR or digital advertising communication when masts are activated/the operators begin offering 4G services (at 800 MHz), and not necessarily in line with when DMSL mails viewers. Prior to mast activations/4g at 800 MHz launch, our primary means of communication with viewers is direct mail postcards. Page 6 of 10

7 SLA C Quality of Service SLA C1: Percentage of households interacting with DMSL that make a complaint. 1% (green); 2% (amber). Result: 0.30% We interacted with a total of 7,583 households in September 2014 Of these, 22 (0.30%) made a complaint Green. SLA C4: Report on total number and categories of complaints Below is a breakdown of the 22 complaints received in September 2014: Compensation: 13 Rollout in general: 3 Filter issuance criteria: 3 Non-4G issue that we cannot help with: 2 Installer-related: 1. [SLA C5:] Vulnerable viewer complaints Out of 833 viewers flagged as being vulnerable with which we interacted in September 2014, none made a complaint. Page 7 of 10

8 SLA D Scale SLA D1: Measure of the numbers of mailings per report of / case of interference The following table shows the number of mailings per reported case and confirmed case of interference to date for September 2014 rollout households that relate to masts planned to be activated up to September Month Installer, non-4g Installer, 4G Reactive filter Initial mailings Mailings per non- 4G Mailings per 4G Mailings per reactive filter Mailings per case Jul ,771 2,342 15,081 7,205,414 1,249 3, Aug , Sep , Oct ,425 1,223 1, Nov ,946 3,610 2,320 1, Dec ,043 2,642 6,440 1, Jan ,796 10,914 19,100 5,243 2,988 Feb ,943 1,592 1, Mar , , Apr ,133 1,033 4, May ,606 1,152 1, Jun ,100 5,881 3,764 2,188 1,120 Jul ,204 1,345 2, Aug ,130 4,008 4,008 1, Sep ,349 2,547 3, Oct ,155 N/A 193, ,718 77,231 Nov ,107 7,562 2,722 1, Dec N/A 230 N/A 230 Nothing within 2km ,613 Total 7,201 3,356 19,148 9,671,592 1,343 2, Note: Data is included for November and December 2014 masts as some of these have been activated out of sequence. SLA D2: Measure of the number of reported cases that occur outside of mailed area Of the 320 households identified as experiencing 4G interference in September 2014: Cases not mailed due to risk profile or low pop in pixel 4 (1.25%) Cases never identified in modelling 7 (2.19%) Total cases outside mailed area 11 (3.44%) Total cases caught by mailing 309 (96.56%) SLA D3: Measure of number of cases of interference per mast remaining between 0.17 and 1.66 (5k to 50k cases of interference for rollout) exceeding or trending towards exceeding the range will be raised to the OB for review/action. Page 8 of 10

9 Note: The above thresholds are based on a total mast count of 50,000. Page 9 of 10

10 Appendix 1: Logic for SLA D1: Identifying the nearest mast Identifying an individual mast that is causing interference is an inexact science. Indeed two masts may individually not result in interference for a household, but when both are active at the same time, interference may result. A household that has interacted with us a number of times (either requesting multiple filters or receiving many installer visits) further complicates the identification of a single affecting mast, as masts may have activated between in between their interactions with us. For the purposes of SLA D1, the logic described below has been used to identify the dates against which households should be measured, and the logic for identifying the mast that is thought most likely to be causing the interference. In-scope households and their effective date The analysis looks separately at households that have received reactive filters and those that have had installer visits. The few households that have received both count in both analyses. For households to which DMSL has sent reactive filters, their analysis is based on the first date on which they made a request for a filter, a request that the triage process indicated warranted the dispatch of a filter. For installer visits, each household is counted once, but all visits are analysed to inform the record. For any households that had a single visit, the date on which that visit was requested is used as the basis for analysis, and the household is categorised as 4G or not based on the outcome of that visit. For any households that had multiple visits, if one or more visits resulted in a 4G diagnosis, it is categorises as 4G and the date on which the earliest of these visits was requested is used as the basis for analysis. If all visits resulted in a non-4g diagnosis, it is categorised such and the request date of the earliest visit is used as the basis for analysis. Identifying mast most likely to be causing interference For each of the above cases, the following process is undertaken: Determine all masts that were activated on or before the household s effective date (see above), but only include masts that are within a 900 metre radius of the property. From these masts, determine the mast that was activated the soonest before the call came in. This is deemed to be the mast most likely to be the cause of the interference If the above yields no masts, undertake the same process using a 1,500 metre radius If the above yields no masts, undertake the same process using a 2,000 metre radius In the event that the above yield no masts (i.e. there was no active mast within 2,000 metres at the point at which the call came in), then the case is not associated with any specific mast and is discounted from the analysis. This logic is used to tag each household with an affecting mast. The rollout month for which this mast was submitted is used as the basis for analysis. Page 10 of 10