AMI Pilot Project. Phase 1 AMI Data and Systems Integration for Market Settlement and Billing Report

Transcription

1 AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing Report August 2008 to November 2009

2 Disclaimer and Limitations This report has been prepared by EnergyAustralia to provide information to interested members of the public on the insights and lessons learnt from EnergyAustralia s Advanced Metering Infrastructure (AMI) Pilot Project, Phase 1 AMI Data and Systems Integration for Market Settlement and Billing (known as Cutover). This report has not been prepared to meet any formal regulatory obligations but it may be provided to regulators, government agencies or other interested persons to provide further support and detail in relation to matters being considered or assessed by those regulators, agencies or persons. It has been prepared with all due care and skill and endeavours to be technically and factually accurate. However anyone proposing to rely on or use information in this report should independently verify and check the accuracy, completeness and suitability of that information for their own purposes. Consequently, EnergyAustralia makes no representations or warranty as to the accuracy, reliability, completeness or suitability for particular purposes of the information in this report. Persons reading or utilising this document acknowledge that EnergyAustralia and its officers, employees, agents and consultants shall have no liability (including liability to any person by reason of negligence or negligent misstatement) for any statements, opinions, information or matters (expressed or implied) arising out of, or contained in or derived from, or for any omissions from, the information in this document, except so far as liability under any New South Wales and Commonwealth statute cannot be excluded. This publication may only be reproduced with the permission of EnergyAustralia. All correspondence should be directed to Managing Director EnergyAustralia GPO Box 4009 NSW 2001 Published by EnergyAustralia March 2010 AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing

3 Contents 1 Introduction Purpose of Report Scope and Objective Background of Pilot Project Report Summary Planning and Design Vendor and Technology Overview Meter Combinations Site Selection and Geographic Locations Testing and Reporting of Meter Combinations Overview Tests 1-7: Meter Management Systems Reliability and Performance Tests 8-13: Back Office Systems Validation and Reconciliation Test 14: End to End Process AMI Read to Billing Invoice Meter Selection Criteria for Cutover (to Production) Stakeholder Approval and Support Retailers Australian Energy Market Operator Australian Energy Regulator Cutover to Production Processes and Activities Ongoing Reporting and Monitoring of AMI Pilot Meters in Production Results and Key Findings Cutover Testing Summary Overview Tests 1-13 Key Findings Tests 1-7 Results for Vendor Tests 1-7 Results for Vendor Tests 1-7 Results for Vendor Tests 1-7 Results for Vendor Tests 8-13 Results for Vendor Tests 8-13 Results for Vendor Test 14 Results and Key Findings for Vendor Stakeholder Approval and Support Cutover of AMI Pilot Meters to Production Summary Overview Key Findings AMI Pilot Meters in Production Next Steps Customer Research Program...56 AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing

4 Contents 5 Glossary...58 Appendix 1: Tests 1-14 Reporting Matrix and Metrics...60 Appendix 2: List of Report Figures and Tables...62 AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing

5 Introduction 1 1 Introduction 1.1 Purpose of Report This report summarises the key findings and testing results from EnergyAustralia s Advanced Metering Infrastructure (AMI) Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing (known as Cutover). 1.2 Scope and Objective The AMI Pilot Project Phase 1 Cutover was undertaken to gain an understanding of the issues for using ½ hour remotely read metering data delivered daily for billing and market settlement. The program required integration of AMI meter data into EnergyAustralia s back office systems. The scope of the AMI Pilot Cutover trialling included testing and assessing the performance of the AMI communications systems, data collection systems and integration into back office systems to provide an end to end solution from the smart meter to the market as shown in Figure 1 below. Figure 1: EnergyAustralia AMI Overview Home Area Network Controlled Load AMI Meter 2. AMI Communications 1. Point to Point C An intermediary device, such as: - Concentrator in a PLC system - Gateway in Mesh Radio system Communications Network Management System (CNMS) Meter Data Collection System (MDCS) Meter Business System (MBS) Meter Management System (MMS) Meter Data Management (MDM) Distribution Business (LNSP) Systems Customers Retailers (FRMP) B2B Hub Market Operator (AEMO) Manual reading of meter data AMI Pilot Project Phase 1 Technology Trial AMI Pilot Project Phase 1 Cutover 1.3 Background of Pilot Project In early 2006 EnergyAustralia recognised the emerging potential of smart meters and initiated the AMI Pilot Project Phase 1 Technology Trial. Potential benefits of AMI identified at this early stage were: Engaging customers in more empowered consumption decisions based on timely price signals and customer awareness of their consumption Operational improvements through remote meter reading and control Providing a solution for the next evolution of Controlled Load services, including hot water Facilitating the replacement of aging meters. EnergyAustralia commenced in early 2006 to undertake the planning, procurement, development, implementation and testing of new AMI technologies and communications. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 1

6 Introduction 1 The AMI technologies and communications selected for Phase 1 were relatively advanced at the time of selection, however in many cases have since been superseded by more developed solutions. This report acknowledges that the equipment tested in these trials may not be current. It is also acknowledged that the trials were conducted over a relatively short duration and that some ongoing operational issues and the performance of components over time have not been fully assessed. The AMI Pilot Project Phase 1 included extensive laboratory and field testing, training, documentation and planning for more than thirty combinations of meter configurations, communication technologies and meter vendors. By September 2007, EnergyAustralia had installed 6756 smart meters from four of Australia s leading meter vendors. Further details on the key findings and testing results of the AMI technology trials undertaken can be found in the AMI Pilot Project Phase 1 Technology Report (June 2006 to July 2008) released in October The pilot study was designed to provide information regarding the capabilities of the emerging technologies, the likely costs of those systems and the potential benefits delivered to networks, retailers, customers and other stakeholders. The meter sites were strategically identified to enable representative customer sampling, alignment with network configurations (i.e. connected to strategic feeders and substations to allow useful network impact analysis), and simulated large scale deployment and operating scenarios. Information from the pilot continues to feed into EnergyAustralia s Cost Benefit Analysis and business case for AMI. A summary of the three key stages of the AMI Pilot Project is shown in Figure 2 below. Figure 2: AMI Pilot Project End to End Process AMI Pilot Project End-to-End Process Customer Research Program Cutover Technology Trial Planning and Design (early 06) Planning and Design Cutover Testing (Tests 1-13) on second group of meter combinations Planning and Design Vendor / Technology Selection and Procurement Select smart meters for Cutover Testing Select smart meters for Cutover Report 1: Impact of TOU Pricing Report 2: Customer Survey Study EnergyAustralia 3239 smart meters in production Testing: Laboratory Field Statistical Stress and Volume Cutover Cutover Testing (Tests 1-13) on first group of meter combinations Stakeholder Approval and Support Design SMI Pricing & Products Customer Trial Program Deployment, Installation and Commissioning 6756 smart meters Sept 07 Select 15 smart meters for End to End Testing (Test 14) Jan 09 June 09 Cutover 1697 smart meters Recruit target customers Ongoing Reporting and Monitoring Record Lessons Learned Test 14 Completed End-to-End process confirmed April 09 Sept 09 Cutover 1542 smart meters Conduct Trials Public Technology Report Oct 08 Monitoring and Evaluation of smart meters for further testing and Cutover consideration Ongoing Reporting and Monitoring Record Lessons Learned Report 3: Customer Responses to SMI Pricing & Products Public Cutover Report Dec 09 AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 2

7 Introduction Report Summary This report summarises the key findings and testing results from EnergyAustralia s AMI Pilot Project AMI Data and Systems Integration for Market Settlement and Billing (known as Cutover). The specific objectives of this part of the Phase 1 AMI Pilot Project were: To identify and understand the issues of using ½ hour remotely read metering data delivered daily for billing and market settlement. To integrate AMI meter data into EnergyAustralia s back office systems. The project was successful in regards to these objectives and the AMI meters are operating in full live production mode. These meters will continue to operate in this manner for the duration of the pilot project. As shown in Figure 2, several key activities were completed during the AMI Pilot Project Phase 1 Cutover phase including: Planning and designing cutover Selecting meters for cutover testing (selection criteria developed) Cutover testing of meters: - Reliability and performance of Meter Management Systems (MMS) - Validation and reconciliation of back office systems - End to end process of reading the AMI meter to generating the billing invoice Continual monitoring and evaluation of meters for cutover consideration Selecting meters for cutting over to production (selection criteria developed) Gaining key stakeholder approval and support for cutover (developed new processes with the Australian Energy Regulator (AER), Australian Energy Market Operator (AEMO) and retailers Changes to EnergyAustralia s back office systems to cutover each meter to production Ongoing reporting and monitoring of meters in production Recording lessons learned and insights gained (this report) The 6756 smart meters originally deployed in the pilot were rationalised to approximately 4200 during the cutover testing phase as a result of the withdrawal of two vendors who did not to continue with the trial. In June 2009, EnergyAustralia cutover 1697 smart meters to full end to end AMI processing and operation. EnergyAustralia communicated with the retailers, AEMO and the AER to ensure endorsement for us to operate those smart meters as Type 5 with daily remote reading capabilities.. The AER advised EnergyAustralia in writing that (on the basis of the information provided by EnergyAustralia) it would not enforce the National Electricity Rules requirements relating to remotely read meters so far as the requirements relate to meters used in the pilot. The cutover meters have been operating successfully since June and EnergyAustralia provides ½ hour remotely read metering data to the market on a daily basis for settlement and billing. The smart meters included in that migration were assessed over a long period and demonstrated data quality and integrity levels equal to or better than normal quarterly probe read results for manually read interval meters. EnergyAustralia has continued to monitor and evaluate the remaining portfolio of smart meters. Those meters have continued to demonstrate excellent performance and data accuracy. Commencing in late September 2009, a further 1542 smart meters have been migrated into live production to join those of the first cutover stage. This now provides a cutover smart meter pilot consisting of 3239 smart meters which will enable a sufficient sample size to not only properly assess the processes and systems for data capture and billing, but also to conduct effective customer trials using a range of smart meter enabled pricing and products as envisaged when the pilot project began. Of the remaining meters not cutover (approximately 961 meters), the majority have been removed from the pilot except for a small group of meters (20% or 190 meters) which continue to be monitored for cutover consideration. EnergyAustralia sought and received approval from the AER to include this additional tranche of smart meters in the no action advice it provided in relation to the first tranche. The AEMO and retailers involved were notified in both instances and confirmed their ongoing support of the AMI Pilot Project Phase 1 Cutover. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 3

8 Introduction 1 Key findings and insights gained include: Early engagement with retailers, AEMO and the AER is critical - Learned what information is required and the support / approval timeframes - Developed and documented an understanding of the process required with AEMO and the AER for cutover support and approval. The process is suitable for small scale trialling but is not recommended, nor suitable for future large scale pilots. - Retailers appreciated the opportunity to be involved in the process as it assisted in their preparation for future large scale smart meter rollouts Gained an understanding of the inflexibility of existing back office systems to accept the vendor MMS data - Back office systems only accept meter data files in ascending date order - There was a high impact on existing processes and workarounds were required - Additional application tools were identified and developed to manage and schedule daily exports of data from the vendor MMS to the back office systems Highly manual process to cutover each meter to production Learned timeframes required for data collection and integration to existing back office systems and data delivery to market. Figure 3 below shows the reporting timeline for the AMI Pilot Project from early 2006 till the end of Figure 3: AMI Pilot Project Reporting Timeline AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 4

9 Planning and Design 2 2 Planning and Design 2.1 Vendor and Technology Overview AMI technologies typically have three main areas of capability: Two-way communications with the smart meter Functionality and intelligence at the electricity meter, including remote supply disconnection and reconnection Meter Data Collection System (MDCS) for the management of both the smart meters and the communications, including reporting, data collection, command management, load control, troubleshooting and interfacing to billing. The combined meter and communications systems needed to be fit for the purpose of revenue metering - which meant the smart meters had to be capable of operating as Type 5 (manually read interval meters also known as MRIM). The smart meters which are cutover will be read remotely with the ½ hour data submitted to AEMO for settlement and to retailers for billing purposes on a daily basis. Cutover of the smart meters does not affect existing network tariff arrangements or network billing cycle and the smart meters remain as Type Meter Combinations The AMI Pilot Project commenced with multiple combinations of meter vendors, communications technologies and meter configurations. From these sixteen were found to be suitable for further evaluation and analysis through cutover testing (shown below in Table 1). Table 1: Meter Combinations for Cutover Testing and Evaluation Meter Vendor Communications Technology Meter Configuration Vendor 1 Vendor 2 General Packet Radio Service (GPRS) Distribution Line Carrier (DLC) Multi-Drop RS485 (using GPRS Modem Gateway) GPRS E1 single phase meters E3 three phase meters E1 single phase meters E3 three phase meters E1 single phase meters E1 single phase meters E2 single phase dual element meters (for load control) E3 three phase meters DLC E1 single phase meters E2 single phase dual element meters (for load control) E3 three phase meters Vendor 3 DLC E1 single phase meters E2 single phase dual element meters (for load control) E3 three phase meters Vendor 4 GPRS E1 single phase meters E3 three phase meters AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 5

10 Planning and Design 2 Communications Technologies A range of communications options were selected, representative of technologies available in 2006 and generally utilised as off the shelf systems for remote communications to electricity meters. Table 2 below provides an overview of the communication technologies used in the cutover phase of the AMI Pilot Project. Table 2: Communications Technologies used in Cutover Communications Technology Distribution Line Carrier (DLC) Multi-Drop RS485 General Packet Radio Service (GPRS) Description DLC utilised the low voltage power line for communications back to a data concentrator at the distribution substation, with subsequent backhaul using higher bandwidth communications (GPRS). DLC technologies used were: - AMIS: AMIS S-FSK PLC Modem - Echelon: PLT-22 Powerline Tranceiver A two wire serial communication link is established with each device, communicating to the outside world via a gateway; typically a GPRS modem. This is an electricity meter with an embedded cellular mobile phone operating in GPRS mode using existing Global System for Mobile (GSM) communication networks. Justification (as at 2006) The most widely utilised remote meter reading systems that could be suitable for high volume data collection. Utilised the electricity network as a private communications network eliminating telecommunications charges to individual meters. Simple and cost-effect method of achieving reliable communications to a network of electricity meters that are installed in a common location, such as office blocks or apartments. A standalone AMI system suitable for low density installations such as semirural, as well as gap filling where a particular DLC system would not operate. Better understanding of operational costs when utilising public telephony systems Site Selection and Geographic Locations The sites selected for the AMI Pilot Project covered rural, suburban, as well as various density areas, with a mixture of installation ages and characteristics. Within each zone substation, selected 11kV feeders were chosen based upon diversity criteria with a general target of around 1,000 customers per feeder. Single feeders were chosen so as to represent a slice of the network infrastructure. Consideration was made such that a proportion of the high voltage and low voltage was above ground and underground and represented a mix of high, medium and low density sites. The geographic areas of the AMI Pilot Project cutover phase are described in Table 3 below and shown on the EnergyAustralia network map (Figure 4 below). Table 3 also shows the zone substations, suburbs and installation characteristics of the sites for the cutover phase of the AMI Pilot Project. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 6

11 Planning and Design 2 Table 3: AMI Pilot Project Cutover Phase Site and Installation Characteristics Zone Substation Suburbs Installation Characteristics Burwood Chatswood Concord Homebush Bay Meadowbank Homebush, Concord, Concord West Longueville, Northwood, Lane Cove, Lane Cove North, Artarmon North Strathfield, Homebush, Concord West, Liberty Grove Homebush, Strathfield Ryde, Putney Mixed suburbs with domestic and some commercial/industrial sites Premise age range from 1920 s to modern Overhead LV (underground HV) Paddington Paddington, Surry Hills Premise age 100+ years old, all Overhead LV and HV Inner-city urban location Somersby Kariong New residential area Premise age approximately 8 to 9 years old Modern Underground Residential Development (Underground LV and HV) Figure 4: Geographic locations of the AMI Pilot Project Cutover Phase AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 7

12 Planning and Design Testing and Reporting of Meter Combinations Overview The planning and design for the cutover testing phase of the AMI Pilot Project was completed in early 2008 with the development of several key project documents, including the Cutover Plan, reporting templates and detailed test scripts. The process of testing for cutover was divided into three key phases: 1. Reliability and performance testing of the Meter Management Systems (MMS) 2. Validation and reconciliation testing of the back office systems 3. End to end testing Initial planning for cutover testing involved using the vendor s MMS off-the-shelf reporting tools however preliminary testing and evaluation of these products found that they were not suitable for EnergyAustralia s requirements. As a result further product development and experimentation was required to enable the cutover testing reports detailed below to be produced. The first seven reports (known as Tests 1-7) cover statistical collection (from the vendor s MMS) for the metering combinations and the results then compared to an agreed set of reliability and performance criteria. Meter combinations had to successfully meet the criteria for each report before moving onto the next report category. Meter combinations that complete testing successfully at this checkpoint then move onto the next phase of cutover testing and reporting. The next six reports (known as Tests 8-13) cover the extraction of meter data from the vendor MMS and the results were subject to validation and reconciliation analysis to meter data collected by alternative processes (e.g., probe reads). Meter combinations had to successfully meet the criteria for each report before moving onto the next report category. The final report (known as Test 14) involved testing the end to end process of reading the meter through to generating the billing invoice. Those meter combinations that successfully completed cutover testing and met the reporting criteria were then subject to the next phase of meeting the selection criteria for cutover (covered later in section 2.3). The reports required to complete testing for cutover were developed using the following sources: Ministerial Council on Energy (MCE) AMI Minimum Functionality Specification December 2007 Current operational requirements on Type 5 meters EnergyAustralia s experience of testing and reporting on the performance of metering configurations and meter reading (using existing operational standards and metrics). A brief description of test reports 1-14 can be found in Table 4 below and further details can be found in Appendix 1: Tests 1-14 Reporting Matrix and Metrics. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 8

13 Planning and Design 2 Table 4: Summary of Cutover Testing Report Categories Group No Test Report Category Test Report Description Test Report Performance Target 1 Meter Collection Daily Statistics Illustrates the trend of the quantity of data received by the MMS at two hourly intervals over the 6 hour period from 0145 to Snapshots of the backhauled data record the number of interval readings returned for each meter on any business day for the previous day per Concentrator /Modem/Point To Point (PTP) device at the vendor MMS software. 100% data collection for all meters without a communications fault 2 Meter Power On to MMS Registration Times Time taken for the MMS to find the meter (first run of power till recognised by the MMS) and register per Concentrator/Modem. No target information gathering only MMS 3 Gaps in Load Profile Illustrates the gaps in load profile data over 48 intervals for the previous day. Snapshots are taken of the MMS database each working day to record the load profile status for each meter for the previous day. The snapshot data is used to populate the 48*7 reports and reports are used to identify any meter with missing data triggering the missing data manual collect business process. 100% data collection for all meters without a communications fault 4a Meter Data Status Reported by Meter Report on meter data status as recorded by the meter, for example: Acceptable = OK/Valid Not Acceptable = Invalid/Error/Corrupt 100% data collection for all meters without a communications fault 4b Negative Consumption Report on meter data where the cumulative total decreases over time (or the interval value is a negative value). 0% meters with regressive cumulative totals 5 Concentrator to Meter Communication Re-try Statistics Concentrator communication to meter statistics on the number of retries. No target information gathering only 6 Response Times - MMS to Meter Command response time to allow comparison between communication methods such as DLC vs PTP. No target information gathering only 7 Communication Faults Communications Fault System (CFS) reports on AMI meters experience communications issues recording details such as prefix, channels and meter technology. No target information gathering only Back Office 8 Hand Held Format (HHF) Pre-Processor (HHF PP) Exceptions Exceptions generated during producing HHF files and the HHF Error Report (exceptions generated during the HHF PP processing of HHF files). Zero exceptions 9 MV90 Loading Exceptions HHF Log Extractor which examines exceptions on MV90 loading such as HHF Load Errors and Time Tolerance. Not to exceed current operational criteria of 3% exceptions 10 MV90 Validation Exceptions Validator produces standard reports to view gaps in data in MV90, Hi/Lo Limit breaches and energy tolerance exceptions. Not to exceed current operational criteria of 3% exceptions AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 9

14 Planning and Design 2 Group No Test Report Category Test Report Description Test Report Performance Target 11 NemSTAR Loading Exceptions Exceptions produced during the loading XV9 files (standard MV90 transfer files) from MV90 into NemSTAR. Not to exceed current operational criteria of 3% exceptions 12 AMI Read vs Probe Read Load Profile Comparison Data interrogation analysis between production National Metering Identifier (NMI) data (NemSTAR) versus same NMI (in test) with remotely read data for meter data values and quality flags for each interval. 100% alignment other than intervals held out due to exceptions above 13 Missing Data Report - NemSTAR NemSTAR report set up to run the Missing Data Report on a meter group (AMI NMIs in test). 100% alignment other than intervals held out due to exceptions above End to End 14 End-to-End - AMI Read to Billing Invoice Execution of test steps on manually read files for any meter vs AMI read file for the same meter. No target information gathering only Tests 1-7: Meter Management Systems Reliability and Performance Tests 1-7 were developed by EnergyAustralia to report the statistical collection (from the vendor s MMS) for the metering combinations and the results then compared to an agreed set of reliability and performance criteria. Detailed test scripts were developed over a six month period for each of the reports and were vendor specific. It was planned that an initial set of fifty meters per meter combination would be tested and the results used to produce the reports 1-7. Meter combinations had to successfully meet the criteria for each report before moving onto the next report category. Meter combinations that complete testing successfully at this checkpoint then move onto the next phase of cutover testing and reporting Tests 8-13: Back Office Systems Validation and Reconciliation Tests 8-13 were selected from existing back office reporting processes. As with Reports 1-7, a detailed set of test scripts were developed and the testing results were used to produce the reports. These tests cover the extraction of meter data from the vendor MMS and the results were subject to validation and reconciliation analysis to meter data collected by alternative processes such as probe reads. Meter combinations had to successfully meet the criteria for each report before moving onto the next report category. Meter combinations that complete testing successfully at this checkpoint then move onto the next phase of cutover testing and reporting Test 14: End to End Process AMI Read to Billing Invoice This testing covered the whole process from remotely reading the meter through to generating the bill for the customer and preparing the data for submission to the market operator. All meter combinations that met the criteria of Tests 1-13 were considered ready for end to end testing (known as Test 14). End to end testing was outlined in the original Cutover Plan and further detailed planning and design would be required at a later date. EnergyAustralia reviewed the planned end to end testing activities in November 2008 and determined that there was no additional value of completing end to end testing on multiple meter combinations that had successfully passed Tests The Cutover Plan and the Report 14 testing scripts were amended to reflect this decision and that one small group of meters would complete end to end testing. The detailed planning and designing of Test 14 identified the following: Criteria to select the meters for testing Learning outcomes expected from the end to end testing Reporting required to monitor and analyse the test results Meter Business System (MBS) system notations to identify customers with cutover meters AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 10

15 Planning and Design 2 Meter Selection Criteria for Test 14 EnergyAustralia developed a set of selection criteria to identify the best performing and most suitable meters for end to end testing. It should be noted that the selection criteria developed here for Test 14 was used as the basis for developing the selection criteria for meters to be cutover (see later section 2.3). The meter selection criteria for Test 14 included: 1. If there are two or more meters at a site, all meters must be either involved in the trial or not involved in the trial there cannot be a combination of meters at the one site. 2. The retailer for the NMI/meter must be an EnergyAustralia Retail customer. (Note that this criteria only applied for the Test 14 sample set of meters. The final cutover program did not discern between retailers). 3. The Retail contract for the meter must have a minimum of four months until expiry date. This would allow for one full meter read and billing cycle to be completed and mitigate risk of the customer changing retailer before the test has been completed. 4. The meter must be selected from a meter combination that successfully met all of the criteria for Tests The Next Scheduled Read Date (NSRD) must be within the next two months (for expediency of testing a full cycle). 6. The meter must have high performance on daily read data availability. a) Meters must have had a history of 100% data available to be suitable for end to end testing. b) Meters that have met all of the criteria 1-5 above will be subjected to further analysis using Report 3 Gaps in Load Profile. Expected Learning Outcomes from Test 14 EnergyAustralia identified a set of expected learning outcomes from end to end testing. These included: 1. Consistent processing of the HHF Exporter on an automated schedule from the vendor MMS. 2. Consistent processing of the HHF file transfer from the MMS host server to the Production HHF PP host server directory. 3. Minimal impact on back office processes for handling exceptions. 4. No impact on the overall data quality levels of MRIM data in the back office. 5. Satisfactory processing timeframes for: a) Billing requests and responses to and from MBS (Manual Requests) b) End-to-end timings through to customer invoice % accuracy of the quality splits (Estimated, Substituted, Actual/Final) for billing data. 7. Determine impact of late supplied AMI data on estimation and substitution process and other business processes. 8. Sufficient evidence to determine if each meter combination is ready for cutover. Reporting for Test 14 EnergyAustralia developed a detailed set of reports to ensure that the monitoring and analysis of the Test 14 results was thorough and complete. It should be noted that the reporting developed here for Test 14 was used as the basis for developing the ongoing reporting and monitoring framework for meters in production, that is the meters that have been cutover (see later section 2.6). AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 11

16 Planning and Design 2 Table 5 below provides an overview of the reporting established for Test 14. Table 5: Test 14 Reporting No Report Title Report Description 1 Test 14 Meter List One off report produced that lists the set of meters being used for Test 14 including NMI, meter and location (address) details. 2 48*7 Report Report showing (48 intervals per day over the last seven days) will be produced and display data received by the MMS as at 0800 each day for the previous day. 3 Data Received over Time Report Report will display data recorded every 2 hours for the prior 24 hours for the previous month. 4 Communications Faults Report Report will provide data on any communication faults raised, closed or actioned by meter/by date. 5 Meter Data Agency (MDA) Exceptions Report Report any exceptions by type and date in MDA back office systems. 6 MDA Operational Support Log Report will provide information on any support activity provided or required including any manual intervention undertaken to progress the flow of data from the vendor MMS to the MDA back office systems. 7 MDA MRIM Billing Register Register will record any MRIM Billing Request or Response file received by the MDA or generated by the MDA for these meters including: receipt of the billing invoice by Revenue Billing (via the billing block) and any action(s) taken invoice release date to the customer actual vs estimated data record once the MRIM Billing Response has been processed AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 12

17 Planning and Design 2 Figure 5 below illustrates the relationships and interactions of the expected meter data flow through the back office systems with the Test 14 reporting. Figure 5: End to End Testing Data Flow and Reporting Overview Reports Mass Market Billing Reports - MDA Reports - SMI Data Flow MBS System Notations to Identify Customers for Test 14 Additional system notations and account checks in MBS were identified to ensure that meter readers do not read meters undergoing end to end testing. The system notations and checks include: Verify that customer has a Meter Installation Type of Manually Read Interval Meter Type 5 Confirm that communications equipment has been installed Set the Hazard code to AMR DO NOT READ, this is a standard hazard code in MBS and informs the meter reader via their hand held device to not read this meter. The meter reader would acknowledge this action by selecting the AMR Skip code in their hand held device Confirm that the comments COMM5 - Do Not Read are noted in the permanent comments field. These comments are also displayed to the meter reader via their hand held device. Confirm type of installation is noted in the Comments field (e.g., AMI DLC Installation). AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 13

18 Planning and Design 2 Figures 6 and 7 below provide examples of the system notations required for meters undergoing end to end testing. Figure 6: Notations Required to Maintain Installation Screen Figure 7: Notations Required to Maintain Installation Comments Screen AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 14

19 Planning and Design Meter Selection Criteria for Cutover (to Production) The set of criteria developed in April 2009 to select meters for cutover consideration was based on the Test 14 meter selection criteria described in section One of the major changes to the meter selection criteria for cutover was gaining key stakeholder support and approval of the cutover process (see section 2.4 below), as Test 14 only involved the one Retailer and that was EnergyAustralia. The meter selection criteria for cutover consideration can be found below in Table 6. Table 6: Meter Selection Criteria for Cutover No Selection Criteria Description 1 If there are two or more meters at a site, all meters must be either involved in the trial or not involved in the trial there cannot be a combination of meters at the one site. 2 The retailer for the NMI/meter must be a retailer who has agreed to have their meters/customers included in the cutover program with EnergyAustralia. 3 The meter must have high performance on daily data read availability. No more than five days of partial data received from 1 st December 2008 to 14 th April 2009 for June cutover No more than eight days of partial data received from 1 st December 2008 to 14 th August 2009 for September cutover 4 The meter must be selected from a meter combination that successfully met all of the criteria for cutover testing. Note: For Criteria No.3, it was found that the main cause for partial data being received for any one day was power outages. Consequently, it was agreed to change the acceptable criteria to no more than eight days of partial data for the September cutover. The detailed activities required to identify meters for Selection Criteria No.3 in the table above are: Produce a Missing Data Report from the vendor MMS for each meter in the Vendor s MMS as at 14 th April (for June cutover) and 14 th August 2009 (for September cutover) Review the meter s Missing Data Report and group into Good, Possible and Not Suitable categories - Good = no more than 3 days of partial data received - Possible = no more than 8 days of partial data received - Not Suitable = greater than 8 days of partial data received Review Possible category to identify meters that have no more than 5 days of partial data received and analyse identified meters history to determine if they are suitable to be marked as Good Produce a 48*7 report for each of the meters in the Good category to check past week s performance and confirm the meter is ready for cutover to production. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 15

20 Planning and Design Stakeholder Approval and Support EnergyAustralia identified the key stakeholders in the cutover process and the need for the support or approval for cutover to be obtained. The communication program along with the support and approval processes were developed in consultation with the stakeholders (shown below in Figure 8) as there was no existing process to follow. The key stakeholders are: Retailers (nine in total, including EnergyAustralia) who were suppliers for customers where selected AMI meters were installed their support and agreement to participate in this phase of the pilot project AEMO support for the smart meters to operate with daily remote reading capabilities for NMI market settlement AER approval for these meters to operate as Type 5 in the market (similar to difficult access sites) under the existing Rule requirements. Figure 8: Stakeholder Approval and Support Process Overview Retailers Existing contacts (as per other market data communications) with retailers were utilised to inform these businesses of the proposed changes to meter data provision arrangements and discuss potential issues. The following activities were identified: Confirm final list of NMIs to be cutover Identify retailers from the final list of NMIs Contact retailers to establish communications and provide cutover program brief Discuss any issues or concerns as required Establish each retailers support to participate in cutover Australian Energy Market Operator EnergyAustralia s Network business made contact with the Head of Metering and Settlements within the previous NEMMCO organisational structure. This enabled effective discussions relating to the key issues, appropriate timing and proper processes. This activity commenced following initial discussions with Retailers involved in cutover. The following activities were identified: Confirm support received from the retailers Contact AEMO to establish communications and provide cutover program brief Discuss any issues or concerns as required Agree cutover program and processes Establish AEMO support for cutover AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 16

21 Planning and Design Australian Energy Regulator The approval of the AER to implement the cutover program was identified as a crucial step in the process. The AER is responsible for ensuring compliance with the National Electricity Rules (NER). Chapter 7 of the NER contains requirements with respect to remotely read meters. AER support to ensure that NER requirements would not be enforced with respect to remotely read meters used in the trial was seen as crucial firstly to allow proper live testing of the data delivery and billing capabilities of AMI and secondly, to enable future customer trialling of AMI pricing and product offerings. The following activities were identified: Confirm final list of NMIs to be cutover Confirm support received from retailers and AEMO Contact AER to establish communications and provide cutover program brief Establish AER requirements and discuss any issues or concerns Request the AER approval for cutover to occur under existing Rule requirements AER to confirm support for EnergyAustralia to cutover selected NMI sites Cutover approved cutover activities can now commence. 2.5 Cutover to Production Processes and Activities The approval and support from stakeholders as detailed in section 2.4 was the final step required prior to commencing the process of cutting over meters to production that is, changing the meters in the MMS to daily remote reading of ½ hour data processed through the existing back office systems and sent to the market on a daily basis for settlement and billing. EnergyAustralia confirmed the final list of meters to be cutover by reviewing a final run of 48*7 reports and the actual process of cutting over each meter can commence. The following activities were identified during cutover testing (see section 2.2.4, Figure 5 End to End Testing Data Flow and Reporting) and confirmed during the months preceding the June cutover. These activities include: Identify gaps in load profile data using the 48*7 reports and vendor MMS reporting Using data sourced from laptop reads or vendor MMS tools, substitute data in the MMS for gaps identified in the load profile data Identify the last read date and time for each meter in MV90 Access vendor s HHF Exporter tool to produce a HHF file from the last read date and time above to the previous day s midnight read and load this data into MV90 Access MDA Back Office Validator tool to confirm loading of meter data in MV90 and transfer to NemSTAR. Missing data reports are reviewed to confirm successful loading of data through to NemSTAR. Update the MMS data collection schedule for each meter to generate 24 hours of data each day for the prior day Access the HHF File Mover (file transfer system) to move the generated HHF Files from the vendor MMS to the MDA back office repository for import into the MDA back office MV90 system Preparation of reports logging progress in conversion and success of meter data provision. During this process, the Cutover Statistics Report monitors the progress of meters being cutover. Once cutover has been completed and all meters are in production, this report will no longer be required. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 17

22 Planning and Design Ongoing Reporting and Monitoring of AMI Pilot Meters in Production The reports developed to monitor the AMI Pilot meters in production were based on the reports used for Test 14 described in section 2.2.4, Table 5. The production reports are produced weekly and grouped by vendor. The primary objective of this reporting activity is to ensure that all meter data generated by the vendor MMS is accurate and delivered on-time to the Retailer and AEMO for settlement and billing. The reporting framework of AMI Pilot meters in production can be found in Table 7 below. Table 7: Ongoing Reporting of AMI Meters in Production Overview No Report Title Report Description 1 Production Meter List Report lists the set of meters currently in production including NMI, meter and location (address), retailer, NSRD details and meters removed from trial. 2 48*7 Report Report will be produced and display data received by the vendor MMS as at 0800 each day for the previous day. 3 Communications Faults Report Report will provide data on any communication faults raised, closed or actioned by meter/by date. 4 Cutover Statistics Report Report will only be required during production reporting to monitor the progress of meters being cutover from the final NMI list. Once cutover has been completed and all meters are in production, this report will no longer be required. 5 MDA MRIM Billing Requests Report Report will provide data (number received and number fulfilled) on all MRIM billing requests for these meters and will be sorted by Billing Request Type (Final, Off Reading, Routine, Sundry). 6 Service Orders Report will list all open service orders and their current status. 7 Lessons Learnt Ongoing recording of lessons learnt, issues raised and resolutions and reporting. 8 End to End Process A one-off process diagram detailing the end to end flow of data from meter read through the back office systems to delivery to external parties (AEMO and market participants) along with the host environment and activity timings. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 18

23 Planning and Design 2 The following diagram (Figure 9) shows the relationships and interactions of the expected meter data flow through the back office systems with the reporting for AMI Pilot Project meters in production. Figure 9: AMI Pilot Project Meters in Production Data Flow and Reporting Overview AMI Pilot Project Cutover Production Data and Reporting Flow End To End Reports - MDA Reports - SMI Data Flow HHF Verifier / HHF Vendor MMS Exporter from MV90 NemSTAR Pre-Proc MMS Report 1 Production Meter List Report 2 48*7 Report HHF Viewer Report 3 Cutover Statistics Report Validator Report 5 Communications Faults Report TCA Database Script Task Manager MBS Raise Billing Request Missing Data Report Report 6 Service Order Maintenance Report MDA Billing Engine MBS Receive Billing Response Report 5 MDA MRIM Billing Register Customer Care & Service (SAP-CCS) XX EnergyAustralia system XX Third Party System XX Verifier Vendors 1 & 2 Exporter Vendor 1 & EnergyAustralia XX Production Report (Weekly) XX Production Report (until completion of cutover) AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 19

24 3 Results and Key Findings 3.1 Cutover Testing Summary Overview As detailed in section 2, the cutover testing phase of the AMI Pilot Project was a detailed process involving the testing of multiple meter combinations initially selected from the 6756 meters installed in September Leading up to consideration of each combination for cutover the technologies and vendors were reviewed. It was recognised that there had been significant developments in the AMI environment over the preceding 3 years and that any consideration of solutions for cutover would require commitment from vendors to maintain the deployed equipment as well as conform to the draft National Minimum Functionality Specifications for smart meters. From this review and following discussions with vendors, two vendors elected to withdraw from the pilot program. Solutions provided by the remaining two vendors continued to be subjected to the cutover testing criteria described in section 2 of this report. A summary of results and key findings from the three phases of cutover testing is provided below in the following sub sections: Tests 1-13 Key Findings Tests 1-7 Results for Vendors 1, 2, 3 and 4 (Meter Management Systems reliability and performance testing) Tests 8-13 Results for Vendors 1 and 2 (Back Office Systems validation and reconciliation testing) Test 14 Results for Vendor 1 (end to end testing) Tests 1-7 were completed by all vendors covering fourteen meter combinations, for all meters installed per combination at the time of testing. Tests 8-13 were completed by Vendors 1 and 2 each with five meter combinations of up to 50 meters per combination. Test 14 was completed by Vendor 1 with 15 meters covering two meter combinations. Cutover testing was undertaken in two groups, the first group in the first half of 2008 and the second group in April and August Table 8 below provides a summary of the cutover testing results completed for this phase of the AMI Pilot Project. Figure 10 below provides an overview of the cutover testing timeline for the multiple meter combinations. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 20

25 Table 8: Summary of Cutover Testing Results Group No Test Category V 1 GPRS V1 RS485 V1 DLC V2 DLC V2 GPRS V3 DLC V4 GPRS MMS Back Office End to End 1 Meter Data Collection Daily Statistics X 2 Meter Power On to MMS Registration Times 3 Gaps in Load Profile X X 4a Meter Data Status reported by Meter N/A 4b Negative Consumption 5 Concentrator to Meter Communication Re-try Statistics N/A* N/A* N/A N/A* N/A N/A* 6 Response Times - MMS to Meter 7 Communications Faults 8 HHF Pre-Processor Exceptions 9 MV90 Loading Exceptions 10 MV90 Validation Exceptions X 11 NemSTAR Loading Exceptions 12 AMI Read vs Probe Read Load Profile Comparison 13 Missing Data Report - NemSTAR 14 End-to-End - AMI Read to Billing Invoice * * X * Ø Ø Key Passed * Passed based on Vendor 1 GPRS Test 14 Results X Ø N/A N/A* Failed Did not continue in Trial MMS did not support the data required for this report Test is not applicable for this metering configuration AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 21

26 Figure 10: Cutover Testing Timeline AMI Cutover Testing Timeline Vendor 1 GPRS Tests 1-7 (258 meters) May 08 Tests 8-13 (50/27 meters) July 08 Test 14 (15 meters) Jan - April 09 Cutover June 09 Cutover Sept 09 Vendor 1 DLC Tests 1-7 (2784 meters) April 09 Test 8-13 (2x50 meters) April 09 Vendor 1 RS485 Tests 1-7 (322 meters) May 08 Tests 8-13 (50 meters) April 09 Vendor 2 GPRS Tests 1-7 (10 meters) Aug 09 Tests 8-13 (20 meters) Aug 09 Vendor 2 DLC Tests 1-7 (343 meters) May 08 Tests 8-13 (3x50 meters) July 08 Removed from Trial Vendor 3 DLC Tests 1-7 (602 meters) Mar 08 Removed from Trial Vendor 4 GPRS Tests 1-7 (319 meters) Sept 08 Removed from Trial AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 22

27 3.1.2 Tests 1-13 Key Findings This section provides a selection of key findings from Tests 1-13 including an overview in Figure 11 of the data flow and back office system integration developed for the AMI Pilot meters. The following sub-sections include: Data flow and back office system integration process flow illustrating: - data flow from the AMI Pilot meter to billing, AEMO and market participants - system applications developed for testing to be successful - integration with existing back office systems and tools General findings HHF File Generation Requirements Specification HHF Verifier and Exporter Modules AMI HHF Verifier and MDA Launch Pad AMI HHF File Mover Figure 11: Cutover Testing Data Flow and Back Office System Integration AMI Pilot Project Cutover Testing Data Flow and Back Office System Integration Existing Back Office process AMI Pilot Project new process AMI Pilot Meter TCA Gateway MBS Files batched for sending to MBS What s Up Gold Monitoring tool Meter to MMS Communication Technology (GPRS, DLC, RS485) HHF Pre-Processor Vendor 1 MMS Vendor 2 MMS MV90 MDA Launch Pad (User interface for HHF Verifier) HHF Verifier HHF Verifier HHF Exporter HHF Exporter NemSTAR AMI HHF File Mover Market Participants AEMO AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 23

28 General Findings Planned enhancements will remove the daily batch process and enable the HHF Pre-Processor to collect HHF files approximately every 5 minutes. This enhancement will reduce the overall processing time of HHF files by at least 24 hours and will further improve processing partial data sporadically received throughout the day. The use of 48*7 reports (which identified gaps in load profile data received) proved to be critical during testing to identify poorly performing concentrators or meters more easily and enabling a faster fix to data collection problems. The standard reporting provided by the MMS was not capable of providing the level of detail required, nor the flexibility to conduct more detailed analysis. High probe read failure rate for some meter combinations meant that difficulties were experienced when running Tests 8-13 because remotely read data could not be compared with accurate probe read data. A key learning was to only select meters for testing where probe read data was robust. The use of an existing tool What s up Gold was invaluable in monitoring and resolving telecommunications issues with Vodafone and Telstra. It allows a greater level of visibility to connections with these providers and also enables ping and FTP trials to be conducted. MBS changed their HHF file collection process from the TCA Gateway to cater for the large volume of daily files being received for processing through the billing systems. The existing MBS process of collecting individual files was changed to batching the individual files into larger files for more efficient processing by MBS. Due to the myriad of specific challenges relating to integration of vendor and EnergyAustralia systems, it was found that EnergyAustralia was often dependent on vendors to be available and responsive when issues arose. Vendors having dedicated resources with the required expertise and available locally to support this testing phase was critical to both the timing and success of the program. HHF File Generation Requirements Specification Development of the HHF File Generation Requirements document by EnergyAustralia assisted the vendors with their product and system development. The Requirements document established one standard for all vendors to use (e.g., pulse units versus engineering units caused problems when comparing probe or laptop read data to the vendor MMS data. MV90 accepts HHF files with engineering units, this required a software upgrade by one vendor to change HHF file format to engineering units) The full Device ID from the meter needs to be exported from the vendor MMS. It was found that one vendor initially removed the first character causing MV90 rejections due to no meter being found by MV90. The requirements document was updated and a software upgrade release by the vendor to correct the problem. Vendors had difficulty complying with the original specification as there was insufficient detail. Enhancements to the requirements document assisted vendors in their ongoing product development such as the HHF Verifier/Exporter modules. HHF Verifier and Exporter Modules Vendors 1 and 2 MMS contained generic HHF Verifier and Exporter modules that required enhancements to process the HHF files. Vendor 2 developed their HHF Verifier and Exporter modules whilst for Vendor 1, EnergyAustralia developed the AMI HHF Verifier and Vendor 1 developed their HHF Exporter module. Without these modules, meters would not be able to cutover to production. HHF Verifier and Exporter modules need to support the extraction of partial days of data, EnergyAustralia developed tools (MDA Launch Pad and AMI File Mover) to assist with managing and processing partial data extracted throughout the day from the vendor s MMS. HHF file data was processed in ascending date order (this is an MV90 system requirement however the vendor s original MMS did not provide this requirement and enhancements to the HHF Verifier modules assisted in correcting this issue). Reading and interpreting the HHF file in its binary format made it difficult to complete some tests and to investigate errors and rejections with the HHF files. It was found that Vendor 2 s MMS HHF interpretation module was compatible to other vendor s HHF files and was successfully used to investigate HHF file errors and rejections. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 24

29 AMI HHF Verifier and MDA Launch Pad EnergyAustralia developed the AMI HHF Verifier and MDA Launch Pad (user interface) along with a dedicated server to support these requirements for Vendor 1 as their MMS did not provide a tool for managing and scheduling daily exports of data from the MMS in HHF files. The development of these applications required greater resources and capabilities than planned. The AMI HHF Verifier covers two areas of functionality: Daily Run: Each day at a time determined by a preset parameter, the AMI HHF Verifier determines when the last Daily Run is completed. The application then generates a new HHF request for each production meter for all previous full days (i.e. to 00:00 of the day of execution) from the date of the last run. Cyclic Run: Every cycle time is determined by a wait by the application of a number of minutes since the previous cyclic run completed. This is determined by a preset parameter. For each cycle, the application checks all outstanding HHF requests and determines which requests can be fulfilled. A series of Business Rules are applied to ensure that requests for days later than other outstanding requests for the same meter are held until all earlier data can be generated. Figure 12 provides a screen shot of the AMI HHF Verifier confirmation of a cyclic run being completed. Figure 12: AMI HHF Verifier The MDA Launch Pad (user Interface application) was also developed to enable monitoring and maintenance capabilities during the generating of HHF files. As shown in Figure 13 below, the MDA Launch Pad toolbar was a key user requirement that provided easy access to a set of functions and screens to manage, create and modify requests for HHF Files as required. The MDA Launch Pad also provides functionality including: viewing of archived requests as well as control of various parameters used by the AMI HHF Verifier reporting to allow monitoring and investigation of meters failing to read allows ad-hoc HHF file requests to be created to provide data to the back office if it was not received in the daily batch run AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 25

30 The MDA Launch Pad is a client application running locally on workstations in the MDA Local Area Network. Figure 13: MDA Launch Pad AMI HHF File Mover In order to deliver the HHF files to the back office for processing, EnergyAustralia developed an interface between the vendor s MMS and the existing back office systems via the TCA Gateway. The AMI HHF File Mover runs cyclically (current business rules are every 5 minutes from 8am to 6pm) and archives all copied files to allow for future investigation if required. The current version of the AMI HHF File Mover is a client application running locally on workstations (one per vendor) in the MDA Local Area Network. There are enhancements currently under development to combine the AMI HHF Verifier and the AMI File Mover to further streamline the process of moving the HHF file data to the back office for processing Tests 1-7 Results for Vendor 1 Vendor 1 underwent Tests 1-7 on five meter combinations consisting of: GPRS communications 258 meters (229 single phase and 29 three phase) - testing completed in May 2008 DLC communications 2686 meters (2465 single phase and 221 three phase) - testing completed in April 2009 RS485 communications 326 meters (single phase) - testing completed in May 2008 Overall Vendor 1 meters performed well for Tests 1-7 (see Section above, Table 7: Summary of Test Results) and were selected to progress to Tests Detailed below is a summary of the results found during testing. AMI Pilot Project Phase 1 AMI Data and Systems Integration for Market Settlement and Billing 26