NATIONAL GRID COMPANY PLC - (A wholly owned subsidiary of National Grid Transco plc)
NATIONAL GRID COMPANY PLC - (A wholly owned subsidiary of National Grid Transco plc)
National Grid Company plc 10 September 2003 Page 1 NATIONAL GRID COMPANY PLC (A wholly owned subsidiary of National Grid Transco plc) Investigation Report into the Loss of Supply Incident affecting parts of South London at 18:20 on Thursday, 28 August 2003 This report has been produced by National Grid Company plc (National Grid) to record the investigation findings concerning the loss of supply in south London on 28 August 2003. The purpose of the investigation is to enable National Grid to identify the cause or causes of the incident so it may seek to prevent a recurrence. The purpose of the report is not, however, to identify legal liability; therefore the data and information within it have not been compiled in accordance with rules of evidence and cannot be treated as determining either National Grid’s nor any individual’s legal liability.
National Grid Company plc 10 September 2003 Page 2 Index page Executive summary 3 Introduction Transmission System in South London Maintenance Activity in the Area South London Transmission System (diagram) The First Fault 4 The Second Fault Restoration 5 Communication Investigation 6-7 Actions being pursued 7 Investigation Report 9 Introduction Overview of the incident Background 10 Transmission system in south London (Map of affected substations) Investment programme in the area 11 (London area investment –graph) Key policies and procedures relevant to the incident 12 Findings Operating arrangements on the day 13-14 (Table of circuits out of service on 28 August) Sequence of Events 14-18 Communication during the incident 18 Response to the Buchholz alarm 19 Disconnection of the transformer 20 Unexpected operation of the protection 20-21 Maintenance of assets 21 Other factors 22 Configuration of EDF Energy’s substation 22-23 Communications 23 Investigation 24-25 Actions being pursued 26 Appendices 27-43
National Grid Company plc 10 September 2003 Page 3 Investigation Report into the Loss of Supply Incident affecting parts of South London at 18:20 on Thursday, 28 August 2003. Executive Summary Introduction 1 A combination of events led to an electricity power supply failure in south London that occurred at 18.20 on 28 August. Restoration began at 18.26 and power supplies from National Grid were fully restored at 18.57. This report describes the circumstances leading to the loss of supply, the steps taken to restore supplies and the measures in hand to minimise the risk of a recurrence.
Transmission System in South London 2 The transmission system in south London consists of four substations at Littlebrook, Hurst, New Cross and Wimbledon.
Normal demands of around 1,100MW are drawn by EDF Energy to supply domestic customers and London Underground, together with supplies for other large users including NetworkRail. Following the incident supplies were lost from Hurst, New Cross and part of Wimbledon. Maintenance Activity in the Area 3 On 28 August 2003, scheduled maintenance was underway on one circuit from Wimbledon to New Cross and one from Littlebrook to Hurst. This level of maintenance is usual during the summer months, when demand for electricity is generally lower.
4 In line with normal practice, the arrangement of the transmission system to accommodate the maintenance had been agreed with the operator of the distribution system for the London region, EDF Energy, well in advance, during July 2002. Routine weekly communication between EDF Energy and National Grid resulted in the planned outage at Wimbledon proceeding on 1 July 2003. EDF Energy confirmed that it could arrange its distribution system to accommodate this outage securely for the maintenance period. Figure 1: Schematic of the south London transmission system Wimbledon New Cross Hurst Littlebrook Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance
National Grid Company plc 10 September 2003 Page 4 The First Fault 5 The sequence of events started at 18:11. Engineers at the Electricity National Control Centre (National Control) received an alarm indicating that a transformer, or its associated shunt reactor, at Hurst substation was in distress and could fail, potentially with significant safety and environmental impacts. This “Buchholz alarm”, told National Control that gas had accumulated within the oil inside the equipment, which can lead to a major failure. National Grid has approximately 1,000 transformers with associated equipment connected to its transmission system and on average only 13 Buchholz alarms are received each year.
6 National Control contacted EDF Energy to discuss the Buchholz alarm and asked EDF Energy to disconnect the distribution system from the transformer. Then, as is normal practice in this situation, National Control initiated a switching sequence to disconnect the transformer from the transmission system. This switching sequence temporarily left supplies dependent on a single transmission circuit from Wimbledon that feeds New Cross and Hurst substations. Under National Grid operating procedures a Buchholz alarm is sufficiently serious to warrant the isolation of equipment and reduced security is acceptable for “switching time”.
This is a period of time, normally around five to ten minutes, during which the transmission system is rearranged, by connecting and disconnecting circuits, so that the affected equipment can be taken out of service.
7 The switching sequence to remove the transformer began at 18:20, disconnecting Hurst substation from Littlebrook substation. This enabled a safe shutdown of the transformer which had suffered the alarm, but left Hurst supplied only from Wimbledon via New Cross. The Second Fault 8 Unexpectedly, a few seconds after the switching, the automatic protection equipment on the number two circuit from Wimbledon to New Cross operated, interpreting the change of power flows, due to the switching, as a fault. 9 The transmission system is extensively fitted with many levels of automatic protection equipment, aimed at isolating faults and preventing damage to equipment or even a complete shutdown of the transmission system.
They measure system characteristics, such as voltage and current and, in the event of a fault, will automatically disconnect affected equipment. On the National Grid transmission system there are approximately 43,000 such pieces of equipment, each with its individual settings to meet local requirements.
10 The automatic protection relay disconnected the circuit from Wimbledon to New Cross. This disconnected New Cross, Hurst and part of Wimbledon from the rest of the transmission system, causing the loss of supply. 724MW of supplies were lost, amounting to around 20% of total London supplies at that time. This affected around 410,000 of EDF Energy’s customers, with supplies being lost to parts of London Underground and NetworkRail.
National Grid Company plc 10 September 2003 Page 5 Restoration 11 Restoration actions began at 18:26, re-energising the Hurst substation from Littlebrook and then isolating the Wimbledon to New Cross circuit, that had automatically disconnected itself, to prevent a recurrence.
12 At 18:38 National Control offered to restore supplies to Wimbledon for EDF Energy. EDF Energy requested restoration of that supply at 18:48 and restoration was completed at 18:51. From this point onwards, London Underground could restore electricity to the underground network, when they considered it was safe to do so. 13 At 18:41 EDF Energy restored supplies via National Grid’s Hurst substation to approximately one third of the consumers.
14 Some 30 switching actions enabled National Grid to restore overall supplies to all substations concluding with New Cross at 18:57 which restored the remaining supplies for NetworkRail. The substations remained connected to the rest of the transmission system via a single circuit until 23:00, the time at which the automatic protection equipment that had operated at Wimbledon was successfully isolated. The number two circuit from Wimbledon to New Cross was then safely returned to service and normal levels of security were restored. A rapid check was made to similar automatic protection equipment.
Communication 15 During the incident there was significant operational communication between National Grid and EDF Energy. Communications were initiated at 18:17, following the Buchholz alarm being reported, and EDF Energy were requested to remove the demand from the transformer. At 18:21 EDF Energy called National Grid to confirm that there was a problem on the transmission system. 16 Such operational communications continued throughout the restoration, with continuous telephone conversations between control engineers at National Grid and EDF Energy’s control centre, working together to reconnect the affected area.
Some 17 minutes later National Grid offered to restore supplies to Wimbledon and New Cross.
17 At 18:51 National Grid was called by New Scotland Yard and National Control informed them that this was a system incident with no third party involvement. 18 The complex and rapidly changing chain of events affected a large number of organisations. In the wider communication exercise through Thursday night and Friday, in addition to briefing the media, National Grid was in contact with the emergency services, the DTI, Ofgem, the London Mayor, energywatch and others.
National Grid Company plc 10 September 2003 Page 6 Investigation 19 The planning of maintenance works had been carried out in accordance with National Grid’s policies and that the maintenance work could not be regarded as a cause of the incident.
The investigation confirmed that the transmission system arrangement and the communication with the distribution system operator regarding this maintenance complied with the relevant National Grid planning standards and operating procedures. 20 All actions in configuring and switching the transmission system complied with National Grid’s planning standards and operating procedures and that the restoration process was carried out quickly and professionally without further incident. The response by control engineers to re-secure the network and restore the balance of generation and demand ensured that the disturbance was contained within the affected substations.
21 The reason that the second fault occurred was that an incorrect protection relay was installed when old equipment was replaced in 2001. This incorrect installation was not discovered despite extensive quality control and commissioning procedures followed by both supplier’s and National Grid’s specialist staff. This piece of equipment has been replaced. Once the cause was known an extensive survey of similar equipment was immediately initiated. To date 20% (9,000 items) of this type of equipment on the National Grid system has been surveyed and there have been no similar cases. The remaining equipment will be surveyed within four weeks.
22 The engineers involved in the commissioning of the automatic protection equipment had the appropriate training, authorisation, experience and skills to undertake the task. There is evidence that the detailed commissioning procedures were followed correctly at all stages and that no part of the process had been omitted. However, the rating of the automatic protection equipment that is included on the documentation used for commissioning could have been more clearly visible to the commissioning engineers.
23 The actions to remove the Hurst transformer did not directly contribute to the cause of the incident. The consequential increase in flows on the Wimbledon to New Cross circuit, which were within operational limits, initiated the operation of the protection relay at Wimbledon. National Grid engineers would not expect their actions to remove the equipment would have caused the loss of supply. 24 The impact of the incident on the areas of south London was exacerbated by the loss of supplies to underground and railway transport services.
25 From the 20 July, EDF Energy’s distribution system was arranged such that a significant supply to London Underground was dependent on a single transmission circuit.
This meant that in the event of a fault occurring on one of National Grid’s transformers at Wimbledon the distribution system configuration would result in a loss of supply. However, National Grid understands that EDF Energy had contingency arrangements for immediate restoration of supplies to London Underground in such an eventuality.
National Grid Company plc 10 September 2003 Page 7 26 Following normal practice, during the incident there was extensive communication between National Control and the EDF Energy Control Centre, with both control rooms working effectively together during the incident. Actions being pursued 27 This is the largest loss of supply from National Grid for over ten years and the company has expressed its deep regret. This incident involved a number of other parties and National Grid will be working closely with them in the coming weeks to examine the consequences and identify improvements in systems or procedures.
National Grid has reviewed its part in the incident and is committed to the following steps: · National Grid will work closely with other network operators to identify any improvements in co-ordination to enhance the overall security of electricity supplies, particularly to city centres and transport systems. · National Grid will work closely with EDF Energy, the Mayor, London Underground, NetworkRail and other London emergency and public service agencies to establish improved and more responsive communications in the event of major loss of supply. · National Grid is urgently surveying all installations as a further check on the integrity of the automatic protection equipment.
National Grid will carry out a further comprehensive investigation examining all aspects of the management of the protection systems so as to eliminate, as far as possible, the risk of incorrect installation or operation of automatic protection equipment. · National Grid will work to review operational procedures, and control room systems, including alarm presentation, in close consultation with Ofgem, DTI and other associated parties, to ensure that there is the right balance between safety risks and supply security.
National Grid Company plc 10 September 2003 Page 8 NATIONAL GRID COMPANY PLC (A wholly owned subsidiary of National Grid Transco plc) Investigation Report into the Loss of Supply Incident affecting parts of South London at 18:20 on Thursday, 28 August 2003 This report has been produced by National Grid Company plc (National Grid) to record the investigation findings concerning the loss of supply in south London on 28 August 2003.
The purpose of the investigation is to enable National Grid to identify the cause or causes of the incident so it may seek to prevent a recurrence. The purpose of the report is not, however, to identify legal liability; therefore the data and information within it have not been compiled in accordance with rules of evidence and cannot be treated as determining either National Grid’s nor any individual’s legal liability.
National Grid Company plc 10 September 2003 Page 9 Investigation Report into the Loss of Supply Incident affecting parts of South London at 18:20 on Thursday, 28 August 2003 INTRODUCTION 28 National Grid Company plc (National Grid), a wholly owned subsidiary of National Grid Transco plc, transports electricity and balances the system on a second by second basis. National Grid delivers electricity from generators and interconnectors to 12 distribution network operators for local distribution to over 24 million consumers and directly to a small number of large industrial users. National Grid is the sole holder of an electricity transmission licence for England and Wales and has a statutory duty under the Electricity Act 1989 (as amended by the Utilities Act 2000) to develop and maintain an efficient, coordinated and economical system of electricity transmission and to facilitate competition in the supply and generation of electricity.
OVERVIEW OF THE INCIDENT 29 On 28 August 2003, two events occurred on the National Grid electricity transmission system in south London, resulting in an electricity supply failure on the transmission system from 18.20 until 18.57.
30 The loss of supply occurred following a switching operation to remove a transformer at Hurst 275kV substation from the transmission system in response to an indication that a serious alarm had been activated on the transformer or its associated shunt reactor. Actions were taken to remove the transformer from the system which required a controlled disconnection of the circuit between Littlebrook and Hurst. For a short period of 5 to 10 minutes (switching time) this resulted in the supply at New Cross, Hurst and parts of Wimbledon being dependent on a single transmission circuit. Within seconds of this operation the circuit between Wimbledon and New Cross substations automatically disconnected itself.
The combination of these two events was to isolate Hurst, New Cross and a part of Wimbledon 275kV substations from the main transmission system, disconnecting 724MW of supplies to EDF Energy’s distribution network.
31 The loss of supply affected 410,000 of EDF Energy’s customers in an area of south London approximately bounded by Bexley in the east, Kingston in the west, Bankside in the north and Beckenham in the south, and led to significant disruption to London Underground and NetworkRail. Supplies from the transmission system to EDF Energy were restored within 37 minutes. 32 Roger Urwin, National Grid Transco’s Chief Executive Officer initiated a incident investigation chaired by Nick Winser, Group Director Transmission and Chief Executive of National Grid Company plc.
33 This is the outcome of the incident investigation into the events of the 28 August 2003.
National Grid Company plc 10 September 2003 Page 10 BACKGROUND The transmission system in south London 34 The National Grid transmission system provides an integrated network for the bulk transfer of power across England and Wales. The transmission system, which is operated at 400,000 volts and 275,000 volts, connects major power stations and delivers electricity to the regional distribution networks. The peak demand on the England and Wales transmission system is around 54,400MW. The demand for electricity in the Greater London area represents about 20% of the total transmission system demand in England and Wales.
35 The transmission system has been designed and built for an expected life of between 15 years and 80 years, depending on the type of asset. 36 There are no large generation stations connected directly to the transmission system in the Central London area, although large power stations exist close to London at Barking, Grain, Littlebrook and Kingsnorth. The transmission system facilitates the transmission of power from these and more remote generating stations to London. Figure 2: Transmission system in south London © Crown Copyright, National Grid Transco EL273384 37 The south London 275kV network between Wimbledon and Littlebrook is shown in the above figure.
The network is made up of substations, which include switchgear, transformers, shunt reactors and protection and control equipment. These are connected by circuits comprising overhead lines and cables. A description of the assets that make up the south London network is provided in appendix 1.
National Grid Company plc 10 September 2003 Page 11 Investment Programme in the Area 38 Since 1990, around £3,600m has been invested in the National Grid transmission system. Of this, approximately £700m has been invested in the transmission system in the Greater London area. 39 Major elements of this work include connection works for new generation at Barking and Kingsnorth, construction of two new substations at West Ham and St Johns Wood, and major infrastructure reinforcement including the new 20km cable between Elstree and St Johns Wood.
40 Since 1995/96 investment in the London area has been increasing and has been around £100m per year for the last 2 years as the new Elstree-St Johns Wood tunnel and cable circuit has been constructed.
Figure 3: Investment in the London area London Area Investment 20 40 60 80 100 120 95/96 96/97 97/98 98/99 99/00 00/01 01/02 02/03 £m Outturn 41 Of the investment in the London area, around £75m has been invested in the Littlebrook to Wimbledon 275kV system and the adjacent network in south London. Specific projects have included: · Supply point reinforcement works at Littlebrook and New Cross · 275kV cable works on the circuits between between Hurst, New Cross and Wimbledon over the period 1995 to 2002 · Works to provide a new tunnel under the River Thames at Dartford and new 275kV cables for the Littlebrook to West Thurrock 275kV circuits · Switchgear replacement · Automatic protection and control system replacement.
Environmental improvement works at several sites, such as enhancing oil containment works.
42 As part of National Grid’s planned asset replacement programme, future work in the Littlebrook to Wimbledon 275kV system and the adjacent network in south London
National Grid Company plc 10 September 2003 Page 12 includes the replacement of shunt reactors, replacement of the Beddington-Rowdown 275kV cable, the extension of the supply point at New Cross and improvements to cable cooling systems and joint bays. 43 The average expenditure in the London area is planned to be over £50m per year over the next 5 years. 44 There has been a considerable investment programme in the transmission system in and around London since 1990, and this programme is set to continue at a high level in future years.
Key Policies and Procedures Relevant to the Incident 45 As part of National Grid’s responsibility to operate a safe, secure and reliable transmission system it has a responsibility to ensure that the asset related safety, environmental and operational risks are managed and acceptable. In addition, the company has an obligation to carry out this duty in an efficient manner. To this end National Grid has developed an asset management approach which uses a combination of maintenance, refurbishment and replacement strategies. The procedures, which provide a framework for the delivery of this approach, are well established and defined and subject to external audit as part of National Grid’s ISO 9001 accreditation.
Details on National Grid’s maintenance, asset replacement and commissioning policy are included in appendix 2.
National Grid Company plc 10 September 2003 Page 13 FINDINGS Operating Arrangements on the day 46 On the evening of Thursday 28 August 2003, the transmission system in the south of London was secure and was operating in accordance with the relevant National Grid planning standards and operating procedures (Appendix 3 provides details). The substations were all configured in a secure manner supplying normal demands of around 1,100MW. 47 The transmission system in the area was arranged with a number of circuits out of service for scheduled maintenance.
Circuit Reason Dates Number two circuit from Littlebrook to Hurst Installation of thermal monitoring on cables, maintenance of circuit breakers and other planned maintenance at Hurst and Littlebrook 26 August to 19 September Number one circuit from Wimbledon to New Cross Major refurbishment of the cables and installation of new protection and control systems 1 July to 28 September 48 The Wimbledon, New Cross and Hurst substations, that were to be affected by the incident, were connected to the rest of the transmission system by two circuits, ensuring that a single transmission fault would not result in a loss of supply.
49 A simple diagram of the transmission network in this area of London is illustrated in the figure below.
Figure 4: Schematic of the Transmission System in South London Wimbledon New Cross Hurst Littlebrook Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance 50 If a fault did occur and left supplies dependent on a single transmission circuit, in most cases, National Grid was able to restore security of supply within switching time or by returning one of the circuits that was out on maintenance. 51 To maintain an efficient transmission system it is necessary to undertake planned maintenance. The disconnection of circuits and network configuration is planned and agreed following a rigorous process that involves studies to both optimise the coordination of outages with parties connected to the system and to ensure the network configuration is secure against all credible faults that may occur.
52 The planning process to agree these outages and the configuration of the system began in July 2002. As part of the process regular liaison meetings were held and exchanges of information undertaken with EDF Energy (on a daily basis in the last two months). These discussions concluded in a formal agreement for the release of
National Grid Company plc 10 September 2003 Page 14 the circuits before the work was due to commence. As is normal, the maintenance schedules of both EDF Energy and National Grid were subject to change during the early part of the summer. EDF Energy were fully engaged in this process and fully aware of the configuration of the transmission system and its impact on its system. As part of this process the contingency plans for EDF Energy’s 132kV Wimbledon substation were discussed, as the planned outage would entail a reduction in the number of feeds from the National Grid transmission system from four to three.
National Grid understood that due to limitations on the EDF Energy’s system, some distribution supplies from Wimbledon would be dependent on a single transmission circuit. National Grid understands that in the event of the loss of this transformer EDF Energy’s post fault action would be to immediately switch its Wimbledon Grid 132kV substation to reinstate supplies, from the remaining two National Grid transformers.
53 If the circuits that were out for maintenance had been available, clearly there would have been no loss of supply. However maintenance is an essential part of sustaining an efficient transmission system and to increase security above the current standards would require a huge investment in new transmission assets. 54 The investigation confirmed that the configuration of the transmission system was not a contributory factor to the loss of supply. Sequence of Events 55 The sequence of events commenced on Thursday 28 August at 18:11 when an alarm was received at the Electricity National Control Centre (National Control) at Wokingham.
The system configuration at the start of the incident and the power flows are illustrated below.
Figure 5: Transmission system configuration and power flows Wimbledon New Cross Hurst Littlebrook 72 MW 486 MW Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance Supplying EDF 478 MW Supplying EDF 359 MW Supplying EDF 199 MW Supplying EDF 84 MW 56 The sequence of events during the incident is described below. The detailed switch operations is attached in appendix 4. 57 At 18:11 staff at the National Control received an indication that a transformer or shunt reactor at Hurst substation was in distress and could fail, with potentially significant safety and environmental impacts.
The indication, called a “Buchholz alarm”, told National Control that gas had accumulated within the oil inside the transformer or shunt reactor, which can lead to equipment failure.
National Grid Company plc 10 September 2003 Page 15 58 At 18:17 discussions were held with EDF Energy regarding the Hurst transformer or shunt reactor. National Control informed EDF Energy that the transformer was to be switched out of service. To achieve this the transmission system had to be rearranged by switching equipment and circuits in and out, so the affected equipment could be safely and securely taken out of service. During switching time (typically 5 – 10 minutes) one circuit would supply New Cross and Hurst substations. 59 EDF Energy confirmed that they had disconnected the transformer from the distribution system.
There was no impact on supplies to EDF Energy as these remained connected to Hurst substation via two other transformers. 60 The immediate priority was security of supplies. At 18:19 the circuit from Littlebrook to Beddington and Kemsley was switched in. This reconfiguration of the network ensured power flows at Littlebrook substation would be secure once the number one circuit from Hurst to Littlebrook was switched to take the Hurst transformer and associated Hurst shunt reactor three out of service.
61 At 18:20 two circuit breakers were opened at Hurst to remove the transformer or shunt reactor from service. At this point, the Hurst and New Cross substations were supplied from Wimbledon 275kV substation and were dependent on the single, number two circuit, from Wimbledon to New Cross. Figure 6: Transmission system at 18.20 Wimbledon New Cross Hurst Littlebrook 558 MW Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance Supplying EDF 478 MW Supplying EDF 359 MW Supplying EDF 199 MW Supplying EDF 84 MW National Control action 62 Immediately following the opening of the two circuit breakers at Hurst, the automatic protection relay operated on the number two circuit from Wimbledon to New Cross automatically opening two circuit breakers at Wimbledon and removing the number two circuit from Wimbledon to New Cross from service.
This isolated New Cross, Hurst and part of Wimbledon substations from the rest of the system. All supplies were lost at Hurst and New Cross substations and 35% of the supplies were also lost to EDF Energy’s Wimbledon Grid 132kV substation. Two transformers at Wimbledon continued to supply Wimbledon Grid 132kV substation.
63 The transmission network was now configured as in the diagram below.
National Grid Company plc 10 September 2003 Page 16 Figure 7: Transmission system immediately following the incident Wimbledon New Cross Hurst Littlebrook Supplying EDF 359 MW lost Supplying EDF 166 MW lost 312 MW maintained Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance Automatic action National Control action Supplying EDF 199 MW lost Supplying EDF 84 MW maintained 64 Following the event EDF Energy transferred 72MW of demand supplied by Wimbledon Grid 132kV substation to another supply point by reconfiguring the distribution network.
65 Assessing the alarms received, National Control concluded that the automatic protection equipment on the number two circuit from Wimbledon to New Cross had most likely operated incorrectly. 66 At 18:21 National Control and EDF Energy discussed the loss of supply and the substations affected. 67 At 18:22 standby engineers were called out to Wimbledon, New Cross and Hurst substations to investigate and help restore supplies. 68 At 18:23 National Control commenced the sequence of operations to restore security for the remaining transmission system and restore supplies. It was decided to keep the number two circuit from Wimbledon to New Cross out of service until the cause of the operation of the automatic protection relay was established, as it was highly probable that if switched into service the automatic disconnection would recur.
69 At 18:25 the network was reconfigured to isolate the number two circuit from Wimbledon to New Cross, while Wimbledon substation was fully energised by closing the two circuit breakers which had automatically opened following the earlier operation of the automatic protection equipment. These actions re-secured the transmission system against further faults, minimising the chance of further losses of supply. Due to uncertainty over the cause of the protection operation, the reenergised transformer feeding Wimbledon Grid 132kV substation was not immediately made available to EDF Energy, although two transformers at Wimbledon capable of carrying the entire demand remained in service throughout the incident.
70 Having re-secured the system at 18:25 the restoration strategy was to configure the network for a phased re-energisation starting at Littlebrook. A complex switching sequence was required to prepare the transmission and distribution network and ensure that the risk of further faults was minimised by carefully switching cable circuits to control the voltage.
71 By 18:30 further reconfiguration of the network had taken place and the first sections of Hurst and New Cross substations had been re-energised.
National Grid Company plc 10 September 2003 Page 17 72 At 18:31 National Control confirmed those developments to EDF Energy Control and informed them that it would contact them soon to start restoring supplies. 73 At 18:38 National Control informed EDF Energy Control that supplies could be restored to Wimbledon Grid 132kV substation. During the same conversation National Control also informed EDF Energy Control that supplies at New Cross could be restored.
At this time EDF Energy Control requested time to assess the distribution network and agreed to call back.
74 By 18:40 further reconfiguration of the network had taken place energising further sections at New Cross and Hurst. 75 At 18:40 National Control contacted EDF Energy Control to offer restoration of supplies at Hurst. EDF Energy reconfigured the distribution network and EDF Energy’s Bromley supplies were restored at 18:44. This restored approximately a third of the 410,000 customers lost at 18:20. 76 Between 18:44 and 18:50 further reconfiguration of the transmission system took place. 77 At 18:48 EDF Energy Control contacted National Control to complete restoration of supplies to the Wimbledon Grid 132kV substation.
At 18:52 all remaining supplies to Wimbledon Grid 132kV substation were restored.
78 At 18:51 a further Buchholz alarm was received relating to the transformer or shunt reactor at Hurst. No indications were received that the transformer had been disconnected by automatic protection, indicating that the shunt reactor was the faulty equipment. 79 At 18:51 New Scotland Yard contacted the National Grid control room and it was confirmed that the loss of supply was a system incident, with no third party involvement. 80 At 18:52 National Control contacted EDF Energy Control to restore supplies at New Cross. EDF Energy Control requested time to assess the distribution network prior to restoring supplies and agreed to call back.
81 At this stage 29 switching operations had been planned and successfully executed in 26 minutes. 82 At 18:56 EDF Energy Control called National Control back and EDF Energy supplies to New Cross were restored at 18:57. At this point all supplies from the transmission system were available to the distribution network. 83 At 19:10 EDF Energy Control contacted National Control and asked for the Hurst transformer to be returned to increase security on the distribution network. The original Buchholz alarm was attributable to the shunt reactor at Hurst which was isolated and the transformer was made available to EDF Energy Control.
National Grid Company plc 10 September 2003 Page 18 84 At 19:14 EDF Energy Control confirmed all supplies to consumers had been restored. 85 During the period 19.00 to 19.45 standby site engineers arrived at the three sites. On arrival each standby engineer checked protection relay indications and alarm logs and confirmed the situation with National Control. To support the restoration process and initial investigation, two further engineers were also called to attend site. 86 The network was now configured as follows: Figure 8: Transmission system after restoration Wimbledon New Cross Hurst Littlebrook Supplying EDF 262 MW Supplying EDF 422 MW Willesden Beddington Beddington & Kemsley Northfleet West Beddington Out of service for scheduled maintenance Automatic action National Control action Supplying EDF 175 MW Supplying EDF 84 MW 437 MW 87 At 20:02 restoration of all supplies was confirmed with New Scotland Yard.
88 At 20:55 EDF Energy Control requested a formal report on the incident. 89 After assessment by engineers on site at Wimbledon, it was confirmed the automatic protection relay had been taken out of service and the number two Wimbledon to New Cross circuit was returned to service at 23:00. This action further enhanced security at Wimbledon and Hurst. No further switching was carried out to allow the system to be re-assessed and minimise the risk of any further faults. Full security was completed by reconfiguring the network overnight, with full security reestablished at 01:05 on 29 August.
90 The Standby engineers called to site remained on each site until restoration was complete. 91 Following the initial loss of supply National Control engineers correctly assessed the risks and restored transmission supplies within 37 minutes. Communication during the incident 92 Throughout the incident close communication was maintained between National Control and EDF Energy. Communications relating to this incident commenced at 18:17 following receipt of the initial alarm and were maintained throughout the incident and well into the night. In addition, National Control received a call from New Scotland Yard, and confirmed that this was a system incident.
National Grid Company plc 10 September 2003 Page 19 93 The incident generated substantial media interest, with National Grid handling some 300 media calls on Thursday evening. 94 A verbal statement was given to journalists within 25 minutes of the incident. Written media statements were issued at 21.25 on Thursday evening and 13.30 on Friday afternoon. 95 During a major incident, National Grid would normally communicate with bodies such as Ofgem, DTI, energywatch and others with a direct interest, depending on the incident. Contact was made with Ofgem, DTI and other parties as soon as possible during the incident and communication continued through the evening and the following days.
96 Following the event there were high-level contacts with Ofgem, the Energy Minister, DTI officials, the Mayor of London, EDF Energy and energywatch, among others. Response to the Buchholz Alarm 97 National Control actions were in line with National Grid procedures for responding to an indication that a Buchholz alarm had been activated on a transformer or shunt reactor at Hurst 275kV substation. 98 The causes and implications of a Buchholz alarm are set out in appendix 5, but in summary, the alarm provides a warning of potential problems in the transformer or its associated shunt reactor that could result in a major failure.
Hence, due to the nature of the consequences of such a failure, National Grid procedures specify the equipment is to be disconnected from the transmission system, except in a limited number of circumstances. These exceptions include any action that would result in a loss of supply.
99 As is normal in the design of control room systems, to avoid “alarm flooding” in the event of major system incidents, alarms are combined to reduce the total number displayed in the control room. The investigation noted that the grouping and nomenclature of the alarms for the transformer and shunt reactor did not clearly indicate whether the transformer or the associated shunt reactor was the origin of the gas alarm. 100 When the alarm was received, National Control took immediate action to begin the process to remove the transformer by asking EDF Energy to disconnect it from the distribution system.
National Grid Company plc 10 September 2003 Page 20 Disconnection of the Transformer 101 National Control undertook further switching actions in order to disconnect the transformer from the transmission system. The specific design of the substation, called a “mesh”, required the disconnection of the circuit from Littlebrook to Hurst. The switching plan undertaken was in accordance with National Grid procedures on operating mesh substations which, for the five to ten minutes taken to switch, left the electricity supply at New Cross and Hurst substations dependent on a single circuit. 102 The disconnection of the Littlebrook to Hurst circuit re-routed power and as expected increased the power flows on the number two circuit from Wimbledon to New Cross from 72MW (213MVA) to 558MW (695 MVA).
This is comfortably within the design ratings of 815 MVA for the circuit.
103 The investigation confirmed that the operational decision to switch out the Littlebrook to Hurst circuit and supply Hurst and New Cross from a single circuit from Wimbledon, for the five to ten minutes required to complete the switching, was in accordance with operating procedures and took account of the need to remove the safety risk of a major failure of a transformer. 104 The investigation has also confirmed that the configuration and capability of the system was in accordance with National Grid’s standards and procedures, and that National Grid undertakes an average of 2,700 annual switching operations at mesh corners without incident.
105 The investigation has found that National Grid engineers would not expect their actions in removing the equipment to have caused a loss of supply. Unexpected Operation of the Protection 106 At National Grid’s Wimbledon substation, the automatic protection equipment associated with the number two circuit from Wimbledon to New Cross, detected the change in power flows as a result of the switching at Hurst, as a fault. The protection equipment disconnected the circuit to prevent damage to other equipment and/or the propagation of the fault through the transmission system.
107 In this case the protection relay that operated was being used for backup protection.
Backup protection is fitted to the transmission network, in conjunction with the main protection and is designed to disconnect faults not cleared by the main protection equipment. 108 The protection equipment that operated was an Inverse Definite Minimum Time (IDMT) relay, a commonly used type. It does not operate immediately, but starts to operate when the electric current on the circuit exceeds a certain threshold. The speed of operation depends on how far the measured current is above the threshold level.
109 The protection relay had been correctly specified during the design process and the settings sheet had been correctly produced. However the relay that had been physically supplied and installed at Wimbledon was a 1 ampere rated relay, not the 5
National Grid Company plc 10 September 2003 Page 21 ampere relay specified on the settings sheet. In all other respects the settings on the relay were correct and were confirmed during several check points in the construction and commissioning process. 110 The effect of installing a 1 ampere relay instead of a 5 ampere, meant that the current flow at which the protection would operate was five times lower than the correct rating and below the rating of the circuit itself.
111 The 1 ampere protection relay was set to operate at a current of 1,020 amperes on the transmission circuit and was triggered on the day by a current of 1,460 amperes. This is significantly below the operating capability of the cable, at 4,450 amperes and the original specification of the protection relay, at 5,100 amperes (see appendix 6). 112 The protection relay was commissioned in June 2001 as part of a replacement scheme. Following a survey conducted as a result of the incident, all the automatic protection equipment in the area was surveyed and found to be correctly installed. A full survey of similar equipment at all substations in England and Wales has been initiated, and to date, having completed 20% of the total, no further cases have been revealed.
113 The incident investigation found that despite rigorous processes for commissioning protection equipment, the wrong protection relay was installed and commissioned at Wimbledon substation and this caused the number two circuit from Wimbledon to New Cross to automatically disconnect unexpectedly, and caused the loss of supply. 114 The commissioning of the automatic protection equipment involved a number of stages as set out in appendix 2. The investigation has found evidence to support that the relay settings had been correctly calculated. The setting sheets were correctly produced and signed by both the engineer who calculated the settings and the engineer who confirmed the application of those settings to the protection equipment.
Furthermore the investigation confirmed the protection equipment had been tested by the manufacturer in accordance with industry practice, and that preenergisation inspection tests were carried out. There is evidence that the rating of the automatic protection equipment that is included on the documentation used for commissioning could have been more clearly set out and hence visible to the commissioning engineers. The investigation found no evidence that any part of the commissioning process had been omitted.
115 The investigation has found that the direct cause of the loss of supply was the incorrect operation of a backup protection relay on the number two circuit from Wimbledon to New Cross. Maintenance of Assets 116 National Grid has an established maintenance policy and the assets involved in the incident have all been maintained according to that policy. 117 The following table summarises the maintenance undertaken on the assets involved in the incident at Wimbledon and Hurst.
National Grid Company plc 10 September 2003 Page 22 Item Description Last Maintenance undertaken Next scheduled maintenance Number two circuit from Wimbledon to New Cross, backup protection Commissioned 2001 2007 Hurst Transformer 3 September 02 2006 Hurst Transformer 3 Shunt Reactor March 03 2009 118 The investigation found that an appropriate level of maintenance had been carried out on the assets and poor asset condition was not a contributing factor to this incident.
Other Factors 119 The investigation has determined that together with the above factors that are directly attributable to the operation of the transmission system, there were a number of other factors, external to the transmission system, that may have contributed to the duration or scale of the incident.
Configuration of EDF Energy’s Wimbledon Grid 132kV Substation 120 EDF Energy own and operate the 132kV substation at Wimbledon, which is physically located on different site to National Grid’s substation. Maintenance outages were agreed between National Grid and EDF Energy as part of a well defined process and significant information was exchanged on network configuration and contingency plans for faults. 121 Four transformers from National Grid’s 275kV substation at Wimbledon supply EDF Energy’s 132kV substation. Normally all four transformers are connected ensuring that supplies can be maintained for the loss of any one transformer.
National Grid understands that EDF Energy splits its Wimbledon substation into two parts to reduce fault currents and prevent over-stressing the equipment. Normally, with two transformers supplying each part.
122 When National Grid requires one of the transformers to be taken out of service for maintenance, EDF Energy configures its network with one transformer on one part and two on the second (figure 9). Two transformers supply the majority of demand for Wimbledon and Wandsworth. The remaining transformer supplies the remaining demand at Wimbledon and Wandsworth, together with demand at Lots Road for London Underground. 123 If one transformer is out of service for maintenance the Lots Road circuits will always be dependent on a single transmission circuit, because they can only be connected to the single transformer.
National Grid Company plc 10 September 2003 Page 23 Figure 9: Configuration of EDF Energy's distribution system 124 The configuration of Wimbledon Grid 132kV substation on 28 August 2003 is illustrated in the diagram above. National Grid maintained supplies to the section connected to transformers two and four throughout the incident. 125 National Grid understands that when one of its transformers is out of service and in the event of a fault on the remaining transformer, the normal arrangement would be for EDF Energy to connect the two parts of the Wimbledon Grid 132kV substation. However, National Grid does not know whether, in these particular circumstances, EDF Energy would have been able to take such post-fault action.
126 The investigation found that the configuration of the EDF Energy’s distribution system was not a contributory factor to the initiation of the incident. However, a more rapid implementation of post-fault actions or an alternative configuration could have mitigated the overall impact of the incident, reducing the duration and perhaps the scale of the loss of supply. Communications 127 Following normal practice, during the incident there was extensive communication between National Control and the EDF Energy Control Centre. Communications were initiated at 18:17, when the initial Buchholz alarm was reported, and EDF Energy were requested to remove the demand from the transformer.
At 18:21 EDF Energy called National Control to confirm that there was a problem on the network, and 17 minutes later National Control called back offering to restore supplies to Wimbledon and New Cross.
128 Such operational communications continued throughout the restoration, with numerous telephone conversations between National Control engineers and EDF Energy’s control engineer, working together to reconnect the affected area. 129 Following the restoration of supply, communications with the control rooms continued as further reconfiguration of the systems took place to ensure full security was restored. 130 During the incident National Control managers were confident that this was a system incident and this was confirmed to New Scotland Yard at 18.51. Wimbledon 275kV Lots Road Wimbledon Grid 132kV Out of Service for scheduled maintenance London Underground EDF Energy National Grid