RO-2012-102 Train control power failure, 26 April 2012

At about 1600 on 26 April 2012, the four train control workstations in the Auckland control room located in the Wellington National Train Control Centre (train control) suddenly lost power and shut down. As a consequence, all of the signals in the Auckland metropolitan area automatically reverted to red ("Stop") and all rail movements in Auckland progressively stopped.

Train controllers were unable to communicate with the stranded trains and could not issue control instructions. Veolia was advised of the power outage and immediately activated its emergency plan. Veolia sent messages to all its on-board train managers, explaining the situation. All passengers were retained on board trains that were prevented from reaching their next stations.

The power outage lasted for about one hour and scheduled passenger services were affected for the rest of the evening. There were 27 train services travelling within the Auckland metro area at the time, with an estimated passenger load of between 1000 and 2000.

The power outage occurred when an electrical fault caused an electrical circuit breaker that was feeding power to all four Auckland workstations to trip. The electrical fault should have first tripped a different circuit breaker, which would have resulted in only one of the four workstations being lost.

The control of all signals for the Auckland metropolitan area had been centralised into Wellington train control since 1997. The last phase of centralisation was the provision of the four new workstations that subsequently lost power in this incident. All train control functions for the Auckland metropolitan area had been managed from these four workstations since late 2010, about 16 months before the incident.

The Commission identified the following safety issues:
- the project team responsible for the Auckland train control centralisation project lacked the appropriate expertise for designing and installing the emergency power supply system
- the management and maintenance of the emergency power supply system for train control were not sufficient to ensure the integrity of what had been designated an "essential service"
- KiwiRail's Risk Management Policy for "continuity of core services" did not give proper consideration to the safety of passengers and crew when a core service such as train control failed, causing the widespread stoppage of an entire metropolitan passenger rail system.

KiwiRail took the necessary safety action to improve the management and maintenance of the power supply system for train control.

The Commission made one recommendation to the Chief Executive of KiwiRail to review its risk assessment matrix to improve the focus on safety risk.

The key lessons learnt from the inquiry into this occurrence were:
- projects involving essential core services must be appropriately scoped and resourced to ensure that the service integrity is not disrupted at any time
- essential core services must be subjected to a rigorous safety risk assessment process that ensures that the risks to people and infrastructure are appropriately managed and tested
- power distribution systems for essential core services must be properly managed and serviced to ensure that the integrity of the service is maintained.