Seattle Averts Downtown Power Outage

Bookmark and Share
City of Seattle Data Center under normal power - click to see more

Seattle's Emergency Operations Center (EOC) was activated Friday, August 15th, for a downtown Seattle power emergency - several banks of transformers failed at City Light's Union Street substation, one of two substations serving the downtown core. City Light (Seattle's municipal electric utility - a department of City Government) shut down power to some portions of the waterfront, and asked downtown buildings to significantly reduce their electrical use in order to avoid a complete failure of the downtown grid.

All this occurred on one of the hottest days of the year (95 degrees) for Seattle. There was no extensive power failure, so the headline "City Light Avoids 90 Degree Outage" in the Seattle PI was buried in the local section, and the problem was just a footnote in the Seattle Times.

 How do City government information technology workers respond to emergencies like this? We've had lots of practice - WTO riots, 2001 Nisqually earthquake, electrical vault fires, windstorms, and actual cyberattacks.  We intentionally conduct emergency operations drills both as a Department of Information Technology (DoIT) and as part of City-wide or region-wide drills such as Soundshake.

On August 15th we went through our well-drilled disaster response: First, IT directors and managers alerted all employees. On our own - even before the EOC was activated - we sent desktop and server technical staff plus telephone, data communication and radio system technicians to the EOC to prepare it for activation. The EOC was activated using a DoIT-maintained "community notification" or telephone call-out system message to all critical City government executives. When the EOC was activated, we sent an executive there to support the City's leadership in making crucial decisions about the event. (See also my previous blog entry about the City's new EOC facility.)

Because this was a power emergency - and because the City's data centers are a major consumer of downtown power - we activated a long-standing protocol to reduce power consumption. All servers and equipment in the data center are color coded based on their importance to government operations. We shut those systems down in an orderly fashion as rapidly as possible. We also have uninterruptible power supplies and a one-megawatt backup generator for the main data center, plus many other backup generators for critical technology services throughout the City of Seattle. 

August 15th was a hot day in Seattle. It didn't get hotter, thanks to well-practiced disaster drills and pre-planning!

1 Comments

I wanted to research this subject and write a paper. Your post did what what a thousand words would not. Nice job.

Leave a comment