Deep in the heart of Texas’ collapsing power grid

Everything in Texas went wrong at once.

Texas is now entering its third day of widespread power outages and, although supplies of electricity are improving, they remain well short of demand. For now, the state's power authority suggests that, rather than restoring power, grid operators will try to shift from complete blackouts to rolling ones. Meanwhile, the state's cold weather is expected to continue for at least another day. How did this happen?

To understand what's going on in Texas, and how things got so bad, you need quite a bit of arcane knowledge—including everything from weather and history to the details of grid structure and how natural gas contracts are organized. We've gathered details on as much of this as possible, and we also talked to grid expert Jeff Dagle at Pacific Northwest National Lab (PNNL). What follows is an attempt to organize and understand an ongoing, and still somewhat chaotic, situation.

Why is Texas so much worse off?

While other states have seen customers lose power, Texas has been hit the hardest, with far more customers losing power for substantially longer.

One key reason for this is because Texas maintains its own power grid largely in isolation from those of its neighbor states. In North America, most customers are served by two major grids that operate on the same alternating current frequency—one serving the eastern half of the continent (including the US, Canada, and parts of Mexico) and the other serving the western half. However, Texas—along with Quebec—both maintain power grids that are largely separate from these larger networks.

So, while problems elsewhere in the Midwest were partly buffered by generating capacity elsewhere in the country, Texas was on its own. It does have interconnections with neighboring grids, but they don't offer much in the way of capacity—and they weren't built for importing power anyway. According to Jeff Dagle of PNNL, these interconnections were mostly built by utilities near the border between grids so that the utilities could use power from whatever source happened to be cheapest at the time. Only half a dozen of these interconnects exist, and they can only handle a few hundred Megawatts each. This is simply "too small to matter," as Dagle put it.

Given that Texas and Quebec are both fierce defenders of their independence, it's tempting to view their insistence on maintaining their own grids as something of a caricature. But it's a caricature rooted in reality. Grids began to integrate because the easiest places to generate power—near large coal fields, for example—weren't necessarily close to large population centers. Texas remained relatively isolated during this buildout period because there weren't real advantages to integrating with its neighbors. Its grid ended up managed by ERCOT, the Electric Reliability Council of Texas, a nonprofit with a complicated state/private governance structure.

Then, when some degree of national power grid regulation began, it was done under the federal government's constitutional ability to regulate "interstate commerce." By purposely keeping its grid within the borders of Texas, the state limited the impact of federal standards and regulations. This deep-seated aversion to regulation recently prompted former US Energy secretary and Texas Governor Rick Perry to quip, "Texans would be without electricity for longer than three days to keep the federal government out of their business.”

How did a cold snap trigger this crisis?

The typical power demand profile in Texas would show a big peak in the summer, when most of the state has to run air conditioning 24 hours a day to keep its inhabitants from melting. Because of this, power generators schedule needed maintenance and upgrades for the winter months, when demand is generally lower. As a result, the Texas grid is less able to match spikes in demand during the winter.

And the recent cold snap set off a huge spike in demand. Lots of buildings in Texas are heated by electricity, and the cold, which blanketed the entire state, sent demand for heating through the roof. Natural gas is typically used where electricity isn't, and its increased use also set off a competition for gas supplies, which quickly became limited for reasons we'll get back to in a moment.

Typically, grid managers have a set of reserve plants that can be brought online if demand spikes suddenly. In a competitive electricity market, prices will rise if demand threatens to exceed supply, inducing producers to activate idle plants. For some reason, this didn't work out in Texas. ERCOT places a cap of $9,000 per Megawatt-hour on the cost of power in its grid. At various times after the cuts began, however, prices were only $1,200 per Megawatt-hour, and the Public Utilities Commission of Texas isn't sure why.

But even if reserves and economics had all kicked in, Texas wouldn't have avoided problems, because generating sources were shutting down left and right.

What shut down in Texas?

Pretty much everything, though for different reasons. Wind turbines iced up and stopped generating, cutting into the grid contribution of renewable power. But wind (and solar) don't produce as much power during the Texas winter under normal circumstances, so the loss of some capacity there wasn't as much of a problem as it might have been in summer; they weren't expected to produce much anyway.

As we mentioned above, the natural gas market saw power generators competing with home users for a limited supply of natural gas. That gas supply ended up being even more limited by the fact that as much as half the state's natural gas production may have ground to a halt. Natural gas doesn't come out of the wells as a pure gas, and one of its major contaminants is water. As temperatures dropped, that water froze in inopportune places, choking off the flow of gas.

Texas happens to have the second-largest capacity to store natural gas of all the states. It's not clear whether storage suffered from similar problems or if there was some other reason it couldn't buffer the loss of supply.

In any event, in the competition between power plants and residential users for the limited gas supplies, residential users won. PNNL's Dagle told Ars that most of the contracts for natural gas delivery can be broken without penalty if the supply isn't there. That's partly because there are alternate ways for grid operators to generate power, while many residences have no other way to produce heat. But Dagle said it was also for practical reasons; it's easy and safe to restart a couple of gas turbines when supply returns, but it's a nightmare to ensure that every consumer gas-fueled appliance is operating safely when gas is restored.

"When the gas supply's constrained, the contract calls for them to be curtailed on the generation-side," Dagle told Ars. "That's quite normal."

A somewhat larger surprise was that Texas also lost nuclear and coal capacity. Rick Perry, while at the Department of Energy, tried to pay coal and nuclear plants to keep fuel stored onsite (a policy that ended up being rejected), under the assumption that it would ensure a reliable supply of energy. Clearly, that didn't work out. And there's some reason to expect it wouldn't help in cases like this, as giant piles of coal can completely ice over in cold weather, making it difficult to use.

Beyond that, Dagle said that both coal and nuclear rely on water to generate power. This water has to replace any of the material that's lost as steam from the portion used for generation, and it may be used as part of a cooling or condensing system. If the water intakes freeze up, then the plant will inevitably have to shut down.

Shouldn’t Texas have been ready for this?

Yes and no—it depends on how you define "this."

Power plants obviously operate much farther north than Texas, in areas where the conditions Texas is facing now are normal for weeks or months at a time. There are ways to cold-proof various systems; wind turbines, for example, can have heaters embedded in the blades to shed ice when needed, intake pipes can be heated by exhaust from power plants, etc. But all of these measures cost additional money, which may be difficult to justify if the conditions they're needed for are extremely rare.

It turns out that these conditions are rare in Texas, but not extremely so; Texas faced something similar a decade earlier, in 2011, when its grid suffered similar failures. In the wake of these earlier problems, the Federal Energy Regulatory Commission (FERC) issued a report suggesting changes to the grid in order to prevent a recurrence. Obviously, if there's a similar event today, it must mean those changes weren't made, right?

Not quite. Dagle told Ars that the earlier event was probably not as cold as the conditions Texas has faced this week. Even if the people operating power plants had made changes that would have gotten them through the 2011 event, those changes might not have been sufficient to handle this week. In addition, the Texas grid, like the rest of the US, has become increasingly reliant on natural gas supplies over the last decade. According to the FERC report, in 2011, Texas lost over a million Megawatt-hours to frozen hardware and mechanical failures; it lost only 120,000 Megawatt-hours to fuel supply problems. The reported problems with natural gas supplies this time around suggest that those numbers will now look very different.

Finally, Dagle noted that, in the absence of actual cold weather, it's hard to test whether the hardware you've put in place to protect against it is actually effective. "It's kind of hard to find all these problems when you can't test it," Dagle said. In Minnesota, you will know if things work in the next winter. In Texas, you might have to wait a decade for a stress test.

"Clearly, they didn't do enough," Dagle told Ars, "but I don't know how fair it is to be too critical."

What do we do now?

Over the course of Wednesday, ERCOT seems to have brought roughly 7 Gigawatts of generating capacity back on line, and supplies have continued to ramp up on Thursday. Its measure of demand has also gone up, but that's not an indication that actual demand is lagging supply; instead, it means more areas of the grid are receiving power. In other words, ERCOT is still managing demand by keeping power cut off to many locations.

Demand has to be kept well below supply because ERCOT plans to start shifting back to rolling blackouts and away from permanent shutdowns. This presents a big challenge, because lots of appliances—heaters, refrigerators, and such—will immediately shift to operating at maximum levels the moment that power is restored. Doing this safely, by ensuring that there's enough excess power to cover this surge, requires building lots of models based on a neighborhood's typical usage—a process that grid operators have hopefully been engaged with since the blackouts began in earnest.

There's good news on the way, though, with our own Eric Berger indicating that warm weather should take hold on Friday, reducing power demand, potentially restoring more hardware to service, and hopefully getting more natural gas into the pipelines. For those who have been suffering in the dark for days, Friday remains a long way off, but the end is at least in sight.

Key steps will still need to be taken months from now. That's when the data should be gathered to analyze what went wrong and figure out how to build a more resilient—and necessarily more expensive—grid. It may also involve Texas giving up a bit of its "go it alone" attitude and forging deeper connections with neighbors, possibly accepting a greater degree of federal oversight. Dagle told Ars that the North American Electric Reliability Corporation, which sets standards for the combined US-Canadian grids, has new standards out for cold-weather operations. These might provide a useful guide for how to avoid events like this in the future, but the standards aren't mandatory yet.

Dagle highlighted the concept of resilience. "Resilience is imagining the things that could go wrong and making sure that we're prepared to accommodate that," he said. That means thinking beyond simply matching the last major cold wave. "Maybe we could have done a better job envisioning temperatures even more extreme than what we saw last time," he said. "Have we put enough counter measures there to handle that?"

While such resilience might make electricity somewhat more expensive, it's also a strong insurance policy against the staggering losses that are being incurred with most of the state shut down entirely.

So far, the signs for change aren't good. In addition to Perry's quip about Texans being willing to suffer through blackouts to avoid any oversight, the present Texas governor, Greg Abbott, is busy blaming renewable power for failures that disproportionately affected fossil fuel generation.