This site is an always-in-progress repository for my thoughts and ideas. You'll generally find me writing about sustainability 🍃, urbanism 🏙️, programming 💾, vegan cooking 🍛, and whatever else pops into my mind.
The Data Behind New York's Increasingly Dirty Electricity Peaks
Published on Oct 31st, 2021.

Photo by Ella Ivanescu.

For many New Yorkers, the hottest days of the summer are ones of refuge: staying indoors with our air conditioners on full-blast in an attempt to escape the heat. While flicking our ACs on during a hot day seems like an entirely mundane aspect of life, behind the scenes an immensely complex machine, our electricity grid, works in overtime to generate the electricity required to keep us cool.

In this post I’d like to provide a glimpse into what happens within this complex machine when it is strained to its limits: on the hottest and most energy intensive days of the year. More specifically, we will look at how New York’s electricity grid has turned to increasingly dirty forms of electricity generation as time has gone on. This is all while its political leaders have pledged to transition away from fossil fuels and meet strict carbon emissions goals.

Finding days of peak demand

The first step in learning about how the grid responds to strain is to discover when moments of peak electricity demand occur. Thankfully NYISO (the organization that manages New York’s electricity grid) has an archive of load data at five-minute intervals available for the public to download.

With some data wrangling in R1, we can scrape and reformat these archives into a table of daily demand values (measured in gigawatt hours) and rank them in descending order for the past three years: 2021, 2020, and 2019:

Date Load
July 20, 2019 623 GWh
July 21, 2019 619 GWh
June 29, 2021 618 GWh
June 30, 2021 614 GWh
July 28, 2020 610 GWh

You may be wondering why we’re only looking at the past three years of data. This was a deliberate decision to analyze the effects of a relevant recent event: the closing of a major nuclear generation facility at Indian Point, in April 2021.

Gathering generation data

Next up we’ll need to gather data about the breakdown of fuel types being used to power the grid. Similarly to the demand archives used above, NYISO also publishes archives of fuel mix reports at five-minute intervals. We can scrape and manipulate these archives in order to create a continuous dataset of the fuel mix used to generate electricity on the grid. Here’s an example of the fuel breakdown on a relatively average set of days, February 2nd through 3rd, 2021:

Generation area chart

The fuel mix of New York's generation from Feb 2nd and 3rd. Data from NYISO.

The dataset divides up the total grid’s generation output into seven categories: wind, hydro, nuclear, natural gas, dual fuel (generators that can switch between natural gas and other petroleum fuels in response to market conditions), other fossil fuels, and other renewables. Since we’re trying to determine how “dirty” the grid is at any given time, I’ve chosen to categorize each of these fuel types into two categories based on their resulting carbon emissions:

This list may prove controversial to some, particularly in my categorization of natural gas and nuclear, however I have chosen to be opinionated in my categorizations based soley on carbon emissions per MWh of electricity generated. While nuclear may have a bad reputation amongst some environmentalist groups, it is undeniably a major source of reliable carbon-free electricity.

Returning to our average February days from above, let’s look at what they look like when we graph the simplified clean/dirty mix as a percentage of total generation per hour. This form of visualization will allow us to better analyze how clean or dirty generation is, regardless of how much electricity is being generated.

Generation clean/dirty mix

Clean vs dirty generation per hour on our average February days. Data from NYISO.

Not too bad! Our average February days managed to have more than 50% of electricity generation come from carbon-free sources. As we’ll soon see, things get messier as demand starts trending upwards during hotter months.

“Cleanliness” of generation during times of peak demand

With both load and generation data at our disposal, we can now look at what happens to the grid’s “cleanliness” on days where electricity demand peaks. Let’s start by looking at the worst peak on our ranking, July 19th and 20th, 2019:

Generation Mix for 2019's July Peak Demand

Data from NYISO.

We can see that the proportion of generation coming from “dirty” sources of electricity are quite a bit higher when compared to our average February days. During this peak the percentage of dirty generation ranges from 43 to 62%.

This increase in carbon-intensive generation is a result of the financial incentives that crop up when electricity demand is high. Though most of us are used to seeing a flat rate per KWh on our electricity bills, electricity is actually bought and sold on a real-time market. This market has fluctuating prices which can become especially volatile on days of peak demand, increasing to wallet-shattering prices as electricity on the grid becomes more scarce. While consumers generally don’t notice these price increases due to layers of financial infrastructure shielding them from real-time markets2, companies who own electricity generators see opportunity for profit. For them, increased prices means it becomes profitable to burn costlier (and generally dirtier) forms of fuel and sell the resulting electricity onto the market.

What we see on the chart above is the result of these strong financial incentives. When electricity on the grid becomes scarce, the market will pay handsome profits to whoever can fulfill demand regardless of the carbon-intensity of its source.

Now that we have an initial idea of what the grid can look like in times of stress, let’s turn to a more recent peak: June 29th and 30th of 2021.

Generation Mix for 2021's June Peak Demand

Clean vs dirty generation mix on June 29th and 30th. Data from NYISO.

When compared to the 2019 peak above, we can see generation has gotten dirtier after two years. On these days the dirty fuels account for 61 to 73% of generation! What gives?

If we compare the more granular visualization of fuel mixes for both of these date ranges we can find the culprit:

Comparing 2019 and 2021 Peak Demand Generation Mix

Fuel mix for generation on our 2019 and 2021 peak demand days, compared. Data from NYISO.

Although both peaks see a spike in fossil-fuel based generation, the nuclear “base generation” in 2019 is noticeably higher3. With no more carbon-free energy coming from Indian Point, we see that natural gas and dual-fuel generation has filled the void and caused our grid to become more reliant on fossil fuels than it was two years ago. For all of the tough talk from New York’s politicians about tackling climate change, this is not the trajectory we should expect.

Unfortunately this low note is where our journey must end. As abnormally hot days become more commonplace events, the question of how to create a more resilient and sustainable grid will become increasingly important. Unfortunately, the data tells a story of regression in New York, where carbon-free nuclear power has been replaced by fossil-fuel generators. If our leaders are serious about their talks of strong action towards sustainability, we need to change course. Ideally more knowledge about what is happening behind the scenes can serve as a wake-up call for all of us to demand better from our government.

  1. All code used to generate the analysis and figures in this article can be found in this repository

  2. Note my use of the word generally. There have been cases where energy customers in certain markets have been burned hard by prices during hours of peak demand. 

  3. Readers with a careful eye may notice that the amount of electricity generated on these peak days doesn’t match up with the amount demanded. This is because New Yorks’s grid also imports electricity from its neighboring grids. For simplicity this post is only looking at generation within NYISO’s market, however a more complete telling of this story would also look at the fuel mix for electricity coming in from other grids. For any who are curious, electricityMap does a good job of collecting and combining this data for electricity markets around the world.