Validation of MCOE

This notebook runs sanity checks on the results of the calculations of marginal cost of electricity (MCOE) that we do based on EIA 923 and EIA 860. Currently this only includes per-generator fuel costs, which also necessitates the calculation of per-generator heat rates and capacity factors. These are the same tests which are run by the mcoe validation tests using PyTest. The notebook and visualizations are meant to be used as a diagnostic tool, to help understand what's wrong when the PyTest based data validations fail for some reason.



In [ ]:

    
%load_ext autoreload
%autoreload 2



In [ ]:

    
import sys
import pandas as pd
import numpy as np
import sqlalchemy as sa
import pudl



In [ ]:

    
import warnings
import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
handler = logging.StreamHandler(stream=sys.stdout)
formatter = logging.Formatter('%(message)s')
handler.setFormatter(formatter)
logger.handlers = [handler]



In [ ]:

    
import matplotlib.pyplot as plt
import matplotlib as mpl
%matplotlib inline



In [ ]:

    
plt.style.use('ggplot')
mpl.rcParams['figure.figsize'] = (10,4)
mpl.rcParams['figure.dpi'] = 150
pd.options.display.max_columns = 56



In [ ]:

    
pudl_settings = pudl.workspace.setup.get_defaults()
ferc1_engine = sa.create_engine(pudl_settings['ferc1_db'])
pudl_engine = sa.create_engine(pudl_settings['pudl_db'])
pudl_settings

Perform the MCOE calculation

In order to validate the output from the MCOE calculation we first have to... do that calculation. We can do it at both monthly and annual resolution. Because we are testing the overall calculation, we don't want to impose the min/max heat rate and capacitiy factor constraints -- that would artificially clean up the outputs, which is what we're trying to evaluate.



In [ ]:

    
pudl_out_year = pudl.output.pudltabl.PudlTabl(pudl_engine, freq="AS")
pudl_out_month = pudl.output.pudltabl.PudlTabl(pudl_engine, freq="MS")

Some of the tests only really work for the monthly case, so that's the default here. Uncomment if you need annual



In [ ]:

    
%%time
#mcoe_year = pudl_out_year.mcoe(
#    update=True,
#    min_heat_rate=None,
#    min_fuel_cost_per_mwh=None,
#    min_cap_fact=None,
#    max_cap_fact=None
#)



In [ ]:

    
%%time
mcoe_month = pudl_out_month.mcoe(
    update=True,
    min_heat_rate=None,
    min_fuel_cost_per_mwh=None,
    min_cap_fact=None,
    max_cap_fact=None
)

Validation Against Fixed Bounds

Some of the MCOE outputs have a fixed range of reasonable values, like the generator heat rates or capacity factors. These varaibles can be tested for validity against external standards directly. In general we have two kinds of tests in this section:

Tails: are the exteme values too extreme? Typically, this is at the 5% and 95% level, but depending on the distribution, sometimes other thresholds are used.
Middle: Is the central value of the distribution where it should be?

Fields that need checking:

heat_rate_mmbtu_mwh (gas, coal)
capacity_factor (gas, coal)
fuel_cost_per_mmbtu (gas, coal)
fuel_cost_per_mwh (gas, coal)



In [ ]:

    
# mcoe = mcoe_year
mcoe = mcoe_month

MCOE vs Self



In [ ]:

    
pudl.validate.plot_vs_self(mcoe, pudl.validate.mcoe_self)



In [ ]:

    
pudl.validate.plot_vs_self(mcoe, pudl.validate.mcoe_self_fuel_cost_per_mmbtu)



In [ ]:

    
pudl.validate.plot_vs_self(mcoe, pudl.validate.mcoe_self_fuel_cost_per_mwh)

Natural Gas Heat Rates (2015+)

Unfortunately EIA fuel / generator data only becomes usable for natural gas as of 2015.



In [ ]:

    
pudl.validate.plot_vs_bounds(mcoe, pudl.validate.mcoe_gas_heat_rate)

Coal Heat Rates



In [ ]:

    
pudl.validate.plot_vs_bounds(mcoe, pudl.validate.mcoe_coal_heat_rate)

Fuel Cost per MWh



In [ ]:

    
pudl.validate.plot_vs_bounds(mcoe, pudl.validate.mcoe_fuel_cost_per_mwh)

Fuel Cost per MMBTU



In [ ]:

    
pudl.validate.plot_vs_bounds(mcoe, pudl.validate.mcoe_fuel_cost_per_mmbtu)

Gas Capacity Factors



In [ ]:

    
mcoe_gas = mcoe.query("fuel_type_code_pudl=='gas'")
nonzero_cf = mcoe_gas.query("capacity_factor!=0.0")
idle_gas_capacity = 1.0 - (nonzero_cf.capacity_mw.sum() / mcoe_gas.capacity_mw.sum())
logger.info(f"Idle gas capacity: {idle_gas_capacity:.2%}")

pudl.validate.plot_vs_bounds(mcoe, pudl.validate.mcoe_gas_capacity_factor)

Coal Capacity Factors



In [ ]:

    
mcoe_coal = mcoe.query("fuel_type_code_pudl=='coal'")
nonzero_cf = mcoe_coal.query("capacity_factor!=0.0")
idle_coal_capacity = 1.0 - (nonzero_cf.capacity_mw.sum() / mcoe_coal.capacity_mw.sum())
logger.info(f"Idle coal capacity: {idle_coal_capacity:.2%}")

pudl.validate.plot_vs_bounds(mcoe[mcoe.capacity_factor!=0.0], pudl.validate.mcoe_coal_capacity_factor)



In [ ]: