© Wachiraphorn/iStock/Getty Images Plus
Some reports gain traction in policymaking communities even though they lack any sort of formal process for rigorous peer review. I recently came across an example that seems especially problematic and offers a cautionary tale. The report, commissioned by the Institute for Energy Research and conducted by Capital Alpha Partners, LLC, (CAP) demonstrates how some papers can introduce confusion and misinformation. The authors conclude that only very high carbon prices can reduce emissions sufficiently to meet the US Paris targets in 2025. This conclusion, however, is not to be trusted because it appears to be based on flawed assumptions that are inconsistent with the peer-reviewed literature on carbon taxes.
To predict changes in emissions given a carbon tax, the authors use the Carbon Tax Analysis Model (CTAM), an open-source model maintained by the Washington State Energy Office. The model was built to analyze state-based carbon pricing policies but can be altered to analyze federal-level policies. The authors offer that they use the US Energy Information Administration’s (EIA’s) Annual Energy Outlook (AEO) 2016 reference case (without the Clean Power Plan). Presumably, they replaced the built-in Washington state energy consumption data with national energy consumption data, but the report does not specifically address this.
Open source models provide a public good, but they can also be misused. In the case of CTAM, this is especially true because the model allows users to adjust a large number of key parameters. With this flexibility, a user could derive literally any emissions path for a given carbon tax. It therefore becomes especially important to document all parameter values in a report. Yet the CAP authors failed to document any policy or parameter assumptions they used in their CTAM simulations.
Without knowing the answers to a few questions, the CTAM results reported by the authors of this study are simply uninformative. These ambiguities are important to mention because the model offers users a variety of options and the CAP authors do not tell readers that the options exist—or which options they selected. Did they assume policy exemptions for jet fuels, marine fuels, or “transition coal”? (Transition coal is a CTAM specific term.). Did they use the model’s default elasticities or “stickiness”? (Stickiness refers to the length, in years, of a linear ramp over which a particular elasticity is fully rolled into the model.) We don’t know, and therefore we can’t rely on this study to inform any related policy debate.
To illustrate the importance of CTAM model assumptions and options, I downloaded and edited the CTAM model myself. Whereas the CAP report uses AEO 2016 energy consumption forecasts, I used AEO 2018 forecasts (more on this below). I also used CTAM default parameter values unless otherwise noted. Figure 1 reports historical and projected energy-related carbon dioxide emissions under business-as-usual from both the CAP results (reported in Table B-1 of their report) and my results (labeled MH CTAM). For comparison, I also included AEO 2016 and 2018 emissions projections.
Figure 1. Historical and Projected Energy-Related CO2 Emissions (mmt), Business as Usual, 2005-2040
Most compellingly, there is a significant difference in benchmark emissions between the CAP projections and my own projections—over 300 million metric tons in 2025. By using an older set of forecasts, the authors of the CAP study have arbitrarily increased the baseline and therefore increased the carbon price necessary to meet any particular target. I also find it interesting that CTAM projections are higher than EIA emissions projections for either year. The difference between CTAM and EIA projections appears to be due to differences in emissions rates assumed for the industrial sector—and therefore we would also expect higher emissions levels for a given carbon price when using CTAM.
Figure 2 displays emissions projections under a carbon tax. The CAP report models a range of prices trajectories. To be concise, I’ve modeled only one carbon tax path from the CAP report: a $40 carbon tax, rising annually at 2 percent above inflation, beginning in 2019. In my analysis, I did not allow for jet fuel, marine fuel, or transition coal exemptions. To highlight the importance of key parameters, I used two alternative set of assumptions. The first, MH CTAM – D uses the CTAM default elasticity and stickiness parameters; for MH CTAM – A, I set all the stickiness values to 1 (implying no stickiness). I’ve also included AEO 2018’s forecast of emissions under a $25 constant carbon tax and I’ve marked the level of emissions that is consistent with the US commitment under the Paris Agreement (26 percent below 2005 emissions by 2025).
Figure 2. Energy-Related CO2 Emissions (mmt) under a Carbon Tax
Reductions from the baseline emissions in the CAP projections and the MH CTAM – D projections look similar; the differences appear to be due to differences in the level of emissions in the baseline. It therefore seems very likely that the CAP report used the CTAM benchmark stickiness parameters. But if I turn off all stickiness in the CTAM model (MH CTAM - A), reductions are much larger in the early years and the policy easily meets the Paris target. Finally, it is worth noting that the EIA’s own forecast of emissions from a much smaller carbon tax both meets the Paris targets and has greater long-run emissions reductions. This indicates to me that CTAM’s default elasticities are smaller (in absolute terms) relative to the elasticities used in another well-known model (NEMS). This may also be driving the small emissions response in the CTAM model projections.
The NEMS comparison brings me to another problem with these types of papers: they often fail to compare their results to the broader academic literature. The CAP report is no exception. The Stanford Energy Modeling Forum exercise 32 (EMF32) provides a good example of the type of literature that should be cited. The results and comparison from 11 different modeling groups have been published in a special issue of the peer-reviewed journal Climate Change Economics. Crucially, the participating models generally report large and immediate reductions to carbon taxes ($25 or $50 initial prices in 2020). This evidence strongly suggests that the default CTAM stickiness parameters are inappropriate for an analysis of a federal-level carbon tax.
To sum it up, determining the exact source of results in the CAP report would be an impossible exercise due to the authors’ failure to report key assumptions in the modeling analysis. Regardless, it seems clear that one of the key takeaways featured in the CAP report—that very large carbon taxes would be required to meet 2025 emissions targets—is a direct result of using an outdated set of forecasts and parameters inconsistent with most evidence in the existing literature.
It is unimportant whether the use of old forecasts and CTAM default stickiness parameters was due to ignorance or was an attempt to generate results that supported a preconceived hypothesis. Not all analyses are created equal and those that simply do not meet fundamental scientific standards for research should carry less credibility to inform policy decisions. The CAP report should be ignored by the policymaking community.
