A critique of “Do Environmental Markets Cause Environmental Injustice? Evidence from California’s Carbon Market,” a 2020 NBER working paper by Danae Hernández-Cortés and Kyle C. Meng

Joint post by Danny Cullenward and Katie Valenzuela


Danny Cullenward a Lecturer and Affiliate Fellow at Stanford Law School, where he teaches courses on energy and climate law. He is also the Policy Director at CarbonPlan and the California Senate’s appointee to the Independent Emissions Market Advisory Committee, which is charged with reviewing California’s cap-and-trade program. Danny holds a JD from Stanford Law School and a PhD in Environment and Resources from Stanford University. He can be reached at dcullenward@ghgpolicy.org.

Katie Valenzuela is an environmental justice advocate based in Sacramento. She previously served as Policy & Political Director for the California Environmental Justice Alliance, Principal Consultant for the Joint Legislative Committee on Climate Change Policies, and Co-Chair of California’s AB 32 Environmental Justice Advisory Committee. She can be reached at kbvale@gmail.com.


When this working paper was released, it was our ardent hope that the technical flaws and clear research bias would prevent it from being used to inform deliberations on climate policy. As time has passed, however, and it has become apparent that this paper is being used to undermine legitimate critiques of carbon pricing programs like California’s cap-and-trade program, we feel it is our responsibility to highlight the errors in method and assumptions present in this paper.

Below, we review five key errors in the Hernández-Cortés and Meng analysis. These errors demonstrate that the paper’s findings cannot be relied upon in the California context nor as an indicator of likely outcomes in other jurisdictions that are considering similar climate policy strategies. 

First, a caveat: we fully understand that greenhouse gas emissions trends do not necessarily correlate perfectly with the emission of other co-pollutants and that optimal strategies for reducing climate and local air pollution might be very different from one another. However, no one can argue that greenhouse gases and the co-pollutants that harm public health do not come from the same sources. This is why investments that reduce greenhouse gas emissions hold significant potential to reduce co-pollutants and why environmental justice movements focus on reducing emissions at the source as much as feasible.

Some will argue that, while this particular paper may not be accurate, there is still no evidence that cap-and-trade harms environmental justice communities. We would argue, however, that simply not harming environmental justice communities was never the goal of California’s landmark climate law, AB 32. The goal of this law, as clearly outlined in the text and in subsequent legislation, was to “[c]onsider the potential for direct, indirect, and cumulative emission impacts from these mechanisms, including localized impacts in communities that are already adversely impacted by air pollution.” In that respect, the cap-and-trade program has fundamentally failed to protect communities burdened by pollution by failing to ensure that facilities located in environmental justice communities reduce emissions. California can — and must — do better. Any state or country seeking to follow our example should heed this valuable lesson: low carbon prices are not sufficient to make a large impact on either greenhouse gas emissions or local air quality.

Error 1: Inaccurate definition of environmental justice communities

The working paper identifies environmental justice communities by zip code, not by census tract — a critical mistake that creates a fundamental flaw for the entire analysis that follows. For example, in CalEnviroScreen 3.0 — the state tool used to identify environmental justice communities — Sacramento zip code 95818 includes three census tracts: 6067002200, 6067002300, and 6067002400. Census tract 6067002200 (shown in red) is in the highest 10% of CalEnviroScreen scores in the state, representing significant pollution burden and social risks. The other two census tracts (shown in green) are in the lowest 30-40% brackets of scores, meaning they have significantly lower environmental burdens and social risks. 

Analysis of what community is an environmental justice community at the zip code level blunts the disparities that exist at times within zip codes, which is why California opted to use census tracts rather than zip codes for analyzing environmental justice impacts.

Error 2: Data quality issues

There are important but poorly understood technical problems with the paper’s underlying pollution data, which come from 35 different regional air districts that use 35 different methods for data collection. Some of those methods are not good or reliable, which is part of the reason a major statewide effort is underway to enhance and standardize monitoring, particularly in environmental justice communities. The data set used for this working paper comes with explicit government caveats about the process of air pollution permit updating and potential incompatibility between districts' reported data and facilities' reported data. 

At the same time, Table S2 shows that the authors' modeled pollution concentrations do not correlate well with EPA's observed data. Across four different local pollutants the correlation between modeled “HYSPLIT” and observed ambient pollution exposure data is very weak — statistically different from zero, but hardly a reliable predictor of what actual pollution monitoring data show.

It should also be noted that industrial sector greenhouse gas emissions have been roughly constant since 2006 (the year AB 32 became law). As illustrated in the chart below, nearly all of the state’s emission reductions have come from the electrical power sector, not the industrial sector. 

It is curious that the authors claim progress in industrial emissions under cap-and-trade when California’s own data shows that notion to be false with respect to greenhouse gases. It is possible that industrial greenhouse gas emissions would be higher (counterfactually) without the cap-and-trade program and with no other policy alternatives. But even if the cap-and-trade program has prevented an increase in emissions relative to having no program at all, it hasn’t substantially reduced total emissions. Any suggestion that maintaining air pollution levels in some of the country’s most polluted communities is an environmental justice success story strains credulity.

Error 3: Control group bias

One of the more remarkable methodological problems with this paper is that the authors’ "treatment" and synthetic "control" groups are not at all similar. This is a major red flag because the authors use a "difference in differences" statistical model, which assumes that that the two groups experience identical trends before the policy applies and therefore that the observed trend in the synthetic "control" group (non-C&T regulated facilities) can predict the counterfactual baseline for the treatment group (C&T regulated facilities). If constructed correctly, the counterfactual baseline can then be used to estimate the impacts of the cap-and-trade policy on the treatment group. 

Whether a facility is included or excluded in cap-and-trade is based on a standard emissions threshold (25,000 tCO2e/year) that is intended to pick up nearly all of the sources in some sectors while covering only some of the sources in others. Meanwhile, the control group includes all facilities that are large enough to be required to report emissions (greater than 10,000 tCO2e/year) but smaller than the cap-and-trade coverage threshold. As a result, the control group has almost no representation from the refining and hydrogen sectors, most of which are individually large sources (>25,000 tCO2e) and which collectively represent a major driver of local air pollution in many environmental justice communities. If any of the sectors that are not well distributed between the "treatment" and synthetic "control" groups and experience divergent trends, then the "differences in differences" method could wrongly attribute the effects of sector-specific differences to the cap-and-trade policy. 

You can see from Table S1 that the authors’ data approach has grouped nearly all of the state’s hydrogen and refinery sources into the treatment group (C&T regulated facilities), without any significant representation from these sectors in the control group (non-C&T regulated facilities). This means that any policies or exogenous trends that have an important impact on the hydrogen and refining sectors will be wrongly interpreted by the authors’ model as an impact caused by the cap-and-trade program.

Error 4: Omitted variable bias

The control group bias problem compounds the authors' assumption that no other policies affect their treatment group, when in fact at least one major policy overlaps the time period and treatment groups the authors address in their model. This is called omitted variable bias. The authors incorrectly assume that no other policies disproportionately affect their treatment group:

We use the facility eligibility criteria and the 2013 introduction of the GHG C&T program to isolate stationary, facility-level, local air pollution emissions driven by the program.[5]

[5] CARB also developed other regulations to meet the AB 32 target, such as a Low Carbon Fuel Standard and Advanced Clean Car Standards. These programs, however, were not market-based, were mostly introduced prior to 2012, and did not have the same facility-level eligibility criteria as the cap-and-trade program. Thus, while our estimated C&T effects take place in the presence of these other regulations, it is unlikely that C&T effects are conflated with the effects of these other regulations.

That is factually incorrect. The Low Carbon Fuel Standard (LCFS) is a "market-based" program — it's quite literally another cap-and-trade program that applies specifically to transportation fuels, and includes fuels' associated emissions from refining and hydrogen production. The LCFS began to affect refinery and hydrogen sector emissions at almost the same time as cap-and-trade came into operation, and LCFS prices ramped up significantly to more than 10 times the cap-and-trade carbon price during the authors' study period. 

Readers who have heard success stories about California’s cap-and-trade program might wonder at how a system that is so widely celebrated has produced such a modest carbon price signal. The answer is not at all surprising: the program has far too many allowances and carbon offsets in circulation to restrict pollution levels. This finding is now widely documented in the academic literature (e.g. Cullenward et al., 2019; Mastrandrea et al., 2020; Inman et al, 2020), but the working paper doesn’t cite any of these peer-reviewed articles.

Because the refinery and hydrogen sectors are nearly 100% clustered in the "treatment" group, a much stronger "signal" that applies to these same sources from the LCFS program is picked up by the authors' statistical model as being attributed to the statewide cap-and-trade program. (Again, there are very few small refinery players whose operations are affected by the LCFS program, so the LCFS effects manifest almost entirely at facilities that are subject to both cap-and-trade and LCFS—hence the effects of the LCFS are strongly correlated with the effects from cap-and-trade, contrary to the authors' assumptions.)

The working paper also failed to control for the effect of other overlapping policies with more direct emissions reductions than cap-and-trade. An example is the Renewable Portfolio Standard, which drove early procurement of zero emission power projects in solar and wind and dramatically reduced emissions from fossil fuel power plants through reduced demand. Attributing all reductions observed at refineries and power plants to the cap-and-trade program ignores the impacts of those stronger policies on emissions trends, giving the reader the false impression that the cap-and-trade program drove those reductions.

Error 5: The question is not cap-and-trade versus nothing

Finally, the paper asks whether cap-and-trade has an impact relative only to the modeled absence of cap-and-trade, but that's not the policy and political question in California — nor is it the right question to ask anywhere else. The real question is whether the program California has — which is massively oversupplied and therefore insufficient to drive GHG pollution down in line with California's climate laws (see here and here for an overview) — is enough. The better alternatives are (1) a stronger cap-and-trade program, (2) more direct regulations on polluters, and (3) more of both kinds of policy. It's not news that a flawed program is better than no program at all, but that's all the paper looks at.

If California had chosen a path of more prescriptive, direct emissions reductions in accordance with state law, we would likely be seeing far more emissions reductions at the source, and far more improvements for environmental justice communities than we're seeing under the cap-and-trade program today. To say it's better than nothing ignores the fact that adopting a weak cap-and-trade program has led to prolonged and higher emissions in environmental justice communities than if California had adopted a stringent carbon pricing policy or relied instead on non-market mechanisms that would have been targeted at the pollution reductions our communities need.

California cap-and-trade research updates