Do companies report reliable GHG emissions?

As part of her thesis at ESCP, supervised by Aurélien Acquier, on the tools and methodologies developed and used by carbon consultants, Camille Habé, with support from Hélène Chauviré and Melodie Pitre, presents a series of articles offering an overview of what the research says about carbon accounting.

Introduction

Alongside discussions on what motivates companies to measure and report their greenhouse gas (GHG) emissions^[1], some of the researchers examined the quality of the reported data. Indeed, the very principle of a carbon reporting system requires that the data be sufficiently understandable to stakeholders so that they can make use of it—whether to favor one company over another or to monitor and regulate  (Kolk, Levy, and Pinkse 2008).

However, this issue is difficult to address because the estimates published by companies cannot be verified through a direct measurement system but are based on models.

The Issue of the Comparability of Reported Emissions

The term "data quality" most often refers to the issue of comparability, defined as the ability to distinguish one company’s climate performance from that of another based on published data. Indeed, this comparability is seen as essential for encouraging and/or enabling the selection of companies that are more responsible than others.

However, it is clear that studies on this subject have concluded that, at present, The published GHG emissions are not comparable between companies (Kolk, Levy, and Pinkse 2008; Talbot and Boiral 2018) are also not consistent for a single company across different reporting years (Comyns and Figge 2015; Talbot and Boiral 2018) or between reports issued by a single company through different channels (Depoers, Jeanjean, and Jérôme 2016; Wegener, Labelle, and Jerman 2019).

Researchers point in particular to methodological factors that explain this lack of comparability:

• The scopes considered can vary considerably (Depoers, Jeanjean, and Jérôme 2016; Kolk, Levy, and Pinkse 2008; Wegener, Labelle, and Jerman 2019). In fact, some companies report only their direct and indirect energy-related emissions (known as Scope 1 and 2). Other variations in reporting scopes can be more subtle and difficult to detect: for example, the GHG Protocol considers certain categories to be optional, such as travel by visitors and customers. Consequently, two companies following the same standard may have different reporting scopes.
• The quality of the data provided can vary widely (Wegener, Labelle, and Jerman 2019). For example, for a given type of product purchased, a company may be able to provide either the amounts spent or the units purchased. In the first case, given possible price variations for the same product or factors such as inflation, the resulting emissions estimate will be less accurate than if the company were able to provide the actual quantities purchased.
• Emissions factors^[2] The data used may come from different sources, which are not always consistent with one another (Harangozo and Szigeti 2017; Kolk, Levy, and Pinkse 2008; Wegener, Labelle, and Jerman 2019)
• Finally, specific methodological choices are necessary for any carbon accounting exercise. There are two in particular: whether to account for emissions from energy purchases on a lease-based or market-based basis^[3] and to define the scope of its entity based on operational or financial controls^[4] (Haslam et al. 2014; Kolk, Levy, and Pinkse 2008).

In France, it is worth noting that ADEME and the Association for the Low-Carbon Transition share the view that the published GHG data are not comparable across organizations and believe that this should not be a goal in itself: “ The Bilan Carbone® comparison doesn't really make sense because there are too many calculation assumptions and choices made, which mean that not all activities within the same sector will have the same carbon impacts. […]. “Year-over-year comparisons make sense when the scope is similar for the same company. Ultimately, it’s not necessarily desirable to compare ourselves to others, because the common goal is to be the one that reduces its emissions by 80% by 2050.” (Association for Low-Carbon Transition, 2023).

The Issue of Transparency in Methodological Choices

Since it is not possible to compare the GHG data published by companies on a like-for-like basis, a high degree of transparency regarding methodological choices could address this issue. However, many companies do not always specify the methodologies they have followed (Comyns and Figge 2015; Talbot and Boiral 2018), do not always specify which categories were excluded from the scope (Depoers, Jeanjean, and Jérôme 2016; Kolk, Levy, and Pinkse 2008; Talbot and Boiral 2018), and sometimes do not specify whether the reported emissions pertain solely to CO₂ or to all GHGs (Kolk, Levy, and Pinkse 2008). The lack of transparency is such that Talbot and Boiral go so far as to suggest that this could be a case of strategic omission or data manipulation on the part of companies (2018).

Changes in Data Quality Over Time

Finally, one might argue that these difficulties in harmonizing practices and ensuring transparency regarding methodological choices stem from the relatively recent nature of carbon accounting—the first methodological frameworks dating back to the late 1990s and early 2000s. However, the most recent studies on the subject are less optimistic: they suggest that, over the past decade, the quality of data published by a given company did not consistently improve (Comyns and Figge 2015; Talbot and Boiral 2018) and that for one-third of the companies studied by Talbot and Boiral, the published data actually became less and less transparent over time (2018).

Conclusion

In this context, should we conclude that reporting standards are not—and will never be—sufficiently harmonized and/or transparent to allow for a meaningful comparison of one company’s climate performance relative to another’s?  It seems clear that, for the time being, investors and NGOs are struggling to make sense of the published GHG data (Kolk, Levy, and Pinkse 2008; Talbot and Boiral 2018).

However, researchers are not unanimous regarding the prospects for the evolution of carbon accounting in this area. Some believe that it is possible to measure emissions in a sufficiently standardized and accurate manner to enable comparisons with other companies, and that this requires improvements to existing methodologies. Others believe that the complexity of estimating emissions—due to the dispersed nature of companies’ activities, the complexity of their value chains, and so on—makes the goal of ultimately comparing them with one another unrealistic. Finally, still others believe that, in any case, organizations’ control over the reporting process means that the data will never be reliable without extremely strict regulations (Boiral et al. 2022).

A Message from Carbone 4

Carbon accounting has continually evolved during the course of these scientific studies, so it is not surprising that researchers observe significant variability in scopes, methods, and results. For example, existing methods and guidelines such as the GHG Protocol have been refined, emission factor databases have been structured, and the SBTi has established itself as a benchmark with very high standards for transparency and rigor. These developments can help improve the quality of published data. For example, according to the Task Force on Climate-related Financial Disclosures (TCFD)^[5], in 2022, 58% of companies published information that complied with at least five of its eleven recommendations, compared with just 18% in 2020^[6]. It might therefore be worthwhile to revisit these comparisons in light of these developments.

However, the issue of comparing companies’ carbon performance remains a powerful driver of emulation, as we see with the companies we work with. We remain very cautious about these GHG footprint comparisons because, unless two organizations are perfectly identical, we quickly recognize their limitations. Ratio-based approaches (such as GHG footprint per unit sold, or per euro of revenue) exhibit the same limitations in terms of comparability. To assist financial stakeholders for whom these comparisons are, however, essential to their investment decisions, our sister company Carbon4 Finance instead offers aggregated indicators that provide a broader analysis of the past, present, and future climate strategies of companies within the same sector, using a common methodology.^[7].

Beyond the issue of comparability, Carbone 4’s experience has identified a key factor in improving the quality of published data: better reporting of operational data within companies. This requires significant changes to establish processes for collecting and centralizing information, as well as for interacting with stakeholders (physical purchase and sales volumes, number of visitors, customers’ geographic origins, etc.). This is a major challenge, particularly for CIOs, as few organizations report being organized and/or consulted on this matter, and the volume of data to be collected is substantial.^[8].

In addition, external stakeholders must be trained to read and understand the reported data. Certain methodological choices seem unavoidable, and it is essential that the teams relying on this data be able to recognize them and understand their implications. This is one of the reasons behind the creation of the Carbone 4 Academy, and we hope this will help raise the standards of rigor in this field.

Finally, it seems important to point out that carbon accounting is not intended solely to publish data that can be used by external stakeholders, but must, first and foremost, be used to guide its efforts to reduce emissions. In this regard, comparing a company’s performance year over year—while clearly explaining, as some companies do with great clarity and transparency, changes in the scope and methods of their carbon accounting—makes it possible to highlight an actual reduction in emissions. New regulatory requirements, such as those under the CSRD, will encourage a wider range of companies to take on the challenge of accounting for and reducing their environmental impacts.^[9][10].