Marcus
04/12/2023
Despite being cited as a critical part of the fight against climate change (by the UN IPCC), voluntary carbon markets have been maligned as opaque. For much of its 30-year history, they are devoid of liquid benchmark prices, are barren of transparency regarding the fee structure, and lacks any information on the available credits on the market. The trade is primarily dominated by private, over-the-counter (OTC) deals.
In recent years, significant progress in demand-side access and transparency has been driven by blockchain technology. Organizations like KlimaDAO and Toucan Protocol ushered in a new era of on-chain liquidity and programmable retirements, and now traditional carbon credit registries like Verra and Gold Standard are rapidly adopting policies for utilizing blockchain technology in the VCM.
By combining data from The Graph network about tokenized carbon credits with data from the Verra registry, KlimaDAO uses open-source software, including Subgrounds, to build a dashboard that brings unprecedented transparency to carbon markets. In this post, we'll walk through this real-world example using Subgrounds and Dash to build a dashboard that provides an interactive and user-friendly representation of carbon credit data.
In the traditional market, this type of data would be collected via phone or online surveys of intermediaries like brokers and traders, then aggregated in quarterly or annual reports. By combining real-time blockchain data with Python data science tooling, KlimaDAO is bringing unprecedented transparency and data access to ecological benefit assets like carbon credits.
Why Subgrounds?
Raw blockchain data is difficult to use directly for analysis since it is composed of transactions, e.g., Bob sent X amount of Y tokens to Alice. In the parlance of data engineering, blockchains are “Online Transactional Processing (OLTP)” systems optimized for handling large volumes of transactions with extreme reliability.
But this raw data format is not very easy to make sense of for data scientists exploring protocols to understand user behavior or optimize mechanics, nor for engineers building applications that need to integrate advanced analytics or interactive visualizations. Instead, data analytics is typically conducted in “Online Analytical Processing (OLAP)” systems. In the context of blockchain data, The Graph Network offers a crowd-sourced set of data models for many common protocols - called Subgraphs, built-in AssemblyScript.
This amounts to an OLAP system that could be hooked up to existing Python data science and analytics tooling - but there is a critical piece of glue code required to transform Subgraph GraphQL queries into Python DataFrames or similar data structures used in mainstream data science stacks. Subgrounds provides that missing piece, bringing preprocessed blockchain data to a new class of users: Python data professionals.
Why Dash?
Dash is a Python framework for building web-based analytical applications. It allows Python developers to build an interactive dashboard in a matter of minutes and integrates with popular data science libraries such as Pandas, Numpy, and Matplotlib. Its simple syntax is also an accessible solution for data scientists who may not be experienced front-end developers but understand how to visualize their data.
While Dash provides a useful prototyping environment for dashboards and similar data-driven applications, the solutions provided by Playgrounds are flexible enough to support production-grade systems. For instance, KlimaDAO also maintains a set of Python-based Discord bots that query Subgraphs using Subgrounds to provide real-time metrics for our community, embedded directly in the Discord chat experience.
Data Sources and Integrations
For the KlimaDAO Digital Carbon Dashboard, the underlying data comes from two sources: on-chain data about carbon credit prices and retirements and data about the underlying carbon credit projects and off-chain retirements from the Verra carbon credit registry.
With Subgrounds, we can easily fetch all the data we need from Subgraphs, from swaps on a DEX like Sushiswap to tokenized carbon credit retirements. Since the resulting data is a standard pandas DataFrame, we can easily join it with the Verra registry data we retrieved from a traditional web2 API endpoint in JSON or CSV format.
Once we have all the data we need, the Subgrounds wrappers for Plotly visualizations make it easy to generate visualizations, with access to the full power of Plotly should the need arise. Visualizations built from both on- and off-chain data sit right next to each other in the Dash application, and we can also generate components that combine data from both sources thanks to the interoperability of pandas DataFrames.
Conclusion
Building a Dash application powered by Subgrounds provides a flexible solution for overcoming the frustrations associated with traditional data products like BI tools in an accessible syntax familiar to most Python data scientists. Combined with Dash, Subgrounds is bringing unparalleled transparency to real-world assets like carbon credits - empowering market participants with a clear and interactive representation of the Digital Carbon Market.
To find out more about Playgrounds, join the Playgrounds Discord or reach out on Twitter. To find out more about KlimaDAO’s pioneering work in the Digital Carbon Market, check out their website https://klimadao.finance or join their Discord
If you’re interested in participating in the Digital Carbon Market, for instance, by offsetting your personal or organizational carbon emissions, please complete this form, and someone from the KlimaDAO team will be in touch: https://share-eu1.hsforms.com/1L1HfszYXTkGP6q-GkRb9sAfhhlr