Crypto market data has specific quality issues that affect what analysis is actually possible. The quality issues differ from those that affect traditional financial market data because crypto market data infrastructure differs structurally from traditional financial market data infrastructure. Understanding the specific quality issues is foundational for serious analytical work.
This piece documents the most consequential quality issues and what analytical responses they call for.
The exchange data inconsistency problem
Crypto exchange data is the foundation for most market analysis but has significant inconsistency issues across exchanges and across time.
Different exchanges report different data fields for similar trading pairs. Some report price, volume, and various derived metrics. Others report different combinations. Cross-exchange aggregation requires reconciliation work that is methodologically non-trivial.
The same exchange may change its data reporting over time as the exchange evolves. Historical data from a specific exchange may not be directly comparable to current data from the same exchange because the underlying methodology has changed.
Wash trading is widespread on certain exchanges and contaminates volume metrics significantly. Filtering for wash trading is technically possible but requires sustained methodological work that not all data providers do. The volume metrics from non-filtered sources overstate genuine trading activity in ways that affect downstream analysis.
The implication is that exchange-derived metrics require sustained data quality attention. Analyses that treat aggregated exchange data as straightforward inputs without engaging with the underlying quality issues produce conclusions that may not be credibly supported.
The on-chain data interpretation problem
On-chain data is more reliable than exchange-derived data in certain respects but has its own interpretation challenges that affect data quality.
Raw on-chain data captures transaction-level events accurately but doesn't capture the meaning of those events in straightforward ways. Inferring economic activity from transaction data requires interpretive work that varies in quality across data providers.
The specific labeling that data providers apply to transactions — categorizing them as DeFi activity, NFT activity, exchange activity, etc. — varies across providers and may be inaccurate for specific transactions. Analytical conclusions that depend on these labels inherit the label accuracy problem.
The aggregation across raw events to higher-level metrics involves methodological choices that affect what the metrics actually measure. "Total value locked" calculations, "active user" counts, and various other commonly cited metrics depend on aggregation choices that may not be transparent.
Better practice involves either reproducing the aggregation work from raw on-chain data or using aggregations from providers whose methodology is documented and credible.
The time-stamp consistency problem
Time-stamp consistency across different data sources is harder than it sounds. Exchange data, on-chain data, and various third-party data sources use different time-stamp conventions and have different sources of clock skew.
Cross-source analyses that require time alignment encounter time-stamp consistency issues that affect analytical conclusions. Events that appear simultaneous in one data source may appear sequential in another. Analysis of cause-effect relationships depends on getting the time alignment right.
The corrections for time-stamp consistency are technically straightforward but require sustained attention. Analyses that don't engage with time-stamp consistency produce conclusions that may be artifacts of time-alignment errors rather than genuine relationships in the data.
The price data accuracy problem
Price data for crypto assets has accuracy issues that affect downstream analysis. Different data sources report different prices for the same asset at the same time. The differences result from various methodological choices including which exchanges to include, how to aggregate across exchanges, and how to handle outlier transactions.
For high-volume assets traded across many exchanges, the price discrepancies are typically small. For lower-volume assets, the discrepancies can be substantial. Analyses that assume single-source price accuracy can produce misleading conclusions for assets where the underlying price quality is weak.
The data sources also differ in how they handle "price" for assets that don't have continuous trading. Specific NFTs, illiquid tokens, and various other assets have prices that are conceptually ambiguous. Different methodologies produce different prices for the same asset, and the appropriate methodology depends on the analytical context.
The historical data revision problem
Historical crypto data is sometimes revised by data providers as their methodology improves or as data quality issues are identified. The revisions affect analyses that were conducted using the previously published data.
Analyses that are reproduced using current data from the same provider may produce different results than the original analysis because the underlying data has been revised. The reproducibility of published analyses depends on data version stability that is often not provided.
Better practice involves snapshotting the data used for any specific analysis and documenting the data version. This enables reproduction of the original analysis even when the underlying data has subsequently been revised.
The address-clustering quality problem
Many analyses depend on address-clustering work that groups multiple on-chain addresses into entity-level units. The clustering is methodologically challenging and the results vary across providers.
Different clustering approaches produce different entity counts and different entity activity patterns from the same underlying address activity. Analyses that depend on entity-level metrics inherit the clustering methodology choices of whichever provider's clustering they use.
The clustering quality varies by entity type. Exchange clusters are typically reliable because the heuristics for identifying exchange wallets are well-developed. Other entity clusters are more uncertain. The reliability of entity-level analysis varies correspondingly.
The NFT-specific data quality problem
NFT market data has specific quality issues that affect this segment of the broader crypto market analysis.
Wash trading is particularly prevalent in NFT markets and has produced documented inflation of activity metrics for certain projects. The unfiltered NFT activity metrics overstate genuine market activity significantly.
The pricing of NFTs is conceptually complicated because individual NFTs are unique. Floor prices, average prices, and various other commonly cited metrics depend on aggregation choices that affect what the metrics measure.
The royalty enforcement on NFT transactions has changed over time and varies across marketplaces. Historical analyses that assume consistent royalty patterns may not be accurate.
The NFT market data quality requires particular attention from analysts working in this segment. The casual treatment of NFT activity metrics produces conclusions that may not survive methodological scrutiny.
The DeFi protocol data quality problem
DeFi protocol data has specific quality issues including the metrics calculation methodology, the distinction between active user activity and bot activity, and the handling of cross-protocol composability.
Total value locked calculations depend on price data accuracy and on what assets are included in the scope. Different methodologies produce different TVL figures for the same protocol.
Active user metrics depend on how user activity is defined and on whether bot activity is filtered out. Many DeFi protocols have substantial bot activity that may or may not be appropriately filtered in published metrics.
Cross-protocol composability means that activity in one protocol may be triggered by activity in another protocol. Attribution of activity to specific protocols requires methodology choices that affect protocol-level conclusions.
What analytical responses these issues call for
The data quality issues described above require specific analytical responses to produce credible conclusions.
Methodological transparency. Analyses should document their data sources, their data quality assumptions, and the methodological choices that affect their conclusions. The transparency allows readers to evaluate the conclusions appropriately.
Cross-source validation where possible. Conclusions that hold across multiple independent data sources are more credible than conclusions from single-source analysis. The validation work is operationally complex but is appropriate for serious analytical work.
Data version documentation. Analyses should snapshot the data they use and document the version. This supports reproduction and enables identification of conclusions that are sensitive to subsequent data revisions.
Sensitivity analysis. Conclusions that depend on specific methodological choices should be evaluated for sensitivity to alternative choices. Conclusions that hold across reasonable alternative methodologies are more robust than conclusions that depend on specific choices.
Appropriate scoping. Conclusions should be scoped to what the underlying data actually supports rather than extended to broader claims that the data doesn't credibly support.
Honest engagement with limitations. Analyses should acknowledge data quality limitations explicitly rather than treating their conclusions as if the limitations didn't exist.
What this means for the analytical community
The crypto analytical community has produced enormous amounts of analytical work, much of which is affected by the data quality issues described above. Improving the data quality engagement across the community would improve the credibility of conclusions available for serious work.
The improvement requires sustained investment in methodological practice. Better data quality engagement is more expensive than casual treatment. The returns to the investment are credibility and analytical rigor that the casual treatment doesn't produce.
For analysts, the productive direction is to engage with data quality as a foundational element of analytical practice rather than as an occasional concern. The engagement compounds over time as the analyst develops better data quality intuition and better methodological practice.
For consumers of analytical work, the productive direction is to evaluate data quality engagement as part of evaluating analytical conclusions. Analyses that engage seriously with data quality are more credible than analyses that don't. The evaluation skill is learnable and pays dividends across years of consuming analytical work in this space.
The crypto market analytical infrastructure will continue to develop. The data quality issues will continue to require attention. The analysts and the readers who engage seriously with these issues will continue to produce and consume better analytical work than those who treat data quality as a peripheral concern.
