The use of this site and the content contained therein is governed by the Terms of Use. When you use this site you acknowledge that you have read the Terms of Use and that you accept and will be bound by the terms hereof and such terms as may be modified from time to time.
All text, graphics, audio, design and other works on the site are the copyrighted works of nasscom unless otherwise indicated. All rights reserved.
Content on the site is for personal use only and may be downloaded provided the material is kept intact and there is no violation of the copyrights, trademarks, and other proprietary rights. Any alteration of the material or use of the material contained in the site for any other purpose is a violation of the copyright of nasscom and / or its affiliates or associates or of its third-party information providers. This material cannot be copied, reproduced, republished, uploaded, posted, transmitted or distributed in any way for non-personal use without obtaining the prior permission from nasscom.
The nasscom Members login is for the reference of only registered nasscom Member Companies.
nasscom reserves the right to modify the terms of use of any service without any liability. nasscom reserves the right to take all measures necessary to prevent access to any service or termination of service if the terms of use are not complied with or are contravened or there is any violation of copyright, trademark or other proprietary right.
From time to time nasscom may supplement these terms of use with additional terms pertaining to specific content (additional terms). Such additional terms are hereby incorporated by reference into these Terms of Use.
Disclaimer
The Company information provided on the nasscom web site is as per data collected by companies. nasscom is not liable on the authenticity of such data.
nasscom has exercised due diligence in checking the correctness and authenticity of the information contained in the site, but nasscom or any of its affiliates or associates or employees shall not be in any way responsible for any loss or damage that may arise to any person from any inadvertent error in the information contained in this site. The information from or through this site is provided "as is" and all warranties express or implied of any kind, regarding any matter pertaining to any service or channel, including without limitation the implied warranties of merchantability, fitness for a particular purpose, and non-infringement are disclaimed. nasscom and its affiliates and associates shall not be liable, at any time, for any failure of performance, error, omission, interruption, deletion, defect, delay in operation or transmission, computer virus, communications line failure, theft or destruction or unauthorised access to, alteration of, or use of information contained on the site. No representations, warranties or guarantees whatsoever are made as to the accuracy, adequacy, reliability, completeness, suitability or applicability of the information to a particular situation.
nasscom or its affiliates or associates or its employees do not provide any judgments or warranty in respect of the authenticity or correctness of the content of other services or sites to which links are provided. A link to another service or site is not an endorsement of any products or services on such site or the site.
The content provided is for information purposes alone and does not substitute for specific advice whether investment, legal, taxation or otherwise. nasscom disclaims all liability for damages caused by use of content on the site.
All responsibility and liability for any damages caused by downloading of any data is disclaimed.
nasscom reserves the right to modify, suspend / cancel, or discontinue any or all sections, or service at any time without notice.
For any grievances under the Information Technology Act 2000, please get in touch with Grievance Officer, Mr. Anirban Mandal at data-query@nasscom.in.
Blockchain has been evolving constantly, transforming into a matured technology, empowering diverse enterprises with next-generation dApps and decentralized solutions. Web3 developers and companies working with blockchain rely on different types of blockchain data for various purposes, such as building dApps, integrating crucial data for smart contracts, and providing relevant data into decentralized databases.
Querying these data from the blockchain is challenging as enormous data are stored within blocks, which remain scattered across a decentralized distributed ledger. With blockchain data indexing, seamless access to data is granted while the reach and usability are enhanced significantly. Hence, let’s dive deeper into the concept of blockchain data indexing to explore its working mechanism, challenges, and efficient solutions to overcome the issues.
What is blockchain data indexing and querying?
Blockchain indexing is a method of organizing on-chain data in a way that users can query the information very quickly and without hassle. To fully understand the concept of blockchain data indexing, you should have complete familiarity with how blockchain technology works, the underlying infrastructure, blockchain’s way of enabling data privacy, and use cases of smart contracts powering dApps.
Looking at a real-world example, imagine a book with hundreds of chapters. Whenever readers want to find a specific chapter, they must go through all the pages individually. This approach is quite time-consuming and complex. As a solution, the publisher provides an index at the front of the book followed by the page number so that readers can easily search for the chapter they are looking for.
Regarding blockchain, it’s an immutable, shared, and distributed ledger that records encrypted data across an open, temper-proof, and decentralized network. As evident from the name, blockchain has a series of blocks chained together, creating a time-ordered structure in which transactions are securely stored into blocks, and each block is linked to the preceding blocks, creating a chain-link structure.
While data privacy and security is concealed on the distributed ledger, blockchain lacks a default mechanism to enable high-level query and identification of data. Even when using APIs, you need to be very specific with the queries to be able to fetch the correct data information they need. Blockchain data indexing simplifies how users can search, filter, and locate data for very specific queries as easily as searching information on popular search engines— Google.
To achieve this, blockchain adopts account transaction trace chain (ATTC) and subchain-based account transaction chain (SCATC) index structure of blockchain data indexing, improving the overall efficiency of data querying while systematically indexing all the data such as account details, blocks, and transactions.
How blockchain data indexing works?
As discussed, blockchain data indexing is an approach that automatically reads, filters, and indexes all the data available on blockchain networks, including Ethereum, Polygon, Polkadot, Avalanche, or custom blockchains (AppChains) such as BNB Sidechains, Parachains, Supernets, and Subnets. Hence, data indexing provides an easy-to-query structure for bulk non-compiled data. The following step-by-step process represents the working mechanism of blockchain data indexing:
Data extracted from the blockchain and stored in a distributed database: This process involves parsing the blockchain’s data structures, such as blocks and transactions, to extract relevant information. This information could include transaction details, contract interactions, token transfers, and more.
Data is indexed in the database: Next, data structures are created to facilitate quick and efficient access to specific types of data. This process employs various indexing techniques based on the type of blockchain and the data being indexed.
Some common indexing methods include:
Indexing transactions-related information such as sender, recipient, timestamp, and transaction hash.
Indexing addresses (accounts) to track their balances, transaction history, and other related data.
Indexing various crypto tokens and NFTs to track token activities and balances.
Indexing smart contract interactions, events, and parameters.
Indexing blocks by block height, timestamp, and other relevant parameters.
3. APIs integration: The indexed data is then provided with powerful APIs and user interfaces that allow users and applications to access the indexed data easily. These APIs mainly include GraphQL endpoints and RESTful APIs.
4. Data served to the users: Finally, users and dApps developers can query the indexed data using predefined queries or search criteria. The indexed data allows for much faster and more efficient querying than scanning the entire blockchain.
The challenges with blockchain data indexing and querying
While blockchain powers decentralization, security and transparency into the web3 ecosystem, it lacks native capabilities to support direct and instant retrieval of specific data from the blockchain’s huge and complicated data sets. Hence, Web3 developers or enterprises looking to query the indexed blockchain data have to use external data sources to query the data, which may lead to the following challenges related to blockchain data indexing, and, ultimately, for the querying:
High data volume
Blockchain comprises a digital ledger that stores transactions, which is opposed to the working of traditional data storage systems containing data packets. Blockchain receives data from endless sources, including blockchain ecosystems, centralized, decentralized exchanges, and dApps.
The volume of data increases with more blockchains getting added to the chain over time. Looking at April 2022’s data, blockchain recorded a whopping 389 gigabytes and an increment of 60GB data since the previous year. Data indexing in blockchain becomes challenging to filter data from such a high storage volume. Likewise, querying data from blockchain becomes problematic as well.
Decentralization of data
Decentralized blockchain infrastructure is beneficial in terms of enabling high security, transparency, and data protection against tempering. However, decentralization may create challenges for blockchain data indexing as it distributes data across diverse nodes rather than gathering it at one single point, making it difficult for indexers to access and filter the data.
Lack of efficient query language
Each type of database must have a supported query language to provide correct information based on the user query. Taking the example of a centralized database, they have an initial query language such as SQL. The blockchain’s immutable and decentralized nature does not support the current query languages. Thereby making it difficult for indexers and data consumers to get relevant information.
In the case of dApps, accessing or reading data from the blockchain becomes even more difficult since dApps utilize a different set of data in large amounts. Hence, developers are forced to implement heavy codes to interact with and access desired data.
Data complexity and entanglement
The issues we mentioned in the previous points, including decentralization and lack of query languages, lead to another major issue related to blockchain data access. For example, most second-generation smart contract blockchains record and distribute historical data across separate nodes (externally from the blockchain and its decentralized storage). Therefore, indexing and retrieving these data is usually very complicated because some public nodes need access to such events, resulting in slow data queries.
API Limitations
Current APIs (application programming interfaces) are limited in ensuring public safety against performance-related issues and denial-of-service attacks. Hence, APIs can currently support only simple queries like top-k queries. That’s where Standard APIs turn advantageous, which are fast and efficient APIs to query all types of data ranging from NFTs, Tokens, and multi-chains. For example, developers can retrieve data from NFT marketplaces, virtual worlds, galleries, on-chain data, and multiple cryptocurrency exchanges.
Solution to blockchain indexing issues- Data Indexing Protocols
Data indexing protocols in blockchain can be considered Google for the internet. These protocols are globally-distributed, decentralized infrastructures aiming to solve the challenges of blockchain data indexing while allowing web3 developers to access an open, transparent ecosystem of blockchain data. The Graph Network and Subquery Network are two popular decentralized blockchain indexing protocols for enabling seamless blockchain data indexing in line with offering highly performant and instant data querying.
With The Graph network, enterprises can leverage a unique blockchain tool — indexed “Subgraphs”. A Subgraph, at its core, is an open API that allows dApp developers to retrieve data from advanced blockchains like Ethereum and Avalanche using a unique API query language, GraphQL. Similarly, SubQuery Network offers reliable, flexible, and fully decentralized APIs that allow application developers and data consumers to extract data from blockchain networks. Therefore, SubQuery simplifies the creation of data-rich, future-proof decentralized solutions. With SubQuery SDK, enterprises can build reliable data APIs and start retrieving data indexed from hundreds of networks.
Any developer with expertise in the web3 domain can build Subquery API and Subgraph. For Subgraph, you need to write a custom Subgraph Manifest. This description defines events in the smart contracts regarding the Subgraph, such as the type of event that the contract should focus on, the approach of mapping event data to data, and so on. Talking about SubQuery Network, its open-source SDK enables the effortless creation of APIs that fulfill the unique data requirements of dApps. Let’s have a look at the top benefits of Blockchain indexing protocols for users:
Faster dApp development: Create groundbreaking applications without needing to run your own data server, build indexing infrastructure, or parse through raw data.
Cost reduction: Cut down on costs and time spent running expensive infrastructure by tapping into the competitive data market.
Maximum resilience: Ensure your application’s uptime and keep its data flowing 24/7 with a globally distributed network of contributors.
Conclusion
As evident, blockchain technology is being adopted rapidly, and tons of data is getting produced regularly. This signifies that data indexing protocols will remain important for businesses to perform blockchain data indexing in an efficient and less-complicated way. As such, Web3 enterprises and developers can leverage the technology underlying The Graph and the SubQuery networks. They can create a Shared Subgraph, dedicated indexer, and custom SubQuery API to leverage specific data. This way, data indexing protocols transform the way businesses utilize blockchain data through data indexing and information querying.
About The Author
Dr. Ravi Chamria is co-founder CEO of Zeeve Inc, an Enterprise Blockchain company. He has an experience of 18+ years in IT consulting spanning across Fintech, InsureTech, Supply Chain and eCommerce. He is an executive MBA from IIM, Lucknow and a prolific speaker on emerging technologies like Blockchain, IoT and AI/ML.
Passionate About: Blockchain, Supply Chain Management, Digital Lending, Digital Payments, AI/ML, IoT
That the contents of third-party articles/blogs published here on the website, and the interpretation of all information in the article/blogs such as data, maps, numbers, opinions etc. displayed in the article/blogs and views or the opinions expressed within the content are solely of the author's; and do not reflect the opinions and beliefs of NASSCOM or its affiliates in any manner. NASSCOM does not take any liability w.r.t. content in any manner and will not be liable in any manner whatsoever for any kind of liability arising out of any act, error or omission. The contents of third-party article/blogs published, are provided solely as convenience; and the presence of these articles/blogs should not, under any circumstances, be considered as an endorsement of the contents by NASSCOM in any manner; and if you chose to access these articles/blogs , you do so at your own risk.
Zeeve is an enterprise-grade Blockchain Infrastructure Automation Platform. Join the growing list of clients that trust us with their Blockchain initiatives
Ethereum is a blockchain-based decentralized platform with smart contract capabilities. As of the platform, ether is the native cryptocurrency. Being the second biggest blockchain network in the blockchain space, Ethereum has a lot of…
The word cross-border refers to payments across different countries. Cross-border payments are vital to individuals, businesses, merchants, industries, and international development organizations. However, cross-border transactions are often…
NFTs will be huge in 2021. In 2022, they will still be the hottest trend.NFTs are gaining popularity at an incredible rate, and it is now possible to create the Metaverse, a digital future. Metaverse allows users to experience a virtual world by…
Driven by the blockchain technology, web 3.0 is already creating ripples and also bringing in changes in the way how businesses operate. Some businesses have already taken initiatives, while many are still exploring opportunities to adopt the new…
There are billions of people who rely on the internet, whether for personal or professional purposes. The Internet enabled people who lived thousands of miles apart to communicate with one another. It aided education, healthcare, and business,…
Web3 is considered as the next iteration of the internet, and is still in its infancy, and will serve as the base for Metaverse. While we are talking about transitioning to decentralised means of storing data, smart contracts and DAOs to host…
