Topics In Demand
Notification
New

No notification found.

Generative AI: information retrieval and query answering
Generative AI: information retrieval and query answering

444

2

Generative AI (GenAI) has received significant attention since the launch of ChatGPT by OpenAI. Many business units, small and large, want to leverage it for their business needs. The widespread adoption of such GenAI offerings by the industry has led to newer ones being added at a rapid pace.

Many organizations have structured data stored in sources such as databases (DB) and spreadsheets. Retrieval of information from databases is generally performed by writing DB queries. 

Search techniques are commonly employed to retrieve information from unstructured data sources such as documents. Users input keywords or terms and the system displays matching parts of the documents to read and interpret. But this can be time consuming, especially when dealing with large volumes of documents. Additionally, the summarization process can be subjective and prone to manual errors.

The rise … and rise … of Large Language Models

Chat applications are vital to respond to user queries, and cater to business users who seek answers from both structured and unstructured data sources. Basic chatbots rely on predefined questions and answers, requiring human intervention for unanswered queries. But as scaling chat operations with human help isn’t always feasible, this is where GenAI emerges as a scalable and expedient solution. 

Large Language Models (LLM), trained on a vast corpus of text, are effective for many Natural Language Processing (NLP) tasks. LLMs can capture the semantic meaning of text and to perform searches. They are also trained to perform tasks such as text summarization and converting text to DB queries. Hence, LLMs can provide effective responses to queries from both structured and unstructured sources.

LLMs are trained on either proprietary or public data, enabling them to effectively generalize to unseen data. However, in the context of structured data, the DB query should correspond to the business schema, and summary must follow basic business guidelines. Additional information to the user query, commonly termed “prompt engineering,” guides LLMs to provide desired outputs.

User query in structured data

Structured data are stored in DBs like SQL and Neo4j. Publicly available source code repositories contain DB queries that LLMs are trained to understand. They can generate DB queries from user text, enabling text-based interactions with DBs.

Off-the-shelf LLM models lack knowledge of business application DB schemas, requiring schema to be provided to generate pertinent DB queries. For example, if numerous database tables exist, their schema should accompany the user query. Trained on data with this capability, LLMs can generate database queries that combine information from multiple tables.

While off-the-shelf LLMs excel at generating DB queries, they are challenged when presented with complex DB schemas. For example, with numerous tables, inaccurate results may arise from incorrect table choices. Similarly, complex queries spanning multiple tables may challenge their performance. This can be addressed by efficient prompt design or by fine-tuning the LLM.

User query in unstructured data

Business users seek answers from various documents like user manuals, policy documents, reports, and knowledge articles, and across formats like word, PDF, PPT, or spreadsheets. The information scattered across these multiple documents requires a cohesive summary in response to user queries. LLMs can be used to effectively locate and summarize pertinent information from these documents.

There are two tasks that need to be performed by the LLM, which it can perform inherently, as outlined in the next section:

  • Locate relevant information
  • Summarize found data

LLM: the options for efficient query answering

In business applications, user queries rely on proprietary and/or public data. Strategies for optimizing LLMs vary depending on the use case, volume of data, timelines and available budget. You can adopt a suitable strategy based on pros and cons.

Below are the strategies suggested to choose from:

  1. Retrieval Augmented Generation (RAG): LLMs can retrieve relevant information from sources and generate a response to the user query. RAG approaches typically use LLMs to retrieve data:
  • For structured sources like databases, the DB schema, including table details, is used with the user query.  This affirms that the generated DB query matches the schema and can be readily executed.
  • For unstructured documents, the content is indexed and stored for semantic retrieval, often chunked at section, page, paragraph or sentence level, with embeddings or vectors appropriately computed and stored. When a user query must be processed, an embedding is generated and compared with those from the documents and the matching ones are retrieved. Choosing the right LLM is crucial to avoid missing key information needed to answer queries.

Once retrieved, information can be summarized to respond to the user query. Irrelevant retrieved information can be handled as summarization is contextual. RAG, a common strategy, uses pre-trained models and eliminates the need for additional training. It can be easily scaled to new data and offers short development cycles, saving time and cost.

  1. Fine-tuning LLM for business application: Fine-tuning is an option to customize a LLM for a complex or a domain-specific problem. Usually, trained LLMs can be fine-tuned for specific tasks for better performance than an off-the-shelf LLM. For example, LLMs can be fine-tuned to convert a user query to a SQL query relevant to the tables and, or data.  

Fine-tuned models require regular monitoring, retraining and re-deployment to stay current with new data. Choosing between RAG and fine-tuning depends on the tradeoffs specific to the business application.

  1. Train an LLM on data to be queried: Consider training LLMs on specific data for higher accuracy, especially when they are significantly different from data used for pre-training. For unstructured sources, LLMs respond directly to user queries like ChatGPT. With structured sources, there is no need to provide the DB schema along with the user query.

LLMs such as GPT-4 require significant compute power like Graphics Processing Units (GPU) to train, which is expensive. Offering clean data with minimal noise and bias for training requires significant effort.

With millions of parameters, LLMs risk overfitting or underfitting the data used for training. This requires careful preparation, training, and evaluation. Also, regular retraining keeps LLMs updated. In the context of documents, training on the entire text corpus of available documents will suffice, while a few hundred or thousand example pairs are needed for user to DB query conversion, depending on the LLM architecture.  

Transforming business applications with evolving LLMs

Given the vast potential of LLMs to retrieve information from structured and unstructured sources, we see the development of applications where users can query them in natural language to find the details they need. With their rapid progress, more advanced models providing higher performance will be released regularly.  

We've discussed three strategies — RAG, fine-tuning, and training — that can be adopted. The choice will depend on the business application and the performance that is required. With regular advances being made in LLMs, information retrieval using them will become more accurate and efficient.

About the author:

Vishnu Vardhan Makkapati is an Associate Director at EY Global Delivery Services India LLP.

He leads the Client Technology AI Center of Excellence in India. He holds more than 35 patents, with many others pending.

Vishnu has published several papers in reputed international conferences and journals.

The views reflected in this article are the views of the author and do not necessarily reflect the views of the global EY organization or its member firms.


That the contents of third-party articles/blogs published here on the website, and the interpretation of all information in the article/blogs such as data, maps, numbers, opinions etc. displayed in the article/blogs and views or the opinions expressed within the content are solely of the author's; and do not reflect the opinions and beliefs of NASSCOM or its affiliates in any manner. NASSCOM does not take any liability w.r.t. content in any manner and will not be liable in any manner whatsoever for any kind of liability arising out of any act, error or omission. The contents of third-party article/blogs published, are provided solely as convenience; and the presence of these articles/blogs should not, under any circumstances, be considered as an endorsement of the contents by NASSCOM in any manner; and if you chose to access these articles/blogs , you do so at your own risk.


© Copyright nasscom. All Rights Reserved.