Saturday, May 24, 2025
News PouroverAI
Visit PourOver.AI
No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
News PouroverAI
No Result
View All Result

Stanford Researchers Introduce RAPTOR: A Novel Tree-based Retrieval System that Augments the Parametric Knowledge of LLMs with Contextual Information

February 8, 2024
in AI Technology
Reading Time: 3 mins read
0 0
A A
0
Share on FacebookShare on Twitter


Retrieval-augmented language models often retrieve only short chunks from a corpus, limiting overall document context. This decreases their ability to adapt to changes in the world state and incorporate long-tail knowledge. Existing retrieval-augmented approaches also need fixing. The one we tackle is that most existing methods retrieve only a few short, contiguous text chunks, which limits their ability to represent and leverage large-scale discourse structure. This is particularly relevant for thematic questions that require integrating knowledge from multiple text parts, such as understanding an entire book.

Recent developments in Large Language Models (LLMs) demonstrate their effectiveness as standalone knowledge stores, encoding facts within their parameters. Fine-tuning downstream tasks further enhances their performance. However, challenges arise in updating LLMs with evolving world knowledge. An alternative approach involves indexing text in an information retrieval system and presenting retrieved information to LLMs for current domain-specific knowledge. Existing retrieval-augmented methods are limited to retrieving only short, contiguous text chunks, hindering the representation of large-scale discourse structure, which is crucial for thematic questions and a comprehensive understanding of texts like in the NarrativeQA dataset.

The researchers from Stanford University propose RAPTOR, an innovative indexing and retrieval system designed to address limitations in existing methods. RAPTOR utilizes a tree structure to capture a text’s high-level and low-level details. It clusters text chunks, generates summaries for clusters, and constructs a tree from the bottom up. This structure enables loading different levels of text chunks into LLMs context, facilitating efficient and effective answering of questions at various levels. The key contribution is using text summarization for retrieval augmentation, enhancing context representation across different scales, as demonstrated in experiments on long document collections.

\"\"

RAPTOR addresses reading semantic depth and connection issues by constructing a recursive tree structure that captures both broad thematic comprehension and granular details. The process involves segmenting the retrieval corpus into chunks, embedding them using SBERT, and clustering them with a soft clustering algorithm based on Gaussian Mixture Models (GMMs) and Uniform Manifold Approximation and Projection (UMAP). The resulting tree structure allows for efficient querying through tree traversal or a collapsed tree approach, enabling retrieval of relevant information at different levels of specificity.

\"\"

RAPTOR outperforms baseline methods across three question-answering datasets: NarrativeQA, QASPER, and QuALITY. Control comparisons using UnifiedQA 3B as the reader show consistent superiority of RAPTOR over BM25 and DPR. Paired with GPT-4, RAPTOR achieves state-of-the-art results on QASPER and QuALITY datasets, showcasing its effectiveness in handling thematic and multi-hop queries. The contribution of the tree structure is validated, demonstrating the significance of upper-level nodes in capturing a broader understanding and enhancing retrieval capabilities.

\"\"
\"\"

In conclusion, Stanford University researchers introduce RAPTOR, an innovative tree-based retrieval system that enhances the knowledge of large language models with contextual information across different abstraction levels. RAPTOR constructs a hierarchical tree structure through recursive clustering and summarization, facilitating the effective synthesis of information from diverse sections of retrieval corpora. Controlled experiments showcase RAPTOR’s superiority over traditional methods, establishing new benchmarks in various question-answering tasks. Overall, RAPTOR proves to be a promising approach for advancing the capabilities of language models through enhanced contextual retrieval.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and Google News. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our Telegram Channel


Source link

Tags: AugmentsContextualInformationIntroduceKnowledgeLLMsParametricRAPTORResearchersretrievalStanfordsystemTreebased
Previous Post

Former Psagot CEO Yohan Kadoche buying control of Sigma Clarity

Next Post

This AI Paper Introduces PirateNets: A Novel AI System Designed to Facilitate Stable and Efficient Training of Deep Physics-Informed Neural Network Models

Related Posts

How insurance companies can use synthetic data to fight bias
AI Technology

How insurance companies can use synthetic data to fight bias

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset
AI Technology

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper
AI Technology

Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper

June 9, 2024
How Game Theory Can Make AI More Reliable
AI Technology

How Game Theory Can Make AI More Reliable

June 9, 2024
Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs
AI Technology

Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs

June 9, 2024
Deciphering Doubt: Navigating Uncertainty in LLM Responses
AI Technology

Deciphering Doubt: Navigating Uncertainty in LLM Responses

June 9, 2024
Next Post
This AI Paper Introduces PirateNets: A Novel AI System Designed to Facilitate Stable and Efficient Training of Deep Physics-Informed Neural Network Models

This AI Paper Introduces PirateNets: A Novel AI System Designed to Facilitate Stable and Efficient Training of Deep Physics-Informed Neural Network Models

35 CSS Animation Libraries

35 CSS Animation Libraries

US Army ends Future Attack Reconnaissance Aircraft program By Reuters

US Army ends Future Attack Reconnaissance Aircraft program By Reuters

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Trending
  • Comments
  • Latest
Is C.AI Down? Here Is What To Do Now

Is C.AI Down? Here Is What To Do Now

January 10, 2024
23 Plagiarism Facts and Statistics to Analyze Latest Trends

23 Plagiarism Facts and Statistics to Analyze Latest Trends

June 4, 2024
A faster, better way to prevent an AI chatbot from giving toxic responses | MIT News

A faster, better way to prevent an AI chatbot from giving toxic responses | MIT News

April 10, 2024
Porfo: Revolutionizing the Crypto Wallet Landscape

Porfo: Revolutionizing the Crypto Wallet Landscape

October 9, 2023
Implementing User Authentication in React Apps with Appwrite — SitePoint

Implementing User Authentication in React Apps with Appwrite — SitePoint

January 30, 2024
NousResearch Released Nous-Hermes-2-Mixtral-8x7B: An Open-Source LLM with SFT and DPO Versions

NousResearch Released Nous-Hermes-2-Mixtral-8x7B: An Open-Source LLM with SFT and DPO Versions

January 25, 2024
Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

June 10, 2024
AI Compared: Which Assistant Is the Best?

AI Compared: Which Assistant Is the Best?

June 10, 2024
How insurance companies can use synthetic data to fight bias

How insurance companies can use synthetic data to fight bias

June 10, 2024
5 SLA metrics you should be monitoring

5 SLA metrics you should be monitoring

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

June 10, 2024
Facebook Twitter LinkedIn Pinterest RSS
News PouroverAI

The latest news and updates about the AI Technology and Latest Tech Updates around the world... PouroverAI keeps you in the loop.

CATEGORIES

  • AI Technology
  • Automation
  • Blockchain
  • Business
  • Cloud & Programming
  • Data Science & ML
  • Digital Marketing
  • Front-Tech
  • Uncategorized

SITEMAP

  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us

Copyright © 2023 PouroverAI News.
PouroverAI News

No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing

Copyright © 2023 PouroverAI News.
PouroverAI News

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In