Sunday, June 8, 2025
News PouroverAI
Visit PourOver.AI
No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
News PouroverAI
No Result
View All Result

Do Language Models Know When They Are Hallucinating? This AI Research from Microsoft and Columbia University Explores Detecting Hallucinations with the Creation of Probes

January 1, 2024
in Data Science & ML
Reading Time: 3 mins read
0 0
A A
0
Share on FacebookShare on Twitter


Large Language Models (LLMs), the latest innovation of Artificial Intelligence (AI), use deep learning techniques to produce human-like text and perform various Natural Language Processing (NLP) and Natural Language Generation (NLG) tasks. Trained on large amounts of textual data, these models perform various tasks, including generating meaningful responses to questions, text summarization, translations, text-to-text transformation, and code completion.

In recent research, a team of researchers has studied hallucination detection in grounded generation tasks with a special emphasis on language models, especially the decoder-only transformer models. Hallucination detection aims to ascertain whether the generated text is true to the input prompt or contains false information.

In recent research, a team of researchers from Microsoft and Columbia University has addressed the construction of probes for the model to anticipate a transformer language model’s hallucinatory behavior during in-context creation tasks. The main focus has been on using the model’s internal representations for the detection and a dataset with annotations for both synthetic and biological hallucinations.

Probes are basically the instruments or systems trained on the language model’s internal operations. Their job is to predict when the model might provide delusional material when doing tasks involving the development of contextually appropriate content. For training and assessing these probes, it is imperative to provide a span-annotated dataset containing examples of synthetic hallucinations, purposely induced disparities in reference inputs, and organic hallucinations derived from the model’s own outputs.

The research has shown that probes designed to identify force-decoded states of artificial hallucinations are not very effective at identifying biological hallucinations. This shows that when trained on modified or synthetic instances, the probes may not generalize well to real-world, naturally occurring hallucinations. The team has shared that the distribution properties and task-specific information impact the hallucination data in the model’s hidden states.

The team has analyzed the intricacy of intrinsic and extrinsic hallucination saliency across various tasks, hidden state kinds, and layers. The transformer’s internal representations emphasize extrinsic hallucinations- i.e., those connected to the outside world more. Two methods have been used to gather hallucinations which include using sampling replies produced by an LLM conditioned on inputs and introducing inconsistencies into reference inputs or outputs by editing.

 

The outputs of the second technique have been reported to elicit a higher rate of hallucination annotations by human annotators; however, synthetic examples are considered less valuable because they do not match the test distribution.

The team has summarized their primary contributions as follows.

A dataset with more than 15,000 utterances has been produced that have been tagged for hallucinations in both natural and artificial output texts. The dataset covers three grounded generation tasks.

Three probe architectures have been presented for the efficient detection of hallucinations, which demonstrate improvements in efficiency and accuracy for detecting hallucinations over several current baselines.

The study has explored the elements that affect the accuracy of the probe, such as the nature of the hallucinations, i.e., intrinsic vs. extrinsic, the size of the model, and the particular encoding components that are being probed.

 

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 35k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, LinkedIn Group, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

Tanya Malhotra is a final year undergrad from the University of Petroleum & Energy Studies, Dehradun, pursuing BTech in Computer Science Engineering with a specialization in Artificial Intelligence and Machine Learning.She is a Data Science enthusiast with good analytical and critical thinking, along with an ardent interest in acquiring new skills, leading groups, and managing work in an organized manner.

🎯 Meet AImReply: Your New AI Email Writing Extension…. Try it free now!.



Source link

Tags: ColumbiacreationDetectingExploresHallucinatingHallucinationslanguageMicrosoftmodelsProbesResearchUniversity
Previous Post

Bitcoin’s Next Bull Run: Three Big Trends To Watch

Next Post

This Paper Explores AI-Driven Hedging Strategies in Finance: A Deep Dive into the Use of Recurrent Neural Networks and k-Armed Bandit Models for Efficient Market Simulation and Risk Management

Related Posts

AI Compared: Which Assistant Is the Best?
Data Science & ML

AI Compared: Which Assistant Is the Best?

June 10, 2024
5 Machine Learning Models Explained in 5 Minutes
Data Science & ML

5 Machine Learning Models Explained in 5 Minutes

June 7, 2024
Cohere Picks Enterprise AI Needs Over ‘Abstract Concepts Like AGI’
Data Science & ML

Cohere Picks Enterprise AI Needs Over ‘Abstract Concepts Like AGI’

June 7, 2024
How to Learn Data Analytics – Dataquest
Data Science & ML

How to Learn Data Analytics – Dataquest

June 6, 2024
Adobe Terms Of Service Update Privacy Concerns
Data Science & ML

Adobe Terms Of Service Update Privacy Concerns

June 6, 2024
Build RAG applications using Jina Embeddings v2 on Amazon SageMaker JumpStart
Data Science & ML

Build RAG applications using Jina Embeddings v2 on Amazon SageMaker JumpStart

June 6, 2024
Next Post
This Paper Explores AI-Driven Hedging Strategies in Finance: A Deep Dive into the Use of Recurrent Neural Networks and k-Armed Bandit Models for Efficient Market Simulation and Risk Management

This Paper Explores AI-Driven Hedging Strategies in Finance: A Deep Dive into the Use of Recurrent Neural Networks and k-Armed Bandit Models for Efficient Market Simulation and Risk Management

Vitalik Buterin Talks about Ethereum’s 2024 Roadmap

Vitalik Buterin Talks about Ethereum's 2024 Roadmap

Top 20 Prompt Engineering Interview Questions and Answers

Top 20 Prompt Engineering Interview Questions and Answers

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Trending
  • Comments
  • Latest
23 Plagiarism Facts and Statistics to Analyze Latest Trends

23 Plagiarism Facts and Statistics to Analyze Latest Trends

June 4, 2024
Accenture creates a regulatory document authoring solution using AWS generative AI services

Accenture creates a regulatory document authoring solution using AWS generative AI services

February 6, 2024
Managing PDFs in Node.js with pdf-lib

Managing PDFs in Node.js with pdf-lib

November 16, 2023
Graph neural networks in TensorFlow – Google Research Blog

Graph neural networks in TensorFlow – Google Research Blog

February 6, 2024
13 Best Books, Courses and Communities for Learning React — SitePoint

13 Best Books, Courses and Communities for Learning React — SitePoint

February 4, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

June 10, 2024
AI Compared: Which Assistant Is the Best?

AI Compared: Which Assistant Is the Best?

June 10, 2024
How insurance companies can use synthetic data to fight bias

How insurance companies can use synthetic data to fight bias

June 10, 2024
5 SLA metrics you should be monitoring

5 SLA metrics you should be monitoring

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

June 10, 2024
Facebook Twitter LinkedIn Pinterest RSS
News PouroverAI

The latest news and updates about the AI Technology and Latest Tech Updates around the world... PouroverAI keeps you in the loop.

CATEGORIES

  • AI Technology
  • Automation
  • Blockchain
  • Business
  • Cloud & Programming
  • Data Science & ML
  • Digital Marketing
  • Front-Tech
  • Uncategorized

SITEMAP

  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us

Copyright © 2023 PouroverAI News.
PouroverAI News

No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing

Copyright © 2023 PouroverAI News.
PouroverAI News

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In