Monday, June 2, 2025
News PouroverAI
Visit PourOver.AI
No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
News PouroverAI
No Result
View All Result

Researchers from ETH Zurich and Microsoft Introduce SliceGPT for Efficient Compression of Large Language Models through Sparsification

February 3, 2024
in AI Technology
Reading Time: 3 mins read
0 0
A A
0
Share on FacebookShare on Twitter


Large language models (LLMs) like GPT-4 require substantial computational power and memory, posing challenges for their efficient deployment. While sparsification methods have been developed to mitigate these resource demands, they often introduce new complexities. For example, these techniques may require extra data structures to support the sparse representations, complicating the system architecture. The potential speedups from sparsification are only partially realized due to limitations in current hardware architectures, which are typically optimized for dense computations.

LLM compression methods include sparsification, low-rank approximation, and structured pruning. Methods like Optimal Brain Surgeon (OBS) are impractical due to high computational demands. GPTQ and SparseGPT focus on quantization and pruning. Low-rank approximation simplifies weight matrices, while other methods propose eliminating specific rows and columns. Techniques like ThiNet and LLM-pruner use linear operations and fine-tuning.

Researchers at ETH Zurich and Microsoft Research have proposed SliceGPT. This post-training sparsification scheme reduces the embedding dimension of the network by replacing each weight matrix with a smaller dense matrix. The sliced models of SliceGPT run on fewer GPUs and achieve faster inference without additional code optimization. The method utilizes computational invariance in transformer networks.

The research approach focuses on RMSNorm operations, which maintain transformation invariance, allowing for the application of orthogonal transformations without altering the model’s function. Networks with LayerNorm can be converted to RMSNorm by integrating LayerNorm’s linear components into adjacent blocks. Principal Component Analysis (PCA) is pivotal in this process and is used to identify and project signals onto their principal components at each layer. Minor components are then sliced off, reducing the network size without compromising performance. This technique, validated through experiments, has been shown to outperform SparseGPT, offering significant speedups across various models and tasks.

SliceGPT demonstrates a breakthrough in compressing LLMs like LLAMA-2 70B, OPT 66B, and Phi-2. It efficiently cuts down up to 25% of model parameters, including embeddings, while preserving high task performance. This increases efficiency, enabling the models to run on fewer GPUs and achieve faster inference times without additional code optimization. On consumer and high-end GPUs, SliceGPT significantly reduces compute requirements during inference to 64% and 66%, respectively. The research highlights that OPT models are more compressible than LLAMA-2 models, with larger models showing less accuracy reduction. SliceGPT is a promising approach for reducing LLMs’ resource demands without compromising effectiveness.

SliceGPT allows for structured pruning of LLMs, reducing the cost of inference and maintaining better performance than SparseGPT. Opportunities for improvement include exploring combined methods with SparseGPT, improving Q computation, and using complementary methods like quantization and structural pruning. Observing computational invariance in SliceGPT can contribute to future research in improving the efficiency of deep learning models and inspire new theoretical insights.

Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and Google News. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our Telegram Channel

Source link

Tags: CompressionEfficientETHIntroducelanguageLargeMicrosoftmodelsResearchersSliceGPTSparsificationZurich
Previous Post

Real-Time Anomaly Detection For Quality Control | by Anthony Cavin | Feb, 2024

Next Post

Trump Warns of AI and Deepfake Dangers in Fox Business Interview

Related Posts

How insurance companies can use synthetic data to fight bias
AI Technology

How insurance companies can use synthetic data to fight bias

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset
AI Technology

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper
AI Technology

Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper

June 9, 2024
How Game Theory Can Make AI More Reliable
AI Technology

How Game Theory Can Make AI More Reliable

June 9, 2024
Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs
AI Technology

Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs

June 9, 2024
Deciphering Doubt: Navigating Uncertainty in LLM Responses
AI Technology

Deciphering Doubt: Navigating Uncertainty in LLM Responses

June 9, 2024
Next Post
Trump Warns of AI and Deepfake Dangers in Fox Business Interview

Trump Warns of AI and Deepfake Dangers in Fox Business Interview

How Gen Z’s social media-fueled status obsession disrupted the luxury watch market

How Gen Z's social media-fueled status obsession disrupted the luxury watch market

Survey Shows that More than 90% of Insurers Plan to Increase AI Investment – Top 4 Trends for Insurers in 2024

Survey Shows that More than 90% of Insurers Plan to Increase AI Investment - Top 4 Trends for Insurers in 2024

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Trending
  • Comments
  • Latest
Is C.AI Down? Here Is What To Do Now

Is C.AI Down? Here Is What To Do Now

January 10, 2024
Accenture creates a regulatory document authoring solution using AWS generative AI services

Accenture creates a regulatory document authoring solution using AWS generative AI services

February 6, 2024
Managing PDFs in Node.js with pdf-lib

Managing PDFs in Node.js with pdf-lib

November 16, 2023
23 Plagiarism Facts and Statistics to Analyze Latest Trends

23 Plagiarism Facts and Statistics to Analyze Latest Trends

June 4, 2024
Azul cloud service spots dead code in Java apps

Azul cloud service spots dead code in Java apps

October 7, 2023
The 15 Best Python Courses Online in 2024 [Free + Paid]

The 15 Best Python Courses Online in 2024 [Free + Paid]

April 13, 2024
Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

June 10, 2024
AI Compared: Which Assistant Is the Best?

AI Compared: Which Assistant Is the Best?

June 10, 2024
How insurance companies can use synthetic data to fight bias

How insurance companies can use synthetic data to fight bias

June 10, 2024
5 SLA metrics you should be monitoring

5 SLA metrics you should be monitoring

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

June 10, 2024
Facebook Twitter LinkedIn Pinterest RSS
News PouroverAI

The latest news and updates about the AI Technology and Latest Tech Updates around the world... PouroverAI keeps you in the loop.

CATEGORIES

  • AI Technology
  • Automation
  • Blockchain
  • Business
  • Cloud & Programming
  • Data Science & ML
  • Digital Marketing
  • Front-Tech
  • Uncategorized

SITEMAP

  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us

Copyright © 2023 PouroverAI News.
PouroverAI News

No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing

Copyright © 2023 PouroverAI News.
PouroverAI News

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In