Monday, June 2, 2025
News PouroverAI
Visit PourOver.AI
No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing
News PouroverAI
No Result
View All Result

A decoder-only foundation model for time-series forecasting – Google Research Blog

February 2, 2024
in AI Technology
Reading Time: 3 mins read
0 0
A A
0
Share on FacebookShare on Twitter



Posted by Rajat Sen and Yichen Zhou, Google Research

Time-series forecasting is widely used in various domains, including retail, finance, manufacturing, healthcare, and natural sciences. Improving the accuracy of demand forecasting in retail, for example, can significantly reduce inventory costs and increase revenue. Deep learning (DL) models have become popular in forecasting rich, multivariate, time-series data because they have shown excellent performance in different settings (e.g., DL models dominated the M5 competition leaderboard). Additionally, there have been significant advancements in large foundation language models used for natural language processing (NLP) tasks like translation, retrieval-augmented generation, and code completion. These models are trained on massive amounts of textual data from various sources, enabling them to identify language patterns. This makes them powerful zero-shot tools, capable of answering questions and summarizing current events when combined with retrieval.

However, DL-based forecasters still face challenges. Most DL architectures require extensive training and validation cycles before customers can test them on new time-series data. In contrast, a foundation model for time-series forecasting can provide accurate forecasts on unseen time-series data without additional training, allowing users to focus on refining forecasts for specific tasks, such as retail demand planning. In our paper, “A decoder-only foundation model for time-series forecasting,” we introduce TimesFM, a single forecasting model pre-trained on a large time-series corpus of 100 billion real-world time-points. Despite being much smaller (200M parameters) than the latest large language models (LLMs), TimesFM demonstrates impressive zero-shot performance on a variety of unseen datasets from different domains and temporal granularities, rivaling state-of-the-art supervised approaches trained explicitly on these datasets. We plan to make this model available for external customers in Google Cloud Vertex AI later this year.

TimesFM follows a decoder-only training approach similar to LLMs, consisting of three steps. First, text is divided into subwords called tokens. Then, these tokens are processed by stacked causal transformer layers that generate an output corresponding to each input token. Finally, the output summarizes information from previous tokens and predicts the next token. During inference, the model generates output tokens one at a time. For example, when given the prompt “What is the capital of France?” the model might generate the token “The,” then condition on “What is the capital of France? The” to generate the next token “capital,” and so on until it produces the complete answer: “The capital of France is Paris.”

In the context of time-series forecasting, we treat a patch of time-points as a token, similar to recent long-horizon forecasting work. The goal is to forecast the next patch of time-points given the previous output. Unlike language models, we need a multilayer perceptron block with residual connections to convert a patch of time-series into a token that can be input to the transformer layers. At the other end, the output can be used to predict a longer length of subsequent time-points than the input patch length. This flexibility allows for better performance in long-horizon forecasting.

To train TimesFM, we use a large volume of legitimate time series data. Synthetic data helps establish fundamental temporal patterns, while real-world data from public time series datasets, including Google Trends and Wikipedia Pageviews, provides domain-specific contexts that enhance generalization. We evaluate TimesFM’s zero-shot performance on unseen data using popular time-series benchmarks and find that it outperforms most statistical methods and even powerful DL models specifically trained on the target time-series. We also compare TimesFM to GPT-3.5 for forecasting and demonstrate that TimesFM performs better despite being significantly smaller.

In conclusion, we present TimesFM, a decoder-only foundation model for time-series forecasting trained on a large pretraining corpus. Despite its smaller size, TimesFM exhibits impressive zero-shot performance on various public benchmarks. We would like to acknowledge the contributions of our research team to this work.

Acknowledgements:
This work is…



Source link

Tags: BlogdecoderonlyForecastingFoundationGooglemodelResearchtimeseries
Previous Post

Examples of sustainability: How new efforts are shaping the environmental future

Next Post

IBM Databand: Self-learning for anomaly detection

Related Posts

How insurance companies can use synthetic data to fight bias
AI Technology

How insurance companies can use synthetic data to fight bias

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset
AI Technology

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper
AI Technology

Decoding Decoder-Only Transformers: Insights from Google DeepMind’s Paper

June 9, 2024
How Game Theory Can Make AI More Reliable
AI Technology

How Game Theory Can Make AI More Reliable

June 9, 2024
Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs
AI Technology

Buffer of Thoughts (BoT): A Novel Thought-Augmented Reasoning AI Approach for Enhancing Accuracy, Efficiency, and Robustness of LLMs

June 9, 2024
Deciphering Doubt: Navigating Uncertainty in LLM Responses
AI Technology

Deciphering Doubt: Navigating Uncertainty in LLM Responses

June 9, 2024
Next Post
IBM Databand: Self-learning for anomaly detection

IBM Databand: Self-learning for anomaly detection

Does ExpressVPN Work in China in 2024? Expert Explains

Does ExpressVPN Work in China in 2024? Expert Explains

Agtech startup Small Robot Company shutting down

Agtech startup Small Robot Company shutting down

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Trending
  • Comments
  • Latest
Is C.AI Down? Here Is What To Do Now

Is C.AI Down? Here Is What To Do Now

January 10, 2024
Accenture creates a regulatory document authoring solution using AWS generative AI services

Accenture creates a regulatory document authoring solution using AWS generative AI services

February 6, 2024
23 Plagiarism Facts and Statistics to Analyze Latest Trends

23 Plagiarism Facts and Statistics to Analyze Latest Trends

June 4, 2024
Managing PDFs in Node.js with pdf-lib

Managing PDFs in Node.js with pdf-lib

November 16, 2023
NVIDIA’s Marketing Strategy Case Study

NVIDIA’s Marketing Strategy Case Study

October 25, 2023
25+ CSS Pricing Tables

25+ CSS Pricing Tables

October 28, 2023
Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

Can You Guess What Percentage Of Their Wealth The Rich Keep In Cash?

June 10, 2024
AI Compared: Which Assistant Is the Best?

AI Compared: Which Assistant Is the Best?

June 10, 2024
How insurance companies can use synthetic data to fight bias

How insurance companies can use synthetic data to fight bias

June 10, 2024
5 SLA metrics you should be monitoring

5 SLA metrics you should be monitoring

June 10, 2024
From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

From Low-Level to High-Level Tasks: Scaling Fine-Tuning with the ANDROIDCONTROL Dataset

June 10, 2024
UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

UGRO Capital: Targeting to hit milestone of Rs 20,000 cr loan book in 8-10 quarters: Shachindra Nath

June 10, 2024
Facebook Twitter LinkedIn Pinterest RSS
News PouroverAI

The latest news and updates about the AI Technology and Latest Tech Updates around the world... PouroverAI keeps you in the loop.

CATEGORIES

  • AI Technology
  • Automation
  • Blockchain
  • Business
  • Cloud & Programming
  • Data Science & ML
  • Digital Marketing
  • Front-Tech
  • Uncategorized

SITEMAP

  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us

Copyright © 2023 PouroverAI News.
PouroverAI News

No Result
View All Result
  • Home
  • AI Tech
  • Business
  • Blockchain
  • Data Science & ML
  • Cloud & Programming
  • Automation
  • Front-Tech
  • Marketing

Copyright © 2023 PouroverAI News.
PouroverAI News

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In