PyTorch has recently introduced the alpha version of torchtune, a new library created to simplify the fine-tuning process of large language models (LLMs) using PyTorch.
Click here to visit the GitHub repository.
This library is based on PyTorch’s fundamental principles, providing modular components and customizable training methods for fine-tuning popular LLMs on various GPUs, ranging from consumer-grade to professional ones.
torchtune offers a complete fine-tuning workflow, which includes downloading and preparing datasets and model checkpoints, customizing training with flexible building blocks, tracking progress, quantizing models post-tuning, evaluating fine-tuned models, conducting local inference for testing, and ensuring compatibility with common production inference systems.
With the goal of meeting the growing demand for fine-tuning LLMs, torchtune allows users to easily incorporate customizations and optimizations to tailor models for specific use cases, including memory-efficient techniques suitable for machines with single 24GB gaming GPUs.
The design of torchtune emphasizes on easy expandability, making fine-tuning accessible to users with varying levels of expertise, and fostering interoperability within the open-source LLM ecosystem.
The library seamlessly integrates with popular tools like Hugging Face Hub, PyTorch FSDP, Weights & Biases, EleutherAI’s LM Evaluation Harness, ExecuTorch, and torchao for tasks such as model and dataset access, distributed training, logging, evaluation, inference, and quantization.
torchtune currently supports Llama 2, Mistral, and Gemma 7B models, with plans to introduce additional models, features, and fine-tuning methods in the near future, including models with 70 billion parameters and Mixture of Experts models.
