LLMs (Large Language Models) are trained on vast volumes of textual data to comprehend and produce language similar to that of humans. The GPT-3, GPT-4, and PaLM-2 are few examples. These models perform complex language tasks, including text generation, conversational interaction, and question answering. They have been used in various domains, enhancing user experiences in chatbots, coding, web search, customer support, and content production.
However, as the AI community delves into the vast landscape of smaller models, Microsoft has introduced the next version of Orca called Orca 2, designed to amplify the capacities of compact AI models. Orca 1, through the integration of detailed explanation, traces, surpasses traditional instruction-tuned models in performance on challenging benchmarks like BigBench Hard and AGIEval. Orca 2 further delves into the potential of enhanced training signals to boost the reasoning capabilities of smaller language models
Imitation learning has been a prevalent approach in refining small language models. These smaller models often need to catch up in reasoning and comprehension skills, even though they can produce content in a manner akin to that of their teachers. Although imitation learning has some benefits, it has drawbacks that may limit smaller models’ ability to reach their full potential and prevent them from using the best possible solutions given the particular problem and the model’s capabilities. They often need help matching their larger counterparts’ reasoning and comprehension skills, hindering their full potential.
Instead of simply imitating, Orca instructs the model in various reasoning techniques. These include step-by-step processing, recall then generate, recall-reason-generate, and direct answers. The objective is to guide the model in acquiring the ability to discern the most effective solution strategy tailored to the nuances of each specific task.
Orca 2’s zero-shot reasoning ability highlights the possibility of improving smaller neural networks. Microsoft continues to believe that specialized training methods, like the one used for Orca 2, may reveal new useful applications. This method seeks to improve the effectiveness of these neural network deployments.
Most importantly, Orca 2 is protected from the initial cues that elicited particular behaviors during the training phase. Orca 2 transforms into a Cautious Reasoner through the innovative Prompt Erasure technique. Unlike blind imitation, this method uses larger models as a source of behaviors from which the best ones are chosen for the given task.
The researchers tested Orca 2 on comprehensive benchmarks. They showed that it outperforms other equivalent models related to language understanding, common sense reasoning, multi-step math problems, reading comprehension, summarization, and more. For instance, on zero-shot reasoning tasks, Orca 2-13B achieves over 25% higher accuracy than comparable 13B models and is on par with a 70B model.
Orca 2 marks a significant stride in the evolution of small language models. Its departure from conventional imitation learning, coupled with a focus on teaching diverse reasoning techniques, showcases a new approach to unleashing the potential of compact AI models.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter..
Rachit Ranjan is a consulting intern at MarktechPost . He is currently pursuing his B.Tech from Indian Institute of Technology(IIT) Patna . He is actively shaping his career in the field of Artificial Intelligence and Data Science and is passionate and dedicated for exploring these fields.
