NousResearch Released Nous-Hermes-2-Mixtral-8x7B: An Open-Source LLM with SFT and DPO Versions

In artificial intelligence and language models, users often face challenges in training and utilizing models for various tasks. The need for a versatile, high-performing model to understand and generate content across different domains is apparent. Existing solutions may provide some level of performance, but they need to catch up in achieving state-of-the-art results and adaptability. The problem is for an advanced language model that can excel in understanding and generating content across many tasks. While other models are available, the existing options may only partially meet the criteria of achieving cutting-edge performance and versatility.

NousResearch just released Nous-Hermes-2-Mixtral-8x7B. It has 2 versions, including an SFT and a DPO version of this model. Nous Hermes 2 Mixtral 8x7B DPO aims to address these challenges by offering a state-of-the-art solution. Trained on a vast dataset comprising primarily GPT-4 generated data and supplemented with high-quality information from open datasets in the AI field, this model exhibits exceptional performance across various tasks. It introduces a novel SFT + DPO version, and for those who prefer a different approach, an SFT-only version is also made available.

The Nous Hermes 2 Mixtral 8x7B SFT is a specialized version of the latest Nous Research model, designed exclusively for supervised fine-tuning. It’s built on the Mixtral 8x7B MoE LLM architecture. This model has been trained using more than one million entries, predominantly generated by GPT-4, along with other high-quality data from various open datasets in the AI field. It demonstrates exceptional performance across a wide range of tasks, setting new benchmarks in the industry.

The Nous-Hermes-2-Mixtral-8x7B model has undergone benchmark testing against GPT4All, AGIEval, and BigBench tasks. The results demonstrate significant improvements over the base Mixtral model, surpassing even the flagship Mixtral Finetune by MistralAI. The average performance across these benchmarks is an impressive 75.70 for GPT4All, 46.05 for AGIEval, and 49.70 for BigBench.

The introduction of ChatML as the prompt format allows for a more structured and engaging interaction with the model, particularly in multi-turn chat dialogues. System prompts enable steerability, providing users with a nuanced way to guide the model’s responses based on roles, rules, and stylistic choices. This format, which aligns with the OpenAI endpoint compatibility, enhances the user experience and makes the model more accessible.

In conclusion, Nous Hermes 2 Mixtral 8x7B DPO is a powerful solution to language model training and utilization challenges. Its comprehensive training data, innovative versions, and impressive benchmark results make it a versatile and high-performing model. With a focus on user interaction through ChatML and a commitment to surpassing existing benchmarks, this model stands out as an advanced and effective tool in artificial intelligence.

Niharika is a Technical consulting intern at Marktechpost. She is a third year undergraduate, currently pursuing her B.Tech from Indian Institute of Technology(IIT), Kharagpur. She is a highly enthusiastic individual with a keen interest in Machine learning, Data science and AI and an avid reader of the latest developments in these fields.