Today, we are thrilled to announce that the DBRX model, an open-source, versatile large language model (LLM) created by Databricks, can now be accessed by customers through Amazon SageMaker JumpStart for easy deployment to run inference with just one click. The DBRX LLM utilizes a fine-grained mixture-of-experts (MoE) architecture, pre-trained on 12 trillion tokens of meticulously curated data with a maximum context length of 32,000 tokens. You can experiment with this model using SageMaker JumpStart, a machine learning (ML) hub that grants access to algorithms and models for a quick start in ML projects. In this article, we guide you through the discovery and deployment process of the DBRX model.
The DBRX model is an advanced decoder-only LLM constructed on transformer architecture. It employs a fine-grained MoE architecture with a total of 132 billion parameters, 36 billion of which are active for any given input. The model was pre-trained on a dataset containing 12 trillion tokens of text and code. Unlike other open MoE models like Mixtral and Grok-1, DBRX utilizes a fine-grained approach with a higher quantity of smaller experts for enhanced performance. With 16 experts to choose from, DBRX stands out among other MoE models. The model is available under the Databricks Open Model license, allowing unrestricted use.
SageMaker JumpStart is a fully managed platform offering cutting-edge foundation models for various applications like content writing, code generation, question answering, copywriting, summarization, classification, and information retrieval. It provides a range of pre-trained models that can be quickly and easily deployed, speeding up the development and deployment of ML applications. The Model Hub in SageMaker JumpStart features a wide array of pre-trained models, including DBRX, for various tasks. You can now discover and deploy DBRX models with just a few clicks in Amazon SageMaker Studio or programmatically through the SageMaker Python SDK, enabling you to leverage model performance and MLOps controls with features like Amazon SageMaker Pipelines, Amazon SageMaker Debugger, or container logs. The model is deployed in a secure AWS environment under your VPC controls, ensuring data security.
To access the DBRX model through SageMaker JumpStart in the SageMaker Studio UI, navigate to the JumpStart section in the navigation pane. From the SageMaker JumpStart landing page, search for “DBRX” in the search box to find DBRX Instruct and DBRX Base. Click on the model card to view details such as license information, training data, and usage instructions. You can then deploy the model by clicking on the Deploy button and creating an endpoint.
Deployment starts when you choose the Deploy button, and upon completion, an endpoint will be created. You can test the endpoint by sending a sample inference request payload or by using the testing option with the SDK. The code provided in the SDK allows you to run inference against the deployed endpoint. Additionally, you can deploy the DBRX Base model using its model ID with the following code:
“`python
from sagemaker.jumpstart.model import JumpStartModel
accept_eula = True
model = JumpStartModel(model_id=”huggingface-llm-dbrx-base”)
predictor = model.deploy(accept_eula=accept_eula)
“`
This code deploys the model on SageMaker with default configurations, but you can customize these configurations by specifying non-default values in JumpStartModel. Remember to set the Eula value to True to accept the end-user license agreement (EULA) and check your account-level service limit for using specific instances for endpoint usage. After deployment, you can interact with the deployed endpoint using the SageMaker predictor.
In conclusion, the DBRX model, available through Amazon SageMaker JumpStart, offers a powerful and efficient solution for deploying large language models with ease. Its fine-grained MoE architecture and extensive pre-training make it a valuable asset for various ML applications. Experience the capabilities of the DBRX model today and accelerate your ML projects with SageMaker JumpStart.
Source link
