Today, I am pleased to announce the general availability of the Mistral Small foundation model (FM) from Mistral AI in Amazon Bedrock. This follows our recent introductions of Mistral 7B and Mixtral 8x7B in March, and Mistral Large in April. With Mistral Small now accessible in Amazon Bedrock, users have a choice of four high-performing models from Mistral AI, expanding the range of models available.
Developed by Mistral AI, Mistral Small is a highly efficient large language model (LLM) optimized for high-volume, low-latency language-based tasks. It is well-suited for tasks such as classification, customer support, or text generation that can be performed in bulk. Mistral Small offers exceptional performance at a cost-effective price point.
Here are some key features of Mistral Small that you should be aware of:
- Retrieval-Augmented Generation (RAG) specialization – Mistral Small retains important information in long context windows of up to 32K tokens.
- Coding proficiency – Mistral Small excels in code generation, review, and commenting, supporting major coding languages.
- Multilingual capability – Mistral Small delivers high performance in English, French, German, Spanish, and Italian, as well as support for dozens of other languages.
Getting started with Mistral Small: To begin using Mistral Small, you first need to access the model in the Amazon Bedrock console. Choose “Model access,” then “Manage model access.” Expand the Mistral AI section, select Mistral Small, and save your changes.
Once you have model access to Mistral Small, you can start using it in Amazon Bedrock. Refresh the Base models table to view the current status.
Use the following template to build a prompt for the model to generate sub-optimal outputs:
<s>[INST] Instruction [/INST]
Note that <s> is a special token for the beginning of a string (BOS), while [INST] and [/INST] are regular strings.
Try the following prompt for a classification example:
Prompt:
<s>[INST] Classify the following email to determine if it is spam or not. Only respond with the exact words “Spam” or “Not spam”. Limited time offer! Buy 2, Get 1 Free! Offer ends today! Don’t miss out the CHANCE TO SAVE, please click on the link below: https://bit.ly/buy-2-get-1-free [/INST]
Mistral 7B, Mixtral 8x7B, and Mistral Large can all correctly classify this email as “Spam.” Mistral Small can also classify it accurately, just like the larger models. When performing tasks such as generating a Bash script from a text prompt or creating a recipe for preparing yogurt, Mistral Small proves to be the most cost-effective and efficient option among the Mistral AI models in Amazon Bedrock.
Mistral Small excels in multilingual tasks, delivering superior performance not only in English but also in French, German, Spanish, and Italian. Test the model’s understanding of the German language by asking it to provide two sentences about sustainability:
Prompt:
<s>[INST] Geben Sie mir zwei Sätze zum Thema Nachhaltigkeit. [/INST]
Output:
1. Nachhaltigkeit bedeutet, die Bedürfnisse der heutigen Generation zu befriedigen, ohne die Möglichkeiten künftiger Generationen zu gefährden. 2. Ein nachhaltiger Lebensstil umfasst den bewussten Umgang mit Ressourcen, wie Wasser, Energie und Rohstoffen, sowie den Schutz von Umwelt und Klima. Diese Sätze sollten Ihnen einen guten Überblick über das Thema Nachhaltigkeit geben.
Programmatically interact with Mistral Small: You can use the AWS Command Line Interface (AWS CLI) and AWS Software Development Kit (SDK) to interact with Mistral Small programmatically using Amazon Bedrock APIs. The Python code snippet below interacts with Amazon Bedrock Runtime APIs with AWS SDK, asking, “What is the color of the sky?”:
“`python
# Python code snippet to interact with Mistral Small
import argparse
import boto3
from botocore.exceptions import ClientError
import json
accept = “application/json”
content_type = “application/json”
def invoke_model(model_id, input_data, region, streaming):
client = boto3.client(‘bedrock-runtime’, region_name=region)
try:
if streaming:
response = client.invoke_model_with_response_stream(body=input_data, modelId=model_id, accept=accept, contentType=content_type)
else:
response = client.invoke_model(body=input_data, modelId=model_id, accept=accept, contentType=content_type)
status_code = response[‘ResponseMetadata’][‘HTTPStatusCode’]
print(json.loads(response.get(‘body’).read()))
except ClientError as e:
print(e)
if __name__ == “__main__”:
parser = argparse.ArgumentParser(description=”Bedrock Testing Tool”)
parser.add_argument(“–prompt”, type=str, help=”prompt to use”, default=”Hello”)
parser.add_argument(“–max-tokens”, type=int, default=64)
parser.add_argument(“–streaming”, choices=[“true”, “false”], help=”whether to stream or not”, default=”false”)
args = parser.parse_args()
streaming = False
if args.streaming == “true”:
streaming = True
input_data = json.dumps({
“prompt”: f”<s>[INST]{args.prompt}[/INST]”,
“max_tokens”: args.max_tokens
})
invoke_model(model_id=”mistral.mistral-small-2402-v1:0″, input_data=input_data, region=”us-east-1″, streaming=streaming)
“`
You will receive the following output:
{‘outputs’: [{‘text’: ‘ The color of the sky can vary depending on the time of day, weather,’, ‘stop_reason’: ‘length’}]}
Now available: The Mistral Small model is now available in Amazon Bedrock in the US East (N. Virginia) Region.
To learn more, visit the Mistral AI in Amazon Bedrock product page. For pricing details, review the Amazon Bedrock pricing page.
To get started with Mistral Small in Amazon Bedrock, visit the Amazon Bedrock console and Amazon Bedrock User Guide.
— Esra
