Today, we are thrilled to reveal that DeepSeek R1 distilled Llama and Qwen models are available through Amazon Bedrock Marketplace and Amazon SageMaker JumpStart. With this launch, you can now deploy DeepSeek AI’s first-generation frontier model, DeepSeek-R1, together with the distilled versions varying from 1.5 to 70 billion criteria to construct, experiment, and properly scale your generative AI ideas on AWS.

In this post, we show how to get started with DeepSeek-R1 on Amazon Bedrock Marketplace and SageMaker JumpStart. You can follow similar steps to release the distilled variations of the models also.

of DeepSeek-R1

DeepSeek-R1 is a large language model (LLM) developed by DeepSeek AI that uses reinforcement learning to improve thinking capabilities through a multi-stage training procedure from a DeepSeek-V3-Base foundation. A key differentiating feature is its reinforcement learning (RL) step, which was used to improve the design’s actions beyond the basic pre-training and fine-tuning process. By integrating RL, DeepSeek-R1 can adjust more successfully to user feedback and objectives, eventually improving both importance and clarity. In addition, DeepSeek-R1 employs a chain-of-thought (CoT) technique, implying it’s geared up to break down complicated inquiries and factor through them in a detailed way. This guided reasoning procedure enables the design to produce more accurate, transparent, and detailed answers. This design integrates RL-based fine-tuning with CoT abilities, aiming to produce structured reactions while focusing on interpretability and wiki.vst.hs-furtwangen.de user interaction. With its extensive abilities DeepSeek-R1 has caught the market’s attention as a flexible text-generation model that can be incorporated into various workflows such as agents, rational thinking and data analysis tasks.

DeepSeek-R1 utilizes a Mix of Experts (MoE) architecture and is 671 billion criteria in size. The MoE architecture enables activation of 37 billion parameters, making it possible for effective inference by routing inquiries to the most pertinent professional “clusters.” This technique allows the model to focus on different problem domains while maintaining total efficiency. DeepSeek-R1 requires a minimum of 800 GB of HBM memory in FP8 format for inference. In this post, we will use an ml.p5e.48 xlarge instance to release the model. ml.p5e.48 xlarge includes 8 Nvidia H200 GPUs offering 1128 GB of GPU memory.

DeepSeek-R1 distilled models bring the thinking abilities of the main R1 model to more effective architectures based upon popular open models like Qwen (1.5 B, 7B, 14B, and 32B) and Llama (8B and 70B). Distillation describes a process of training smaller, more effective designs to imitate the behavior and thinking patterns of the larger DeepSeek-R1 model, utilizing it as an instructor design.

You can release DeepSeek-R1 model either through SageMaker JumpStart or Bedrock Marketplace. Because DeepSeek-R1 is an emerging model, we suggest releasing this model with guardrails in location. In this blog site, we will utilize Amazon Bedrock Guardrails to introduce safeguards, avoid hazardous material, and evaluate designs against crucial safety criteria. At the time of composing this blog site, for DeepSeek-R1 releases on SageMaker JumpStart and Bedrock Marketplace, Bedrock Guardrails supports only the ApplyGuardrail API. You can create several guardrails tailored to different usage cases and apply them to the DeepSeek-R1 design, improving user experiences and standardizing security controls across your generative AI applications.

Prerequisites

To release the DeepSeek-R1 model, you need access to an ml.p5e circumstances. To check if you have quotas for P5e, open the Service Quotas console and under AWS Services, pick Amazon SageMaker, and verify you’re utilizing ml.p5e.48 xlarge for endpoint usage. Make certain that you have at least one ml.P5e.48 xlarge instance in the AWS Region you are releasing. To ask for a limit increase, develop a limit boost request and connect to your account team.

Because you will be releasing this design with Amazon Bedrock Guardrails, make certain you have the right AWS Identity and Gain Access To Management (IAM) permissions to use Amazon Bedrock Guardrails. For directions, see Establish authorizations to use guardrails for gratisafhalen.be material filtering.

Implementing guardrails with the ApplyGuardrail API

Amazon Bedrock Guardrails enables you to present safeguards, avoid harmful content, [forum.batman.gainedge.org](https://forum.batman.gainedge.org/index.php?action=profile