DeepSeek-Coder-V2-Lite-Instruct-20B Model

Version:

20B GPU: BF16

PyTorch

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for DeepSeek-Coder-V2-Lite-Instruct/20B:

max-pipelines serve --huggingface-repo-id=deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

1. Introduction

DeepSeek-Coder-V2 is an open-source Mixture-of-Experts (MoE) code language model offering performance similar to GPT4-Turbo in code-specific tasks. Enhanced with 6 trillion additional training tokens, it surpasses its predecessor, DeepSeek-Coder-33B, in coding and mathematical reasoning tasks. The model supports 338 programming languages and extends context length from 16K to 128K, excelling in coding and math benchmarks against closed-source models (Fig. 1).

DeepSeek-Coder-V2 Performance

2. Model Downloads

DeepSeek-Coder-V2 is available in variants with 16B and 236B parameters. Based on the DeepSeekMoE framework, models have active parameters of 2.4B and 21B. Below is a summary of model versions:

Model	#Total Params	#Active Params	Context Length	Download
DeepSeek-Coder-V2-Lite-Base	16B	2.4B	128k	[Download Link]
DeepSeek-Coder-V2-Lite-Instruct	16B	2.4B	128k	[Download Link]
DeepSeek-Coder-V2-Base	236B	21B	128k	[Download Link]
DeepSeek-Coder-V2-Instruct	236B	21B	128k	[Download Link]

3. Chat Website

Chat with DeepSeek-Coder-V2 at DeepSeek's official site: coder.deepseek.com.

4. API Platform

DeepSeek offers an OpenAI-compatible API at platform.deepseek.com, with competitive pricing.

API Pricing

5. How to run locally

Inference with vLLM (recommended)

To use vLLM for model inference, integrate relevant updates into your codebase. Setup and instructions are dependent on system capabilities.

6. License

The code repository is licensed under the MIT License. Usage of models follows the Model License, permitting commercial use.

7. Contact

For questions, please raise an issue or email service@deepseek.com.

Citations

For the framework referenced in this summary: DeepSeekMoE publication.

Metadata

architectures.0	DeepseekV2ForCausalLM
auto_map.AutoConfig	configuration_deepseek.DeepseekV2Config
auto_map.AutoModel	modeling_deepseek.DeepseekV2Model
auto_map.AutoModelForCausalLM	modeling_deepseek.DeepseekV2ForCausalLM
model_type	deepseek_v2

Version: 20B GPU BF16

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy DeepSeek-Coder-V2-Lite-Instruct-20B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running DeepSeek-Coder-V2-Lite-Instruct-20B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

LICENSE: deepseek-license

DETAILS

CodeMODEL CLASS

PyTorch

HARDWARE

GPU

QUANTIZATION

BF16

ARCHITECTURE

PyTorch

MAX GITHUB

Modular / MAX

MODEL

deepseek-ai

deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running DeepSeek-Coder-V2-Lite-Instruct-20B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

DeepSeek-Coder-V2-Lite-Instruct-20B

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

1. Introduction

2. Model Downloads

3. Chat Website

4. API Platform

5. How to run locally

Inference with vLLM (recommended)

6. License

7. Contact

Citations

Metadata

Learn more

Resources & support for running DeepSeek-Coder-V2-Lite-Instruct-20B

Resources & support for
running DeepSeek-Coder-V2-Lite-Instruct-20B