deepseek-v2-16b

PyTorch

1 versions

A strong, economical, and efficient Mixture-of-Experts language model.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for deepseek-v2/16b:

    max-serve serve --huggingface-repo-id deepseek-ai/DeepSeek-Coder-V2-Instruct

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "deepseek-v2/16b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. šŸŽ‰ Hooray! Youā€™re running Generative AI. Our goal is to make this as easy as possible.

About

DeepSeek-V2 is a powerful Mixture-of-Experts (MoE) language model designed to combine economical training processes with efficient inference capabilities. It has been developed to operate bilingually in both English and Chinese, catering to diverse linguistic applications.

The model is available in two scalable configurations to accommodate various computational needs:

  • 16B Lite: A lightweight version designed for cost-effective and efficient deployment.
  • 236B: A larger, more sophisticated version for demanding natural language processing tasks requiring higher performance.

DeepSeek-V2ā€™s architecture leverages the MoE approach to optimize resource utilization while maintaining strong language modeling abilities, making it a cutting-edge choice for applications requiring bilingual proficiency and scalable efficiency.

References

GitHub

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

deepseek-ai

MODEL

deepseek-ai/DeepSeek-Coder-V2-Instruct

TAGS

arxiv:2401.06066
autotrain_compatible
base_model:deepseek-ai/DeepSeek-Coder-V2-Base
base_model:finetune:deepseek-ai/DeepSeek-Coder-V2-Base
conversational
custom_code
deepseek_v2
endpoints_compatible
license:other
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024