qwen2-0.5b

PyTorch

3 versions

Qwen2 is a new series of large language models from Alibaba group

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for qwen2/0.5b:

    max-serve serve --huggingface-repo-id Qwen/Qwen2-0.5B

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "qwen2/0.5b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Qwen2 is a state-of-the-art multilingual AI model trained on data in 29 languages, including English and Chinese. It is available in four parameter sizes—0.5B, 1.5B, 7B, and 72B—with the 7B and 72B versions supporting extended context lengths of up to 128k tokens.

Models Qwen2-0.5B Qwen2-1.5B Qwen2-7B Qwen2-72B
Params 0.49B 1.54B 7.07B 72.71B
Non-Emb Params 0.35B 1.31B 5.98B 70.21B
GQA True True True True
Tie Embedding True True False False
Context Length 32K 32K 128K 128K

Supported Languages

In addition to English and Chinese, Qwen2 supports:

Regions Languages
Western Europe German, French, Spanish, Portuguese, Italian, Dutch
Eastern & Central Europe Russian, Czech, Polish
Middle East Arabic, Persian, Hebrew, Turkish
Eastern Asia Japanese, Korean
South-Eastern Asia Vietnamese, Thai, Indonesian, Malay, Lao, Burmese, Cebuano, Khmer, Tagalog
Southern Asia Hindi, Bengali, Urdu

Performance

image.png

image.png

image.png

image.png

License

Most models are Apache 2.0 licensed, except for Qwen2 72B (instruct and base), which uses the Qianwen License.

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

Qwen

MODEL

Qwen/Qwen2-0.5B

TAGS

autotrain_compatible
conversational
en
endpoints_compatible
license:apache-2.0
pretrained
qwen2
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024