qwq-32b

PyTorch

1 versions

QwQ is an experimental research model focused on advancing AI reasoning capabilities.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for qwq/32b:

    max-serve serve --huggingface-repo-id Qwen/QwQ-32B-Preview

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "qwq/32b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

QwQ is a 32-billion-parameter experimental research model developed by the Qwen Team to advance AI reasoning capabilities.

image.png

image.png

The model showcases remarkable performance in technical reasoning benchmarks:

  • 65.2% on GPQA, highlighting graduate-level scientific reasoning.
  • 50.0% on AIME, demonstrating strong mathematical problem-solving.
  • 90.6% on MATH-500, excelling in diverse mathematical topics.
  • 50.0% on LiveCodeBench, verifying programming capabilities in real-world applications.

These scores emphasize substantial advancements in analytical and technical problem-solving skills.

Despite its strengths, QwQ faces notable limitations:

  1. Language Mixing: Responses may include unintended language switching, reducing clarity.
  2. Recursive Reasoning Loops: Lengthy, circular reasoning patterns may emerge without a clear conclusion.
  3. Safety Concerns: Enhanced safety measures are needed to ensure secure and reliable deployment.
  4. Performance Gaps: While strong in math and coding, it underperforms in areas like common sense reasoning and nuanced language understanding.

This preview demonstrates promising capabilities while identifying areas for improvement.

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

Qwen

MODEL

Qwen/QwQ-32B-Preview

TAGS

arxiv:2407.10671
autotrain_compatible
base_model:Qwen/Qwen2.5-32B-Instruct
base_model:finetune:Qwen/Qwen2.5-32B-Instruct
chat
conversational
en
endpoints_compatible
license:apache-2.0
qwen2
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024