starcoder2-3b

PyTorch

3 versions

StarCoder2 is the next generation of transparently trained open code LLMs that comes in three sizes: 3B, 7B and 15B parameters.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for starcoder2/3b:

    max-serve serve --huggingface-repo-id bigcode/starcoder2-3b

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "starcoder2/3b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

StarCode 2

Supporting a context window of up to 16,384 tokens, StarCoder2 represents the next generation of transparently trained open-code Large Language Models (LLMs).

  • starcoder2-instruct is a 15B parameter model designed to effectively follow natural and human-written instructions.
  • starcoder2-15b was trained on over 600 programming languages and 4+ trillion tokens, offering state-of-the-art performance in its size category, matching 33B+ models on numerous evaluations.
  • starcoder2-7b was trained on 17 programming languages and 3.5+ trillion tokens, delivering robust capabilities at a smaller scale.
  • starcoder2-3b was trained on 17 programming languages and 3+ trillion tokens, achieving performance comparable to the original StarCoder1 15B model.

StarCoder2 is optimized for a broad range of programming and natural language processing tasks, providing high performance while maintaining transparency and accessibility.

References

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

bigcode

MODEL

bigcode/starcoder2-3b

TAGS

arxiv:2004.05150
arxiv:2205.14135
arxiv:2207.14255
arxiv:2305.13245
arxiv:2402.19173
autotrain_compatible
code
dataset:bigcode/the-stack-v2-train
endpoints_compatible
license:bigcode-openrail-m
model-index
region:us
safetensors
starcoder2
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024