Version:

15B GPU: BF16

PyTorch

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for starcoder2-instruct-v0.1/15B:

max-pipelines serve --huggingface-repo-id=bigcode/starcoder2-15b-instruct-v0.1

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "bigcode/starcoder2-15b-instruct-v0.1",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

StarCoder2-Instruct

Model Summary

StarCoder2-15B-Instruct-v0.1 is an innovative self-aligned code Large Language Model (LLM) leveraging a transparent pipeline for code generation. Using StarCoder2-15B, it generates instruction-response pairs, optimizing the model without human annotations or proprietary LLM data.

Model: StarCoder2-15B-Instruct-v0.1
Code: StarCoder2-self-align project
Dataset: Self-oss-instruct-sc2-exec-filter-50k
Authors: Yuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding, Naman Jain, Harm de Vries, Leandro von Werra, Arjun Guha, Lingming Zhang.

Use

Intended Use

The model primarily responds to coding-related, single-turn instructions, particularly in Python, which may limit versatility across different programming languages and complex queries.

Bias, Risks, and Limitations

StarCoder2-15B-Instruct-v0.1 is optimized for Python, which may introduce biases and output variability, particularly with different languages and instruction styles.

Evaluation

The model has been evaluated using benchmarks like EvalPlus, LiveCodeBench, and DS-1000, demonstrating its capabilities in code generation tasks.

Training Details

Hyperparameters

Optimizer: Adafactor
Learning rate: 1e-5
Epochs: 4
Batch size: 64
Warmup ratio: 0.05
Scheduler: Linear
Sequence length: 1280
Dropout: Not applied

Hardware

Trained on 1x NVIDIA A100 80GB.

Resources

Full Data Pipeline

Original seed dataset: Filtered from The Stack v1.
Bad docstring removal using StarCoder2-15B as a judge.
Concept extraction from seeds.
Instruction generation from concepts.
Unfiltered response generation.
Execution-based response filtering.
Final dataset filtering through deduplication.

Citation

@article{wei2024selfcodealign,
  title={SelfCodeAlign: Self-Alignment for Code Generation},
  author={Yuxiang Wei and Federico Cassano and Jiawei Liu and Yifeng Ding and Naman Jain and Zachary Mueller and Harm de Vries and Leandro von Werra and Arjun Guha and Lingming Zhang},
  year={2024},
  journal={arXiv preprint arXiv:2410.24198}
}

Metadata

architectures.0	Starcoder2ForCausalLM
model_type	starcoder2

Version: 15B GPU BF16

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy starcoder2-instruct-v0.1-15B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running starcoder2-instruct-v0.1-15B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id bigcode/starcoder2-15b-instruct-v0.1

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

The model is licensed under the BigCode OpenRAIL-M v1 license agreement. You can find the full agreement here.

DETAILS

CodeMODEL CLASS

PyTorch

HARDWARE

GPU

QUANTIZATION

BF16

ARCHITECTURE

PyTorch

MAX GITHUB

Modular / MAX

MODEL

bigcode

bigcode/starcoder2-15b-instruct-v0.1

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running starcoder2-instruct-v0.1-15B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

starcoder2-instruct-v0.1-15B

StarCoder2-Instruct

Model Summary

Use

Intended Use

Bias, Risks, and Limitations

Evaluation

Training Details

Hyperparameters

Hardware

Resources

Full Data Pipeline

Citation

Metadata

Learn more

Resources & support for running starcoder2-instruct-v0.1-15B

Resources & support for
running starcoder2-instruct-v0.1-15B