Version:

13B GPU:

MAX Model

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for vicuna-v1.1/13B:

max-pipelines serve --huggingface-repo-id=lmsys/vicuna-13b-delta-v1.1

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "lmsys/vicuna-13b-delta-v1.1",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

Vicuna Model Card

Model Details

Vicuna is a chat assistant model fine-tuned on user-contributed conversations from ShareGPT.

Developed by: LMSYS
Model Type: Auto-regressive language model using transformer architecture
License: Non-commercial
Finetuned From Model: LLaMA

Model Sources

Repository: GitHub Repository
Blog Post: LMSYS Blog
Research Paper: arXiv Paper
Demo: Chat Demo

Uses

Vicuna is primarily intended for research purposes related to large language models and chatbots, targeting researchers and hobbyists in NLP, machine learning, and AI.

How to Get Started with the Model

To use the model, you can refer to these resources:

Command Line Interface: CLI Instructions
APIs: API Details

Training Details

Vicuna v1.1 is derived from LLaMA using supervised instruction on a dataset containing approximately 70K conversations sourced from ShareGPT.com. More information is available in the training details section of the paper.

Evaluation

The model's performance is assessed using standard benchmarks, human preference evaluations, and LLM-as-a-judge methodologies. Further details and results can be found in the research paper.

Difference Between Different Versions of Vicuna

For details on version differences, refer to the vicuna_weights_version.md.

Acknowledgement

Thanks to @TheBloke for previously hosting the merged version of weights.

Citations

Metadata

architectures.0	LlamaForCausalLM
model_type	llama

Version: 13B GPU undefined

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy vicuna-v1.1-13B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running vicuna-v1.1-13B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id lmsys/vicuna-13b-delta-v1.1

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

Developed by: LMSYS
Model type: An auto-regressive language model based on the transformer architecture.
License: Non-commercial license
Finetuned from model: LLaMA.

DETAILS

ChatMODEL CLASS

MAX Model

MAX Models are popular open-source models converted to MAX’s native graph format. Anything with the label is either SOTA or being worked on. Learn more about MAX Models.

Browse all MAX Models

HARDWARE

GPU

QUANTIZATION

ARCHITECTURE

MAX Model

MAX GITHUB

Modular / MAX

MODEL

lmsys

lmsys/vicuna-13b-delta-v1.1

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running vicuna-v1.1-13B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

vicuna-v1.1-13B

Vicuna Model Card

Model Details

Model Sources

Uses

How to Get Started with the Model

Training Details

Evaluation

Difference Between Different Versions of Vicuna

Acknowledgement

Citations

Metadata

Learn more

Resources & support for running vicuna-v1.1-13B

Resources & support for
running vicuna-v1.1-13B