Version:

7B GPU:

PyTorch

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for llava-v1.5/7B:

max-pipelines serve --huggingface-repo-id=liuhaotian/llava-v1.5-7b

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "liuhaotian/llava-v1.5-7b",
    "stream": true,
    "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "What is in this image?"
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "http://images-assets.nasa.gov/image/iss072e571418/iss072e571418~orig.jpg"
              }
            }
          ]
        }
    ]

}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

LLaVA

Model Details

Model Type:
LLaVA is a chatbot developed by fine-tuning LLaMA/Vicuna on GPT-generated multimodal instruction data. It functions as an auto-regressive language model utilizing the transformer architecture.

Model Date:
Training for LLaVA-v1.5-7B was completed in September 2023.

License

Questions or Comments:
For inquiries regarding the model, please visit the GitHub issue page.

Intended Use

Primary Intended Uses:
This model is primarily intended for research purposes focusing on large multimodal models and the development of chatbots.

Primary Intended Users:
The model is designed for researchers and hobbyists specializing in computer vision, natural language processing, machine learning, and artificial intelligence.

Training Dataset

The training dataset consists of the following components:

558K filtered image-text pairs from LAION/CC/SBU, captioned by BLIP.
158K GPT-generated multimodal instruction data.
450K academic-task-oriented Visual Question Answering (VQA) data mixture.
40K ShareGPT data.

Evaluation Dataset

LLaVA was evaluated on a set of 12 benchmarks, comprising 5 academic VQA benchmarks and 7 new benchmarks created specifically for the evaluation of instruction-following large multimodal models (LMMs).

Citations

LLaVA Website

Metadata

architectures.0	LlavaLlamaForCausalLM
model_type	llava

Version: 7B GPU undefined

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy llava-v1.5-7B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running llava-v1.5-7B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id liuhaotian/llava-v1.5-7b

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

DETAILS

VisionMODEL CLASS

PyTorch

HARDWARE

GPU

QUANTIZATION

ARCHITECTURE

PyTorch

MAX GITHUB

Modular / MAX

MODEL

liuhaotian

liuhaotian/llava-v1.5-7b

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running llava-v1.5-7B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

llava-v1.5-7B

LLaVA

Model Details

License

Intended Use

Training Dataset

Evaluation Dataset

Citations

Metadata

Learn more

Resources & support for running llava-v1.5-7B

Resources & support for
running llava-v1.5-7B