Install our magic package manager:

curl -ssL https://magic.modular.com/ | bash

Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines

Start a local endpoint for bakllava/7b:

max-serve serve --huggingface-repo-id SkunkworksAI/BakLLaVA-1

The endpoint is ready when you see the URI printed in your terminal:

Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "bakllava/7b",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n/
/g'