2 versions
A large language model built by the Technology Innovation Institute (TII) for use in summarization, text generation, and chat bots.
Install our magic
package manager:
curl -ssL https://magic.modular.com/ | bash
Then run the source
command that's printed in your terminal.
Install Max Pipelines in order to run this model.
magic global install max-pipelines
Start a local endpoint for falcon/7b:
max-serve serve --huggingface-repo-id tiiuae/falcon-7b-instruct
The endpoint is ready when you see the URI printed in your terminal:
Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Now open another terminal to send a request using curl
:
curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "falcon/7b",
"stream": true,
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the World Series in 2020?"}
]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n/
/g'
🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.
Falcon is a family of advanced large language models developed by the Technology Innovation Institute (TII), a research center under Abu Dhabi’s Advanced Technology Research Council. These models deliver high performance and are designed for a range of applications.
Parameter Count | Recommended Memory | ||
---|---|---|---|
7 billion | 8GB | View | |
40 billion | 32GB | View | |
180 billion | 192GB | View |
chat |
Fine-tuned for conversational datasets. |
instruct |
Optimized for instructions using the baize dataset. |
text |
Base models best suited for text completion. |
The Falcon 180B model, featuring 180 billion parameters, is the most advanced openly available LLM as of September 2023. It ranks between GPT-3.5 and GPT-4 in performance. Running it requires a system with at least 192GB of memory. Falcon 180B has licensing restrictions that limit certain commercial uses.
DETAILS
MODULAR GITHUB
ModularCREATED BY
tiiuae
MODEL
tiiuae/falcon-7b-instruct
TAGS
@ Copyright - Modular Inc - 2024