falcon-7b

PyTorch

2 versions

A large language model built by the Technology Innovation Institute (TII) for use in summarization, text generation, and chat bots.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for falcon/7b:

    max-serve serve --huggingface-repo-id tiiuae/falcon-7b-instruct

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "falcon/7b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Technology Innovation Institute Logo

Falcon is a family of advanced large language models developed by the Technology Innovation Institute (TII), a research center under Abu Dhabi’s Advanced Technology Research Council. These models deliver high performance and are designed for a range of applications.

Parameter Counts

Parameter Count Recommended Memory
7 billion 8GB View
40 billion 32GB View
180 billion 192GB View

Variations

chat Fine-tuned for conversational datasets.
instruct Optimized for instructions using the baize dataset.
text Base models best suited for text completion.

Falcon 180B

The Falcon 180B model, featuring 180 billion parameters, is the most advanced openly available LLM as of September 2023. It ranks between GPT-3.5 and GPT-4 in performance. Running it requires a system with at least 192GB of memory. Falcon 180B has licensing restrictions that limit certain commercial uses.

More Information

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

tiiuae

MODEL

tiiuae/falcon-7b-instruct

TAGS

arxiv:1911.02150
arxiv:2005.14165
arxiv:2104.09864
arxiv:2205.14135
arxiv:2306.01116
autotrain_compatible
conversational
coreml
custom_code
dataset:tiiuae/falcon-refinedweb
en
endpoints_compatible
falcon
license:apache-2.0
pytorch
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024