Choose Version

(2 versions)

PyTorch

F32

GPU

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for c4ai-command-r-v01/4B:

max-pipelines serve --huggingface-repo-id=CohereForAI/c4ai-command-r-v01-4bit

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "CohereForAI/c4ai-command-r-v01-4bit",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

C4AI Command-R

Model Summary

C4AI Command-R is a cutting-edge 35 billion parameter generative model designed for various applications, including reasoning, summarization, and question answering. It supports multilingual generation in 10 languages and excels in retrieval-augmented generation capabilities.

Key Features:

Model Developer: Cohere and Cohere For AI
Model Size: 35 billion parameters
Context Length: 128K

Experimentation: The model supports both non-quantized and quantized (8-bit precision) versions using bitsandbytes, offering versatility in implementation and performance enhancements.

Model Details

Input and Output

Input: Text
Output: Text

Architecture

Command-R leverages an optimized transformer architecture with auto-regressive language modeling capabilities. After initial pretraining, supervised fine-tuning and preference training are utilized to align model behavior with human preferences for helpfulness and safety.

Multilingual Capabilities

The model performs exceptionally well in languages such as:

English
French
Spanish
Italian
German
Brazilian Portuguese
Japanese
Korean
Simplified Chinese
Arabic

Grounded Generation and RAG Capabilities

Command-R features grounded generation capabilities, enabling response generation based on document snippets with citations included.

Single-Step Tool Use Capabilities ("Function Calling")

The model supports interaction with external tools like APIs, databases, or search engines through single-step tool use, enhancing its functional range.

Multi-Step Tool Use Capabilities ("Agents")

Command-R can plan and execute multi-step actions using multiple tools, allowing for advanced agent-like behaviors.

Code Interaction

The model is optimized for code interaction, with capabilities such as code requests, explanations, and rewrites.

Contact and Licensing

For more information or questions, contact info@for.ai. The model is available under a CC-BY-NC license with an acceptable use policy.

Citation

If you use Command-R in your work, please cite:

Cohere and Cohere For AI. "C4AI Command-R."

Metadata

architectures.0	CohereForCausalLM
auto_map.AutoConfig	CohereForAI/c4ai-command-r-v01--configuration_cohere.CohereConfig
auto_map.AutoModel	CohereForAI/c4ai-command-r-v01--modeling_cohere.CohereModel
auto_map.AutoModelForCausalLM	CohereForAI/c4ai-command-r-v01--modeling_cohere.CohereForCausalLM
model_type	cohere
quantization_config.load_in_4bit	true
quantization_config.load_in_8bit	false
quantization_config.quant_method	bitsandbytes

Version: 4B GPU F32

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy c4ai-command-r-v01-4B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running c4ai-command-r-v01-4B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id CohereForAI/c4ai-command-r-v01-4bit

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

Point of Contact: Cohere For AI: cohere.for.ai
License: CC-BY-NC, requires also adhering to C4AI's Acceptable Use Policy
Model: c4ai-command-r-v01
Model Size: 35 billion parameters
Context length: 128K

DETAILS

CodeMODEL CLASS

PyTorch

HARDWARE

GPU

QUANTIZATION

F32

ARCHITECTURE

PyTorch

MAX GITHUB

Modular / MAX

MODEL

CohereForAI

CohereForAI/c4ai-command-r-v01-4bit

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running c4ai-command-r-v01-4B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

c4ai-command-r-v01-4B

C4AI Command-R

Model Summary

Model Details

Input and Output

Architecture

Multilingual Capabilities

Grounded Generation and RAG Capabilities

Single-Step Tool Use Capabilities ("Function Calling")

Multi-Step Tool Use Capabilities ("Agents")

Code Interaction

Contact and Licensing

Citation

Metadata

Learn more

Resources & support for running c4ai-command-r-v01-4B

Resources & support for
running c4ai-command-r-v01-4B