granite-code-3b

MAX Model

4 versions

A family of open foundation models by IBM for Code Intelligence

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for granite-code/3b:

    max-serve serve --huggingface-repo-id ibm-granite/granite-3b-code-instruct-2k

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "granite-code/3b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

image.png

Granite Code is a family of advanced decoder-only code models specifically designed to address various code generation tasks, including automatic code creation, explanation, and bug fixing. These models are optimized for high performance across different parameter sizes and context windows, making them versatile for diverse coding needs.

Parameter Sizes

  • 34B Parameters
  • 20B Parameters
  • 8B Parameters (supports 128K context window)
  • 3B Parameters (supports 128K context window)

Granite Code provides a robust framework for coding professionals and researchers, empowering users to handle complex programming challenges efficiently.

Resources

DETAILS

MODEL CLASS
MAX Model

MAX Models are extremely optimized inference pipelines to run SOTA performance for that model on both CPU and GPU. For many of these models, they are the fastest version of this model in the world.

Browse 18+ MAX Models

MODULAR GITHUB

Modular

CREATED BY

ibm-granite

MODEL

ibm-granite/granite-3b-code-instruct-2k

TAGS

arxiv:2405.04324
autotrain_compatible
base_model:finetune:ibm-granite/granite-3b-code-base-2k
base_model:ibm-granite/granite-3b-code-base-2k
code
conversational
dataset:TIGER-Lab/MathInstruct
dataset:bigcode/commitpackft
dataset:bugdaryan/sql-create-context-instruction
dataset:garage-bAInd/Open-Platypus
dataset:glaive-function-calling-v2
dataset:glaiveai/glaive-code-assistant-v3
dataset:meta-math/MetaMathQA
dataset:nvidia/HelpSteer
granite
license:apache-2.0
llama
model-index
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024