phi3-3.8b

PyTorch

2 versions

Phi-3 is a family of lightweight 3B (Mini) and 14B (Medium) state-of-the-art open models by Microsoft.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for phi3/3.8b:

    max-serve serve --huggingface-repo-id microsoft/Phi-3-mini-4k-instruct-gguf

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "phi3/3.8b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. πŸŽ‰ Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Phi-3 is a family of open AI models developed by Microsoft.

Parameter sizes

Context window sizes

  • 4k
  • 128k

image.png

Phi-3 Mini

Phi-3 Mini is a lightweight 3.8B parameter model offering strong performance in reasoning and language tasks. It was trained with a mix of high-quality public and synthetic data and fine-tuned for alignment with human preferences and safety.

Phi-3 Medium

Phi-3 Medium is a 14B parameter model outperforming similar models in reasoning, math, and long-context tasks.

image.png

Intended Uses

These models are suitable for commercial and research applications, optimized for constrained environments, reasoning tasks, and long-context scenarios.

Responsible AI Considerations

While tuned for safety, Phi-3 models can exhibit biases, inaccuracies, or inappropriate outputs and should be carefully evaluated in high-risk or sensitive use cases.

Training Information

Phi-3 models were trained between February and April 2024 on 3.3 trillion tokens using a dense transformer architecture with supervised fine-tuning and preference optimization.

Resources

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

microsoft

MODEL

microsoft/Phi-3-mini-4k-instruct-gguf

TAGS

code
en
endpoints_compatible
gguf
license:mit
nlp
region:us
text-generation

@ Copyright - Modular Inc - 2024