phi-2.7b

PyTorch

1 versions

Phi-2: a 2.7B language model by Microsoft Research that demonstrates outstanding reasoning and language understanding capabilities.

Run this model

  1. Install our magic package manager:

    curl -ssL https://magic.modular.com/ | bash

    Then run the source command that's printed in your terminal.

  2. Install Max Pipelines in order to run this model.

    magic global install max-pipelines
  3. Start a local endpoint for phi/2.7b:

    max-serve serve --huggingface-repo-id microsoft/phi-2

    The endpoint is ready when you see the URI printed in your terminal:

    Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
  4. Now open another terminal to send a request using curl:

    curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "phi/2.7b",
        "stream": true,
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the World Series in 2020?"}
        ]
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
    ' | sed 's/\n/
    /g'
  5. 🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Phi-2 is a compact yet powerful language model designed for common-sense reasoning and language understanding. Despite having fewer than 13 billion parameters, it achieves state-of-the-art performance within its size class.

Example prompt

Phi-2 supports multi-turn conversational tasks seamlessly with its chat prompt capabilities:

>>> Hello, can you help me find my way to Toronto?
Certainly! What is the exact location in Toronto that you are looking for?

>>> Yonge & Bloor
Sure, Yonge and Bloor is a busy intersection in downtown Toronto. Would you like to take public transportation or drive there?

>>> Public transportation
Great! The easiest way to get there is by taking the TTC subway. You can take Line 1, which runs along Yonge Street and passes through downtown Toronto.

Example prompts (raw mode)

Phi-2 effectively handles diverse prompt formats without default templates using raw mode:

Instruct

Instruct: Write a detailed analogy between mathematics and a lighthouse.
Output:

Code Completion

def print_prime(n):

Text completion

There once was a mouse named

References

HuggingFace

Blog Post

DETAILS

MODEL CLASS
PyTorch

MODULAR GITHUB

Modular

CREATED BY

microsoft

MODEL

microsoft/phi-2

TAGS

autotrain_compatible
code
en
endpoints_compatible
license:mit
nlp
phi
region:us
safetensors
text-generation
text-generation-inference
transformers

@ Copyright - Modular Inc - 2024