1 versions
Phi-2: a 2.7B language model by Microsoft Research that demonstrates outstanding reasoning and language understanding capabilities.
Install our magic
package manager:
curl -ssL https://magic.modular.com/ | bash
Then run the source
command that's printed in your terminal.
Install Max Pipelines in order to run this model.
magic global install max-pipelines
Start a local endpoint for phi/2.7b:
max-serve serve --huggingface-repo-id microsoft/phi-2
The endpoint is ready when you see the URI printed in your terminal:
Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Now open another terminal to send a request using curl
:
curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "phi/2.7b",
"stream": true,
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the World Series in 2020?"}
]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n/
/g'
🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.
Phi-2 is a compact yet powerful language model designed for common-sense reasoning and language understanding. Despite having fewer than 13 billion parameters, it achieves state-of-the-art performance within its size class.
Phi-2 supports multi-turn conversational tasks seamlessly with its chat prompt capabilities:
>>> Hello, can you help me find my way to Toronto?
Certainly! What is the exact location in Toronto that you are looking for?
>>> Yonge & Bloor
Sure, Yonge and Bloor is a busy intersection in downtown Toronto. Would you like to take public transportation or drive there?
>>> Public transportation
Great! The easiest way to get there is by taking the TTC subway. You can take Line 1, which runs along Yonge Street and passes through downtown Toronto.
Phi-2 effectively handles diverse prompt formats without default templates using raw mode:
Instruct: Write a detailed analogy between mathematics and a lighthouse.
Output:
def print_prime(n):
There once was a mouse named
DETAILS
MODULAR GITHUB
ModularCREATED BY
microsoft
MODEL
microsoft/phi-2
TAGS
@ Copyright - Modular Inc - 2024