Phi3-3.8b Model | MAX Builds

Run this model

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.
Install Max Pipelines in order to run this model.
```
magic global install max-pipelines
```

Start a local endpoint for phi3/3.8b:

max-serve serve --huggingface-repo-id microsoft/Phi-3-mini-4k-instruct-gguf

The endpoint is ready when you see the URI printed in your terminal:

Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "phi3/3.8b",
    "stream": true,
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who won the World Series in 2020?"}
    ]
}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n/
/g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

About

Phi-3 is a family of open AI models developed by Microsoft.

Parameter sizes

Phi-3 Mini – 3B parameters
Phi-3 Medium – 14B parameters

Context window sizes

4k
128k

Phi-3 Mini

Phi-3 Mini is a lightweight 3.8B parameter model offering strong performance in reasoning and language tasks. It was trained with a mix of high-quality public and synthetic data and fine-tuned for alignment with human preferences and safety.

Phi-3 Medium

Phi-3 Medium is a 14B parameter model outperforming similar models in reasoning, math, and long-context tasks.

Intended Uses

These models are suitable for commercial and research applications, optimized for constrained environments, reasoning tasks, and long-context scenarios.

Responsible AI Considerations

While tuned for safety, Phi-3 models can exhibit biases, inaccuracies, or inappropriate outputs and should be carefully evaluated in high-risk or sensitive use cases.

Training Information

Phi-3 models were trained between February and April 2024 on 3.3 trillion tokens using a dense transformer architecture with supervised fine-tuning and preference optimization.

Resources

general.architecture	phi3
general.basename	Phi-3
general.file_type	2
general.finetune	128k-instruct
general.languages	[en]
general.license	mit
general.license.link	https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/resolve/main/LICENSE
general.name	Phi 3 Mini 128k Instruct
general.quantization_version	2
general.size_label	mini
general.tags	[nlp, code, text-generation]
general.type	model
phi3.attention.head_count	32
phi3.attention.head_count_kv	32
phi3.attention.layer_norm_rms_epsilon	1e-05
phi3.attention.sliding_window	262144
phi3.block_count	32
phi3.context_length	131072
phi3.embedding_length	3072
phi3.feed_forward_length	8192
phi3.rope.dimension_count	96
phi3.rope.freq_base	10000
phi3.rope.scaling.attn_factor	1.1902381
phi3.rope.scaling.original_context_length	4096
tokenizer.ggml.add_bos_token	false
tokenizer.ggml.add_eos_token	false
tokenizer.ggml.bos_token_id	1
tokenizer.ggml.eos_token_id	32000
tokenizer.ggml.model	llama
tokenizer.ggml.padding_token_id	32000
tokenizer.ggml.pre	default
tokenizer.ggml.scores	[-1000, -1000, -1000, 0, 0, ...]
tokenizer.ggml.token_type	[3, 3, 4, 6, 6, ...]
tokenizer.ggml.tokens	[<unk>, <s>, </s>, <0x00>, <0x01>, ...]
tokenizer.ggml.unknown_token_id	0

Microsoft. Copyright (c) Microsoft Corporation.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

phi3-3.8b