Version:

5B GPU: BF16

PyTorch

This version is not quantized and a GPU is recommended.

Install our magic package manager:
```
curl -ssL https://magic.modular.com/ | bash
```
Then run the source command that's printed in your terminal.

Install Max Pipelines in order to run this model.

magic global install max-pipelines && magic global update

Start a local endpoint for Phi-3.5-vision-instruct/5B:

max-pipelines serve --huggingface-repo-id=microsoft/Phi-3.5-vision-instruct

The endpoint is ready when you see the URI printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

Now open another terminal to send a request using curl:

curl -N http://0.0.0.0:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "microsoft/Phi-3.5-vision-instruct",
    "stream": true,
    "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "What is in this image?"
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "http://images-assets.nasa.gov/image/iss072e571418/iss072e571418~orig.jpg"
              }
            }
          ]
        }
    ]

}' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '
' | sed 's/\n//g'

🎉 Hooray! You’re running Generative AI. Our goal is to make this as easy as possible.

Deploy this model to cloud

Phi 3.5 Vision Instruct

Model Summary

Phi-3.5-vision is a cutting-edge, multilingual multimodal model designed for tasks requiring advanced image and text comprehension. Developed from quality datasets, including synthetic and public web data, it features enhanced reasoning capabilities. It supports a 128K context length and benefits from fine-tuning and preference optimization for precise instructions and robust safety measures.

Intended Uses

Primary Use Cases

Phi-3.5-vision is optimized for commercial and research purposes in English, accommodating various AI systems requiring image and text inputs. Main applications include:

Computation-constrained environments.
Scenarios with latency limitations.
General image understanding.
Optical character recognition.
Chart and table analysis.
Multiple image comparison.
Summarization of multiple images or video clips.

This model aids in accelerating research on language and multimodal models, serving as a foundation for generative AI features.

Use Case Considerations

This model is not tailored for all downstream purposes. Developers must assess language model limitations and address accuracy, safety, and fairness before specific applications, especially in high-risk scenarios. Adherence to applicable laws or regulations (including privacy and trade laws) is crucial.

Release Notes

Phi-3.5-vision enhances multi-frame image understanding, based on customer feedback. Features include detailed image comparisons, multi-image summarization, and video summarization, relevant for Office applications. Performance improvements are observed in single image benchmarks, with increased MMMU, MMBench, and document understanding scores. Broad use cases could benefit from this release, encouraging community feedback.

Responsible AI Considerations

The Phi model family, like others, can exhibit biases, inaccuracies, or offensive content. Developers must ensure responsible AI practices and legal compliance. Key considerations include:

Quality of Service: Performance may vary with non-English languages or English varieties underrepresented in training data.
Representation and Stereotypes: Potential for over- or under-representation, leading to reinforced stereotypes.
Content Offensiveness: Inappropriateness in certain contexts necessitating additional mitigations.
Information Reliability: Potential for nonsensical or outdated content requiring accuracy checks.
Code Scope: Recommendations for verification of Python scripts generated with non-standard packages.

Training

Phi-3.5-vision features a 4.2B parameter architecture with components like an image encoder and a Phi-3 Mini language model. It handles text and image inputs and supports up to 128K tokens. Training utilized 256 A100-80G GPUs over six days, using 500B vision and text tokens. Trained between July and August 2024, it was released in August 2024.

Benchmarks

The model excels across image-related benchmarks, outpacing several competitors in multi-image and video data processing.

Safety Evaluation and Red-Teaming

Safety alignment involves SFT and RLHF approaches with evaluations like red teaming and adversarial simulations. Detailed techniques are outlined in the technical report.

Software & Hardware

Optimized using PyTorch, Transformers, and Flash-Attention libraries, the model is tested on GPUs like NVIDIA A100, A6000, and H100.

License

The model is released under the MIT License.

Citations

Metadata

architectures.0	Phi3VForCausalLM
auto_map.AutoConfig	configuration_phi3_v.Phi3VConfig
auto_map.AutoModelForCausalLM	modeling_phi3_v.Phi3VForCausalLM
model_type	phi3_v

Version: 5B GPU BF16

This code works on compatible Linux machines.
We are actively working on enabling MAX Serve for MacOS ARM64 as well.

You can quickly deploy Phi-3.5-vision-instruct-5B to an endpoint using our MAX container. It includes the latest version of MAX with GPU support and our Python-based inference server called MAX Serve.

With the following Docker command, you’ll get an OpenAI-compatible endpoint running Phi-3.5-vision-instruct-5B:

docker run --gpus 1 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_HUB_ENABLE_HF_TRANSFER=1" \
    --env "HF_TOKEN=" \
    -p 8000:8000 \
    docker.modular.com/modular/max-openai-api:nightly \
    --huggingface-repo-id microsoft/Phi-3.5-vision-instruct

In order to download the model from Hugging Face, you just need to fill in the HF_TOKEN value with your access token, unless the model is from https://huggingface.co/modularai.

Learn more

For more information about the container image, see the MAX container documentation.

To learn more about how to deploy MAX to the cloud, check out our MAX Serve tutorials.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

DETAILS

VisionMODEL CLASS

PyTorch

HARDWARE

GPU

QUANTIZATION

BF16

ARCHITECTURE

PyTorch

MAX GITHUB

Modular / MAX

MODEL

microsoft

microsoft/Phi-3.5-vision-instruct

QUESTIONS ABOUT THIS MODEL?

Resources & support for
running Phi-3.5-vision-instruct-5B

Browse 27+ Tutorials

View Tutorials

Get help using MAX

Modular Forum

Read Documentation

Go to Docs

Phi-3.5-vision-instruct-5B

Phi 3.5 Vision Instruct

Model Summary

Intended Uses

Primary Use Cases

Use Case Considerations

Release Notes

Responsible AI Considerations

Training

Benchmarks

Safety Evaluation and Red-Teaming

Software & Hardware

License

Citations

Metadata

Learn more

Resources & support for running Phi-3.5-vision-instruct-5B

Resources & support for
running Phi-3.5-vision-instruct-5B