Microsoft has unveiled its newest lightweight AI model, the Phi-3 Mini, designed to run on smartphones and other native devices. research case. Trained on 3.8 billion parameters, it is the first of three small Phi-3 language models the company will release in the near future. The goal is to provide a cheaper alternative to cloud-based LLMs, bringing AI to smaller organizations.
According to Microsoft, the new model easily surpasses the previous smaller Phi-2 model and is on par with larger models. Llama 2. In fact, the company says the Phi-3 Mini delivers responses close to the level of a model 10 times its size.
“The innovation is entirely in our dataset for training,” according to the research paper. Based on that database Phi-2 model, but uses “heavily filtered web data and synthetic data,” the team reports. In fact, a separate LLM was used to perform both of these tasks, effectively creating new data that allowed the smaller language model to be more efficient. The team was allegedly inspired by children’s books that use simpler language to get across complex topics. The Verge.
Although it still can’t deliver the results of cloud-powered LLMs, Phi-3 Mini can outperform Phi-2 and other small language models (Mistral, Gemma, Llama-3-In) in tasks ranging from math to programming to academic tests. . At the same time, it works on devices as simple as smartphones without requiring an internet connection.
Its main limitation is the breadth of “factual knowledge” due to its smaller database size – so it doesn’t perform well on the “TriviaQA” test. However, it should only be good for models that require small internal data sets. Microsoft hopes this could enable companies that can’t afford cloud-based LLMs to switch to AI.
Phi-3 Mini is now available in Azure, Hugging Face and Ollama. Microsoft is next set to release the Phi-3 Small and Phi-3 Medium with significantly higher capabilities (7 billion and 14 billion settings, respectively).
This article contains affiliate links; we may earn a commission if you click on such a link and make a purchase.
