Google’s DeepMind team has introduced Gemma, a family of open-source models available in 2B and 7B parameter sizes. These models, trained on English data, support various tasks like text summarization, generation, reasoning, and Q&A on laptops. However, developers must adhere to Gemma’s prohibited use policy despite the commercial licensing.
After Gemini 1.0 Ultra and Gemini 1.5 Pro, Google released compact open-source models called Gemma. Gemma comes in 2B and 7B parameter versions, with a commercial license for unrestricted use. Unlike Gemini, Gemma models are open-source, yet they leverage the same research and technology.
Gemma Models Can Run on Your Laptop Easily
Google highlights the compact nature of the Gemma open-source models, underlining their ease of deployment on laptops or desktops. These models are trained on English datasets, including web documents, code, and mathematics, and are optimized for tasks such as text summarization, generation, reasoning, Q&A, and more. Google specifies that the Gemma models are trained on a vast dataset containing 6 trillion tokens.
Google has extensively tested the models for safety, bias, and risks, ensuring their open-source nature. They rigorously applied a CSAM (Child Sexual Abuse Material) filter to remove any harmful content. Furthermore, they applied extensive sensitive data filtering to exclude personal information from the models.
In addition, Google provides a Responsible Generative AI Toolkit for developers to utilize the model responsibly. While the Gemma models are open-source, they are subject to a prohibited use policy that prohibits developers from engaging in “dangerous, illegal, or malicious activities,” among other restrictions.
When it comes to benchmarks, the Gemma 2B model achieved a score of 42.3 in the MMLU test, while the 7B model scored 64.3. In the HellaSwag test, the 2B model obtained a score of 71.4, and the 7B model scored 81.2. In comparison, Microsoft’s 2.7B Phi-2 model achieved a score of 56.7 in the MMLU test, while Meta’s Llama 2 (7B) scored 45.3. However, Google’s own Gemini Nano 2 (3.2B) model achieved a score of 55.8 in the same test.
Overall, I consider Google’s decision to release open-source models as a positive step toward advancing research and fostering innovation. Interested individuals can begin using Gemma models on Kaggle or explore the official PyTorch implementation of Gemma models available on GitHub. Additionally, Gemma can be accessed on Vertex AI.
I’ll be test-driving this open-source model soon to compare it with other popular models. Stay tuned for a comprehensive, hands-on test of Gemma, Mixtral, and other LLMs.