Gemma2 is Google's latest iteration of open language models (LLMs) released in 2024.
Model sizes: Gemma2 comes in two sizes - 9 billion and 27 billion parameters, with both base (pre-trained) and instruction-tuned versions available.
Training data: The models were trained on approximately 13 trillion tokens for the 27B version and 8 trillion tokens for the 9B version, using web data (primarily English), code, and math.
Performance: Gemma2 outperforms similarly sized open models on multiple benchmarks. The 27B variant offers competitive performance to models more than twice its size, nearly matching the performance of Llama 3 70B while being three times smaller.
Technical features: Gemma2 has a context length of 8192 tokens and uses Rotary Position Embedding (RoPE). It also incorporates new features like sliding window attention and multi-query attention.
Licensing: Gemma2 is released under a permissive license that allows redistribution, fine-tuning, commercial use, and derivative works.
Availability: The models are accessible through various platforms, including Google Cloud's Vertex AI, AI Studio, Hugging Face, and Ollama for local usage.
Evaluation: While benchmark results show impressive performance, real-world testing is ongoing to determine its practical capabilities compared to other models.
Gemma2 represents Google's effort to provide powerful, efficient, and open-source language models for developers and researchers to use and build upon.