LLM.rocks 🤘
Welcome
Welcome to llm.rocks.
I'm a software engineer who finds LLMs (large language models) and their potential in the (near) future like magic. I don't work with LLMs, so this is my site to learn/educate about them.
I also keep this page up to date with LLM news/articles.
Biggest LLMs
| name | release date | company | # params | size |
|---|---|---|---|---|
| BERT | 2018 | 340 million | 3.3 billion words | |
| GPT-2 | 2019 | OpenAI | 1.5 billion | 40 GB, 10 billion tokens |
| GPT-3 | 2020 | OpenAI | 175 billion | 500 billion tokens |
| GBP-Neo | 2021 | EleutherAI | 2.7 billion | 800 GB |
| GPT-J | 2021 | EleutherAI | 6 billion | 800 GB |
| Megatron-Turing NLG | 2021 | Nvidia/Microsoft | 500 billion | 300 billion tokens |
| Ernie 3.0 Titan | 2021 | Baidu | 260 billion | 4tb |
| Claude | 2021 | Anthropic | 25 billion | 400 billion tokens |
| GLaM (Generalist Language Model) | 2021 | 1.2 trillion | 1.6 trillion tokens | |
| Gopher | 2021 | DeepMind | 280 billion | 300 billion tokens |
| LaMDA (Language Models for Dialog Applications) | 2022 | 137 billion | 1.5 trillion words, 160 billion tokens | |
| GPT-NeoX | 2022 | EleutherAI | 20 billion | 800 gb |
| Chinchilla | 2022 | DeepMind | 70 billion | 1.4 trillion |
| PaLM (Pathways Language Model) | 2022 | 540 billion | 768 billion tokens | |
| OPT (Open pretrained transformer) | 2022 | Meta | 175 billion | 180 billion tokens |
| YaLM 100B | 2022 | Yandex | 100 billion | 1.7TB |
| Minerva | 2022 | 540 billion | 38.5 tokens (from math content/academic papers) | |
| BLOOM | 2022 | Hugging Face | 175 billion | 350 billion tokens/1.6TB |
| Galactica | 2022 | Meta | 120 billion | 106 billion tokens |
| AlexaTM (Teacher Models) | 2022 | Amazon | 20 billion | 1.3 trillion |
| LLaMA (large language model meta AI) | 2023 | Meta | 65 billion | 1.4 trillion |
| GPT-4 | 2023 | OpenAI | ||
| Cerebras-GPT | 2023 | Cerebras | 13 billion | |
| Falcon | 2023 | Technology Innocation Institute | 40 billion | 1 Trillion tokens (1TB) |
| BloombergGPT | 2023 | Bloomberg | 50 billion | 360 billion tokens from Bloomberg data + 345 billion tokens from general datasets |
| PanGu | 2023 | Huawei | 1 trillion | 329 billion token |