Llama 3.1 70B
A 70B-parameter dense large language model released by Meta, belonging to the Llama 3.1 series.
Core Facts
Timeline (last 90 days)
The 70B model achieved state-of-the-art (SOTA) performance on all datasets, surpassing prior prompt-based methods.
The study evaluated the token-probability scoring method using Llama 3.1 8B and 70B models on three datasets: TOEFL9, Write and Improve 2024, and BULATS-L1.
Llama 3.1 70B uses 8 KV heads instead of 64 attention heads, reducing the KV cache size to about one-eighth of the original.
Hugging Face calculates that the KV cache under BF16 is about 250 KB per token, and a full 1-million-token context requires about 266 GB.
Dense models such as Llama 3.1 70B must pass every token through all weights, so they cannot run efficiently beyond memory when memory is insufficient.
Models with the same name (such as Llama 3.1 70B) hosted by different service providers may differ in actual inference quality, context window size, and rate limits.
The 78B parameter scale is in a similar range as Meta's Llama 3.1 70B and Mistral's Mixtral series
All Facts (8)
Hugging Face calculates that the KV cache under BF16 is about 250 KB per token, and a full 1-million-token context requires about 266 GB.
65%UnverifiedLlama 3.3 70B is essentially a fine-tuned version of the earlier Llama 3.1 70B, trained using supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF).
60%UnverifiedThe study evaluated the token-probability scoring method using Llama 3.1 8B and 70B models on three datasets: TOEFL9, Write and Improve 2024, and BULATS-L1.
50%UnverifiedThe 70B model achieved state-of-the-art (SOTA) performance on all datasets, surpassing prior prompt-based methods.
50%UnverifiedLlama 3.1 70B uses 8 KV heads instead of 64 attention heads, reducing the KV cache size to about one-eighth of the original.
50%UnverifiedDense models such as Llama 3.1 70B must pass every token through all weights, so they cannot run efficiently beyond memory when memory is insufficient.
50%UnverifiedModels with the same name (such as Llama 3.1 70B) hosted by different service providers may differ in actual inference quality, context window size, and rate limits.
50%UnverifiedThe 78B parameter scale is in a similar range as Meta's Llama 3.1 70B and Mistral's Mixtral series
50%