Merge 1f4350143d into 7050ed204b

Update README.md to specify the active parameters
This is important because users may think they need to have the full 314B at one time, but only 86B which is much more manageable!
2025-09-20 18:29:21 +03:00 · 2024-03-19 08:53:59 -07:00 · 2024-03-18 10:41:38 -07:00
1 changed files with 1 additions and 1 deletions
--- a/README.md
+++ b/README.md
@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp

 Grok-1 is currently designed with the following specifications:

- **Parameters:** 314B
+- **Parameters:** 314B (86B active)
 - **Architecture:** Mixture of 8 Experts (MoE)
 - **Experts Utilization:** 2 experts used per token
 - **Layers:** 64
Author	SHA1	Message	Date
Andrew Kean Gao	1094ba77eb	Merge `1f4350143d` into `7050ed204b`	2024-03-19 08:53:59 -07:00
Andrew Kean Gao	1f4350143d	Update README.md to specify the active parameters This is important because users may think they need to have the full 314B at one time, but only 86B which is much more manageable!	2024-03-18 10:41:38 -07:00