mirror of
https://github.com/xai-org/grok-1.git
synced 2026-10-22 01:51:20 +03:00
Merge 1f4350143d into 7050ed204b
This commit is contained in:
1 file changed
+1
-1
@@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp
|
||||
|
||||
Grok-1 is currently designed with the following specifications:
|
||||
|
||||
- **Parameters:** 314B
|
||||
- **Parameters:** 314B (86B active)
|
||||
- **Architecture:** Mixture of 8 Experts (MoE)
|
||||
- **Experts Utilization:** 2 experts used per token
|
||||
- **Layers:** 64
|
||||
|
||||
Reference in new issue
Block a user