mirror of
https://github.com/xai-org/grok-1.git
synced 2024-11-26 21:49:53 +03:00
Compare commits
2 Commits
1094ba77eb
...
72d3a0dd94
Author | SHA1 | Date | |
---|---|---|---|
|
72d3a0dd94 | ||
|
1f4350143d |
@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp
|
||||
|
||||
Grok-1 is currently designed with the following specifications:
|
||||
|
||||
- **Parameters:** 314B
|
||||
- **Parameters:** 314B (86B active)
|
||||
- **Architecture:** Mixture of 8 Experts (MoE)
|
||||
- **Experts Utilization:** 2 experts used per token
|
||||
- **Layers:** 64
|
||||
|
Loading…
Reference in New Issue
Block a user