AMajor/grok-1
1
0
Fork 0
mirror of https://github.com/xai-org/grok-1.git synced 2024-11-26 13:39:52 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update README.md to specify the active parameters

Browse Source 
This is important because users may think they need to have the full 314B at one time, but only 86B which is much more manageable!
This commit is contained in:
Andrew Kean Gao 2024-03-18 10:41:38 -07:00 committed by GitHub
parent 310e19eee2
commit 1f4350143d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp
Grok-1 is currently designed with the following specifications: Grok-1 is currently designed with the following specifications:
- **Parameters:** 314B - **Parameters:** 314B (86B active)
- **Architecture:** Mixture of 8 Experts (MoE) - **Architecture:** Mixture of 8 Experts (MoE)
- **Experts Utilization:** 2 experts used per token - **Experts Utilization:** 2 experts used per token
- **Layers:** 64 - **Layers:** 64