AMajor/grok-1
1
0
Fork 0
mirror of https://github.com/xai-org/grok-1.git synced 2025-11-21 19:01:41 +03:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: AMajor:1094ba77eb5b498d8f46386d16731afd9e7d3be1
Branches Tags
AMajor:main AMajor:download-instruction
...
compare: AMajor:72d3a0dd94bb9ab84c529514a085d61aca1e9886
Branches Tags
AMajor:main AMajor:download-instruction

2 Commits
1094ba77eb ... 72d3a0dd94

Author SHA1 Message Date
Andrew Kean Gao
72d3a0dd94 Merge 1f4350143d into d6d9447e2d 2024-03-19 00:13:46 +05:30
Andrew Kean Gao
1f4350143d Update README.md to specify the active parameters 
This is important because users may think they need to have the full 314B at one time, but only 86B which is much more manageable!
 2024-03-18 10:41:38 -07:00
1 changed files with 1 additions and 1 deletions
Expand all files Collapse all files

2
README.md
View File

@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp
Grok-1 is currently designed with the following specifications: Grok-1 is currently designed with the following specifications:
- **Parameters:** 314B - **Parameters:** 314B (86B active)
- **Architecture:** Mixture of 8 Experts (MoE) - **Architecture:** Mixture of 8 Experts (MoE)
- **Experts Utilization:** 2 experts used per token - **Experts Utilization:** 2 experts used per token
- **Layers:** 64 - **Layers:** 64