AMajor/grok-1
1
0
Fork 0
mirror of https://github.com/xai-org/grok-1.git synced 2025-09-21 02:39:22 +03:00
Code Issues Packages Projects Releases Wiki Activity

Compare commits

base: AMajor:72d3a0dd94bb9ab84c529514a085d61aca1e9886
Branches Tags
AMajor:main AMajor:download-instruction
..
compare: AMajor:main
Branches Tags
AMajor:main AMajor:download-instruction

1 Commits
72d3a0dd94 ... main

Author SHA1 Message Date
Eddy
7050ed204b Corrected name of package "cuda12-pip" (#194) 
The `cuda12-pip` package was wrongly named `cuda12_pip`
in requirements.txt
 2024-03-19 08:48:22 -07:00
2 changed files with 2 additions and 2 deletions
Expand all files Collapse all files

2
README.md
View File

@ -22,7 +22,7 @@ The implementation of the MoE layer in this repository is not efficient. The imp
Grok-1 is currently designed with the following specifications:
- **Parameters:** 314B (86B active)
- **Parameters:** 314B
- **Architecture:** Mixture of 8 Experts (MoE)
- **Experts Utilization:** 2 experts used per token
- **Layers:** 64

2
requirements.txt
View File

@ -1,4 +1,4 @@
dm_haiku==0.0.12
jax[cuda12_pip]==0.4.25 -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
jax[cuda12-pip]==0.4.25 -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
numpy==1.26.4
sentencepiece==0.2.0