mirror of
https://github.com/xai-org/grok-1.git
synced 2025-04-03 18:00:10 +03:00
25 lines
887 B
Markdown
25 lines
887 B
Markdown
# Grok-1
|
|
|
|
This repository contains JAX example code for loading and running the Grok-1 open-weights model.
|
|
|
|
Make sure to download the checkpoint and place `ckpt-0` directory in `checkpoint`.
|
|
Then, run
|
|
|
|
```shell
|
|
pip install -r requirements.txt
|
|
python run.py
|
|
```
|
|
|
|
to test the code.
|
|
|
|
The script loads the checkpoint and samples from the model on a test input.
|
|
|
|
Due to the large size of the model (314B parameters), a machine with enough GPU memory is required to test the model with the example code.
|
|
The implementation of the MoE layer in this repository is not efficient. The implementation was chosen to avoid the need for custom kernels to validate the correctness of the model.
|
|
|
|
# License
|
|
|
|
The code and associated Grok-1 weights in this release are licensed under the
|
|
Apache 2.0 license. The license only applies to the source files in this
|
|
repository and the model weights of Grok-1.
|