Docker Compose
Get started with the simple Docker Compose infra setup.
The SP1 Cluster is the official multi-GPU prover service implementation for generating SP1 proofs on the Succinct Prover Network. It can coordinate proof generation across tens to hundreds of GPU nodes.
This page explains how to setup the SP1 Cluster using Docker Compose and generate some basic proofs using command line tools. There are several ways to deploy SP1 Cluster, with Docker Compose being the simplest method for small to medium scale cluster deployments. For easy deployment across many machines, please refer to the Kubernetes installation guide.
Prerequisites
SP1 Cluster runs on Linux and has the following software requirements:
Hardware Requirements
The hardware requirements for running SP1 Cluster depend on the node configuration and can change over time as the prover changes or new features are implemented.
| Component | Minimum Requirements | Recommended Requirements | Notes |
|---|---|---|---|
| RAM | 64 GB | 64 GB DDR5 | DDR5 RAM is recommended for better proving performance |
| Storage | 30 GB | >30 GB | - |
| CPU | High single clock speed | High single clock speed + High core count | - High single clock speed is important for optimal VM emulation - High core count helps reduce Groth16/Plonk proving latency |
| GPU | 1-8x NVIDIA GPU with: - ≥24 GB VRAM - CUDA Compute Capability ≥8.6 | Multiple supported GPUs: - GeForce RTX 5090/4090 (Best performance) - NVIDIA L4 - NVIDIA A10G - NVIDIA A5000/A6000 | - Multiple GPUs are supported on a single machine - Each GPU can run a separate instance of the GPU node binary - If you have no GPU, consider using the single-node prover reference implementation |
See the FAQ for detailed hardware recommendations.
Note that if you are using 5090s, there is currently a driver bug which requires you to set the env var MOONGATE_DISABLE_GRIND_DEVICE=true on the GPU node to enable a workaround. Not setting this will greatly reduce proving performance.
Setup
In this section, we'll walk through the steps needed to setup a basic proving service.
1. Clone the repo
git clone https://github.com/succinctlabs/sp1-cluster.git
cd cluster/infra
2. Setup image registry
Create a personal access token with the read:packages scope to access the private images.
docker login ghcr.io -u $GITHUB_USERNAME -p $GITHUB_TOKEN
3. Run the cluster
You can specify gpu{0..n} to run multiple GPU workers and a CPU worker, or specify mixed if you don't have any GPUs.
# Run only 1 GPU + 1 CPU worker
docker compose up -d gpu0
# Run 8 GPUs + 1 CPU worker
docker compose up -d gpu{0..7}
# Run only 1 CPU worker (no GPUs)
docker compose up -d mixed
# Run from source
docker compose -f docker-compose.local.yml up --build -d gpu0
View the logs with:
docker logs -f infra-gpu0-1
4. Submit a test 20m cycle Fibonacci proof
docker run --rm -it --network=infra_default ghcr.io/succinctlabs/sp1-cluster:base-latest \
/cli bench fibonacci 20 \
--cluster-rpc http://api:50051 \
--redis-nodes redis://:redispassword@redis:6379/0
When you run this command, it'll request a simple Fibonacci proof to the cluster and wait for the response:
INFO crates/common/src/logger.rs:111: logging initialized
INFO bin/cli/src/commands/bench.rs:68: Running Fibonacci Compressed for 5 million cycles...
INFO crates/common/src/client.rs:22: connecting to http://127.0.0.1:50051
INFO bin/cli/src/commands/bench.rs:113: using redis artifact store
INFO crates/artifact/src/redis.rs:38: initializing redis pool
INFO serialize: crates/artifact/src/lib.rs:126: close time.busy=62.9µs time.idle=1.55ms
INFO upload: crates/artifact/src/redis.rs:196: close time.busy=2.20ms time.idle=2.15ms artifact_type=Program id="artifact_01jxzhe54bf28v7qk3qr9yedqd"
INFO upload: crates/artifact/src/redis.rs:196: close time.busy=734µs time.idle=442µs artifact_type=Stdin id="artifact_01jxzhe54hfg1bhewm129hw58j"
INFO bin/cli/src/commands/bench.rs:146: proof_id: cli_1750183908498
INFO bin/cli/src/commands/bench.rs:185: Proof request completed after 6.531405385s
INFO bin/cli/src/commands/bench.rs:187: Aggregate MHz: 0.77
Congratulations! You've now successfully generated a SP1 proof end-to-end across your cluster of GPUs.
Prover Network Integration
This section walks through the steps necessary to integrate with the Succinct Prover Network enabling you to fulfill proofs on behalf of requesters and earn fees.
0. Onchain Registration
You should have already registered your prover onchain. If you haven't, please refer to the Introduction for more information.
You also should have added a signer address to your prover so that you can use the private key to sign transactions on behalf of the prover, which we'll use in the next step.
1. Run the fulfiller
export RPC_GRPC_ADDR=https://rpc.sepolia.succinct.xyz
export SP1_PRIVATE_KEY=<key>
docker compose up -d fulfiller
2. Run the bidder
export RPC_GRPC_ADDR=https://rpc.sepolia.succinct.xyz
export SP1_PRIVATE_KEY=<key>
docker compose up -d bidder
Multi Machine Operation
To operate the cluster using Docker Compose across more than one machine, you simply need to ensure that the nodes can all connect to the coordinator and Redis instances over the internal network. Only 1x coordinator/API/bidder/fulfiller is needed for the whole cluster.