Guides

Each guide covers one task. Read the guide that matches your task.

Platforms

• Running on Slurm: Singularity images, bind mounts, modules, and node selection

• Running on Lepton: Endpoints, job configuration, secrets, and platform differences

Running work

• Submitting jobs: Input formats, job classes, and common flags

• Checkpointing and resuming: Resume an interrupted run

• Sharding and concurrency: Parallelism and resume behavior

• Choosing a data backend: pandas, polars, and Ray

Operating

• Managing swarm state: The state database, db commands, and swarm inspection

• Monitoring and troubleshooting: Status output and unhealthy replicas

• Metrics and dashboards: Prometheus, GPU exporters, and domyn-swarm monitor

• Swarm pools: Current limitations of the incomplete pool feature

Extending

• Implementing a backend: Protocols, configuration, and platform-specific features

For why things are built this way rather than how to use them, see Concepts. For exhaustive flag and field lists, see Reference.