domyn-swarm#

A CLI and Python library for launching LLM serving endpoints and running high-throughput batch jobs against them, with first-class support for Slurm and NVIDIA DGX Cloud Lepton.

domyn-swarm up -c config.yaml
domyn-swarm job submit --name my-swarm --input prompts.parquet --output answers.parquet
domyn-swarm down my-swarm

Why domyn-swarm#

Serving a model at scale and running work against it are usually two unrelated chores: one person writes the launch scripts, another writes the batch loop, and the two agree by convention about where the endpoint lives. domyn-swarm makes that one workflow. A YAML file describes the swarm; up stands up replicas behind a load balancer and waits until they actually answer; job submit runs a typed job against them with batching, retries and checkpointing; down removes everything it created.

The same commands work on an HPC cluster and in the cloud. Only the config’s backend section changes.

Get started

Install, launch your first swarm, and write your first custom job.

Getting started
Guides

Checkpointing, sharding, data backends, and the platform-specific details.

Guides
Concepts

How the pieces fit together, and why they are split the way they are.

Concepts
Reference

CLI, configuration and Python API, generated from the source.

Reference

What it does#

  • One CLI across platforms — up, job submit, status, down behave the same on Slurm and Lepton

  • Endpoints that are ready when they say they are — replicas are health-probed before up returns, so a job never starts against a model that is still loading

  • Jobs that survive failure — DataFrame in, DataFrame out, with bounded concurrency, retries with backoff, and Parquet checkpointing that can resume a half-finished run

  • A script escape hatch — when a job class is the wrong shape, submit any Python file to the compute backend instead

  • Swarms you can find again — state is kept in a local SQLite database, so a swarm is addressable by name from any later command

  • Backends behind protocols — serving and compute are separate interfaces, so supporting a new platform adds code rather than changing it

Supported backends#

Serving and compute — where the model runs, and where jobs execute:

Backend

Notes

Slurm

Singularity containers and a job array for replicas, behind an Nginx load balancer

NVIDIA DGX Cloud Lepton

Endpoint plus batch job through the Lepton SDK (domyn-swarm[lepton])

Data — how job input and output are read, written and iterated. See Choosing a data backend:

Backend

Install

pandas

always available; the default

polars

domyn-swarm[polars]

ray

domyn-swarm[ray]

Where to go next#

New here? Installation then Quickstart is about fifteen minutes end to end.

Already running jobs? Checkpointing and resuming and Sharding and concurrency are the two guides that most change how a long run behaves.

Debugging something? Monitoring and troubleshooting starts from the symptom.