Configuration

Every runtime option lives in a single YAML file, loaded into DomynLLMSwarmConfig. The tables below are generated from the Pydantic models, so they always match the installed version.

A field marked required must be present. A field marked computed has a default derived at load time from other fields — see the field’s description for the formula.

Values are resolved highest priority first:

1. the CLI arguments and this YAML config

2. defaults.yaml, written by domyn-swarm init defaults

3. the built-in defaults shown below

Any field you omit inherits the default, so configs can stay minimal.

Models

DomynLLMSwarmConfig

Field	Type	Default	Description
model	str	required	Hugging Face model ID or local path. Passed verbatim to vllm serve; must resolve to a local directory or an offline Hugging Face model in HF_HOME.
name	str	required	Name of the swarm. Stripped of surrounding whitespace, lower-cased and limited to 38 characters so it fits backend resource-name limits. The swarm’s unique id is this name plus a short suffix.
revision	`str	None`	None
replicas	int	1	How many independent vLLM clusters to launch. Useful for A/B tests.
gpus_per_replica	int	4	How many GPUs each replica uses. Also sets vLLM’s --tensor-parallel-size.
gpus_per_node	int	4	Number of GPUs per node (vLLM)
replicas_per_node	`int	None`	None
nodes	`int	None`	None
cpus_per_task	`int	None`	None
mem_per_cpu	`str	None`	None
wait_endpoint_s	int	1200	Seconds the load-balancer script waits for the endpoint to come up before giving up.
image	`str	EnvPath`	computed
args	str	''	Extra CLI flags passed verbatim to python -m vllm.entrypoints.openai.api_server.
port	int	8000	Port on which each replica’s OpenAI-compatible API listens.
home_directory	EnvPath	computed	Home directory where logs and state are stored
backend	`Annotated[LeptonConfig	SlurmConfig, annotation=NoneType required=True discriminator=’type’]	None`
env	`dict[str, str]	None`	None
watchdog	WatchdogConfig	computed	Watchdog settings governing how the spawned vLLM replicas are monitored and restarted.

BackendsConfig

Field	Type	Default	Description
backends	`list[Annotated[LeptonConfig	SlurmConfig, annotation=NoneType required=True discriminator=’type’]]`	required

SlurmConfig

Configuration for SLURM-based deployments.

Field	Type	Default	Description
type	"slurm"	'slurm'	Backend discriminator; always slurm for this model.
partition	str	computed	Slurm partition to submit to.
account	str	computed	Slurm account or charge code.
qos	str	computed	Slurm QoS for the cluster and load-balancer jobs. The load balancer can override it with endpoint.qos.
requires_ray	`bool	None`	None
ray_port	int	6379	Port for Ray’s GCS / head node inside each replica.
ray_dashboard_port	int	8265	Port for the optional Ray dashboard.
modules	list[str]	computed	Environment modules to module load at the top of the generated cluster sbatch script.
preamble	list[str]	computed	Additional lines inserted near the top of the generated cluster sbatch script, before the module loads. Use for extra sbatch directives or shell setup.
template_path	EnvPath	computed	Path to the Jinja2 template for the cluster sbatch script. Auto-filled and normally left alone: a Ray deployment renders from llm_swarm_ray.sh.j2 and a single-node one from llm_swarm.sh.j2. Setting it pins the template and disables that choice.
nginx_template_path	EnvPath	computed	Path to the Jinja2 template for the Nginx config. Auto-filled; there is normally no need to set it.
time_limit	str	'36:00:00'	Overall Slurm wall-clock limit for the allocation.
exclude_nodes	`str	None`	None
node_list	`str	None`	None
mail_user	`str	None`	None
endpoint	SlurmEndpointConfig	computed	Configuration for the Nginx load-balancer job.
venv_path	`EnvPath	None`	None
env	`dict[str, str]	None`	None
mounts	list[str]	computed	Extra Singularity bind mounts for the vLLM containers. Each entry is either ‘/path’ (bound at the same path inside the container) or ‘/host/path:/container/path’ (with an optional ‘:ro’/’:rw’ suffix). Appended verbatim to the container’s bind list.

SlurmEndpointConfig

Configuration for the Nginx load-balancer job fronting the replicas.

Field	Type	Default	Description
cpus_per_task	int	32	vCPUs for the driver process that launches and monitors the swarm.
mem	str	'16GB'	Physical memory for the driver job.
threads_per_core	int	1	SMT threads to request per physical core.
wall_time	str	'24:00:00'	Slurm time limit for the driver job.
enable_proxy_buffering	bool	True	Enable Nginx response and request buffering in the generated load balancer config. Turn it off for streaming responses that should reach the client as they are produced.
nginx_timeout	`str	int`	'60s'
port	int	9000	External port exposed by the Nginx load balancer.
nginx_image	`str	EnvPath`	computed
qos	`str	None`	None
poll_interval	int	10	Seconds between sacct status checks while waiting for the load balancer to become ready.
require_allocated_node	bool	False	Refuse to build an srun command unless already inside a Slurm allocation. Guards against large data jobs accidentally running on the load-balancer node.
monitoring	MonitoringConfig	computed	Prometheus and GPU-exporter sidecars running alongside the load balancer. Off by default, and Slurm-only.

MonitoringConfig

Optional Prometheus-based monitoring sidecars for the load-balancer node. Disabled by default; when disabled the load balancer behaves exactly as it did before monitoring existed. Slurm only.

Field	Type	Default	Description
enabled	bool	False	Run Prometheus and an Nginx exporter alongside the load balancer. Master switch: with it off, every other field here is ignored.
mode	`”container”	“binary”`	'container'
prometheus_image	`str	None`	computed
nginx_exporter_image	`str	None`	computed
prometheus_binary	str	'prometheus'	Prometheus binary name or path, for mode: binary.
nginx_exporter_binary	str	'nginx-prometheus-exporter'	nginx-prometheus-exporter binary name or path, for mode: binary.
port	int	9090	Port Prometheus listens on, on the load-balancer node. Reached through the load balancer rather than directly, so this is not the port you connect to.
exporter_port	int	9113	Port the Nginx exporter serves its metrics on for Prometheus to scrape.
route_prefix	str	'/prometheus'	Path under the swarm’s endpoint where Prometheus is served. A leading slash is added if missing.
scrape_interval	str	'15s'	How often Prometheus scrapes every target.
retention	str	'12h'	How long Prometheus keeps samples. The database is node-local and dies with the load-balancer job, so this only caps a single run.
gpu_exporter	GpuExporterConfig	computed	Per-node GPU metrics exporter. Off by default.
ray_metrics	RayMetricsConfig	computed	Scraping of Ray’s own metrics. Auto-enabled for Ray deployments.

GpuExporterConfig

Optional per-node GPU metrics exporter for replica nodes. Disabled by default. kind selects both the exporter implementation and the metric vocabulary the bundled dashboard expects, so changing it changes which dashboard domyn-swarm monitor --gpu loads.

Field	Type	Default	Description
enabled	bool	False	Run a GPU exporter on every replica node. Requires monitoring.enabled; on its own it does nothing.
kind	`”nvidia_smi”	“dcgm”`	'nvidia_smi'
image	`str	None`	None
binary	`str	None`	None
port	int	9835	Port each node’s exporter listens on for Prometheus to scrape.

RayMetricsConfig

Optional scraping of Ray’s per-node Prometheus metrics (ray_*). Only meaningful for Ray multi-node replicas. Left unset it resolves itself: on when monitoring is on and the deployment requires Ray, off otherwise.

Field	Type	Default	Description
enabled	`bool	None`	None
port	int	8090	Ray’s --metrics-export-port. Fixed rather than ephemeral so the per-node announce files Prometheus reads have stable contents.

LeptonConfig

Field	Type	Default	Description
type	"lepton"	required	Backend discriminator; always lepton for this model.
workspace_id	str	computed	Lepton workspace ID
endpoint	LeptonEndpointConfig	computed	Serving endpoint configuration: image, resource shape, mounts, environment and token secret.
job	LeptonJobConfig	computed	Batch job configuration: image, resource shape, mounts and environment.
env	dict[str, str]	computed	Global environment variables applied to both the endpoint and jobs where relevant.

LeptonEndpointConfig

Field	Type	Default	Description
image	str	'vllm/vllm-openai:latest'	Container image running the serving endpoint, i.e. a vLLM OpenAI-compatible server.
allowed_dedicated_node_groups	`list[str]	None`	None
resource_shape	str	'gpu.8xh200'	Lepton resource shape for endpoint replicas, describing accelerator type, count and memory, e.g. gpu.4xh200.
allowed_nodes	list[str]	computed	Specific nodes within a node group on which the endpoint may run. Empty means any node in the allowed groups.
mounts	list[Any]	computed	Filesystem mounts injected into the endpoint container. Normalised to Lepton Mount objects when the Lepton SDK is installed, otherwise accepted as dicts.
env	dict[str, str]	computed	Environment variables set on the endpoint container.
api_token_secret_name	`str	None`	None
image_pull_secrets	`list[str]	None`	None

LeptonJobConfig

Field	Type	Default	Description
allowed_dedicated_node_groups	`list[str]	None`	None
image	str	'igeniusai/domyn-swarm:latest'	Container image used by domyn-swarm driver jobs on Lepton.
resource_shape	str	'gpu.8xh200'	Lepton resource shape for batch job execution.
allowed_nodes	list[str]	computed	Specific nodes on which batch jobs may run. Empty means any node in the allowed groups.
mounts	list[Any]	computed	Filesystem mounts injected into the job container. Normalised to Lepton Mount objects when the Lepton SDK is installed, otherwise accepted as dicts.
env	dict[str, str]	computed	Environment variables set on the job container.
image_pull_secrets	`list[str]	None`	None

WatchdogConfig

Field	Type	Default	Description
enabled	bool	True	Master switch for the per-replica watchdog process.
probe_interval	int	30	Seconds between watchdog HTTP and Ray health probes.
http_path	str	'/health'	HTTP path probed on the vLLM REST server to determine readiness and health. A leading / is added automatically if missing.
http_timeout	float	2.0	Seconds allowed for each HTTP health probe request.
readiness_timeout	int	600	Seconds the server is given to become ready before it is considered unhealthy.
restart_policy	`”always”	“on-failure”	“never”`
unhealthy_restart_after	int	120	If the replica stays unhealthy for this many seconds, the watchdog forces a restart, or exits, depending on the restart policy.
max_restarts	int	3	Maximum restart attempts before giving up and leaving the replica in the failed state.
restart_backoff_initial	int	5	Seconds to wait before the first restart attempt.
restart_backoff_max	int	60	Upper bound in seconds for the exponential backoff between restart attempts.
kill_grace_seconds	int	10	Seconds to wait after SIGTERM before the watchdog sends SIGKILL to the child process.
log_level	`”debug”	“info”	“warning”
ray	WatchdogRayConfig	computed	Ray-aware health checking, layered on top of the HTTP probe. Disabled unless ray.enabled is set.

WatchdogRayConfig

Field	Type	Default	Description
enabled	bool	False	Enable Ray-aware health checks (cluster liveness and capacity) in addition to the HTTP checks.
expected_tp	`int	None`	None
probe_timeout_s	float	120.0	Seconds allowed for each Ray health probe command, such as ray status or ray list nodes.
status_grace_s	float	10.0	Ray must report healthy for at least this many seconds before the watchdog treats it as fully ready.
probe_interval_s	float	30.0	Seconds between Ray health probes when Ray checks are enabled.

SwarmPoolConfig

Field	Type	Default	Description
pool	list[SwarmPoolElement]	required	The swarms making up the pool, one entry per swarm.

SwarmPoolElement

Field	Type	Default	Description
name	str	required	Label identifying this swarm within the pool.
config_path	str	required	Path to a DomynLLMSwarmConfig YAML file describing this swarm. Read with DomynLLMSwarmConfig.read when the pool is built.