Skip to content

runner

Runners and compute pools: enrolment, and what administrators see of them.

A runner proves who it is with an Ed25519 key pair. It enrols once with a single-use token, sending its public key, and stays pending until an administrator of its pool's organisation compares the key's fingerprint, as the runner's host shows it, and approves it.

The platform signs its answers with its own key, which a root key certifies in a KeyCertificate. The runner pins the root's public key at install.

AnalysisRunOut pydantic-model

Bases: BaseModel

Where an analysis runs or ran, and on what.

The place is known from submission. The image digest and the inputs' SHA-256 come from the runner's attestation once the run's result is accepted; until then, and for a run that predates attestations, they are empty.

Fields:

image_digest = None pydantic-field

The digest of the image that ran.

input_sha256 pydantic-field

The inputs, as the runner read them.

place pydantic-field

On the platform, or at the organisation's own site.

recorded_at = None pydantic-field

When the run's result was accepted.

runner_name = None pydantic-field

The runner that ran it. Shown to the organisation's owners and admins, and to platform administrators; None for anyone else.

ComputePoolListResponse pydantic-model

Bases: PaginatedResponse

A page of compute pools.

Fields:

ComputePoolOut pydantic-model

Bases: BaseModel

A compute pool as administrators see it.

Fields:

held pydantic-field

Jobs the pool's runners hold now, starting or running.

lease_seconds pydantic-field

In force: the pool's own or the default.

org_id pydantic-field

None for the platform's own pool.

queue_timeout_hours pydantic-field

In force: the pool's own or the default.

queued pydantic-field

Jobs waiting for one of the pool's runners.

runner_seen = False pydantic-field

Whether one of the pool's runners is taking work (a runner's taking_work).

EnrolmentTokenOut pydantic-model

Bases: BaseModel

A new enrolment token. The token is shown here once and never again.

With it come the storage addresses the runner reaches, which the installer takes each with --storage-origin, and the runner image to install. The platform's root key is not among them: the runner's release carries it and the install pins it, as a key served here could be changed by a proxy between the site and the platform, which pinning guards against.

Fields:

  • pool_id (UUID7)
  • token (str)
  • expires_at (datetime)
  • storage_origins (list[str])
  • runner_image (str | None)

storage_origins pydantic-field

The storage the runner downloads its jobs' inputs from and uploads their results to, as origins (scheme, host and port). Empty from a server that predates it.

HeaderRule pydantic-model

Bases: BaseModel

The pseudonymisation rule a runner applies to a recording's EDF header.

The runner hashes the recording as fetched, then rewrites the header of its local copy before the container starts: the identification fields are blanked, and the start date is replaced by EDF+'s unknown date unless keep_dates. The container reads only that copy.

Config:

  • extra: forbid
  • frozen: True

Fields:

keep_dates pydantic-field

Keep the start date, or replace it with EDF+'s unknown date.

HeartbeatLease pydantic-model

Bases: BaseModel

One held lease, as its runner reports it.

Config:

  • extra: forbid

Fields:

  • lease_id (UUID7)
  • attempt (int)
  • phase (LeasePhase)

HeartbeatRequest pydantic-model

Bases: BaseModel

A runner reports the leases it holds.

Config:

  • extra: forbid

Fields:

HeartbeatResponse pydantic-model

Bases: BaseModel

An answer for each reported lease.

Fields:

JobClaim pydantic-model

Bases: BaseModel

The resources a leased job may use, as the runner enforces them.

Config:

  • extra: forbid

Fields:

  • cores (float)
  • memory_mib (int)
  • wall_clock_seconds (int)
  • gpus (int)

memory_mib pydantic-field

A hard limit with no swap. Never 0, which a container runtime reads as no limit.

KeyCertificate pydantic-model

Bases: BaseModel

A root key's statement that public_key signs as kid until not_after.

Config:

  • frozen: True
  • extra: forbid

Fields:

public_key pydantic-field

The certified key's 32 raw bytes, base64.

serial pydantic-field

Higher than every earlier certificate's; a runner refuses a serial below the highest it has seen.

signature pydantic-field

The root key's Ed25519 signature, base64.

Lease pydantic-model

Bases: BaseModel

One job a runner now holds, until expires_at unless heartbeated.

Exactly the field of its kind is set; the platform leaves the others out of the answer, so a runner that does not know a kind never meets its field.

Fields:

Validators:

  • _its_job_alone

attempt pydantic-field

Sent back with every report on it.

LeaseAnswer pydantic-model

Bases: BaseModel

What to do with one reported lease.

Fields:

LeaseFile pydantic-model

Bases: BaseModel

A file the container reads or writes, and the URL that moves it.

Fields:

header_rule = None pydantic-field

On the recording among a job's inputs: the rule its local copy's header is rewritten by before the container reads it. Absent on every other file.

url pydantic-field

A presigned URL: GET for an input, PUT for an output. Used exactly as given: normalising it would break its signature.

LeaseRequest pydantic-model

Bases: BaseModel

A runner asks for work that fits its free room and its local policy.

Config:

  • extra: forbid

Fields:

capacity = None pydantic-field

The runner's total room, as its configuration offers it now. The platform keeps it as the runner's capacity, so a change made after enrolment reaches the platform with the runner's next request for work. Unset, the capacity last reported, or the one the runner enrolled with, stands.

kinds pydantic-field

The kinds of job the runner takes.

largest_job pydantic-field

The largest single job the runner's local policy allows.

wait_seconds = 30 pydantic-field

How long the platform may hold the request open for work.

LeaseResponse pydantic-model

Bases: BaseModel

The jobs leased to the runner; empty when none came in time.

Fields:

lease_seconds pydantic-field

How long a lease lasts without a heartbeat. The runner reports every :func:heartbeat_seconds of it.

LeaseResultRequest pydantic-model

Bases: BaseModel

How a leased job ended.

Config:

  • extra: forbid

Fields:

Validators:

  • _one_way_to_end

exit_code = None pydantic-field

Set exactly when the job ended exited.

header_changed = None pydantic-field

For an ingest that fetched its recording: whether the header rule changed the copy's header. The platform refuses an uploaded copy the rule changed.

peak_memory_mib = None pydantic-field

The container's peak memory, as it recorded it on exit.

return_reason = None pydantic-field

Set exactly when the job ended returned.

succeeded property

Whether the container exited 0 by itself.

LeaseResultResponse pydantic-model

Bases: BaseModel

The platform's decision on a posted result.

Fields:

LeasedAnalysis pydantic-model

Bases: BaseModel

Everything a runner needs to run one analysis, and nothing more.

Fields:

image pydantic-field

The container image to run.

manifest pydantic-field

The one document the container receives.

manifest_path pydantic-field

Where to write the manifest, relative to the work directory.

LeasedDelete pydantic-model

Bases: BaseModel

One uploaded copy of a recording to delete from an organisation's bucket.

The bucket is reachable only from the site, so its runner sends the presigned DELETE. No container runs. A copy already gone counts as deleted. The recording itself may be gone by then, so the job names only the object.

Fields:

url pydantic-field

The presigned DELETE for the object.

LeasedIngest pydantic-model

Bases: BaseModel

Everything a runner needs to read one recording, and nothing more.

The runner fetches the recording, hashing it as it arrives, and rewrites its copy's header by the recording's rule. It then runs its own image on the copy, and the container writes the ingest document (:class:~yousleep_common.models.ingest.IngestDocument). The runner uploads the document to document.

Fields:

Validators:

  • _the_rule_applies

annotation_text pydantic-field

Keep the text of annotations that match no label. False for a recording in an organisation's bucket, whose free text stays at the site.

document pydantic-field

Where the ingest document goes.

recording pydantic-field

The recording to read, always with its header rule.

PoolImage pydantic-model

Bases: BaseModel

An image the runners of a pool may be asked to run.

Fields:

analyses pydantic-field

The analyses that run this image, by configuration id. Empty for a runner an administrator has not yet approved.

kind = LeaseKind.ANALYSIS pydantic-field

The kind of job that runs the image.

reference pydantic-field

The image as a runner is asked to run it: its name and tag, and its digest when the analysis declares one.

smallest_job pydantic-field

The least room a job on this image claims: the fewest cores any of its analyses claims, and the least memory any claims for the shortest recording. A runner whose largest job is smaller is never asked to run it.

PoolImagesResponse pydantic-model

Bases: BaseModel

The images a runner's pool may be asked to run.

A runner pulls them ahead of its first job, so a missing registry login or permission shows when the host is set up, not when an analysis waits.

Fields:

ResultAttestations pydantic-model

Bases: BaseModel

What ran, on what, as the runner states it.

Config:

  • extra: forbid

Fields:

  • image_digest (str)
  • input_sha256 (list[Annotated[str, Field(pattern=_SHA256_HEX)]])
  • runner_version (str)
  • input_sizes (list[Annotated[int, Field(ge=0)]] | None)
  • input_etags (list[Annotated[str, Field(max_length=256)] | None] | None)

input_etags = None pydantic-field

Each input's ETag as its storage answered the fetch, in the order of input_sha256, or null where it gave none. Older runners leave it out.

input_sizes = None pydantic-field

Each input's size in bytes as fetched, in the order of input_sha256. Older runners leave it out.

RunRecordListResponse pydantic-model

Bases: PaginatedResponse

A page of run records, newest first.

Fields:

RunRecordOut pydantic-model

Bases: BaseModel

Where a job ran and on what, as its runner attested it.

One per accepted result, of every kind of job. It names the job's subject as its lease did: the analysis an analysis ran, the recording an ingest read, or the object a delete removed from an organisation's bucket, and that bucket's storage location.

Fields:

analysis_id = None pydantic-field

An analysis's.

image_digest pydantic-field

The digest of the image that ran, as the runner pulled it.

input_sha256 pydantic-field

The inputs, as the runner read them.

object_url = None pydantic-field

A delete's.

output_sha256 pydantic-field

The document the job uploaded, as the platform read it: an analysis's events, an ingest's reading.

peak_memory_mib pydantic-field

The container's peak memory, as it recorded it on exit.

recording_id = None pydantic-field

An ingest's.

storage_location_id = None pydantic-field

A delete's.

RunnerApproveRequest pydantic-model

Bases: BaseModel

Approve a pending runner.

The fingerprint is the one the administrator compared with the runner's host. The platform refuses the approval when it differs from the enrolled key's, so an approval cannot land on another runner by mistake.

Config:

  • extra: forbid

Fields:

  • fingerprint (str)

RunnerCapacity pydantic-model

Bases: BaseModel

Room on a runner: cores and memory.

Config:

  • extra: forbid

Fields:

  • cores (float)
  • memory_mib (int)

RunnerEnrolRequest pydantic-model

Bases: BaseModel

What a runner sends to enrol. The request is signed with the new key.

Config:

  • extra: forbid

Fields:

Validators:

name pydantic-field

A label for the host, chosen at install.

public_key pydantic-field

The runner's Ed25519 public key: 32 raw bytes, base64.

RunnerEnrolResponse pydantic-model

Bases: BaseModel

The enrolled runner, as it asks after itself.

Fields:

fingerprint pydantic-field

The runner key's fingerprint, as the platform computed it.

RunnerListResponse pydantic-model

Bases: PaginatedResponse

A page of runners, and what applies to every runner.

The page answers before a pool has its first runner, so it carries the runner image a new runner installs: an install command can be shown before any enrolment token exists.

Fields:

quarantine_after_failures pydantic-field

Failures in a row that quarantine a runner.

RunnerOut pydantic-model

Bases: BaseModel

A runner as administrators see it.

Fields:

booked_cores pydantic-field

Cores its held jobs book, out of its capacity's cores.

booked_memory_mib pydantic-field

Memory its held jobs book, out of its capacity's memory.

failures_in_a_row pydantic-field

Jobs given back in a row, on their first try, for a fault that points at the runner's host: an image that would not pull or verify, an input that would not download, or an error from the container runtime. A job given back for another reason, such as an image the registry says it does not have, leaves it as it is; any other result resets it. At the limit in the list's quarantine_after_failures the runner is quarantined.

fingerprint_hint pydantic-field

The last characters of the key's fingerprint. Approval needs the whole fingerprint, read on the runner's host.

held pydantic-field

Jobs the runner holds now, starting or running.

taking_work = False pydantic-field

Whether the platform counts the runner as taking work: it is approved, not quarantined, and asked for work in the last three minutes. False from a server that predates it.

UploadedFile pydantic-model

Bases: BaseModel

An output the runner uploaded to its presigned URL.

Config:

  • extra: forbid

Fields:

  • container_path (str)
  • sha256 (str)
  • size (int)

heartbeat_seconds(lease_seconds)

Return how often a runner reports its leases: a third of the lease period.