Skip to content

resources

Resource requirements, limits, costs, and priority models for analyses.

Priority pydantic-model

Bases: BaseModel

Represents the priority of the analysis.

A priority is a non-negative integer where 0 is the lowest priority and higher values indicate higher priority.

Fields:

  • base (NonNegativeInt)

base = 0 pydantic-field

The base priority of the analysis for any user.

ResourceRequirements pydantic-model

Bases: BaseModel

Resource requirements for an analysis.

Memory needs are a sum of optional linear terms (all but base default to 0, so each analysis opts into the terms matching how it loads and holds signal data — no analysis is forced into another's shape):

peak_memory_mib =
      base
    + per_hour             * hours
    + per_hour_per_khz     * hours * (max_sample_rate_hz / 1000)
    + per_hour_per_channel * hours * n_channels
    + per_hour_per_channel_per_khz
          * hours * n_channels * (max_sample_rate_hz / 1000)

The per-channel term captures data at the analysis's fixed working rate (one float32 full-signal copy at 128 Hz ~= 1.76 MiB/hour/channel); the per-kHz term captures data held at the recording's native rate (one float32 copy ~= 13.7 MiB/hour/channel/kHz) — e.g. the raw signal before an analysis downsamples it. per_hour is for channel-independent buffers; per_hour_per_khz for a channel-independent buffer at the native rate, such as the working memory of resampling one channel at a time, which every channel reuses.

The api-server uses :meth:estimate_peak_system_memory_mib / :meth:estimate_peak_gpu_memory_mib at submission time to project the expected memory consumption and passes it to the orchestrator as the resource claim.

CPU is claimed in whole cores: cpu_cores is the parallelism the analysis is written for. The platform reserves that many cores for the run, passes the same number to the container as --cpus and as the thread-count environment variables, and the container sizes its thread pools to it. Thread pools do not look at cgroups, so a container that starts more threads than it has cores is slower, not faster; the number here is what keeps a full queue from over-subscribing the host. There is no failure mode for too few cores, only a longer wall time, so no formula or per-tier cap is needed.

Fields:

Validators:

  • _legacy_cpu_mhz

artifacts_memory_mib = 0 pydantic-field

Disk space (in MiB) reserved for output artifacts. Added to the size of any input recording files to determine the total disk requirement.

base_gpu_memory_mib = 0 pydantic-field

Constant term of the GPU-memory formula. Set to 0 if the analysis does not require a GPU.

base_system_memory_mib pydantic-field

Constant term of the system-memory formula: covers fixed overhead (model weights, ONNX runtime, Python, preprocessing libraries) before any signal data is loaded.

cpu_cores pydantic-field

Whole CPU cores the analysis is written to use in parallel. Reserved for the run and passed to the container as --cpus and as the thread-count environment variables; size thread pools to it.

gpu_required = False pydantic-field

Whether GPUs are mandatory or optional for the analysis.

gpus = 0 pydantic-field

The number of GPUs required for the analysis. Set to 0 if the analysis does not require a GPU.

per_hour_artifacts_memory_mib = 0 pydantic-field

Disk space (in MiB) added per signal hour for output artifacts.

per_hour_gpu_memory_mib = 0 pydantic-field

GPU MiB per signal hour, independent of channel count. 0 if N/A.

per_hour_per_channel_gpu_memory_mib = 0 pydantic-field

GPU MiB per signal hour per selected channel (at the working rate). 0 if the analysis needs no GPU or its GPU memory doesn't scale with length x channels.

per_hour_per_channel_per_khz_gpu_memory_mib = 0 pydantic-field

GPU MiB per signal hour per channel per kHz of the native sample rate. 0 if the analysis doesn't hold native-rate signal on the GPU.

per_hour_per_channel_per_khz_system_memory_mib = 0 pydantic-field

System MiB per signal hour per channel per kHz of the recording's NATIVE sample rate — data held at native rate before downsampling (a float32 copy is ~13.7 MiB/h/ch/kHz). 0 if the analysis doesn't hold native-rate signal.

per_hour_per_channel_system_memory_mib = 0 pydantic-field

System MiB per signal hour per selected channel, at the analysis's fixed WORKING sample rate (a float32 full-signal copy at 128 Hz is ~1.76 MiB/h/ch). 0 if system memory does not scale with length x channels.

per_hour_per_khz_system_memory_mib = 0 pydantic-field

System MiB per signal hour per kHz of the recording's NATIVE sample rate, independent of channel count: a buffer at native rate that the channels use one at a time, such as the working memory of resampling. 0 if not applicable.

per_hour_system_memory_mib = 0 pydantic-field

System MiB per signal hour, independent of channel count (e.g. a channel-independent results/accumulator buffer). 0 if not applicable.

wall_clock = None pydantic-field

A wall-clock ceiling raised above the platform's default, with the reason. A platform decision made at registration; an author leaves it out.

estimate_peak_artifacts_memory_mib(hours)

Project disk space (in MiB) for output artifacts.

estimate_peak_gpu_memory_mib(hours, n_channels, max_sample_rate_hz)

Project peak GPU memory in MiB for an analysis of the given size.

Same optional-term structure as :meth:estimate_peak_system_memory_mib.

estimate_peak_system_memory_mib(hours, n_channels, max_sample_rate_hz)

Project peak system memory in MiB for an analysis of the given size.

Sum of the optional linear terms (see the class docstring): base + per_hour*h + per_hour_per_khz*h*(max_sample_rate_hz/1000) + per_hour_per_channel*h*c + per_hour_per_channel_per_khz*h*c*(max_sample_rate_hz/1000).

max_sample_rate_hz is the maximum native sample rate across the selected channels (it only feeds the per-kHz terms). Rounded up so the orchestrator never gets a value below the projection.

max_hours_within_system_memory(memory_mib, n_channels, max_sample_rate_hz)

Return the longest recording whose projected peak fits memory_mib.

The inverse of :meth:estimate_peak_system_memory_mib for a given channel count and native rate: (memory - base) / (the formula's per-hour slope). None when nothing scales with length, so any length fits once the base does; 0.0 when even the base does not fit.

WallClockCeiling pydantic-model

Bases: BaseModel

A wall-clock ceiling raised above the platform's default, with the reason.

The platform bounds how long one run may take, scaling with the length of the recording, and stops a run that exceeds it. An author cannot predict their analysis's scaling reliably, so no configuration is asked to declare one; a configuration that genuinely needs more asks for it here, and the reason is part of the request. Set at registration.

Fields:

reason pydantic-field

Why this analysis needs more than the platform's default.

seconds pydantic-field

The ceiling for one run, in seconds. Applied only when it is above the platform's default for the recording's length.