Skip to content

Analysis manifest

An analysis container receives one argument, --manifest-file <path>, and nothing else. The manifest is a JSON document that names every input by the path the container sees it at and describes it, carries the custom parameters typed as the configuration declared them, states the resources reserved for the run, and names the output the container must write. It holds no credential and no URL, so it can be logged and shared.

The contract is the Pydantic model AnalysisManifest in this package; AnalysisManifest.json_schema() publishes it for containers written in other languages.

Shape

{
  "manifest_version": 1,
  "analysis": {"id": "01a0…", "config_id": "u-sleep-research-v2", "config_sha1": "3f2c…"},
  "inputs": {
    "recording": {
      "path": "/alloc/input/recordings/edf/01a0….edf",
      "format": "edf",
      "start": "2023-01-01T00:00:00",
      "duration_ms": 28800000,
      "channels": [
        {"index": 0, "source_name": "EEG Fpz-Cz", "name": "Fpz-Cz", "type": "EEG",
         "unit": "uV", "sample_rate": 100.0, "slot": null, "laterality": null}
      ]
    },
    "events": null,
    "metadata": {"subject": {"age": 54, "sex": "male", "bmi": 27.5}, "study": {"lights_off_ms": 1200000, "lights_on_ms": null}}
  },
  "slots": [],
  "parameters": {"data-per-prediction": 3840},
  "resources": {"cpus": 4, "memory_mib": 2926, "gpus": 0, "gpu_devices": [], "gpu_memory_mib": []},
  "outputs": {"events": {"path": "/alloc/output/events/json/01a0…-output-events.json.gz"}}
}
  • Channels are loaded by index, never by label. source_name is the header label; name is what the user calls the channel and is what belongs in output.
  • Slots are the ordered, labelled input positions an analysis declares (for example a left and a right EOG), each filled by one channel of the recording. Montages the file does not carry are prepared before upload; the platform never asks a container to derive a channel.
  • Parameters are keyed by the configuration's parameter key and are already validated and typed; a container reads them, it does not parse them.
  • Resources are what the run has: the memory is enforced by the container runtime, and cpus is what a container should size its thread pools to.
  • Metadata is present only when the configuration declares a metadata input. Only the subject values it declares are filled; the others, and any the study does not record, are null, so an analysis must run without them.
  • Paths are what the container sees: on the platform the inputs and the output live under /alloc, the manifest itself under /local. A path is never a URL and never climbs with ...
  • Unknown keys are refused. A misspelt key fails at load time instead of being silently dropped.
  • Enumerated fields are plain text. A channel's type is "EEG", a subject's sex is "male"; compare them as strings.

Reading it in a container

from yousleep_common.utils.manifest import load_manifest

manifest = load_manifest("/local/manifest.json")
indices = [c.index for c in manifest.inputs.recording.channels]
threads = manifest.resources.cpus

A container that keeps its runtime dependencies minimal can read the JSON directly; the schema is small and stable, and manifest_version changes only when a field changes meaning.

Running a container by hand

The platform writes manifests itself. To run a container outside the platform, yousleep-manifest reads an EDF header and the analysis configuration and writes one:

yousleep-manifest --recording night.edf --config u-sleep-research-v2.yaml \
    --channel "EEG Fpz-Cz" --channel "EOG horizontal" \
    --param data-per-prediction=3840 > manifest.json
docker run --rm -v "$PWD:/local" ghcr.io/yousleep-ai/yousleep/usleep:latest \
    --manifest-file /local/manifest.json

With --config the manifest says what the platform's would: the configuration's id and its SHA-1, its cores, the memory its formula projects for this recording, its parameter defaults with --param on top, and its slots bound in the order the channels were selected. analysis.id stays null: a hand run is not an analysis on the platform. The hash is of the file as given; on the platform it is the registered configuration's, which also carries the fields set at registration. A selection the platform would refuse, too few channels of a type, a rate below the requirement, a named channel the file does not carry, is refused here with the reason.

Without --channel, the channels are selected as the platform selects them when a user chooses none:

  • an exact channel is matched by what it is, so Fpz-Cz matches a header's EEG Fpz-Cz, and a channel type takes every channel of its type;
  • a channel at a sampling rate the requirement does not accept is left out;
  • where a type offers more channels than the analysis takes, the first ones in the file's order are kept.

The tool prints what it kept and what it left out, and why. A configuration whose slots name a side (a left and a right EOG) needs --channel, in slot order: the platform binds such slots by each channel's recorded side. Add =TYPE to a chosen channel when its label does not say what it is (--channel "Chin=EMG").

--age, --sex and --bmi give the subject values. As on the platform, only the values the configuration declares under inputs.metadata are written, a declared value not given is null, and there is no metadata when the configuration declares none.

Values given with --param are typed: integers, floats and true/false are read as such. --config-id in place of --config writes a manifest for an analysis that has no configuration yet: every signal but the annotations channel is selected unless --channel says otherwise, the subject values given are written, and the resources are placeholders, which the tool warns about.