Packaging an analysis¶

This guide is for authors of a sleep-analysis method, a script, a trained model or a library, who want to run it on the youSleep platform: on the recordings users upload, across whole cohorts, with its output shown in the viewer and the reports beside every other analysis. It describes how to package a new analysis and how to convert an existing script.

Analyses can be written in any language. The platform communicates with a container through two JSON documents and one command-line argument. All examples and the helpers in this package are written in Python, which we recommend because it is the language we support best.

Start with yousleep-init unless you have particular needs. It writes a project that passes the conformance check as generated, in the layout the platform team reviews and registers most quickly.

Start a project

yousleep-init my-analysis writes a script, a Dockerfile, the configuration and a Makefile. make verify passes before you change a line.

Starting a project
The manifest

The document a container reads. It names the recording, the channels by index, the parameters, the resources and the output path.

Analysis manifest
The events document

The document a container writes, specified in prose and by JSON Schema.

Events document
The check

yousleep-verify runs an image the way the platform does and reports what would fail before submission.

Checking an image
Start from an example

Two complete analyses, a staging analysis and one that consumes another analysis's events, with their configurations, Dockerfiles and recorded expected output.

analysis-example

How the platform runs an image¶

An analysis is a container image plus a configuration file. When a user runs it on a recording, the platform:

fetches the recording, and any upstream events the configuration asks for, into a directory the container will see;
writes a manifest, a JSON document describing this run;
starts the container with one argument, the manifest's path;
collects the events document the container wrote, stores it, and presents it in the viewer and the reports.

flowchart LR
    R[recording and<br/>upstream events] --> C
    M[manifest.json] --> C[your container<br/>--manifest-file /local/manifest.json]
    C --> E[events document]
    E --> V[viewer, reports,<br/>dependent analyses]

The container runs confined:

no network access, so weights, licence checks and telemetry cannot be fetched or sent at run time;
a read-only root filesystem, with a writable /tmp;
no capabilities and an unprivileged user, with no way to gain privileges through a setuid binary;
at most 4,096 processes and threads at once;
the cores and memory the configuration declares, and a wall-clock ceiling.

The container holds no credential and never calls the platform. Every input arrives as a file and every output is a file.

A library that keeps a cache or configuration in the home directory cannot write it on the read-only filesystem, and warns or fails at run time. Point it at /tmp in the Dockerfile: ENV MPLCONFIGDIR=/tmp/matplotlib for matplotlib, for example, or ENV NUMBA_CACHE_DIR=/tmp/numba for numba.

The contract¶

Invocation	`<entrypoint> --manifest-file <path>`, and nothing else: no other arguments, and no environment variables to rely on
Input	The manifest: where the recording is, which channels were selected and at which index, the parameters typed as declared, the resources the run has, and where to write
Output	One events document at `outputs.events.path`
Exit status	`0` after the document is written; non-zero on any failure

Both documents are specified in prose and by JSON Schema, so an analysis can be written in R, Julia, C++ or as a shell script around a binary. An image is registered when it follows the contract; how it was built is not assessed.

Getting started¶

pip install yousleep-common
yousleep-init my-analysis
cd my-analysis
make verify

An analysis is three files: a script, a Dockerfile and a configuration. yousleep-init writes them, with a Makefile that builds the image, runs it by hand and runs the conformance check. Starting a project describes the command; the files it writes are the ones below.

flowchart LR
    N[yousleep-init] --> S[script] --> D[Dockerfile] --> I[image]
    N --> Y[configuration] --> H[make run]
    I --> H
    H --> V[make verify] --> G[registration]

The script. The script below is complete and runs as it stands. It scores every window Sleep stage ?; replace the loop that builds events with your method and keep the rest.

from argparse import ArgumentParser
from pathlib import Path

import mne
from yousleep_common.models.events import Event
from yousleep_common.utils.analysis import select_channels
from yousleep_common.utils.event_blocks import save_event_blocks
from yousleep_common.utils.manifest import load_manifest


def main() -> None:
    parser = ArgumentParser()
    parser.add_argument("--manifest-file", required=True)
    manifest = load_manifest(parser.parse_args().manifest_file)

    raw = mne.io.read_raw_edf(manifest.inputs.recording.path, preload=False, verbose="ERROR")
    selected = select_channels(manifest, header_labels=raw.ch_names)  # refuses the wrong file
    raw.pick([channel.index for channel in selected])
    names = [channel.name for channel in selected]  # what belongs in the output

    window_ms = int(manifest.parameters.get("window-ms", 30000))
    length_ms = int(raw.n_times / raw.info["sfreq"] * 1000)
    events = [
        Event(
            start_time_ms=start,
            end_time_ms=min(start + window_ms, length_ms),
            label="Sleep stage ?",
            channels=names,
        )
        for start in range(0, length_ms, window_ms)
    ]

    output = Path(manifest.outputs.events.path)
    output.parent.mkdir(parents=True, exist_ok=True)
    save_event_blocks(output, events)


if __name__ == "__main__":
    main()

The Dockerfile. Any base image can be used. A wheel may need a system library the slim images do not carry: LightGBM, for example, needs the OpenMP runtime (apt-get install libgomp1 on Debian-based images). An import of every dependency at build time, as below, catches that before the check does.

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt \
 && python -c "import mne, yousleep_common"
COPY analysis.py .
ENTRYPOINT ["python3", "analysis.py"]

with requirements.txt pinning yousleep-common and mne. For a real analysis, build from a lockfile in a multi-stage image so that the runtime layer carries no build tooling; the uv Docker guide describes a suitable pattern.

The configuration. The configuration file tells the platform what it needs to offer, run and present the analysis. The reference below explains every block. The file below is complete.

identification:
  name: "My Staging Analysis v1"

purpose:
  description: "Scores every 30-second window of a recording."
  analysis_types: ["staging"]

provenance:
  origin: "third-party"
  developer: "The authors"
  source_url: "https://github.com/example/my-analysis"
  source_licence: "MIT"

licence:
  type: "non-commercial"

docker:
  image: "ghcr.io/example/my-analysis"
  tag: "1.0.0"

resources:
  cpu_cores: 1
  base_system_memory_mib: 512

inputs:
  recording:
    channel_types:
      - type: "EEG"
        number:
          range: [1, 8]
        requirements:
          sampling_rate:
            range: [100, "+inf"]

outputs:
  events:
    - label: "Sleep stage ?"

parameters:
  custom:
    - name: "Window length"
      key: "window-ms"
      type: "int"
      default: 30000
      description: "The length of each staging window."
      choices:
        - value: 15000
          description: "15 seconds"
        - value: 30000
          description: "30 seconds"

Running by hand. make run RECORDING=night.edf does the following. yousleep-manifest reads an EDF header and your configuration and writes the manifest the platform would write; the working directory is mounted at /local and the image runs on it:

pip install yousleep-common
yousleep-manifest --recording night.edf --config my-staging-analysis.yaml \
    --channel "EEG Fpz-Cz" > manifest.json

docker build -t my-analysis .
docker run --rm --network none -v "$PWD:/local" my-analysis --manifest-file /local/manifest.json

The events document is written to output-events.json.gz in the working directory. The cores, memory and parameter defaults in the manifest are taken from the configuration, and a channel selection the platform would refuse is refused here too. Pass --network none because the platform runs the container without network access; a run that only works with network access will not work on the platform.

Converting an existing script¶

Run yousleep-init for the packaging, then move the method into analysis.py. A working script needs four changes and no restructuring. Keep your main, your loader, your model and your tests.

Today	On the platform
Paths are arguments, environment variables or constants	One argument, `--manifest-file`; every path is a field of the manifest
Channels are picked by label, or by position, or assumed	Channels are the manifest's `inputs.recording.channels`, each with an `index` into the file. Read by index, and check the label there is the `source_name` the manifest says; `select_channels` does exactly this
Parameters are parsed from the command line or a config file	`manifest.parameters`, keyed as the configuration declares them, already typed and validated
Output is a CSV, a NumPy array or a proprietary format	One events document at `outputs.events.path`, with channels named by the manifest's `name`

Also check the following:

Select the channels before reading any signal. In MNE, open the file with read_raw_edf(path, preload=False), which reads only the header, then pick the selected indices, then load_data. A polysomnogram carries dozens of channels the analysis never reads. A script that loads them all and keeps its own afterwards needs memory for the whole file, which grows with its length and its number of channels, and can run out of memory on long recordings that the selection alone would fit. The conformance fixture carries 20 such channels, and the peak it reports shows the difference.
Size thread pools and worker counts from manifest.resources.cpus, not from the host's core count. The run has exactly the cores it declared.
Copy anything the script downloads at run time, model weights in particular, into the image. The container has no network access.
Keep any resampling the method performs. The platform does not resample recordings. The configuration declares the sample rate the analysis accepts, and a recording outside that range is refused at submission.

Helpers for Python¶

yousleep-common contains the platform's models and four functions that cover the steps every Python analysis would otherwise implement itself. All of them are optional.

import mne
from yousleep_common.utils.manifest import load_manifest
from yousleep_common.utils.analysis import select_channels
from yousleep_common.utils.event_blocks import save_event_blocks

manifest = load_manifest(path)                                      # validated, typed
raw = mne.io.read_raw_edf(manifest.inputs.recording.path, preload=False)  # the header only
selected = select_channels(manifest, header_labels=raw.ch_names)    # refuses the wrong file
raw.pick([channel.index for channel in selected])                   # select by index
raw.load_data()                                                     # the selected signals only
save_event_blocks(manifest.outputs.events.path, events)             # the encoding the platform reads

Helper	What it does
`load_manifest(path)`	Reads the manifest and validates it into a typed `AnalysisManifest`
`select_channels(manifest, header_labels=...)`	Returns the channels the manifest names, by index, each with its `name`, `type` (`"EEG"`, `"EOG"`, …) and `slot`. The header labels passed in (`raw.ch_names` in MNE, `getSignalLabels()` in pyedflib) are compared with the manifest's `source_name` at each index. If they differ, the helper raises before any signal is read
`channel_for_slot(selected, "EOG-L")`	Looks up a slot's channel by slot label, so that adding a slot to the configuration does not shift the ones after it
`save_event_blocks(path, events)`	Encodes a list of `Event` as an events document and compresses it when the path ends in `.gz`

An environment that cannot accept the package's dependency floors can read and write the JSON directly. Both paths are supported.

Channels are selected by index, never by label

A recording may repeat a label and a user may rename a channel. An analysis that matches on a name can silently read a different signal than the one the platform selected. select_channels checks the label at each index against the manifest and rejects the file before any signal is read. An analysis that does not use the helper should perform the same check.

The configuration file¶

The configuration is read by the catalogue, the submission form, the dispatcher and the attribution page. It holds only what an author can know. Everything the platform decides is set at registration and does not belong in the file: scheduling priority, the image digest, GPU placement, the memory terms that scale with recording length, and to whom the analysis is offered.

Block	Required	Content
`identification`	`name`	A unique, human-readable name. The id is derived from it and names one lineage of results, see versions
`identification`		Optionally `family`, shared by the versions and variants of one method (for example "U-Sleep"), so the catalogue and the publications page can group and filter by it. Unset, the analysis is a family of its own
`purpose`	`description`, `analysis_types`	What it does, and which of `staging`, `rapid-eye-movements`, `arousals`, `artifacts` it belongs to. `tags` is free-form and optional
`provenance`	`origin`, `developer`	Who developed it. `origin` is `yousleep` or `third-party`; for a third-party analysis, also `source_url` (where it is published), `source_licence` (an SPDX identifier) and `data_rights`: `basis: pending` while developing, `basis: declaration` with the developer's signed Analysis Declaration on record once the analysis is offered beyond its developer, see registering. `source_version` names the upstream version, and the trained model's version when separate. Shown to users, so it has to be true
`evidence`		What the analysis cites: `publications`, each with `title`, `url`, a short `name` of at most 32 characters such as "Vallat and Walker, 2021", and `by: "developer"` for the authors' own work or `by: "independent"` for anyone else's. Optionally `venue`, `year` and a one-line `summary` of who was studied and what was found. Results credit the developer and cite the `developer` publications wherever they are shown or exported, in the viewer, reports and downloads; independent publications are listed as evidence. Set `cite: false` on the authors' own evaluation that users should not cite in place of the method paper, or `cite: true` on an independent publication they must cite. Omit `evidence` if there is nothing to cite yet
`licence`	`type`	The licence class it is offered under: `non-commercial` or `commercial` today. Confirmed at registration
`docker`	`image`, `tag`	The image the platform runs. The digest is pinned at registration
`resources`	`cpu_cores`, `base_system_memory_mib`	Whole cores, and the memory one run needs before any input is counted. Set `base_system_memory_mib` from the peak `yousleep-verify` reports for its five-minute fixture, so that the peak is at most about 77% of it; the check warns when it is more. The terms that scale with recording length are set at registration: the platform team measures them with `yousleep-verify --meter`, which an author can also run. A run is expected to finish within one minute and is stopped after two, regardless of recording length. A configuration that needs more time requests it at registration, with the reason
`inputs.recording`	one of `channels`, `channel_types`	The recording, by exact channel names or by types with a count range `number: [min, max]`; each with a sampling-rate requirement and, optionally, signal-quality requirements. A type with `min` 0 is optional: the analysis runs without it, and the conformance check runs once without it. `minimum_duration_seconds` defaults to 1
`inputs.events`		An upstream events document, by the labels it must contain, for an analysis that builds on another's output
`inputs.metadata`		Subject context the analysis reads: any of `age`, `bmi`, `sex`. Only the declared values reach the analysis, and each is null when the study does not record it; the conformance check runs once with all of them null
`outputs.events`	`label` per entry	The event labels it produces, each optionally `probabilistic` or `valued`, or `instant`
`outputs.supports_full_probabilistic_output`		`true` when the analysis can write every class's probability for each prediction, not only the winning class's. Every output must then be `probabilistic`. The field declares the ability; a user asks for it per run through a custom parameter, by convention a `bool` keyed `store-all-probabilities` with default `false`, which the container reads as `manifest.parameters["store-all-probabilities"]`. The events document shows the encoding
`parameters.custom`	`name`, `key`, `type`, `default`, `description`, and `choices` or `range`	What a user can set. Each reaches the container in the manifest, keyed by `key` and typed as declared. Omit the block if there are none

The file is validated against AnalysisConfig. AnalysisConfig.model_validate(yaml.safe_load(text)) checks a candidate file and reports what is wrong.

Versions¶

An analysis id names one lineage of results that can be compared with each other. Results reference the id, so an id never changes once it is registered.

Change	Same id	New id (`-v2`)
A fix to the code that leaves results unchanged or corrects a defect	Yes
A rebuilt image, new dependency versions, a new memory claim	Yes
New trained weights, new training data, or a different label set		Yes
A change to the method that alters its results		Yes

Every version registered under one id is recorded with its configuration and image digest, so each result can be traced to the exact software that made it.
The name shows a version only once a second lineage exists: the first lineage of a method is registered as its plain name, and a second one as … v2. The id is derived from the name; to rename an analysis and keep its id, set identification.id to the registered id.
provenance.source_version records the upstream version, for example YASA 0.7.0, classifiers 0.5.0, and is shown on the analysis page.
A library that ships its own trained models, such as YASA, chooses among the models of the installed release. The version pinned in the lock fixes which models the image carries; source_version records it. An upgrade that changes results fails the expected check against the recorded output (Checking an image), and new weights need a new id.

Registering the analysis¶

Registration is currently done together with the platform team.

An analysis is first offered privately, to the organisation it is registered for. A third-party analysis is published to every user only with the developer's signed Analysis Declaration on record: a form stating what the analysis is, what data it was developed on and under which terms, for which use it may be offered, and how to credit it. youSleep sends it for signature and keeps the signed record; the configuration then references it:

provenance:
  data_rights:
    basis: "declaration"
    declaration:
      record: "records/declarations/my-analysis-2026-09-25.pdf"
      commit: "…"          # the records repository commit holding it
      sha256: "…"          # of the signed file
      signed_on: 2026-09-25
      signatory: "The developer, as signed"
      offer: "non-commercial"   # or "commercial", as ticked on the form

Until then basis: pending lets you build, verify and use it privately; the check reports it as a warning, and the platform does not publish it.

Run yousleep-verify on the final image (make verify in a generated project). It is the check we run at registration, so an image that passes locally passes there. Set base_system_memory_mib from the peak memory it reports.
Optionally, run yousleep-verify --meter --propose to see how the memory grows with recording length. It takes minutes to hours and needs a few GB of disk. We run it at registration in any case.
Contact us through the portal's contact form and choose Integrate an algorithm. Send the image, as a reference we can pull or as a docker save archive, and the configuration file.
We run the same check on the platform's architecture, pin the digest, measure the memory terms that scale with recording length with yousleep-verify --meter, set the other fields that are ours to set, and register the configuration. The image is copied into a registry the platform controls, so that past analyses remain re-runnable.

Two complete examples, a staging analysis and one that consumes another analysis's events, are published in the analysis-example repository.