EDF headers and pseudonymisation

yousleep_common.edf reads the header of an EDF or EDF+ recording, reads its EDF+ annotations, and holds the rule that pseudonymises a header. Every part of the platform that reads a header uses this package, so a recording is read the same way wherever it is read.

Reading a header

Function	Reads
validate_edf_recording(readable, full_file_size)	The header, validated, and checked against the size of the whole file
read_edf_recording_header_from_readable(readable)	The header, validated
read_edf_raw_headers_from_readable(readable)	The fixed and signal header fields, unvalidated
parse_edf_annotations(readable, …)	The EDF+ annotations of the data records

Each returns or reads only what it names: no function reads signal samples. A header is refused if its version is not 0, if it marks an interrupted recording (EDF+D), if its size does not match its number of signals, or if a signal's samples per record are not a positive multiple of the record duration.

The start date. EDF+ writes the date twice: four digits in the recording field (Startdate 12-OCT-2026) and two in the start date field (12.10.26). The four-digit date is read first. When it is missing or X, the start date field is read, with its two-digit year taken as the latest year not in the future. Two values mean the date is unknown:

Value	Meaning	Time of day
EDF_UNKNOWN_START_DATE, 1985-01-01	EDF+'s marker for an unknown or removed date, 01.01.85	Kept and usable
NO_HEADER_DATE_YEAR, 1900-01-01	The header carries no usable date	Not usable

Reading a recording for an ingest

An ingest job reads every recording once, on a runner, in a container with no network. The container reads the runner's copy, after the pseudonymisation rule, and writes an IngestDocument (yousleep_common.models.ingest): the SHA-256 of the header, and the annotations with the platform's labels.

Function	Does
read_recording(path, annotation_text=…)	Returns the document for a file. Without annotation_text, an annotation that matches no label keeps its label, Unknown, but not its text
header_sha256(data)	Returns the SHA-256 of the header at the start of data: the fixed header and every signal header

The platform hashes the header a client registers with header_sha256, and compares it with the document's. They are equal when the stored file has the registered header: an upload's header as cleaned before upload, or a registered file's header after the rule.

Only the header can make a file unreadable: read_recording then raises NotAnEdfRecording, and the ingest container exits with NOT_AN_EDF_RECORDING_EXIT (65), the one exit the platform refuses an upload for. Annotations come from the uploader's file and never fail the read:

• an onset that is not a number from 0 to 30 days is left out;
• a duration that is negative or not a number is 0, and one longer than 30 days is cut to it;
• text longer than 4,096 characters is cut to that length.

The pseudonymisation rule

The rule rewrites three fields of the 256-byte fixed header and copies every other byte. A file keeps its length, its signals, its annotations and its time of day.

Bytes	Field	Becomes
8–87	local patient identification	X X X X
88–167	local recording identification	Startdate D X X X
168–175	start date	01.01.85 if dates are removed; unchanged if kept

Every field is padded with spaces. D is the header's own dd-MMM-yyyy date when dates are kept and that date is valid, and X otherwise.

• D is kept only when the recording field's first subfield is exactly Startdate and its second is a real calendar date. The month is read in any case and written in upper case, as EDF+ requires.
• A plain EDF file's free-text recording field gives D = X. With dates kept, the date stays in the start date field.
• A header that is not 256 bytes, or whose version is not 0, is refused.
• Bytes outside printable ASCII are not refused: the fields the rule writes are replaced whole, and every other byte is copied.
• Applying the rule twice gives the same bytes as applying it once.

Function	Does
pseudonymise_header(fixed_header, keep_dates=…)	Returns the rewritten 256 bytes
is_pseudonymised(fixed_header, keep_dates=…)	Says whether the rule would change nothing
pseudonymise_file_header(path, keep_dates=…)	Rewrites a file's header in place, and says whether it changed

Annotation text in the data records is outside the rule.

Test vectors

yousleep_common/edf/pseudonymisation_vectors.json ships with the package. Each case gives a fixed header as hex, whether dates are kept, and the expected rewritten header as hex; an expected value of null means the input must be refused. Every implementation of the rule is tested against this file. The package's own tests also rewrite whole EDF and EDF+ files and open them in pyedflib.

yousleep_common/edf/pseudonymisation_corpus.json, in the same format, holds 400 headers generated from a fixed seed by tests/edf/corpus.py:

• real, impossible and lower-case dates;
• whitespace beyond the space;
• bytes outside ASCII;
• free text;
• headers already cleaned;
• wrong versions and lengths.

Its expected outputs are the Python implementation's. The vectors fix the rule case by case; the corpus holds another implementation to this one across the space between those cases. A test fails while the corpus differs from what the rule gives, so it is written again whenever the rule changes.