system

A collection of utility functions for handling system-related tasks.

Examples include: path manipulation, file deletion, system resource querying, etc.

ResourceRange dataclass

Dataclass to store the minimum and maximum resource requirements for an analysis.

Attributes:

Name	Type	Description
min	ResourceSpec	The minimum resource requirements.
max	ResourceSpec	The maximum resource requirements.

__init__(min, max)

Initialize the resource range with the minimum and maximum resource requirements.

Parameters:

Name	Type	Description	Default
min	ResourceSpec	The minimum resource requirements.	required
max	ResourceSpec	The maximum resource requirements.	required

__post_init__()

Validate the resource requirements.

Validates resource requirements by checking that the minimum requirements are less than the maximum requirements and that all requirements are non-negative.

get_satisfied_max(available_resources)

Get the maximum resources in the range that are satisfied by the available resources.

Get the maximum resources in the range that are satisfied by the available resources while checking that the available resources are greater than the minimum requirements.

Parameters:

Name	Type	Description	Default
available_resources	ResourceSpec	The available resources.	required

Returns:

Type	Description
ResourceSpec	The maximum resources in the range that are satisfied by the available resources.

Raises:

Type	Description
ValueError	If the available resources are not greater than the minimum requirements.

Examples:

>>> ResourceRange(
...     min=ResourceSpec(cpus=4, memory=16384, gpus={"0": 8.0}),
...     max=ResourceSpec(cpus=8, memory=32768, gpus={"0": 16.0}),
... ).get_satisfied_max(
...     ResourceSpec(cpus=8, memory=32768, gpus={"0": 16.0})
... )
ResourceSpec(cpus=8, memory=32768, gpus={'0': 16.0})
>>> ResourceRange(
...     min=ResourceSpec(cpus=4, memory=16384, gpus={"0": 8.0}),
...     max=ResourceSpec(cpus=8, memory=32768, gpus={"0": 16.0}),
... ).get_satisfied_max(
...     ResourceSpec(cpus=2, memory=8192, gpus={"0": 4.0})
... )
Traceback (most recent call last):
...
ValueError: Available resources must be greater than minimum requirements.

ResourceSpec dataclass

Dataclass to store resource specfications.

Dataclass to store the resource requirements for an analysis, available resources, and the resources allocated to the analysis.

Attributes:

Name	Type	Description
cpus	int	The number of CPU threads required.
memory	int	The system memory required in MiB.
gpus	dict[str, float]	A dictionary mapping the GPU device ID to the memory required in GiB.

Examples:

>>> ResourceSpec(cpus=4, memory=16384, gpus={"0": 8.0, "1": 8.0})
ResourceSpec(cpus=4, memory=16384, gpus={'0': 8.0, '1': 8.0})

__add__(other)

Add two resource specs, returning a new ResourceSpec.

Parameters:

Name	Type	Description	Default
other	ResourceSpec	The resource spec to add.	required

Returns:

Type	Description
ResourceSpec	A new ResourceSpec with the combined resources.

__post_init__()

Validate the resource requirements by checking that all requirements are non-negative.

__sub__(other)

Subtract a resource spec, returning a new ResourceSpec.

Parameters:

Name	Type	Description	Default
other	ResourceSpec	The resource spec to subtract.	required

Returns:

Type	Description
ResourceSpec	A new ResourceSpec with the remaining resources.

Raises:

Type	Description
ValueError	If the subtraction would result in negative resources.

is_satisfied_by(available_resources)

Check if the resource requirements are satisfied by the available resources.

Parameters:

Name	Type	Description	Default
available_resources	ResourceSpec	The available resources.	required

Returns:

Type	Description
bool	True if the resource requirements are satisfied by the available resources, False
bool	otherwise.

Examples:

>>> ResourceSpec(cpus=4, memory=16384, gpus={"0": 8.0}).is_satisfied_by(
...     ResourceSpec(cpus=8, memory=32768, gpus={"0": 16.0})
... )
True
>>> ResourceSpec(cpus=4, memory=16384, gpus={"0": 8.0}).is_satisfied_by(
...     ResourceSpec(cpus=2, memory=8192, gpus={"0": 4.0})
... )
False

get_cpu_cores()

Get the number of CPU cores available on the system.

Returns:

Type	Description
int	The number of CPU cores available on the system.

Examples:

>>> get_cpu_cores()
4

get_file_ext(filename)

Return the extension/suffixes of a filename or path.

Removes the leading dot. If the filename has multiple suffixes, e.g., 'file.tar.gz' this function consideres all suffixes together, i.e., the string 'tar.gz' will be returned.

Parameters:

Name	Type	Description	Default
filename	str | Path	The filename or path to extract the extension from.	required

Returns:

Type	Description
str	The extension of the filename or path.

Examples:

>>> get_file_ext("file.tar.gz")
'tar.gz'
>>> get_file_ext("file.tar")
'tar'

get_gpu_memory_dict()

Get the approximate amount of memory in MiB available on each NVIDIA GPU device.

Queries the system for any NVIDIA NVML controllable devices and returns a dictionary mapping the device index to the approxiamte total memory in MiB available on that device. If NVML is not available or no devices are found, an empty dictionary is returned.

Returns:

Type	Description
dict[str, PositiveFloat]	A dictionary mapping the device index to the approxiamte total memory in MiB available on
dict[str, PositiveFloat]	that device.

Examples:

>>> get_gpu_memory_dict()
{'0': 8192.0, '1': 8192.0}
>>> get_gpu_memory_dict()
{...}

get_memory_mib()

Get the approximate amount of memory in MiB available on the system.

Returns:

Type	Description
PositiveInt	The amount of memory in MiB available on the system.

Examples:

>>> get_memory_mib()
16384

get_system_resource_spec()

Get the system resource specifications.

Returns:

Type	Description
ResourceSpec	The system resource specifications.

Examples:

>>> get_system_resource_spec()
ResourceSpec(cpus=..., memory=..., gpus={...})