mlrun.package.packagers_manager.PackagersManager

class mlrun.package.packagers_manager.PackagersManager(default_packager: type[Packager] | None = None)

Bases: object

A packager manager holds the project's packagers and sends them objects to pack, and data items to unpack.

It prepares the instructions / log hint configurations and then looks for the first packager that fits the task.

Initialize a packagers manager.

Parameters:

default_packager -- The default packager should be a packager that fits all types. It should fit any packagers that are managed by the manager that do not fit an object or data item. Default to mlrun.DefaultPackager.

Attributes

artifacts	Get the artifacts that were packed by the manager.
results	Get the results that were packed by the manager.

artifacts

Get the artifacts that were packed by the manager.

Returns:

A list of tuples with the artifacts and their context.log_artifact method kwargs.

results

Get the results that were packed by the manager.

Returns:

A results dictionary.

Methods

__init__([default_packager])	Initialize a packagers manager.
clear_packagers_outputs()	Clear the outputs of all packagers.
collect_packagers(packagers[, default_priority])	Collect the provided packagers.
get_bundles_results(logged_outputs)	Get the bundles results with updated store paths according to the logged outputs.
link_packages(additional_artifact_uris, ...)	Link packages to each other according to the provided extra data and metrics spec keys.
pack(obj, log_hint)	Pack an object using one of the manager's packagers.
unpack(data_item, type_hint)	Unpack an object using one of the manager's packagers.

__init__(default_packager: type[Packager] | None = None)

Initialize a packagers manager.

Parameters:

default_packager -- The default packager should be a packager that fits all types. It should fit any packagers that are managed by the manager that do not fit an object or data item. Default to mlrun.DefaultPackager.

clear_packagers_outputs()

Clear the outputs of all packagers. This method should be called at the end of the run, only after logging all artifacts, to ensure that files that require uploading have already been uploaded.

collect_packagers(packagers: list[type[Packager] | str], default_priority: int = 5)

Collect the provided packagers. Packagers passed as module paths are imported and validated to be of type Packager. If it's needed to import all packagers from a module, use the module path with an asterisk "*" at the end. (A packager with a name that starts with an underscore '_' is not collected.)

Notice: Only packagers that are declared in the module are collected (packagers imported in the module scope aren't collected). For example:

from mlrun import Packager
from x import XPackager


class YPackager(Packager):
    pass

Only "YPackager" is collected since it is declared in the module, but not "XPackager", which is only imported.

Parameters:

• packagers -- List of packagers to add.

• default_priority -- The default priority for the packagers that don't have a set priority (equals to ...).

Raises:

MLRunPackageCollectingError -- In case the packager could not be collected.

get_bundles_results(logged_outputs: dict) → dict

Get the bundles results with updated store paths according to the logged outputs.

Parameters:

logged_outputs -- The logged outputs dictionary from the MLRun context.

Returns:

A results dictionary with the bundles updated store paths.

link_packages(additional_artifact_uris: dict, additional_results: dict) → set[Artifact]

Link packages to each other according to the provided extra data and metrics spec keys. A future link is marked with ellipses (...). If no link is found, None is used and a warning is printed.

Parameters:

• additional_artifact_uris -- Additional artifact URIs to link (should come from an mlrun.MLClientCtx).

• additional_results -- Additional results to link (should come from an mlrun.MLClientCtx).

Returns:

A set of the additional artifacts that require updates post linking (the packagers artifacts were not logged yet).

pack(obj: Any, log_hint: LogHint) → Artifact | dict | None | list[Artifact | dict | None]

Pack an object using one of the manager's packagers.

A list unpacking syntax ("*") in the log hint key unbundle the given object to pack each of its item separately. If a number is added before the asterisk ("X*"), it represent the level of unbundling.

For example, if the object is a nested list [[1, 2], [3, 4]] and the log hint key is "1*", the object will be unbundled once to [1, 2] and [3, 4], and each of these items will be packed separately. If the log hint key is "2*", the object will be unbundled twice to 1, 2, 3, and 4, and each of these items will be packed separately.

By default, an asterisk without a number will unbundle all the levels possible.

Parameters:

• obj -- The object to pack as an artifact.

• log_hint -- The log hint to use.

Returns:

The packaged artifact or result. None is returned if there was a problem while packing the object. If unbundling is performed, a list of all the unbundled packaged objects is returned.

Raises:

• MLRunInvalidArgumentError -- If the key in the log hint instructs do not follow the unbundling syntax.

• MLRunPackagePackingError -- If there was an error during the packing.

• MLRunPackageUnbundlingError -- If there was an error during the unbundling.

unpack(data_item: DataItem | dict | list, type_hint: type) → Any

Unpack an object using one of the manager's packagers. The data item can be unpacked in two ways:

• As a package: If the data item contains a package and the type hint provided is equal to the object type noted in the package. Or, if it's a package and a type hint was not provided.

• As a data item: If the data item is not a package or the type hint provided is not equal to the one noted in the package.

If the data_item received is a collection (a dict or list), each item in the collection will be unpacked according to the type hint provided.

If the type hint is a mlrun.DataItem then it won't be unpacked.

Notice: It is not recommended to use a different packager than the one that originally packed the object to unpack it. A warning displays in that case.

Parameters:

• data_item -- The data item holding the package. Can be a collection of data items (the type hint must match a packager that supports initializing a collection).

• type_hint -- The type hint to parse the data item as.

Returns:

The unpacked object parsed as type hinted.