mlrun.package.packagers_manager.PackagersManager#

class mlrun.package.packagers_manager.PackagersManager(default_packager: Type[Packager] | None = None)[source]#

Bases: object

A packager manager holds the project's packagers and sends them objects to pack, and data items to unpack.

It prepares the instructions / log hint configurations and then looks for the first packager that fits the task.

Initialize a packagers manager.

Parameters:

default_packager -- The default packager should be a packager that fits all types. It should fit any packagers that are managed by the manager that do not fit an object or data item. Default to mlrun.DefaultPackager.

Attributes

artifacts

Get the artifacts that were packed by the manager.

results

Get the results that were packed by the manager.
artifacts#

Get the artifacts that were packed by the manager.

Returns:

A list of artifacts.

results#

Get the results that were packed by the manager.

Returns:

A results dictionary.

Methods

__init__([default_packager])

Initialize a packagers manager.

clear_packagers_outputs()

Clear the outputs of all packagers.

collect_packagers(packagers[, default_priority])

Collect the provided packagers.

link_packages(additional_artifacts, ...)

Link packages to each other according to the provided extra data and metrics spec keys.

pack(obj, log_hint)

Pack an object using one of the manager's packagers.

unpack(data_item, type_hint)

Unpack an object using one of the manager's packagers.
__init__(default_packager: Type[Packager] | None = None)[source]#

Initialize a packagers manager.

Parameters:

default_packager -- The default packager should be a packager that fits all types. It should fit any packagers that are managed by the manager that do not fit an object or data item. Default to mlrun.DefaultPackager.

clear_packagers_outputs()[source]#

Clear the outputs of all packagers. This method should be called at the end of the run, only after logging all artifacts, to ensure that files that require uploading have already been uploaded.

collect_packagers(packagers: List[Type[Packager] | str], default_priority: int = 5)[source]#

Collect the provided packagers. Packagers passed as module paths are imported and validated to be of type Packager. If it's needed to import all packagers from a module, use the module path with an asterisk "*" at the end. (A packager with a name that starts with an underscore '_' is not collected.)

Notice: Only packagers that are declared in the module are collected (packagers imported in the module scope aren't collected). For example:

from mlrun import Packager
from x import XPackager

class YPackager(Packager):
    pass

Only "YPackager" is collected since it is declared in the module, but not "XPackager", which is only imported.

Parameters:

  • packagers -- List of packagers to add.

  • default_priority -- The default priority for the packagers that don't have a set priority (equals to ...).

Raises:

MLRunPackageCollectingError -- In case the packager could not be collected.

Link packages to each other according to the provided extra data and metrics spec keys. A future link is marked with ellipses (...). If no link is found, None is used and a warning is printed.

Parameters:

  • additional_artifacts -- Additional artifacts to link (should come from an mlrun.MLClientCtx).

  • additional_results -- Additional results to link (should come from an mlrun.MLClientCtx).

pack(obj: Any, log_hint: Dict[str, str]) → Artifact | dict | None | List[Artifact | dict | None][source]#

Pack an object using one of the manager's packagers. A dict ("**") or list ("*") unpacking syntax in the log hint key packs the objects within them in separate packages.

Parameters:

  • obj -- The object to pack as an artifact.

  • log_hint -- The log hint to use.

Returns:

The packaged artifact or result. None is returned if there was a problem while packing the object. If a prefix of dict or list unpacking was provided in the log hint key, a list of all the arbitrary number of packaged objects is returned.

Raises:

  • MLRunInvalidArgumentError -- If the key in the log hint instructs to log an arbitrary number of artifacts but the object type does not match the "*" or "**" used in the key.

  • MLRunPackagePackingError -- If there was an error during the packing.

unpack(data_item: DataItem, type_hint: Type) → Any[source]#

Unpack an object using one of the manager's packagers. The data item can be unpacked in two ways:

  • As a package: If the data item contains a package and the type hint provided is equal to the object type noted in the package. Or, if it's a package and a type hint was not provided.

  • As a data item: If the data item is not a package or the type hint provided is not equal to the one noted in the package.

If the type hint is a mlrun.DataItem then it won't be unpacked.

Notice: It is not recommended to use a different packager than the one that originally packed the object to unpack it. A warning displays in that case.

Parameters:

  • data_item -- The data item holding the package.

  • type_hint -- The type hint to parse the data item as.

Returns:

The unpacked object parsed as type hinted.