mlrun.package.packagers_manager.PackagersManager#

class mlrun.package.packagers_manager.PackagersManager(default_packager: Optional[Type[Packager]] = None)[source]#

Bases: object

A packager manager is holding the project’s packagers and sending them objects to pack and data items to unpack.

It prepares the instructions / log hint configurations and then looks for the first packager who fits the task. That’s why when the manager collects its packagers, it first collects builtin MLRun packagers and only then the user’s custom packagers, this way user’s custom packagers will have higher priority.

Initialize a packagers manager.

Parameters:

default_packager – The default packager should be a packager that fits to all types. It will be the first packager in the manager’s packagers (meaning it will be used at lowest priority) and it should be found fitting when all packagers managed by the manager do not fit an object or data item. Default to mlrun.DefaultPackager.

Attributes

artifacts

Get the artifacts that were packed by the manager.

results

Get the results that were packed by the manager.
artifacts#

Get the artifacts that were packed by the manager.

Returns:

A list of artifacts.

results#

Get the results that were packed by the manager.

Returns:

A results dictionary.

Methods

__init__([default_packager])

Initialize a packagers manager.

clear_packagers_outputs()

Clear the outputs of all packagers.

collect_packagers(packagers[, default_priority])

Collect the provided packagers.

link_packages(additional_artifacts, ...)

Link packages between each other according to the provided extra data and metrics spec keys.

pack(obj, log_hint)

Pack an object using one of the manager's packagers.

unpack(data_item, type_hint)

Unpack an object using one of the manager's packagers.
__init__(default_packager: Optional[Type[Packager]] = None)[source]#

Initialize a packagers manager.

Parameters:

default_packager – The default packager should be a packager that fits to all types. It will be the first packager in the manager’s packagers (meaning it will be used at lowest priority) and it should be found fitting when all packagers managed by the manager do not fit an object or data item. Default to mlrun.DefaultPackager.

clear_packagers_outputs()[source]#

Clear the outputs of all packagers. This method should be called at the end of the run after logging all artifacts as some will require uploading the files that will be deleted in this method.

collect_packagers(packagers: List[Union[Type, str]], default_priority: int = 5)[source]#

Collect the provided packagers. Packagers passed as module paths will be imported and validated to be of type Packager. If needed to import all packagers from a module, use the module path with a “*” at the end (packager with a name that start with a ‘_’ won’t be collected).

Notice: Only packagers that are declared in the module will be collected (packagers imported in the module scope won’t be collected). For example:

from mlrun import Packager
from x import XPackager

class YPackager(Packager):
    pass

Only “YPackager” will be collected as it is declared in the module, but not “XPackager” which is only imported.

Parameters:

  • packagers – List of packagers to add.

  • default_priority – The default priority to set for the packagers with not set priority (equals to …).

Raises:

MLRunPackageCollectingError – In case the packager could not be collected.

Link packages between each other according to the provided extra data and metrics spec keys. A future link is marked with ellipses (…). If no link was found, None will be used and a warning will get printed.

Parameters:

  • additional_artifacts – Additional artifacts to link (should come from a mlrun.MLClientCtx).

  • additional_results – Additional results to link (should come from a mlrun.MLClientCtx).

pack(obj: Any, log_hint: Dict[str, str]) → Union[Artifact, dict, None, List[Optional[Union[Artifact, dict]]]][source]#

Pack an object using one of the manager’s packagers. A dict (“**”) or list (“*”) unpacking syntax in the log hint key will pack the objects within them in separate packages.

Parameters:

  • obj – The object to pack as an artifact.

  • log_hint – The log hint to use.

Returns:

The packaged artifact or result. None is returned if there was a problem while packing the object. If a prefix of dict or list unpacking was provided in the log hint key, a list of all the arbitrary number of packaged objects will be returned.

Raises:

  • MLRunInvalidArgumentError – If the key in the log hint is noting to log an arbitrary amount of artifacts but the object type does not match the “*” or “**” used in the key.

  • MLRunPackagePackingError – If there was an error during the packing.

unpack(data_item: DataItem, type_hint: Type) → Any[source]#

Unpack an object using one of the manager’s packagers. The data item can be unpacked in two options:

  • As a package: If the data item contains a package and the type hint provided is equal to the object type noted in the package. Or, if it’s a package and a type hint was not provided.

  • As a data item: If the data item is not a package or the type hint provided is not equal to the one noted in the package.

If the type hint is a mlrun.DataItem then it won’t be unpacked.

Notice: It is not recommended to use a different packager than the one who originally packed the object to unpack it. A warning will be shown in that case.

Parameters:

  • data_item – The data item holding the package.

  • type_hint – The type hint to parse the data item as.

Returns:

The unpacked object parsed as type hinted.