mlrun.platforms

mlrun.platforms.VolumeMount

alias of mlrun.platforms.iguazio.Mount

mlrun.platforms.auto_mount(pvc_name='', volume_mount_path='', volume_name=None)[source]

choose the mount based on env variables and params

volume will be selected by the following order: - k8s PVC volume when both pvc_name and volume_mount_path are set - iguazio v3io volume when V3IO_ACCESS_KEY and V3IO_USERNAME env vars are set - k8s PVC volume when env var is set: MLRUN_PVC_MOUNT=<pvc-name>:<mount-path>

mlrun.platforms.mount_pvc(pvc_name, volume_name='pipeline', volume_mount_path='/mnt/pipeline')[source]

Modifier function to apply to a Container Op to simplify volume, volume mount addition and enable better reuse of volumes, volume claims across container ops.

Usage:

train = train_op(...)
train.apply(mount_pvc('claim-name', 'pipeline', '/mnt/pipeline'))
mlrun.platforms.mount_v3io(name='v3io', remote='', mount_path='', access_key='', user='', secret=None, volume_mounts=None)[source]

Modifier function to apply to a Container Op to volume mount a v3io path

Parameters
  • name – the volume name

  • remote – the v3io path to use for the volume. ~/ prefix will be replaced with /users/<username>/

  • mount_path – the volume mount path (deprecated, exists for backwards compatibility, prefer to use mounts instead)

  • access_key – the access key used to auth against v3io. if not given V3IO_ACCESS_KEY env var will be used

  • user – the username used to auth against v3io. if not given V3IO_USERNAME env var will be used

  • secret – k8s secret name which would be used to get the username and access key to auth against v3io.

  • volume_mounts – list of VolumeMount. empty volume mounts & remote will default to mount /v3io & /User.

mlrun.platforms.mount_v3io_extended(name='v3io', remote='', mounts=None, access_key='', user='', secret=None)[source]

Modifier function to apply to a Container Op to volume mount a v3io path

Parameters
  • name – the volume name

  • remote – the v3io path to use for the volume. ~/ prefix will be replaced with /users/<username>/

  • mounts – list of mount & volume sub paths (type Mount). empty mounts & remote mount /v3io & /User

  • access_key – the access key used to auth against v3io. if not given V3IO_ACCESS_KEY env var will be used

  • user – the username used to auth against v3io. if not given V3IO_USERNAME env var will be used

  • secret – k8s secret name which would be used to get the username and access key to auth against v3io.

mlrun.platforms.mount_v3io_legacy(name='v3io', remote='~/', mount_path='/User', access_key='', user='', secret=None)[source]

Modifier function to apply to a Container Op to volume mount a v3io path :param name: the volume name :param remote: the v3io path to use for the volume. ~/ prefix will be replaced with /users/<username>/ :param mount_path: the volume mount path :param access_key: the access key used to auth against v3io. if not given V3IO_ACCESS_KEY env var will be used :param user: the username used to auth against v3io. if not given V3IO_USERNAME env var will be used :param secret: k8s secret name which would be used to get the username and access key to auth against v3io.

mlrun.platforms.pprint(object, stream=None, indent=1, width=80, depth=None, *, compact=False)[source]

Pretty-print a Python object to a stream [default is sys.stdout].

mlrun.platforms.sleep(seconds)

Delay execution for a given number of seconds. The argument may be a floating point number for subsecond precision.

mlrun.platforms.v3io_cred(api='', user='', access_key='')[source]

Modifier function to copy local v3io env vars to task

Usage:

train = train_op(...)
train.apply(use_v3io_cred())
mlrun.platforms.watch_stream(url, shard_ids: Optional[list] = None, seek_to: Optional[str] = None, interval=None, is_json=False, **kwargs)[source]

watch on a v3io stream and print data every interval

example::

watch_stream(‘v3io:///users/admin/mystream’)

Parameters
  • url – stream url

  • shard_ids – range or list of shard IDs

  • seek_to – where to start/seek (‘EARLIEST’, ‘LATEST’, ‘TIME’, ‘SEQUENCE’)

:param interval watch interval time in seconds, 0 to run once and return :param is_json: indicate the payload is json (will be deserialized)