Converting notebooks to function#

MLRun annotations are used to identify the code that needs to be converted into an MLRun function. They provide non-intrusive hints that indicate which parts of your notebook should be considered as the code of the function.

Annotations start a code block using # mlrun: start-code and end a code block(s), with # mlrun: end-code. Use the #mlrun: ignore to exclude items from the code qualified annotations. Make sure that the annotations include anything required for the function to run.

# mlrun: start-code


def sub_handler():
    return "hello world"

The # mlrun: ignore annotation enables you to exclude the cell from the function code.

# mlrun: ignore

# the handler in the code section below will not call this sub_handler
def sub_handler():
    return "I will be ignored!"

def handler(context, event):
    return sub_handler()


# mlrun: end-code

Convert the function with mlrun.code_to_function and run the handler. Notice the returned value under results.

Note

Make sure to save the notebook before running mlrun.code_to_function so that the lateset changes will be reflected in the function.

from mlrun import code_to_function

some_function = code_to_function("some-function-name", kind="job", code_output=".")
some_function.run(name="some-function-name", handler="handler", local=True)

> 2021-11-01 07:42:44,930 [info] starting run some-function-name uid=742e7d6e930c48f3a2f1d6175e971455 DB=http://mlrun-api:8080
project uid iter start state name labels inputs parameters results artifacts
default 0 Nov 01 07:42:45 completed some-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello world
> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-01 07:42:45,214 [info] run executed, status=completed

<mlrun.model.RunObject at 0x7f3fc9ed81d0>

In this section

Named annotations#

The # mlrun: start-code and # mlrun: end-code annotations can be used to convert different code sections to different MLRun, functions in the same notebook. To do so add the name of the MLRun function to the end of the annotation as shown in the example below.

# mlrun: start-code my-function-name


def handler(context, event):
    return "hello from my-function"


# mlrun: end-code my-function-name

Convert the function and run the handler. Notice that the handler that is being used and that there is a change in the returned value under results.

my_function = code_to_function("my-function-name", kind="job")
my_function.run(name="my-function-name", handler="handler", local=True)

> 2021-11-01 07:42:53,892 [info] starting run my-function-name uid=e4bbc3cae21042439cc1c3cb9631751c DB=http://mlrun-api:8080
project uid iter start state name labels inputs parameters results artifacts
default 0 Nov 01 07:42:54 completed my-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from my-function
> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-01 07:42:54,137 [info] run executed, status=completed

<mlrun.model.RunObject at 0x7f3fc9ac71d0>

Note

Make sure to use the name given to the code_to_function parameter (name='my-function-name' in the example above) so that all relevant start-code and end-code annotations are included. If none of the annotations are marked with the function's name, all annotations without any name are used.

Multi section function#

You can use the # mlrun: start-code and # mlrun: end-code annotations multiple times in a notebook since the whole notebook is scanned. The annotations can be named like the following example, and they can be nameless. If you choose nameless, remember all nameless annotations in the notebook are used.

# mlrun: start-code multi-section-function-name

function_name = "multi-section-function-name"

# mlrun: end-code multi-section-function-name

Any code between those sections are not included:

function_name = "I will be ignored!"

# mlrun: start-code multi-section-function-name

def handler(context, event):
    return f"hello from {function_name}"

# mlrun: end-code multi-section-function-name

my_multi_section_function = code_to_function("multi-section-function-name", kind="job")
my_multi_section_function.run(
    name="multi-section-function-name", handler="handler", local=True
)

> 2021-11-01 07:43:05,587 [info] starting run multi-section-function-name uid=9ac6a0e977a54980b657bae067c2242a DB=http://mlrun-api:8080
project uid iter start state name labels inputs parameters results artifacts
default 0 Nov 01 07:43:05 completed multi-section-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from multi-section-function-name
> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-01 07:43:05,834 [info] run executed, status=completed

<mlrun.model.RunObject at 0x7f3fc9a24e10>

Annotation's position in code cell#

# mlrun: start-code and # mlrun: end-code annotations are relative to their positions inside the code block. Notice how the assignments to function_name below # mlrun: end-code don't override the assignment between the annotations in the function's context.

# mlrun: start-code part-cell-function


def handler(context, event):
    return f"hello from {function_name}"


function_name = "part-cell-function"

# mlrun: end-code part-cell-function

function_name = "I will be ignored"

my_multi_section_function = code_to_function("part-cell-function", kind="job")
my_multi_section_function.run(name="part-cell-function", handler="handler", local=True)

> 2021-11-01 07:43:14,347 [info] starting run part-cell-function uid=5426e665c7bc4ba492e0a704c5555fb6 DB=http://mlrun-api:8080
project uid iter start state name labels inputs parameters results artifacts
default 0 Nov 01 07:43:14 completed part-cell-function
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from part-cell-function
> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-01 07:43:14,628 [info] run executed, status=completed

<mlrun.model.RunObject at 0x7f3fc9a2bf50>

Guidelines#

  • Make sure that every # mlrun: start-code has a corresponding # mlrun: end-code before the next # mlrun: start-code in the notebook.

  • Only one MLRun function can have a nameless annotation per notebook.

  • Do not use multiple # mlrun: start-code nor multiple # mlrun: end-code annotations in a single code cell. Only the first appearance of each is used.

  • Using single annotations:

    • Use a # mlrun: start-code alone, and all code blocks from the annotation to the end of the notebook are included.

    • Use a # mlrun: end-code alone, and all code blocks from the beginning of the notebook to the annotation are included.