{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Converting code to function\n", "\n", "MLRun annotations are used to identify the code that needs to be converted into an MLRun function.\n", "They provide non-intrusive hints which indicate which parts of your notebook should be considered as the code of the function." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Annotations start a code block using `# mlrun: start-code` and end a code block(s), with `# mlrun: end-code`.\n", "Use the `#mlrun: ignore` to exclude items from the code qualified annotations.\n", "Make sure that the annotations include anything required for the function to run." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# mlrun: start-code\n", "\n", "def sub_handler():\n", " return \"hello world\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `# mlrun: ignore` annotation enables you to exclude the cell from the function code." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# mlrun: ignore\n", "\n", "# the handler in the code section below will not call this sub_handler\n", "def sub_handler():\n", " return \"I will be ignored!\"" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def handler(context, event):\n", " return sub_handler()\n", "\n", "# mlrun: end-code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Convert the function with `mlrun.code_to_function` and run the handler. Notice the returned value under `results`." ] }, { "cell_type": "markdown", "source": [ "```{admonition} Note\n", "Make sure to save the notebook before running `mlrun.code_to_function` so that the lateset changes will be reflected in the function.\n", "```" ], "metadata": { "collapsed": false } }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:42:44,930 [info] starting run some-function-name uid=742e7d6e930c48f3a2f1d6175e971455 DB=http://mlrun-api:8080\n" ] }, { "data": { "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
projectuiditerstartstatenamelabelsinputsparametersresultsartifacts
default0Nov 01 07:42:45completedsome-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello world
\n", "
\n", "
\n", "
\n", " Title\n", " ×\n", "
\n", " \n", "
\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ " > to track results use the .show() or .logs() methods or click here to open in UI" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:42:45,214 [info] run executed, status=completed\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from mlrun import code_to_function\n", "\n", "some_function = code_to_function('some-function-name', kind='job', code_output='.')\n", "some_function.run(name='some-function-name', handler='handler', local=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**In this section**\n", "- [Named annotations](#named-annotations)\n", "- [Multi section function](#multi-section-function)\n", "- [Annotation's position in code cell](#annotation's-position-in-code-cell)\n", "- [Guidelines](#guidelines)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Named annotations\n", "The `# mlrun: start-code` and `# mlrun: end-code` annotations can be used to convert different code sections to different MLRun, functions in the same notebook.\n", "To do so add the name of the MLRun function to the end of the annotation as shown in the example below." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# mlrun: start-code my-function-name\n", "\n", "def handler(context, event):\n", " return \"hello from my-function\"\n", "\n", "# mlrun: end-code my-function-name" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Convert the function and run the handler. Notice that the handler that is being used and that there is a change in the returned value under `results`." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:42:53,892 [info] starting run my-function-name uid=e4bbc3cae21042439cc1c3cb9631751c DB=http://mlrun-api:8080\n" ] }, { "data": { "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
projectuiditerstartstatenamelabelsinputsparametersresultsartifacts
default0Nov 01 07:42:54completedmy-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from my-function
\n", "
\n", "
\n", "
\n", " Title\n", " ×\n", "
\n", " \n", "
\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ " > to track results use the .show() or .logs() methods or click here to open in UI" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:42:54,137 [info] run executed, status=completed\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_function = code_to_function('my-function-name', kind='job')\n", "my_function.run(name='my-function-name', handler='handler', local=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "```{admonition} Note\n", "Make sure to use the name given to the `code_to_function` parameter (`name='my-function-name'` in the example above) so that all relevant `start-code` and `end-code` annotations are included. If none of the annotations are marked with the function's name, all annotations without any name are used.\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Multi section function\n", "You can use the `# mlrun: start-code` and `# mlrun: end-code` annotations multiple times in a notebook since the whole notebook is scanned.\n", "The annotations can be named like the following example, and they can be nameless. If you choose nameless, remember all nameless annotations in the notebook are used." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# mlrun: start-code multi-section-function-name\n", "\n", "function_name = \"multi-section-function-name\"\n", "\n", "# mlrun: end-code multi-section-function-name" ] }, { "cell_type": "raw", "metadata": {}, "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Any code between those sections are not included:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "function_name = \"I will be ignored!\"" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "# mlrun: start-code multi-section-function-name" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def handler(context, event):\n", " return f\"hello from {function_name}\"" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# mlrun: end-code multi-section-function-name" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:43:05,587 [info] starting run multi-section-function-name uid=9ac6a0e977a54980b657bae067c2242a DB=http://mlrun-api:8080\n" ] }, { "data": { "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
projectuiditerstartstatenamelabelsinputsparametersresultsartifacts
default0Nov 01 07:43:05completedmulti-section-function-name
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from multi-section-function-name
\n", "
\n", "
\n", "
\n", " Title\n", " ×\n", "
\n", " \n", "
\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ " > to track results use the .show() or .logs() methods or click here to open in UI" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:43:05,834 [info] run executed, status=completed\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_multi_section_function = code_to_function('multi-section-function-name', kind='job')\n", "my_multi_section_function.run(name='multi-section-function-name', handler='handler', local=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Annotation's position in code cell\n", "\n", "`# mlrun: start-code` and `# mlrun: end-code` annotations are relative to their positions inside the code block. Notice how the assignments to `function_name` below `# mlrun: end-code` don't override the assignment between the annotations in the function's context." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# mlrun: start-code part-cell-function\n", "\n", "def handler(context, event):\n", " return f\"hello from {function_name}\"\n", "\n", "function_name = \"part-cell-function\"\n", "\n", "# mlrun: end-code part-cell-function\n", "\n", "function_name = \"I will be ignored\"" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:43:14,347 [info] starting run part-cell-function uid=5426e665c7bc4ba492e0a704c5555fb6 DB=http://mlrun-api:8080\n" ] }, { "data": { "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
projectuiditerstartstatenamelabelsinputsparametersresultsartifacts
default0Nov 01 07:43:14completedpart-cell-function
v3io_user=admin
kind=
owner=admin
host=jupyter-8459699595-z544v
return=hello from part-cell-function
\n", "
\n", "
\n", "
\n", " Title\n", " ×\n", "
\n", " \n", "
\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/html": [ " > to track results use the .show() or .logs() methods or click here to open in UI" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "> 2021-11-01 07:43:14,628 [info] run executed, status=completed\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "my_multi_section_function = code_to_function('part-cell-function', kind='job')\n", "my_multi_section_function.run(name='part-cell-function', handler='handler', local=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Guidelines\n", "- Make sure that every `# mlrun: start-code` has a corresponding `# mlrun: end-code` before the next `# mlrun: start-code` in the notebook.\n", "- Only one MLRun function can have a nameless annotation per notebook.\n", "- Do not use multiple `# mlrun: start-code` nor multiple `# mlrun: end-code` annotations in a single code cell. Only the first appearance of each is used.\n", "- Using single annotations:\n", " - Use a `# mlrun: start-code` alone, and all code blocks from the annotation to the end of the notebook are included.\n", " - Use a `# mlrun: end-code` alone, and all code blocks from the beginning of the notebook to the annotation are included." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 4 }