Quick-Start Guide

MLRun is an end-to-end open-source MLOps solution to manage and automate your analytics and machine learning lifecycle, from data ingestion, through model development and full pipeline/model deployment, to model monitoring. Its primary goal is to ease the development of machine learning pipeline at scale and help organizations build a robust process for moving from the research phase to fully operational production deployments.

MLRun is automating the process of moving code to production by implementing a serverless approach, where different tasks or services are executed over elastic serverless functions (read more about MLRun functions), in this quick start guide we will use existing (marketplace) functions, see the tutorial with more detailed example of how to create and use functions.

Table of Contents

Working with MLRun

If you need to install MLRun, refer to the Installation Guide.

Note: If you are using the Iguazio MLOps Platform, MLRun already comes preinstalled and integrated in your system.

If you are not viewing this quick-start guide from a Jupyter Lab instance, open it on your cluster, create a new notebook, and copy the sections below to the notebook to run them.

Set Environment

Before you begin, initialize MLRun by calling set_environment and provide it with the project name. All the work will be saved and tracked under that project.

import mlrun

project_name = 'quick-start'
mlrun.set_environment(project=project_name, user_project=True)
> 2021-11-15 22:56:00,358 [info] loaded project quick-start-admin from MLRun DB
('quick-start-admin', 'v3io:///projects/{{run.project}}/artifacts')

Train a Model

MLRun introduces the concept of functions. You can run your own code in functions, or use functions from the function marketplace. Functions can run locally or over elastic “serverless” engines (as containers over kubernetes).

In the example below, you’ll use the sklearn_classifier from MLRun function marketplace to train a model and use a sample dataset (CSV file) as the input. You can read more on how to use data items from different data sources or from the Feature Store.

Note: When training a model in an air-gapped site:

If you are working in MLRun:

  1. Download your data file and save it locally.

  2. Run:
    import os
    os.environ["env_name"] = 1

  3. Use the same command for the sample data set path, for example:
    source_url = mlrun.get_sample_path("data/iris/iris_dataset.csv")

If your system is integrated with an MLOps Platform:

  1. Download your data file and save it locally.

  2. In the UI, click the settings icon (Settings) in the top-right of the header in any page to open the Settings dialog.

  3. Click Environment variables | Create a new environment variable, and set an environmental variable: SAMPLE_DATA_SOURCE_URL_PREFIX = the relative path to locally-stored data. For example:
    /v3io/bigdata/path/...

  4. Use the same command for the sample data set path, for example:
    source_url = mlrun.get_sample_path("data/iris/iris_dataset.csv")

# import the training function from the marketplace (hub://)
train = mlrun.import_function('hub://sklearn_classifier')

# Get a sample dataset path (points to MLRun data samples repository)
source_url = mlrun.get_sample_path("data/iris/iris_dataset.csv")

# run the function and specify input dataset path and some parameters (algorithm and label column name)
train_run = train.run(name='train',
                      inputs={'dataset': source_url},
                      params={'model_pkg_class': 'sklearn.linear_model.LogisticRegression',
                              'label_column': 'label'})
> 2021-11-15 22:56:01,887 [info] starting run train uid=da2cba1514ee4308b0a84a0372ddabc9 DB=http://mlrun-api:8080
> 2021-11-15 22:56:02,096 [info] Job is running in the background, pod: train-nt8bv
> 2021-11-15 22:56:09,420 [info] run executed, status=completed
final state: completed
project uid iter start state name labels inputs parameters results artifacts
quick-start-admin 0 Nov 15 22:56:07 completed train
v3io_user=admin
kind=job
owner=admin
host=train-nt8bv
class=sklearn.linear_model.LogisticRegression
dataset
model_pkg_class=sklearn.linear_model.LogisticRegression
label_column=label
accuracy=0.9375
test-error=0.0625
auc-micro=0.9921875
auc-weighted=1.0
f1-score=0.9206349206349206
precision_score=0.9047619047619048
recall_score=0.9555555555555556
test_set
confusion-matrix
precision-recall-multiclass
roc-multiclass
model

> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-15 22:56:11,537 [info] run executed, status=completed

The run output above contains a link to the MLRun UI. Click it to inspect the various aspects of the jobs you run:

ui-info

As well as their artifacts:

ui-artifacts

When running the function in a Jupyter notebook, the output cell for your function execution contains a table with run information — including the state of the execution, all inputs and parameters, and the execution results and artifacts.

MLRun quick start train output

Test the Model

Now that you have a trained model, you can test it: run a task that uses the test_classifier function from the function marketplace to run the selected trained model against the test dataset. The test dataset was returned from the training task (train_run) in the previous step.

test = mlrun.import_function('hub://test_classifier')

You can then run the function as part of your project, just as any other function that you have written yourself. To view the function documentation, call the doc method:

test.doc()
function: test-classifier
test a classifier using held-out or new data
default handler: test_classifier
entry points:
  test_classifier: Test one or more classifier models against held-out dataset

Using held-out test features, evaluates the peformance of the estimated model

Can be part of a kubeflow pipeline as a test step that is run post EDA and
training/validation cycles
    context  - the function context, default=
    models_path(DataItem)  - artifact models representing a file or a folder, default=
    test_set(DataItem)  - test features and labels, default=
    label_column(str)  - column name for ground truth labels, default=
    score_method(str)  - for multiclass classification, default=micro
    plots_dest(str)  - dir for test plots, default=
    model_evaluator  - NOT IMPLEMENTED: specific method to generate eval, passed in as string or available in this folder, default=None
    default_model(str)  - , default=model.pkl
    predictions_column(str)  - column name for the predictions column on the resulted artifact, default=yscore
    model_update  - (True) update model, when running as stand alone no need in update, default=True

Configure parameters for the test function (params), and provide the selected trained model from the train task as an input artifact (inputs)

test_run = test.run(name="test",
                    params={"label_column": "label"},
                    inputs={"models_path": train_run.outputs['model'],
                            "test_set": train_run.outputs['test_set']})
> 2021-11-15 22:56:28,306 [info] starting run test uid=b365d3a00cef4773aa7d6aadfd751cad DB=http://mlrun-api:8080
> 2021-11-15 22:56:28,510 [info] Job is running in the background, pod: test-z5qrv
> 2021-11-15 22:56:33,848 [info] run executed, status=completed
final state: completed
project uid iter start state name labels inputs parameters results artifacts
quick-start-admin 0 Nov 15 22:56:32 completed test
v3io_user=admin
kind=job
owner=admin
host=test-z5qrv
models_path
test_set
label_column=label
accuracy=0.9777777777777777
test-error=0.022222222222222223
auc-micro=0.9985185185185185
auc-weighted=0.9985392720306513
f1-score=0.9769016328156113
precision_score=0.9761904761904763
recall_score=0.9791666666666666
confusion-matrix
precision-recall-multiclass
roc-multiclass
test_set_preds

> to track results use the .show() or .logs() methods or click here to open in UI
> 2021-11-15 22:56:34,652 [info] run executed, status=completed

Serve the Model

MLRun serving can take MLRun models or standard model files and produce managed, real-time, serverless functions using the Nuclio real-time serverless framework. Nuclio is built around data, I/O, and compute-intensive workloads, and is focused on performance and flexibility. Nuclio is also deeply integrated into the MLRun framework. See MLRun Serving documentation to learn more about the rich serving capabilities MLRun has to offer.

To deploy your model using the v2_model_server function, run the following code:

serve = mlrun.import_function('hub://v2_model_server')
model_name='iris'
serve.add_model(model_name, model_path=train_run.outputs['model'])
addr = serve.deploy()
> 2021-11-15 22:57:00,042 [info] Starting remote function deploy
2021-11-15 22:57:00  (info) Deploying function
2021-11-15 22:57:00  (info) Building
2021-11-15 22:57:00  (info) Staging files and preparing base images
2021-11-15 22:57:00  (info) Building processor image
2021-11-15 22:57:01  (info) Build complete
2021-11-15 22:57:07  (info) Function deploy complete
> 2021-11-15 22:57:07,998 [info] successfully deployed function: {'internal_invocation_urls': ['nuclio-quick-start-admin-v2-model-server.default-tenant.svc.cluster.local:8080'], 'external_invocation_urls': ['default-tenant.app.yh38.iguazio-cd2.com:32503']}

The invoke method enables to programmatically test the function.

import json

inputs = [[5.1, 3.5, 1.4, 0.2],
          [7.7, 3.8, 6.7, 2.2]]
my_data = json.dumps({'inputs': inputs})
serve.invoke(f'v2/models/{model_name}/infer', my_data)
> 2021-11-15 22:57:12,675 [info] invoking function: {'method': 'POST', 'path': 'http://nuclio-quick-start-admin-v2-model-server.default-tenant.svc.cluster.local:8080/v2/models/iris/infer'}
{'id': '057372bd-518d-4e88-987a-cfb64dd0498a',
 'model_name': 'iris',
 'outputs': [0, 2]}

Open the Nuclio UI to view the function and test it.

Nuclio Functions UI

For a more detailed walk-through, refer to the getting-started tutorial.