Nuclio real-time functions

Nuclio real-time functions#

Nuclio is a high-performance "serverless" framework focused on data, I/O, and compute intensive workloads. It is well integrated with popular data science tools, such as Jupyter and Kubeflow; supports a variety of data and streaming sources; and supports execution over CPUs and GPUs.

You can use Nuclio through a fully managed application service (in the cloud or on-prem) in the Iguazio MLOps Platform. MLRun serving utilizes serverless Nuclio functions to create multi-stage real-time pipelines.

The underlying Nuclio serverless engine uses a high-performance parallel processing engine that maximizes the utilization of CPUs and GPUs, supports 13 protocols and invocation methods (for example, HTTP, Cron, Kafka, Kinesis), and includes dynamic auto-scaling for HTTP and streaming. Nuclio and MLRun support the full life cycle, including auto-generation of micro-services, APIs, load-balancing, logging, monitoring, and configuration management—such that developers can focus on code, and deploy to production faster with minimal work.

Nuclio is extremely fast: a single function instance can process hundreds of thousands of HTTP requests or data records per second. To learn more about how Nuclio works, see the Nuclio architecture documentation.

Nuclio is secure: Nuclio is integrated with Kaniko to allow a secure and production-ready way of building Docker images at run time.

Read more in the Nuclio documentation and the open-source MLRun library.

In this section

Runtimes using Nuclio#

The following MLRun runtimes rely on Nuclio for execution:

  • application

  • nuclio

  • serving

  • remote

Nuclio function example#

You can create your own Nuclio function, for example a data processing function. For every Nuclio function, by default, there is one worker. See Number of GPUs.

The following code illustrates an example of an MLRun function, of kind 'nuclio', that can be deployed to the cluster.

Create a file func.py with the code of the function:

def handler(context, event):
    return "Hello"

Create the project and the Nuclio function:

import mlrun

# Create the project
project = mlrun.get_or_create_project("nuclio-project", "./")

# Create a Nuclio function
project.set_function(
    func="func.py",
    image="mlrun/mlrun",
    kind="nuclio",
    name="nuclio-func",
    handler="handler",
)
project.save()
# Deploy the function in the cluster
project.deploy_function("nuclio-func")

Change the Nuclio runtime#

Nuclio supports multiple runtimes (Python, Node.js, Go, etc.). You can find the full list of supported runtimes in the Nuclio documentation. In order to change the Nuclio runtime of an MLRun function, you can set the nuclio_runtime field in the function spec as shown below:

func = project.set_function("./func.py", name="py11", kind="nuclio", handler="handler")
func.spec.nuclio_runtime = "python:3.11"
func.deploy()

Modify Nuclio trigger configurations#

To modify a trigger configuration in a Nuclio-based function, you can access the Nuclio-based trigger configuration by spec.triggers.<trigger_name>. For example, to set the HTTP trigger mode to async, you can do the following:

func = project.set_function(
    func=f"func.py", name="func", image="python:3.9", kind="nuclio", handler="handler"
)
func.with_http()
func.spec.config["spec.triggers.http"]["mode"] = "async"

All possible trigger configurations are available in the Nuclio documentation.

See more examples in Create functions.

Modifying Nuclio functions#

When working with Nuclio functions (remote / serving / application runtimes) and API gateways in MLRun, any modifications/deletions should be made in MLRun only. This is also relevant for modifying the spec of a function or API gateway. Changes that are made in Nuclio are not guaranteed to be synced to MLRun and can cause unexpected behavior.

For example, if you create a remote function in MLRun (which is then created in Nuclio) and then delete it in Nuclio, the function status might show as ready, but invoking it will fail.

If you get an error similar to:
Error when translating nuclio names to mlrun names in api gateway: number of functions doesn't match the mlrun functions in annotation
it means that the API gateway was updated at least once manually in Nuclio itself, without using the MLRun SDK. To rectify this situation, delete the gateway, and then create it again.

API gateway#

Note

Since API Gateway is only a Nuclio feature, it is supported only for Nuclio runtimes.

API gateway is an entity that Nuclio creates on top of one or two functions. From a Kubernetes perspective, an API gateway is just an ingress that links to the Nuclio function. If an API gateway is created on top of two functions, it's a canary function. The other feature that API gateways provide is authorization. Read more about API gateways in the Nuclio documentation.

Creating and managing API gateways using the SDK#

Note

Requires Nuclio v1.13.1 or higher.

MLRun SDK:

Create an API gateway with basic authorization#

Assume you already deployed a nuclio function:

fn = project.set_function(
            func="nuclio_function.py",
            name=f"nuclio-func",
            kind="nuclio",
            image="python:3.9",
            handler="handler",
        )

Define the API gateway entity:

my_api_gateway = mlrun.runtimes.nuclio.api_gateway.APIGateway(
            mlrun.runtimes.nuclio.api_gateway.APIGatewayMetadata(
                name="gw-with-basic-auth",
            ),
            mlrun.runtimes.nuclio.api_gateway.APIGatewaySpec(
                functions=fn,
                project=project.name,
            ),
    )

Add basic authorization configuration:

my_api_gateway.with_basic_auth(
    username="test",
    password="pass",
)

Add an access key authentication that authenticates users via the Iguazio backend:

my_api_gateway.with_access_key_auth()

Create (or update) the API gateway
It's crucial to update the defined API Gateway entity with the entity returned from the store_api_gateway method. This fills in important fields like host/path, etc.

my_api_gateway = project.store_api_gateway(my_api_gateway)

store_api_gateway checks that api_gateway is ready to be invoked before returning it. You can also check if api gateway is in ready state:

my_api_gateway.is_ready()

If you know that the API gateway was changed (for example, in another jupyter notebook), load the latest changes:

my_api_gateway.sync()

Invoke the API gateway
Since the gateway is configured with basic auth, you need to pass authorization credentials:

response = my_api_gateway.invoke(auth=("test", "pass"), verify=False)

Updating an API gateway#

Add basic auth on top the created API gateway

my_api_gateway.with_basic_auth("test", "pass")

Change the canary function percentages

my_api_gateway.with_canary([fn1, fn2], [10, 90])

Update the API gateway

my_api_gateway = project.store_api_gateway(my_api_gateway)

Create an API gateway in the UI#

To create an API gateway in the UI:

  1. In your project page, press API Gateways tab, then press NEW API GATEWAY.

  2. Select an Authentication Mode:

    • None (default)

    • Basic

    • Access key

    • OAuth2 (Not supported yet in MLRun SDK)

    and fill in any required values.

  3. Type in the API Gateway parameters:

    • Name: The name of the API Gateway. Required

    • Description: A description of the API Gateway.

    • Host: The host of the API Gateway. (Relevant for open-source only.)

    • Path: The path of the API Gateway.

  4. In Primary, type in the function that is triggered via the API Gateway.

Note

Do not change the configuration by using the UI: it breaks the functionality.