.md .pdf

repository open issue suggest edit

Real-time serving pipelines (graphs)

MLRun graphs enable building and running DAGs (directed acyclic graph). Graphs are composed of individual steps. The first graph element accepts an Event object, transform/process the event and pass the result to the next steps in the graph. The final result can be written out to some destination (file, DB, stream, etc.) or returned back to the caller (one of the graph steps can be marked with .respond()).

The serving graphs can be composed of pre-defined graph steps, block-type elements (model servers, routers, ensembles, data readers and writers, data engineering tasks, validators, etc.), custom steps, or from native python classes/functions. A graph can have data processing steps, model ensembles, model servers, post-processing, etc. (see the Advanced Model Serving Graph Notebook Example). Graphs can auto-scale and span multiple function containers (connected through streaming protocols).

Different steps can run on the same local function, or run on a remote function. You can call existing functions from the graph and reuse them from other graphs, as well as scale up and down different components individually.

Graphs can run inside your IDE or Notebook for test and simulation. Serving graphs are built on top of Nuclio (real-time serverless engine), MLRun Jobs, MLRun Storey (native Python async and stream processing engine), and other MLRun facilities.

In this section

• Getting started
  • Steps
  • Create a function
  • Build the graph
  • Visualize the graph
  • Test the function
  • Deploy the function
  • Test the deployed function
• Use cases
  • Data and feature engineering (using the feature store)
  • Example of a simple model serving router
  • Example of advanced data processing and serving ensemble
  • Example of an NLP processing pipeline with real-time streaming
• Graph concepts and state machine
  • The Event object
  • The Context object
  • Topology
  • Building distributed graphs
  • Error handling
• Writing custom steps
• Built-in steps
  • Base Operators
  • Data Transformations
  • External IO and data enrichment
  • Sources
  • Targets
  • Models
  • Routers
  • Other
• Demos and Tutorials
  • Distributed (Multi-function) Pipeline Example
  • Advanced Model Serving Graph - Notebook Example
• Serving graph high availability configuration
  • Using Nuclio with stream triggers
  • Consumer function configuration
  • Remote function retry mechanism
  • Configuration considerations

Build and use function images (Kubernetes) Getting started