Deploying gen AI application serving pipelines

MLRun serving can produce managed ML application pipelines using real-time, auto-scaling, Nuclio serverless functions. The application pipeline includes all the steps including: accepting events or data, preparing the required model features, inferring results using one or more models, and driving actions.

In this section

• Serving gen AI models
• GPU utilization
• Gen AI realtime serving graph
• Integrating an OpenAI LLM with MLRun
• Integrating a Hugging Face image classification model with MLRun

See also

• Deploying an LLM using MLRun

• Model monitoring using LLM

• Model monitoring tutorial

• Model monitoring

• Alerts and notifications