Deploying gen AI application serving pipelines

MLRun serving can produce managed ML application pipelines using real-time auto-scaling Nuclio serverless functions. The application pipeline includes all the steps including: accepting events or data, preparing the required model features, inferring results using one or more models, and driving actions.

In this section

• Serving gen AI models
• GPU utilization
• Gen AI realtime serving graph

See also

• Deploying an LLM using MLRun

• Model monitoring using LLM

• Model monitoring user flow

• Alerts and notifications