Skip to main content
Ctrl+K
Logo image
Ctrl+K

MLRun basics

  • Using MLRun
    • MLRun architecture
    • MLRun ecosystem
  • Tutorials and Examples
    • Deploying an LLM using MLRun
    • Model monitoring using LLM
    • Machine learning tutorials
      • Quick start tutorial for machine learning
      • Train, compare, and register models
      • Serving pre-trained ML/DL models
      • Projects and automated ML pipeline
      • Realtime monitoring and drift detection
      • Batch inference and drift detection
      • Add MLOps to existing code
      • Feature store example (stocks)
      • MLflow tracker
    • Demos
    • MLRun cheat sheet
  • Installation and setup guide
    • Install MLRun on Kubernetes
    • Set up your environment

Gen AI tasks

  • Gen AI development workflow
  • Gen AI data management
    • Using LLMs to process unstructured data
    • Vector databases
    • Guardrails for data management
  • Developing a gen AI pipeline
    • Working with RAG in MLRun
    • Evaluating LLMs with MLRun
    • Fine tuning LLMs with MLRun
  • Deploying gen AI application serving pipelines
    • Serving gen AI models
    • GPU utilization
    • Gen AI realtime serving graph
  • Gen AI liveOps

MLOps tasks

  • MLOps development workflow
  • Ingest and process data
    • Using data sources and items
    • Logging datasets
    • Ingest data using the feature store
    • Ingest features with Spark
  • Develop and train models
    • Model training and tracking
      • Create a basic training job
      • Working with data and model artifacts
      • Automated experiment tracking
      • Using the built-in training function
      • Hyperparameter tuning optimization
    • Training with the feature store
  • Deploy models and applications
    • Real-time serving
      • Using built-in model serving classes
      • Build your own model serving class
      • Test and deploy a model server
      • Model serving API
    • Serving with the feature store
    • Batch inference
    • Canary functions and rolling upgrades
  • CI/CD automation with Git
    • Create a project using a Git source
    • Load project YAML from Git, Zip, Tar source
    • Run pipelines with GitHub Actions, GitLab

Core components

  • Projects and automation
    • Create, save, and use projects
    • Git best practices
    • Load projects
    • Run, build, and deploy functions
    • Build and run workflows/pipelines
    • Working with secrets
    • MLRun project bootstrapping with project_setup.py
  • Functions
    • Kinds of functions (runtimes)
      • Function of type job
      • Function of type serving
      • Application runtime
      • Dask distributed runtime
        • Running Dask on the cluster with MLRun
        • Pipelines using Dask, Kubeflow and MLRun
      • Databricks runtime
      • MPIJob and Horovod runtime
      • Spark Operator runtime
      • Nuclio real-time functions
    • Create and use functions
    • Converting notebooks to function
    • Attach storage to functions
    • Images and their usage in MLRun
    • Build function image
    • Function hub
    • Using a Git repo as a function hub
  • Data and artifacts
    • Data stores
    • Data items
    • Artifacts
    • Model Artifacts
    • Logging artifacts
  • Model monitoring
    • Model monitoring description
    • Model monitoring user flow
    • View model monitoring results in the platform UI and in Grafana
    • Writing a model monitoring application
  • Alerts and notifications
    • Alerts
    • Notifications
  • Batch runs and workflows
    • MLRun execution context
    • Decorators and auto-logging
    • Running a task (job)
    • Running a multi-stage workflow
    • Running multiple functions with ExitHandler
    • Running multiple functions in parallel
    • Running a conditional workflow
    • Configuring runs and functions
    • Scheduled jobs and workflows
    • Notifications
  • Real-time serving pipelines (graphs)
    • Getting started
    • Use cases
    • Graph concepts and state machine
    • Model serving graph
    • Writing custom steps
    • Built-in steps
    • Demos and tutorials
      • Distributed (multi-function) pipeline example
      • Advanced model serving graph - notebook example
    • Serving graph high availability configuration
    • Error handling
  • Feature store
    • Feature store overview
    • Feature sets
    • Sources and targets
    • Feature set transformations
    • Feature vectors
    • Feature store end-to-end demo
      • Part 1: Data ingestion
      • Part 2: Training
      • Part 3: Serving
      • Part 4: Automated ML pipeline

References

  • Index
  • API by module
    • mlrun
    • mlrun.alerts.alert
    • mlrun.artifacts
    • mlrun.common
      • mlrun.common.schemas
        • mlrun.common.schemas.alert
        • mlrun.common.schemas.api_gateway
        • mlrun.common.schemas.artifact
        • mlrun.common.schemas.notification
    • mlrun.config
    • mlrun.datastore
    • mlrun.db
    • mlrun.execution
    • mlrun.feature_store
    • mlrun.frameworks
      • AutoMLRun
      • TensorFlow.Keras
      • PyTorch
      • SciKit-Learn
      • XGBoost
      • LightGBM
    • mlrun.model
    • mlrun.model_monitoring
    • mlrun.package
      • mlrun.package.packager.Packager
      • mlrun.package.packagers.default_packager.DefaultPackager
      • mlrun.package.packagers_manager.PackagersManager
      • mlrun.package.errors
      • mlrun.package.packagers.python_standard_library_packagers
        • mlrun.package.packagers.python_standard_library_packagers.BoolPackager
        • mlrun.package.packagers.python_standard_library_packagers.BytearrayPackager
        • mlrun.package.packagers.python_standard_library_packagers.BytesPackager
        • mlrun.package.packagers.python_standard_library_packagers.DictPackager
        • mlrun.package.packagers.python_standard_library_packagers.FloatPackager
        • mlrun.package.packagers.python_standard_library_packagers.FrozensetPackager
        • mlrun.package.packagers.python_standard_library_packagers.IntPackager
        • mlrun.package.packagers.python_standard_library_packagers.ListPackager
        • mlrun.package.packagers.python_standard_library_packagers.NonePackager
        • mlrun.package.packagers.python_standard_library_packagers.PathPackager
        • mlrun.package.packagers.python_standard_library_packagers.SetPackager
        • mlrun.package.packagers.python_standard_library_packagers.StrPackager
        • mlrun.package.packagers.python_standard_library_packagers.TuplePackager
      • mlrun.package.packagers.numpy_packagers
        • mlrun.package.packagers.numpy_packagers.NumPyNDArrayDictPackager
        • mlrun.package.packagers.numpy_packagers.NumPyNDArrayListPackager
        • mlrun.package.packagers.numpy_packagers.NumPyNDArrayPackager
        • mlrun.package.packagers.numpy_packagers.NumPyNumberPackager
        • mlrun.package.packagers.numpy_packagers.NumPySupportedFormat
      • mlrun.package.packagers.pandas_packagers
        • mlrun.package.packagers.pandas_packagers.PandasDataFramePackager
        • mlrun.package.packagers.pandas_packagers.PandasSeriesPackager
        • mlrun.package.packagers.pandas_packagers.PandasSupportedFormat
    • mlrun.platforms
    • mlrun.projects
    • mlrun.run
    • mlrun.runtimes
      • mlrun.runtimes
    • mlrun.serving
    • storey.transformations - Graph transformations
  • Command-Line Interface
  • Resources
  • Glossary

Change log

  • Change log
  • Repository
  • Suggest edit
  • Open issue
  • .md

Develop and train models

Develop and train models#

In this section

  • Model training and tracking
  • Training with the feature store

previous

Ingest features with Spark

next

Model training and tracking

By Iguazio

© Copyright 2023, Iguazio.