Flow de modelo de clasificación de tickets: procesamiento de datos y features

Curso de MLOPS: Despliegue de Modelos de Machine Learning

Fundamentos de MLOps y tracking de modelos

¿Qué es MLOps y para qué sirve?

Tracking de modelos en localhost con MLflow

Tracking de modelos en localhost: directorio personalizado

Etapas del ciclo de MLOps

Componentes de MLOps

Tracking de modelos con MLflow y SQLite

Tracking de modelos con MLflow en la nube

Tracking del ciclo de vida de modelos de machine learning

Tracking de experimentos con MLflow: preprocesamiento de datos

Tracking de experimentos con MLflow: definición de funciones

Tracking de experimentos con MLflow: tracking de métricas e hiperparámetros

Tracking de experimentos con MLflow: reporte de clasificación

Entrenamiento de modelos baseline y análisis en UI de MLflow

MLflow Model Registry: registro y uso de modelos

Registro de modelos con mlflow.client

Testing de modelo desde MLflow con datos de prueba

¿Para qué sirve el tracking de modelos en MLOps?

Orquestación de pipelines de machine learning

Tasks con Prefect

Flows con Prefect

Flow de modelo de clasificación de tickets: procesamiento de datos y features

Flow de modelo de clasificación de tickets: integración de las tasks

Flow de modelo de clasificación de tickets: ejecución de tasks

¿Cómo se integra la orquestación en MLOps?

Despliegue de modelo de machine learning

Despligue con Docker y FastAPI: configuración y requerimientos

Despligue con Docker y FastAPI: definición de clases y entry point

Despligue con Docker y FastAPI: procesamiento de predicciones en main app

Despligue con Docker y FastAPI: configuración de la base de datos

Despliegue y pruebas de modelo de machine learning en localhost

Despliegue y pruebas de modelo de machine learning en la nube

¿Qué hacer con el modelo desplegado?

Monitoreo de modelo de machine learning en producción

¿Cómo monitorear modelos de machine learning en producción?

Entrenamiento de modelo baseline

Preparar datos para crear reporte con Evidently

Análisis de la calidad de los datos con Evidently

Creación de reportes con Grafana

¿Cómo mejorar tus procesos de MLOps?

You don't have access to this class

Keep learning! Join and start boosting your career

If you already have an account,

Resources

How to implement a pipeline for ticket classification?

Developing a robust machine learning pipeline is essential to handle automated data analysis processes. In this context, we will implement a pipeline focused on ticket classification, integrating data processing and model training processes in an efficient flow using tools such as Visual Studio Code and relevant libraries.

How to set up the working environment?

Before starting to develop our pipeline, it is essential to prepare the working environment. Two important files are provided:

config.py:

Houses crucial variables for the workflow, including data storage paths and model parameter settings.

It allows versatility by including a configurable version, helping to identify specific pipeline executions.

# Basic example structureconfig = { "data_path": "processed_data_path", "version": 2, "language": "English", "filename_input": "input_name", "model_params": {"param1": "value1", "param2": "value2"},}

utils.py:

Includes auxiliary functions not defined as tasks, but useful for general workflow, such as decoding tags or storing and loading objects with picle.

How to integrate tasks in the flow?

Starting with the flow development, a number of libraries are imported:

Scikit-learn for training and model metrics.
Prefect to orchestrate the pipeline.
Custom modules for text processing and feature extraction.

How to define a text processing task?

Creating a task that handles text processing is essential to prepare data for the model.

from prefect import task
 @task(retries=3, retry_delay_seconds=60, name="textProcessingTask", tags=["data_processing"])def text_processing(language: str, filename: str, version: int): processor = TextProcessingProcessor(language) processed_data = processor.run(filename, version) return processed_data

How is feature extraction executed?

Feature extraction allows you to structure the data more effectively for the model.

@task(retries=3, retry_delay_seconds=60, name="featureExtractionTask", tags=["feature_engineering"])def feature_extraction(data_path: str, version: int): feature_processor = FeatureExtractionProcessor() features = feature_processor.run(data_path, version) return features

How is the whole pipeline executed?

Once the tasks are defined, they are integrated into a flow:

from prefect import Flow
 # Flow definitionwith Flow("Ticket Classification Pipeline") as flow: processed_data = text_processing(config['language'], config['filename_input'], config['version']) features = feature_extraction(config['data_path'], config['version'])
flow.run().

This flow first runs the text processing and then the feature extraction, ensuring that the data is properly prepared for the model.

What are the next steps?

With the pipeline in place, the next step will be to integrate new tasks to complete the execution of the processes required for model training and evaluation. This includes tasks to apply transformations to the data, handle hyperparameters, and evaluate the accuracy of the model. Continuing to explore and improve the robustness of your pipeline will ensure not only accurate, but also efficient results, which is critical for any machine learning project.

Contributions 0

Questions 1

Sort by:

Want to see more contributions, questions and answers from the community?