Thank you for your participation

We hope that the materials we used during this time together were useful and gave you a better understanding of OpenShift AI, Workbenches, Pipelines, Model Serving and Model Optimization techniques.

If you notice issues with the content and/or want to send us a pull request, we’ll appreciate it very much.

The instructions of this lab are always available at https://rhpds.github.io/showroom-summit2025-lb2959-neural-magic/.

What you learnt

During this lab, you have gained hands-on experience with:

  • OpenShift AI Platform

    • Creating and managing Data Science Projects

    • Setting up Data Connections

    • Configuring Pipeline Servers

    • Working with Workbenches

    • Designing and implementing Pipelines

  • Model Optimization with llm-compressor

    • Using llm-compressor to optimize models

    • Applying optimization techniques through Workbenches and Pipelines

    • Understanding different quantization approaches (int4, fp8, int8)

  • Model Evaluation

    • Using lm-eval to assess model performance

    • Comparing model metrics before and after optimization

  • Model Deployment

    • Deploying models using vLLM through OpenShift AI

    • Comparing performance between base and optimized models

    • Understanding the impact of optimization on inference efficiency

Acknowledgements

Here is a list of people who have participated to the creation and delivery of this lab.

  • Luis Tomás Bolívar

  • Rob Shaw

  • Alireza Rahmani

  • Juliano Mohr

  • Tony Kay